16:35:33.00 
 16:35:33.01 Welcome to the Bitnami elasticsearch container
 16:35:33.10 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-elasticsearch
 16:35:33.18 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-elasticsearch/issues
 16:35:33.19 
 16:35:33.29 INFO  ==> ** Starting Elasticsearch setup **
 16:35:35.29 INFO  ==> Configuring/Initializing Elasticsearch...
 16:35:36.21 INFO  ==> Setting default configuration
 16:35:36.60 INFO  ==> Configuring Elasticsearch cluster settings...
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
 16:36:21.48 INFO  ==> ** Elasticsearch setup finished! **

 16:36:21.78 INFO  ==> ** Starting Elasticsearch **
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-10-21T16:37:47,383][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/5.4.0-96-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS]
[2022-10-21T16:37:47,480][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] JVM home [/opt/bitnami/java]
[2022-10-21T16:37:47,483][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-13277069663402277153, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2022-10-21T16:38:34,531][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [aggs-matrix-stats]
[2022-10-21T16:38:34,533][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [analysis-common]
[2022-10-21T16:38:34,679][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [geo]
[2022-10-21T16:38:34,680][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [ingest-common]
[2022-10-21T16:38:34,680][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [ingest-geoip]
[2022-10-21T16:38:34,681][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [ingest-user-agent]
[2022-10-21T16:38:34,681][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [kibana]
[2022-10-21T16:38:34,682][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [lang-expression]
[2022-10-21T16:38:34,682][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [lang-mustache]
[2022-10-21T16:38:34,683][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [lang-painless]
[2022-10-21T16:38:34,683][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [mapper-extras]
[2022-10-21T16:38:34,684][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [parent-join]
[2022-10-21T16:38:34,684][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [percolator]
[2022-10-21T16:38:34,685][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [rank-eval]
[2022-10-21T16:38:34,685][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [reindex]
[2022-10-21T16:38:34,686][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [repository-url]
[2022-10-21T16:38:34,687][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [tasks]
[2022-10-21T16:38:34,688][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded module [transport-netty4]
[2022-10-21T16:38:34,689][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-1] loaded plugin [repository-s3]
[2022-10-21T16:38:37,180][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-master-1] using [1] data paths, mounts [[/bitnami/elasticsearch/data (192.168.13.158:/dockerdata-nfs/onap/elastic-master-0)]], net usable_space [95.3gb], net total_space [99.9gb], types [nfs4]
[2022-10-21T16:38:37,283][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-master-1] heap size [123.7mb], compressed ordinary object pointers [true]
[2022-10-21T16:38:39,183][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] node name [onap-sdnrdb-master-1], node ID [smPDS3xtShGp5xZr0iCmrQ], cluster name [sdnrdb-cluster]
[2022-10-21T16:40:27,328][INFO ][o.e.t.NettyAllocator     ] [onap-sdnrdb-master-1] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}]
[2022-10-21T16:40:29,182][INFO ][o.e.d.DiscoveryModule    ] [onap-sdnrdb-master-1] using discovery type [zen] and seed hosts providers [settings]
[2022-10-21T16:40:34,984][WARN ][o.e.g.DanglingIndicesState] [onap-sdnrdb-master-1] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-10-21T16:40:37,481][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] initialized
[2022-10-21T16:40:37,483][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] starting ...
[2022-10-21T16:40:40,786][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][3] overhead, spent [416ms] collecting in the last [1s]
[2022-10-21T16:40:40,982][INFO ][o.e.t.TransportService   ] [onap-sdnrdb-master-1] publish_address {10.233.72.3:9300}, bound_addresses {0.0.0.0:9300}
[2022-10-21T16:40:47,083][INFO ][o.e.b.BootstrapChecks    ] [onap-sdnrdb-master-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-10-21T16:40:50,181][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][12] overhead, spent [500ms] collecting in the last [1s]
[2022-10-21T16:40:52,286][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] setting initial configuration to VotingConfiguration{smPDS3xtShGp5xZr0iCmrQ,{bootstrap-placeholder}-onap-sdnrdb-master-2,0N0iZdQhS8CqWzQH9aUJWg}
[2022-10-21T16:40:56,883][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r},{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr},{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 97, reason: ApplyCommitRequest{term=2, version=97, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T16:40:58,795][INFO ][o.e.c.s.ClusterSettings  ] [onap-sdnrdb-master-1] updating [action.auto_create_index] from [true] to [false]
[2022-10-21T16:41:01,084][INFO ][o.e.h.AbstractHttpServerTransport] [onap-sdnrdb-master-1] publish_address {10.233.72.3:9200}, bound_addresses {0.0.0.0:9200}
[2022-10-21T16:41:01,084][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-1] started
[2022-10-21T16:51:20,989][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][635] overhead, spent [325ms] collecting in the last [1s]
[2022-10-21T17:37:36,020][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 177, reason: ApplyCommitRequest{term=2, version=177, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:37:38,364][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 178, reason: ApplyCommitRequest{term=2, version=178, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:39:35,207][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 179, reason: ApplyCommitRequest{term=2, version=179, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:39:36,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T17:39:36,582][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T17:39:36,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T17:39:36,886][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,080][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,180][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,790][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T17:39:37,985][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T17:39:38,698][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 180, reason: ApplyCommitRequest{term=2, version=180, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:39:39,251][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 181, reason: ApplyCommitRequest{term=2, version=181, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:39:40,313][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 182, reason: ApplyCommitRequest{term=2, version=182, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:40:51,899][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T17:40:51,981][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 248, reason: becoming candidate: onLeaderFailure
[2022-10-21T17:40:52,615][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 249, reason: ApplyCommitRequest{term=2, version=249, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:42:26,059][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T17:42:26,088][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 249, reason: becoming candidate: onLeaderFailure
[2022-10-21T17:42:27,653][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 251, reason: ApplyCommitRequest{term=2, version=251, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T17:49:06,223][WARN ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][4093] overhead, spent [997ms] collecting in the last [1.4s]
[2022-10-21T18:37:38,267][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 319, reason: ApplyCommitRequest{term=2, version=319, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:37:41,442][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 320, reason: ApplyCommitRequest{term=2, version=320, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:39:41,436][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 321, reason: ApplyCommitRequest{term=2, version=321, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:39:41,793][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:41,886][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:41,888][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,084][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,103][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,192][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,303][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,501][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,685][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:42,780][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,091][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,291][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,397][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,488][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,600][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,700][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:43,791][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:44,184][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T18:39:44,280][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 322, reason: ApplyCommitRequest{term=2, version=322, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:39:44,482][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T18:39:44,777][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 323, reason: ApplyCommitRequest{term=2, version=323, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:39:45,543][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 324, reason: ApplyCommitRequest{term=2, version=324, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:40:52,226][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T18:40:52,286][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 394, reason: becoming candidate: onLeaderFailure
[2022-10-21T18:40:52,837][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 395, reason: ApplyCommitRequest{term=2, version=395, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T18:42:29,196][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T18:42:30,662][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 395, reason: becoming candidate: onLeaderFailure
[2022-10-21T18:42:33,627][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 398, reason: ApplyCommitRequest{term=2, version=398, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:37:41,434][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 466, reason: ApplyCommitRequest{term=2, version=466, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:37:43,642][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 467, reason: ApplyCommitRequest{term=2, version=467, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:39:47,164][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 468, reason: ApplyCommitRequest{term=2, version=468, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:39:47,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T19:39:47,480][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:47,592][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:47,690][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:47,805][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:47,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,083][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,388][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,528][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,699][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,797][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,893][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,985][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T19:39:48,987][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,008][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,116][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,201][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,482][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 469, reason: ApplyCommitRequest{term=2, version=469, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:39:49,587][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T19:39:49,846][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 470, reason: ApplyCommitRequest{term=2, version=470, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:39:50,579][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 471, reason: ApplyCommitRequest{term=2, version=471, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:40:52,586][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T19:40:52,590][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 538, reason: becoming candidate: onLeaderFailure
[2022-10-21T19:40:53,112][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 539, reason: ApplyCommitRequest{term=2, version=539, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T19:42:34,553][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T19:42:34,584][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 540, reason: becoming candidate: onLeaderFailure
[2022-10-21T19:42:35,689][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 542, reason: ApplyCommitRequest{term=2, version=542, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:37:43,536][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 611, reason: ApplyCommitRequest{term=2, version=611, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:37:45,999][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 612, reason: ApplyCommitRequest{term=2, version=612, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:39:51,349][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 613, reason: ApplyCommitRequest{term=2, version=613, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:39:51,782][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:51,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:51,790][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T20:39:51,890][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,088][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,187][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,281][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,388][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,488][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,779][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:52,979][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,081][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,289][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,479][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,779][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,784][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,884][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T20:39:53,999][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 614, reason: ApplyCommitRequest{term=2, version=614, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:39:54,509][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 615, reason: ApplyCommitRequest{term=2, version=615, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:39:54,953][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 616, reason: ApplyCommitRequest{term=2, version=616, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:40:52,792][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T20:40:52,794][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 684, reason: becoming candidate: onLeaderFailure
[2022-10-21T20:40:53,004][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 685, reason: ApplyCommitRequest{term=2, version=685, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T20:42:37,532][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T20:42:37,549][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 686, reason: becoming candidate: onLeaderFailure
[2022-10-21T20:42:38,867][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 688, reason: ApplyCommitRequest{term=2, version=688, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:37:46,494][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 757, reason: ApplyCommitRequest{term=2, version=757, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:37:48,777][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 758, reason: ApplyCommitRequest{term=2, version=758, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:39:56,935][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 759, reason: ApplyCommitRequest{term=2, version=759, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:39:57,195][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,288][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,492][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,596][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,789][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:57,923][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,183][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,191][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,288][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,292][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,479][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,685][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T21:39:58,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,089][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,381][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,492][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,692][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T21:39:59,779][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 760, reason: ApplyCommitRequest{term=2, version=760, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:40:00,383][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 761, reason: ApplyCommitRequest{term=2, version=761, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:40:00,800][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 762, reason: ApplyCommitRequest{term=2, version=762, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:40:52,822][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T21:40:52,830][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 830, reason: becoming candidate: onLeaderFailure
[2022-10-21T21:40:53,216][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 831, reason: ApplyCommitRequest{term=2, version=831, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T21:42:40,729][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T21:42:40,742][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 832, reason: becoming candidate: onLeaderFailure
[2022-10-21T21:42:41,519][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 834, reason: ApplyCommitRequest{term=2, version=834, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:37:48,792][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 904, reason: ApplyCommitRequest{term=2, version=904, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:37:51,955][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 905, reason: ApplyCommitRequest{term=2, version=905, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:40:02,750][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 906, reason: ApplyCommitRequest{term=2, version=906, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:40:02,904][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:02,988][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,120][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,289][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,484][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,602][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,681][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,701][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,888][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:03,900][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,004][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,090][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,101][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,183][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,299][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,414][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,498][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T22:40:04,685][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 907, reason: ApplyCommitRequest{term=2, version=907, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:40:05,108][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 908, reason: ApplyCommitRequest{term=2, version=908, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:40:05,930][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 909, reason: ApplyCommitRequest{term=2, version=909, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:40:52,925][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T22:40:52,928][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 976, reason: becoming candidate: onLeaderFailure
[2022-10-21T22:40:53,219][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 977, reason: ApplyCommitRequest{term=2, version=977, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T22:42:43,781][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T22:42:43,794][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 978, reason: becoming candidate: onLeaderFailure
[2022-10-21T22:42:44,815][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 980, reason: ApplyCommitRequest{term=2, version=980, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:37:51,814][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1049, reason: ApplyCommitRequest{term=2, version=1049, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:37:54,663][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1050, reason: ApplyCommitRequest{term=2, version=1050, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:40:07,534][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1051, reason: ApplyCommitRequest{term=2, version=1051, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:40:07,888][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:07,900][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:07,982][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,086][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,197][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,297][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,484][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,587][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,726][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,879][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T23:40:08,986][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,100][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,190][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,384][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,502][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,592][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,684][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,779][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-21T23:40:09,895][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1052, reason: ApplyCommitRequest{term=2, version=1052, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:40:10,121][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1053, reason: ApplyCommitRequest{term=2, version=1053, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:40:11,124][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1054, reason: ApplyCommitRequest{term=2, version=1054, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:40:52,997][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-21T23:40:53,085][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1123, reason: becoming candidate: onLeaderFailure
[2022-10-21T23:40:53,520][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1124, reason: ApplyCommitRequest{term=2, version=1124, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:42:46,114][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-21T23:42:46,183][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1125, reason: becoming candidate: onLeaderFailure
[2022-10-21T23:42:47,656][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1127, reason: ApplyCommitRequest{term=2, version=1127, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-21T23:43:26,804][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][25327] overhead, spent [415ms] collecting in the last [1.1s]
[2022-10-22T00:37:54,946][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1196, reason: ApplyCommitRequest{term=2, version=1196, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:37:57,759][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1197, reason: ApplyCommitRequest{term=2, version=1197, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:40:12,559][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1198, reason: ApplyCommitRequest{term=2, version=1198, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:40:13,192][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,681][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,779][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,804][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,885][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:13,893][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,179][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,286][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,602][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,699][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,880][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,892][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T00:40:14,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,001][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,092][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,179][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,190][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,197][WARN ][o.e.t.TransportService   ] [onap-sdnrdb-master-1] Transport response handler not found of id [37864]
[2022-10-22T00:40:15,529][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,601][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T00:40:15,657][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1199, reason: ApplyCommitRequest{term=2, version=1199, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:40:15,970][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1200, reason: ApplyCommitRequest{term=2, version=1200, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:40:16,629][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1201, reason: ApplyCommitRequest{term=2, version=1201, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:40:53,308][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T00:40:53,314][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1268, reason: becoming candidate: onLeaderFailure
[2022-10-22T00:40:53,604][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1269, reason: ApplyCommitRequest{term=2, version=1269, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T00:42:49,218][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T00:42:49,283][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1270, reason: becoming candidate: onLeaderFailure
[2022-10-22T00:42:50,031][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1272, reason: ApplyCommitRequest{term=2, version=1272, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:37:57,811][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1341, reason: ApplyCommitRequest{term=2, version=1341, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:38:00,418][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1342, reason: ApplyCommitRequest{term=2, version=1342, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:17,844][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1343, reason: ApplyCommitRequest{term=2, version=1343, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:18,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,089][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,109][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,298][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,680][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:18,886][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,080][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,099][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,196][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,296][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,379][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,407][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,485][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,588][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,697][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,783][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:19,887][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T01:40:20,014][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T01:40:20,020][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1344, reason: ApplyCommitRequest{term=2, version=1344, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:20,401][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1345, reason: ApplyCommitRequest{term=2, version=1345, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:20,508][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1346, reason: ApplyCommitRequest{term=2, version=1346, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:20,638][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1347, reason: ApplyCommitRequest{term=2, version=1347, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:21,410][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1348, reason: ApplyCommitRequest{term=2, version=1348, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:40:53,396][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T01:40:53,399][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1416, reason: becoming candidate: onLeaderFailure
[2022-10-22T01:40:53,677][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1417, reason: ApplyCommitRequest{term=2, version=1417, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:42:52,204][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T01:42:52,218][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1418, reason: becoming candidate: onLeaderFailure
[2022-10-22T01:42:54,717][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1422, reason: ApplyCommitRequest{term=2, version=1422, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T01:43:24,086][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][32516] overhead, spent [308ms] collecting in the last [1s]
[2022-10-22T02:38:00,428][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1490, reason: ApplyCommitRequest{term=2, version=1490, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T02:38:03,451][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1491, reason: ApplyCommitRequest{term=2, version=1491, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T02:40:22,150][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1492, reason: ApplyCommitRequest{term=2, version=1492, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T02:40:22,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:22,386][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T02:40:22,489][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:22,685][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:22,894][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,083][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,583][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,587][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,688][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,791][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,802][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,885][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:23,984][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,096][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,281][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,482][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,487][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,497][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T02:40:24,750][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1493, reason: ApplyCommitRequest{term=2, version=1493, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T02:40:53,495][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T02:40:53,499][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1552, reason: becoming candidate: onLeaderFailure
[2022-10-22T02:40:53,700][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1553, reason: ApplyCommitRequest{term=2, version=1553, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T02:42:56,363][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T02:42:56,480][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1562, reason: becoming candidate: onLeaderFailure
[2022-10-22T02:42:57,158][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1564, reason: ApplyCommitRequest{term=2, version=1564, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:38:03,428][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1633, reason: ApplyCommitRequest{term=2, version=1633, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:38:06,171][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1634, reason: ApplyCommitRequest{term=2, version=1634, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:40:25,768][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1635, reason: ApplyCommitRequest{term=2, version=1635, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:40:25,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:25,991][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,095][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,280][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,430][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,579][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,800][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,887][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T03:40:26,993][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,084][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,087][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,280][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,293][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,404][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,490][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,579][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,680][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,782][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,881][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T03:40:27,890][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1636, reason: ApplyCommitRequest{term=2, version=1636, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:40:28,189][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1637, reason: ApplyCommitRequest{term=2, version=1637, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:40:28,793][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1638, reason: ApplyCommitRequest{term=2, version=1638, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:40:53,506][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T03:40:53,509][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1696, reason: becoming candidate: onLeaderFailure
[2022-10-22T03:40:53,837][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1697, reason: ApplyCommitRequest{term=2, version=1697, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T03:42:59,579][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T03:42:59,603][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1705, reason: becoming candidate: onLeaderFailure
[2022-10-22T03:43:00,837][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1707, reason: ApplyCommitRequest{term=2, version=1707, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:38:06,171][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1774, reason: ApplyCommitRequest{term=2, version=1774, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:38:08,779][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1775, reason: ApplyCommitRequest{term=2, version=1775, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:40:29,987][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1776, reason: ApplyCommitRequest{term=2, version=1776, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:40:30,197][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,328][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,414][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,506][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,679][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,706][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,890][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:30,903][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,089][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,197][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,408][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,484][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T04:40:31,780][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:32,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:32,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:32,685][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:32,784][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T04:40:32,985][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T04:40:33,099][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T04:40:33,293][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1777, reason: ApplyCommitRequest{term=2, version=1777, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:40:33,455][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1778, reason: ApplyCommitRequest{term=2, version=1778, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:40:33,812][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1779, reason: ApplyCommitRequest{term=2, version=1779, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:40:53,845][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T04:40:53,850][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1828, reason: becoming candidate: onLeaderFailure
[2022-10-22T04:40:54,240][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1830, reason: ApplyCommitRequest{term=2, version=1830, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T04:43:01,976][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T04:43:01,992][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1848, reason: becoming candidate: onLeaderFailure
[2022-10-22T04:43:02,687][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1850, reason: ApplyCommitRequest{term=2, version=1850, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:38:08,814][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1919, reason: ApplyCommitRequest{term=2, version=1919, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:38:11,798][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 1920, reason: ApplyCommitRequest{term=2, version=1920, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:40:35,099][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1921, reason: ApplyCommitRequest{term=2, version=1921, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:40:35,489][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:35,783][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,089][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,187][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,289][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,322][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,392][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,498][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,787][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,887][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:36,989][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,084][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,087][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,184][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,282][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,411][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,480][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,641][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T05:40:37,726][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1922, reason: ApplyCommitRequest{term=2, version=1922, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:40:38,018][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1923, reason: ApplyCommitRequest{term=2, version=1923, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:40:38,884][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 1924, reason: ApplyCommitRequest{term=2, version=1924, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:40:53,898][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T05:40:53,905][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1968, reason: becoming candidate: onLeaderFailure
[2022-10-22T05:40:54,463][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1969, reason: ApplyCommitRequest{term=2, version=1969, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:43:04,289][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T05:43:04,314][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 1992, reason: becoming candidate: onLeaderFailure
[2022-10-22T05:43:05,005][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 1994, reason: ApplyCommitRequest{term=2, version=1994, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T05:43:38,383][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][46913] overhead, spent [410ms] collecting in the last [1s]
[2022-10-22T06:38:12,045][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2063, reason: ApplyCommitRequest{term=2, version=2063, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:38:13,909][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2064, reason: ApplyCommitRequest{term=2, version=2064, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:40:39,938][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2065, reason: ApplyCommitRequest{term=2, version=2065, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:40:39,991][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,079][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,179][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,193][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,288][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,484][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,586][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:40,906][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,496][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,499][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,584][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,684][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,787][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,830][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,920][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T06:40:41,979][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T06:40:42,018][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T06:40:42,030][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2066, reason: ApplyCommitRequest{term=2, version=2066, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:40:42,426][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2067, reason: ApplyCommitRequest{term=2, version=2067, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:40:42,933][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2068, reason: ApplyCommitRequest{term=2, version=2068, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:40:53,915][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T06:40:53,921][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2096, reason: becoming candidate: onLeaderFailure
[2022-10-22T06:40:54,213][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2097, reason: ApplyCommitRequest{term=2, version=2097, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T06:43:06,994][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T06:43:07,092][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2136, reason: becoming candidate: onLeaderFailure
[2022-10-22T06:43:08,313][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2138, reason: ApplyCommitRequest{term=2, version=2138, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:38:13,915][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2206, reason: ApplyCommitRequest{term=2, version=2206, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:38:17,027][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2207, reason: ApplyCommitRequest{term=2, version=2207, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:40:44,168][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2208, reason: ApplyCommitRequest{term=2, version=2208, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:40:44,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,471][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,565][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,584][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,606][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,780][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,995][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:44,996][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,301][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,304][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,390][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,612][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,684][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,726][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,794][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,913][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:45,914][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T07:40:46,059][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T07:40:46,188][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T07:40:46,373][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T07:40:46,388][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2209, reason: ApplyCommitRequest{term=2, version=2209, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:40:47,604][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2210, reason: ApplyCommitRequest{term=2, version=2210, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:40:48,108][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2211, reason: ApplyCommitRequest{term=2, version=2211, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:40:54,882][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T07:40:54,892][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2225, reason: becoming candidate: onLeaderFailure
[2022-10-22T07:40:56,093][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2227, reason: ApplyCommitRequest{term=2, version=2227, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:43:09,975][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T07:43:09,994][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2281, reason: becoming candidate: onLeaderFailure
[2022-10-22T07:43:12,662][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2287, reason: ApplyCommitRequest{term=2, version=2287, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T07:43:49,795][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][54114] overhead, spent [408ms] collecting in the last [1.3s]
[2022-10-22T08:38:17,138][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2355, reason: ApplyCommitRequest{term=2, version=2355, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:38:20,219][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2356, reason: ApplyCommitRequest{term=2, version=2356, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:51,079][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2357, reason: ApplyCommitRequest{term=2, version=2357, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:51,203][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,386][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,489][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,594][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,780][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,881][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,909][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:51,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,081][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,282][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,480][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,487][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,579][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,592][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,702][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,785][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,882][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T08:40:52,883][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2358, reason: ApplyCommitRequest{term=2, version=2358, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:53,111][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2359, reason: ApplyCommitRequest{term=2, version=2359, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:53,193][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2360, reason: ApplyCommitRequest{term=2, version=2360, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:53,351][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2361, reason: ApplyCommitRequest{term=2, version=2361, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:54,093][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2362, reason: ApplyCommitRequest{term=2, version=2362, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:40:54,998][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T08:40:55,001][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2363, reason: becoming candidate: onLeaderFailure
[2022-10-22T08:40:55,229][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2364, reason: ApplyCommitRequest{term=2, version=2364, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T08:43:15,040][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T08:43:15,064][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2433, reason: becoming candidate: onLeaderFailure
[2022-10-22T08:43:16,757][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 2, version: 2435, reason: ApplyCommitRequest{term=2, version=2435, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:09:03,643][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][59222] overhead, spent [286ms] collecting in the last [1s]
[2022-10-22T09:38:20,036][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2505, reason: ApplyCommitRequest{term=2, version=2505, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:38:22,655][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 2, version: 2506, reason: ApplyCommitRequest{term=2, version=2506, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:40:55,021][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T09:40:55,025][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 2, version: 2506, reason: becoming candidate: onLeaderFailure
[2022-10-22T09:40:55,478][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 2, version: 2507, reason: ApplyCommitRequest{term=2, version=2507, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:40:55,715][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:55,788][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:55,884][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:55,985][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,088][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,183][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,201][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,298][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,398][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,487][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,579][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,679][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:56,799][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,089][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,282][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,782][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T09:40:57,779][INFO ][o.e.c.c.JoinHelper       ] [onap-sdnrdb-master-1] failed to join {onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}, minimumTerm=2, optionalJoin=Optional[Join{term=2, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.NotMasterException: Node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] not master for join request
[2022-10-22T09:40:57,785][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T09:40:58,183][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T09:40:58,385][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T09:40:58,918][INFO ][o.e.c.c.JoinHelper       ] [onap-sdnrdb-master-1] failed to join {onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}, minimumTerm=2, optionalJoin=Optional[Join{term=3, lastAcceptedTerm=2, lastAcceptedVersion=2507, sourceNode={onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}, targetNode={onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.72.3:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 3 does not match current term 4
	at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:1013) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:532) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
[2022-10-22T09:40:58,980][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2508, reason: ApplyCommitRequest{term=4, version=2508, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:43:18,129][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T09:43:18,146][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2579, reason: becoming candidate: onLeaderFailure
[2022-10-22T09:43:18,744][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2581, reason: ApplyCommitRequest{term=4, version=2581, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T09:43:51,182][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][61307] overhead, spent [301ms] collecting in the last [1.1s]
[2022-10-22T10:38:22,699][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2650, reason: ApplyCommitRequest{term=4, version=2650, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:38:24,782][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2651, reason: ApplyCommitRequest{term=4, version=2651, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:40:55,040][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T10:40:55,053][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2651, reason: becoming candidate: onLeaderFailure
[2022-10-22T10:40:55,888][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2652, reason: ApplyCommitRequest{term=4, version=2652, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:40:58,561][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2653, reason: ApplyCommitRequest{term=4, version=2653, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:40:58,695][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:40:58,783][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:40:58,880][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:40:58,913][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,002][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,090][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,184][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,196][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,387][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,486][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,680][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T10:40:59,987][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,883][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T10:41:00,985][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T10:41:01,199][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T10:41:01,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T10:41:01,409][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2654, reason: ApplyCommitRequest{term=4, version=2654, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:41:01,894][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2655, reason: ApplyCommitRequest{term=4, version=2655, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:41:02,139][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2656, reason: ApplyCommitRequest{term=4, version=2656, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T10:43:20,926][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T10:43:20,939][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2722, reason: becoming candidate: onLeaderFailure
[2022-10-22T10:43:21,493][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2724, reason: ApplyCommitRequest{term=4, version=2724, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:38:24,813][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2792, reason: ApplyCommitRequest{term=4, version=2792, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:38:27,774][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2793, reason: ApplyCommitRequest{term=4, version=2793, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:40:55,141][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T11:40:55,146][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2793, reason: becoming candidate: onLeaderFailure
[2022-10-22T11:40:55,987][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2794, reason: ApplyCommitRequest{term=4, version=2794, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:41:03,230][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2795, reason: ApplyCommitRequest{term=4, version=2795, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:41:03,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,487][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,679][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,681][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,879][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,899][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:03,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,082][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,181][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,485][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,684][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:04,788][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T11:41:05,084][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T11:41:05,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T11:41:05,280][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:05,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:06,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T11:41:06,381][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:06,503][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T11:41:06,523][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T11:41:06,587][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2796, reason: ApplyCommitRequest{term=4, version=2796, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:41:06,777][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2797, reason: ApplyCommitRequest{term=4, version=2797, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:41:07,253][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2798, reason: ApplyCommitRequest{term=4, version=2798, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:43:24,101][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T11:43:24,112][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2866, reason: becoming candidate: onLeaderFailure
[2022-10-22T11:43:24,550][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2868, reason: ApplyCommitRequest{term=4, version=2868, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T11:43:53,887][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][68501] overhead, spent [289ms] collecting in the last [1s]
[2022-10-22T12:38:27,782][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2936, reason: ApplyCommitRequest{term=4, version=2936, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T12:38:30,720][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 2937, reason: ApplyCommitRequest{term=4, version=2937, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T12:40:55,302][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T12:40:55,386][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 2937, reason: becoming candidate: onLeaderFailure
[2022-10-22T12:40:56,092][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 2938, reason: ApplyCommitRequest{term=4, version=2938, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T12:41:08,136][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2939, reason: ApplyCommitRequest{term=4, version=2939, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T12:41:08,285][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,292][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,399][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,501][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,509][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,614][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,782][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T12:41:08,885][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,083][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,088][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,101][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,296][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,382][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,396][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,409][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,499][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,579][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,682][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T12:41:09,697][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T12:41:10,296][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 2940, reason: ApplyCommitRequest{term=4, version=2940, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T12:43:27,221][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T12:43:27,233][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3009, reason: becoming candidate: onLeaderFailure
[2022-10-22T12:43:27,834][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3011, reason: ApplyCommitRequest{term=4, version=3011, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:38:30,767][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3080, reason: ApplyCommitRequest{term=4, version=3080, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:38:33,527][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3081, reason: ApplyCommitRequest{term=4, version=3081, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:40:55,524][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T13:40:55,684][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3081, reason: becoming candidate: onLeaderFailure
[2022-10-22T13:40:56,710][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3082, reason: ApplyCommitRequest{term=4, version=3082, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:41:11,486][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3083, reason: ApplyCommitRequest{term=4, version=3083, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:41:12,958][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,482][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,494][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,591][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,693][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,797][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,899][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T13:41:13,994][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,007][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,193][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,281][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,298][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,311][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,394][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,480][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,507][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,683][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,695][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,705][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T13:41:14,792][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3084, reason: ApplyCommitRequest{term=4, version=3084, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:41:15,083][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3085, reason: ApplyCommitRequest{term=4, version=3085, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:41:15,426][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3086, reason: ApplyCommitRequest{term=4, version=3086, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:43:29,765][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T13:43:29,800][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3153, reason: becoming candidate: onLeaderFailure
[2022-10-22T13:43:30,452][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3155, reason: ApplyCommitRequest{term=4, version=3155, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T13:44:01,882][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][75700] overhead, spent [423ms] collecting in the last [1s]
[2022-10-22T14:38:33,543][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3223, reason: ApplyCommitRequest{term=4, version=3223, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:38:36,470][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3224, reason: ApplyCommitRequest{term=4, version=3224, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:40:55,991][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T14:40:56,001][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3224, reason: becoming candidate: onLeaderFailure
[2022-10-22T14:40:56,397][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3225, reason: ApplyCommitRequest{term=4, version=3225, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:41:16,334][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3226, reason: ApplyCommitRequest{term=4, version=3226, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:41:17,599][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:17,686][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T14:41:17,728][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:17,885][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,002][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,090][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,325][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,590][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,602][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,680][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,791][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,885][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,900][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:18,994][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,008][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,093][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,105][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,196][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,287][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,298][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T14:41:19,354][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3227, reason: ApplyCommitRequest{term=4, version=3227, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:41:19,512][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3228, reason: ApplyCommitRequest{term=4, version=3228, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:41:20,544][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3229, reason: ApplyCommitRequest{term=4, version=3229, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T14:43:32,395][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T14:43:32,482][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3299, reason: becoming candidate: onLeaderFailure
[2022-10-22T14:43:33,395][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3301, reason: ApplyCommitRequest{term=4, version=3301, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:38:36,460][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3369, reason: ApplyCommitRequest{term=4, version=3369, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:38:39,135][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3370, reason: ApplyCommitRequest{term=4, version=3370, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:40:56,023][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T15:40:56,080][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3370, reason: becoming candidate: onLeaderFailure
[2022-10-22T15:40:56,285][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3371, reason: ApplyCommitRequest{term=4, version=3371, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:41:21,515][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3372, reason: ApplyCommitRequest{term=4, version=3372, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:41:21,690][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:21,789][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T15:41:21,793][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T15:41:21,893][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:21,989][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:22,087][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T15:41:22,282][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:22,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T15:41:22,484][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T15:41:22,586][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:23,480][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:23,780][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:23,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,290][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,381][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,588][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,685][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:24,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T15:41:25,090][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T15:41:25,180][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T15:41:25,190][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3373, reason: ApplyCommitRequest{term=4, version=3373, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:41:25,603][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3374, reason: ApplyCommitRequest{term=4, version=3374, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:41:26,642][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3375, reason: ApplyCommitRequest{term=4, version=3375, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:43:35,337][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T15:43:35,354][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3442, reason: becoming candidate: onLeaderFailure
[2022-10-22T15:43:35,983][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3444, reason: ApplyCommitRequest{term=4, version=3444, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T15:44:09,196][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][82899] overhead, spent [319ms] collecting in the last [1s]
[2022-10-22T16:38:39,176][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3513, reason: ApplyCommitRequest{term=4, version=3513, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T16:38:42,093][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3514, reason: ApplyCommitRequest{term=4, version=3514, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T16:40:56,101][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T16:40:56,108][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3514, reason: becoming candidate: onLeaderFailure
[2022-10-22T16:40:56,993][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3515, reason: ApplyCommitRequest{term=4, version=3515, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T16:41:27,294][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3516, reason: ApplyCommitRequest{term=4, version=3516, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T16:41:27,395][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,492][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,580][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,591][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,679][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,692][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,707][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,792][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,880][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T16:41:27,986][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,079][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,086][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,088][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,182][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,292][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,379][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,391][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,492][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,505][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,593][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T16:41:28,779][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3517, reason: ApplyCommitRequest{term=4, version=3517, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T16:43:38,397][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T16:43:38,413][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3583, reason: becoming candidate: onLeaderFailure
[2022-10-22T16:43:38,921][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3585, reason: ApplyCommitRequest{term=4, version=3585, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:38:41,756][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3654, reason: ApplyCommitRequest{term=4, version=3654, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:38:44,219][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3655, reason: ApplyCommitRequest{term=4, version=3655, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:40:56,194][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T17:40:56,204][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3655, reason: becoming candidate: onLeaderFailure
[2022-10-22T17:40:57,080][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3656, reason: ApplyCommitRequest{term=4, version=3656, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:41:29,334][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3657, reason: ApplyCommitRequest{term=4, version=3657, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:41:29,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:29,879][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T17:41:29,881][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:29,999][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,081][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,095][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,199][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,322][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,412][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,485][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,505][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,705][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,784][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,794][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,884][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,894][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T17:41:30,993][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:31,095][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T17:41:31,199][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T17:41:31,280][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T17:41:31,948][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3658, reason: ApplyCommitRequest{term=4, version=3658, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:43:40,947][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T17:43:40,980][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3725, reason: becoming candidate: onLeaderFailure
[2022-10-22T17:43:41,708][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3727, reason: ApplyCommitRequest{term=4, version=3727, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T17:44:18,283][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][90100] overhead, spent [408ms] collecting in the last [1s]
[2022-10-22T18:38:44,203][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3795, reason: ApplyCommitRequest{term=4, version=3795, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T18:38:47,303][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3796, reason: ApplyCommitRequest{term=4, version=3796, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T18:40:56,216][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T18:40:56,225][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3796, reason: becoming candidate: onLeaderFailure
[2022-10-22T18:40:56,371][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3797, reason: ApplyCommitRequest{term=4, version=3797, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T18:41:32,825][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3798, reason: ApplyCommitRequest{term=4, version=3798, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T18:41:32,986][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T18:41:32,993][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,086][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,112][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,194][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,286][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,292][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,397][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,488][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,505][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,586][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,695][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,754][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,879][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,886][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,981][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T18:41:33,999][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T18:41:34,381][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:34,390][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:34,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T18:41:34,494][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T18:41:34,517][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3799, reason: ApplyCommitRequest{term=4, version=3799, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T18:43:43,359][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T18:43:43,370][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3866, reason: becoming candidate: onLeaderFailure
[2022-10-22T18:43:45,197][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3868, reason: ApplyCommitRequest{term=4, version=3868, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:04:53,012][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][94929] overhead, spent [425ms] collecting in the last [1s]
[2022-10-22T19:38:47,467][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3937, reason: ApplyCommitRequest{term=4, version=3937, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:38:50,292][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 3938, reason: ApplyCommitRequest{term=4, version=3938, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:40:56,241][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T19:40:56,252][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 3938, reason: becoming candidate: onLeaderFailure
[2022-10-22T19:40:56,551][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 3939, reason: ApplyCommitRequest{term=4, version=3939, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:41:35,039][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3940, reason: ApplyCommitRequest{term=4, version=3940, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:41:35,198][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,398][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,500][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,684][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,785][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:35,983][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,093][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,380][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,391][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,479][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,687][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,700][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,806][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,890][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,899][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,980][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T19:41:36,994][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3941, reason: ApplyCommitRequest{term=4, version=3941, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:41:37,107][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3942, reason: ApplyCommitRequest{term=4, version=3942, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:41:37,628][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 3943, reason: ApplyCommitRequest{term=4, version=3943, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:43:46,601][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T19:43:46,642][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4014, reason: becoming candidate: onLeaderFailure
[2022-10-22T19:43:47,630][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4016, reason: ApplyCommitRequest{term=4, version=4016, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T19:44:17,598][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][97291] overhead, spent [308ms] collecting in the last [1s]
[2022-10-22T20:38:50,281][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4084, reason: ApplyCommitRequest{term=4, version=4084, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:38:52,687][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4085, reason: ApplyCommitRequest{term=4, version=4085, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:40:56,273][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T20:40:56,282][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4085, reason: becoming candidate: onLeaderFailure
[2022-10-22T20:40:56,892][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4086, reason: ApplyCommitRequest{term=4, version=4086, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:41:38,621][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4087, reason: ApplyCommitRequest{term=4, version=4087, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:41:38,702][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:38,881][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,192][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,381][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,481][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,689][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,781][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,888][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:39,992][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,288][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,395][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,485][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,501][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,583][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,608][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,692][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,704][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,883][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,907][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T20:41:40,986][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4088, reason: ApplyCommitRequest{term=4, version=4088, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:41:41,107][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4089, reason: ApplyCommitRequest{term=4, version=4089, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:41:41,588][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4090, reason: ApplyCommitRequest{term=4, version=4090, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T20:43:49,908][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T20:43:49,927][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4157, reason: becoming candidate: onLeaderFailure
[2022-10-22T20:43:51,131][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4159, reason: ApplyCommitRequest{term=4, version=4159, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:38:53,016][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4228, reason: ApplyCommitRequest{term=4, version=4228, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:38:55,411][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4229, reason: ApplyCommitRequest{term=4, version=4229, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:40:56,312][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T21:40:56,327][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4229, reason: becoming candidate: onLeaderFailure
[2022-10-22T21:40:57,682][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4230, reason: ApplyCommitRequest{term=4, version=4230, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:41:43,285][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4231, reason: ApplyCommitRequest{term=4, version=4231, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:41:43,483][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,498][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,601][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,689][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,783][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,793][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,888][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:43,988][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,083][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,097][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,191][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,284][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,295][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,386][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,398][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,482][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,494][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,588][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,595][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,681][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,699][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T21:41:44,711][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4232, reason: ApplyCommitRequest{term=4, version=4232, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:41:44,893][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4233, reason: ApplyCommitRequest{term=4, version=4233, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:41:45,749][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4234, reason: ApplyCommitRequest{term=4, version=4234, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:43:52,828][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T21:43:52,884][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4303, reason: becoming candidate: onLeaderFailure
[2022-10-22T21:43:53,355][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4305, reason: ApplyCommitRequest{term=4, version=4305, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T21:44:19,379][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][104484] overhead, spent [299ms] collecting in the last [1s]
[2022-10-22T22:38:55,522][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4373, reason: ApplyCommitRequest{term=4, version=4373, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:38:58,403][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4374, reason: ApplyCommitRequest{term=4, version=4374, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:40:56,395][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T22:40:56,404][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4374, reason: becoming candidate: onLeaderFailure
[2022-10-22T22:40:56,726][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4375, reason: ApplyCommitRequest{term=4, version=4375, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:41:47,280][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4376, reason: ApplyCommitRequest{term=4, version=4376, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:41:47,598][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:47,791][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:47,893][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:47,979][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,029][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,195][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,306][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,383][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,485][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,681][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,683][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,784][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T22:41:48,887][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,085][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,187][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,283][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,288][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,379][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,482][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,488][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,498][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T22:41:49,583][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4377, reason: ApplyCommitRequest{term=4, version=4377, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:41:49,680][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4378, reason: ApplyCommitRequest{term=4, version=4378, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:41:49,767][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4379, reason: ApplyCommitRequest{term=4, version=4379, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T22:43:55,616][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T22:43:55,679][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4447, reason: becoming candidate: onLeaderFailure
[2022-10-22T22:43:56,367][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4449, reason: ApplyCommitRequest{term=4, version=4449, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:38:58,610][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4518, reason: ApplyCommitRequest{term=4, version=4518, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:39:01,339][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4519, reason: ApplyCommitRequest{term=4, version=4519, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:40:56,417][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-22T23:40:56,424][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4519, reason: becoming candidate: onLeaderFailure
[2022-10-22T23:40:56,901][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4520, reason: ApplyCommitRequest{term=4, version=4520, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:41:50,565][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4521, reason: ApplyCommitRequest{term=4, version=4521, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:41:50,784][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T23:41:50,880][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [eventlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T23:41:50,904][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:50,999][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [connectionlog-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,087][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,189][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance24h-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,279][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,294][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,305][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,384][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,581][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,688][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [guicutthrough-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,786][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,879][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [userdata-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,894][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,908][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [historicalperformance15min-v7][3] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,990][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2022-10-22T23:41:51,998][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [cmlog-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:52,079][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-22T23:41:52,200][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [inventoryequipment-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:52,285][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-1] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations
[2022-10-22T23:41:52,738][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2eOhoyZT36xLcYb_2ztww}{10.233.65.112}{10.233.65.112:9300}{dmr}}, term: 4, version: 4522, reason: ApplyCommitRequest{term=4, version=4522, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:43:57,948][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.117:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-1}{smPDS3xtShGp5xZr0iCmrQ}{9jo_J_BbR3mPIqO4BUhzAg}{10.233.72.3}{10.233.72.3:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-22T23:43:58,094][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4589, reason: becoming candidate: onLeaderFailure
[2022-10-22T23:43:58,729][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4591, reason: ApplyCommitRequest{term=4, version=4591, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-22T23:44:28,101][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-1] [gc][111684] overhead, spent [282ms] collecting in the last [1s]
[2022-10-23T00:39:01,584][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4659, reason: ApplyCommitRequest{term=4, version=4659, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-23T00:39:03,461][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-dzh8n}{ArQV6vwyTrGDZxUU9ocrjA}{imb0xrgpS9aD1VhIhi6_Pg}{10.233.65.92}{10.233.65.92:9300}{r}}, term: 4, version: 4660, reason: ApplyCommitRequest{term=4, version=4660, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-23T00:40:56,509][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-1] master node [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.117:9300][disconnected] disconnected
[2022-10-23T00:40:56,519][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}], current []}, term: 4, version: 4660, reason: becoming candidate: onLeaderFailure
[2022-10-23T00:40:56,986][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] master node changed {previous [], current [{onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}]}, term: 4, version: 4661, reason: ApplyCommitRequest{term=4, version=4661, sourceNode={onap-sdnrdb-master-0}{0N0iZdQhS8CqWzQH9aUJWg}{bHRG2dFiQd-ypxG9O_64LA}{10.233.67.117}{10.233.67.117:9300}{dmr}}
[2022-10-23T00:41:54,733][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-1] removed {{onap-sdnrdb-master-2}{cw5gkwLgTJ-4oVhsYEyq8A}{n2