Results

By type
Logs
raw version
          [38;5;6m [38;5;5m14:49:32.84 [0m
[38;5;6m [38;5;5m14:49:32.85 [0m[1mWelcome to the Bitnami elasticsearch container[0m
[38;5;6m [38;5;5m14:49:32.93 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch[0m
[38;5;6m [38;5;5m14:49:32.94 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch/issues[0m
[38;5;6m [38;5;5m14:49:32.95 [0m
[38;5;6m [38;5;5m14:49:32.96 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch setup **
[38;5;6m [38;5;5m14:49:33.43 [0m[38;5;2mINFO [0m ==> Configuring/Initializing Elasticsearch...
[38;5;6m [38;5;5m14:49:33.75 [0m[38;5;2mINFO [0m ==> Setting default configuration
[38;5;6m [38;5;5m14:49:33.94 [0m[38;5;2mINFO [0m ==> Configuring Elasticsearch cluster settings...
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[38;5;6m [38;5;5m14:49:57.44 [0m[38;5;2mINFO [0m ==> ** Elasticsearch setup finished! **

[38;5;6m [38;5;5m14:49:57.64 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch **
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-10-14T14:50:44,236][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/5.4.0-96-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS]
[2022-10-14T14:50:44,240][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] JVM home [/opt/bitnami/java]
[2022-10-14T14:50:44,332][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-14288337239161090277, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2022-10-14T14:51:04,337][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [aggs-matrix-stats]
[2022-10-14T14:51:04,338][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [analysis-common]
[2022-10-14T14:51:04,346][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [geo]
[2022-10-14T14:51:04,350][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [ingest-common]
[2022-10-14T14:51:04,351][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [ingest-geoip]
[2022-10-14T14:51:04,352][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [ingest-user-agent]
[2022-10-14T14:51:04,352][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [kibana]
[2022-10-14T14:51:04,353][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [lang-expression]
[2022-10-14T14:51:04,354][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [lang-mustache]
[2022-10-14T14:51:04,355][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [lang-painless]
[2022-10-14T14:51:04,356][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [mapper-extras]
[2022-10-14T14:51:04,365][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [parent-join]
[2022-10-14T14:51:04,434][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [percolator]
[2022-10-14T14:51:04,441][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [rank-eval]
[2022-10-14T14:51:04,443][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [reindex]
[2022-10-14T14:51:04,445][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [repository-url]
[2022-10-14T14:51:04,447][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [tasks]
[2022-10-14T14:51:04,448][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded module [transport-netty4]
[2022-10-14T14:51:04,532][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-master-0] loaded plugin [repository-s3]
[2022-10-14T14:51:05,544][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-master-0] using [1] data paths, mounts [[/bitnami/elasticsearch/data (192.168.13.82:/dockerdata-nfs/onap/elastic-master-0)]], net usable_space [95.5gb], net total_space [99.9gb], types [nfs4]
[2022-10-14T14:51:05,545][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-master-0] heap size [123.7mb], compressed ordinary object pointers [true]
[2022-10-14T14:51:06,435][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] node name [onap-sdnrdb-master-0], node ID [7um74vIcSyewFiD9B7M30w], cluster name [sdnrdb-cluster]
[2022-10-14T14:52:05,939][INFO ][o.e.t.NettyAllocator     ] [onap-sdnrdb-master-0] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}]
[2022-10-14T14:52:07,042][INFO ][o.e.d.DiscoveryModule    ] [onap-sdnrdb-master-0] using discovery type [zen] and seed hosts providers [settings]
[2022-10-14T14:52:11,741][WARN ][o.e.g.DanglingIndicesState] [onap-sdnrdb-master-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-10-14T14:52:14,233][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] initialized
[2022-10-14T14:52:14,236][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] starting ...
[2022-10-14T14:52:16,634][INFO ][o.e.t.TransportService   ] [onap-sdnrdb-master-0] publish_address {10.233.72.39:9300}, bound_addresses {0.0.0.0:9300}
[2022-10-14T14:52:18,457][INFO ][o.e.b.BootstrapChecks    ] [onap-sdnrdb-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-10-14T14:52:21,034][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] setting initial configuration to VotingConfiguration{7um74vIcSyewFiD9B7M30w,{bootstrap-placeholder}-onap-sdnrdb-master-2,mPUdSz6LTq-eI9TuIEvEqg}
[2022-10-14T14:52:25,041][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr},{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r},{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}, term: 2, version: 4, reason: ApplyCommitRequest{term=2, version=4, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T14:52:25,834][INFO ][o.e.h.AbstractHttpServerTransport] [onap-sdnrdb-master-0] publish_address {10.233.72.39:9200}, bound_addresses {0.0.0.0:9200}
[2022-10-14T14:52:25,835][INFO ][o.e.n.Node               ] [onap-sdnrdb-master-0] started
[2022-10-14T14:52:32,739][INFO ][o.e.c.s.ClusterSettings  ] [onap-sdnrdb-master-0] updating [action.auto_create_index] from [true] to [false]
[2022-10-14T14:52:39,051][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][24] overhead, spent [307ms] collecting in the last [1s]
[2022-10-14T15:52:07,466][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 88, reason: ApplyCommitRequest{term=2, version=88, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:52:09,978][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 89, reason: ApplyCommitRequest{term=2, version=89, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:52:20,646][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T15:52:20,665][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 89, reason: becoming candidate: onLeaderFailure
[2022-10-14T15:52:21,580][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 90, reason: ApplyCommitRequest{term=2, version=90, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:52:25,646][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T15:52:25,736][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 90, reason: becoming candidate: onLeaderFailure
[2022-10-14T15:52:29,541][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 94, reason: ApplyCommitRequest{term=2, version=94, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:45,057][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 163, reason: ApplyCommitRequest{term=2, version=163, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:45,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:45,733][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,237][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,638][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,734][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,938][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T15:53:46,942][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,148][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,334][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,446][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,633][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,646][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,932][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T15:53:47,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T15:53:49,553][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 164, reason: ApplyCommitRequest{term=2, version=164, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:50,241][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 165, reason: ApplyCommitRequest{term=2, version=165, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:50,562][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 166, reason: ApplyCommitRequest{term=2, version=166, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:50,943][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 167, reason: ApplyCommitRequest{term=2, version=167, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T15:53:51,537][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 168, reason: ApplyCommitRequest{term=2, version=168, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:52:10,174][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 238, reason: ApplyCommitRequest{term=2, version=238, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:52:13,010][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 239, reason: ApplyCommitRequest{term=2, version=239, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:52:20,855][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T16:52:20,934][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 239, reason: becoming candidate: onLeaderFailure
[2022-10-14T16:52:21,172][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 240, reason: ApplyCommitRequest{term=2, version=240, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:52:31,206][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T16:52:31,219][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 240, reason: becoming candidate: onLeaderFailure
[2022-10-14T16:52:32,360][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 242, reason: ApplyCommitRequest{term=2, version=242, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:53:00,837][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][7240] overhead, spent [487ms] collecting in the last [1s]
[2022-10-14T16:53:52,515][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 312, reason: ApplyCommitRequest{term=2, version=312, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T16:53:52,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:52,645][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T16:53:52,742][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T16:53:52,832][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:52,938][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,135][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,239][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,439][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,540][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,635][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,638][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,654][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,752][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,753][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,843][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,855][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,938][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T16:53:53,949][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T16:53:54,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:54,135][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T16:53:54,145][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T16:53:54,600][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 313, reason: ApplyCommitRequest{term=2, version=313, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:52:13,311][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 383, reason: ApplyCommitRequest{term=2, version=383, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:52:16,408][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 384, reason: ApplyCommitRequest{term=2, version=384, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:52:21,057][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T17:52:21,064][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 384, reason: becoming candidate: onLeaderFailure
[2022-10-14T17:52:21,465][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 385, reason: ApplyCommitRequest{term=2, version=385, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:52:34,509][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T17:52:34,521][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 385, reason: becoming candidate: onLeaderFailure
[2022-10-14T17:52:35,200][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 387, reason: ApplyCommitRequest{term=2, version=387, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:53:55,749][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 458, reason: ApplyCommitRequest{term=2, version=458, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T17:53:55,946][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,137][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,146][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,239][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,346][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,452][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,633][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,642][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,739][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T17:53:56,940][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,048][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,145][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,242][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,251][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,437][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,556][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T17:53:57,861][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 459, reason: ApplyCommitRequest{term=2, version=459, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:52:16,645][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 529, reason: ApplyCommitRequest{term=2, version=529, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:52:19,726][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 530, reason: ApplyCommitRequest{term=2, version=530, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:52:21,146][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T18:52:21,153][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 530, reason: becoming candidate: onLeaderFailure
[2022-10-14T18:52:21,851][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 531, reason: ApplyCommitRequest{term=2, version=531, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:52:36,924][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T18:52:36,937][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 531, reason: becoming candidate: onLeaderFailure
[2022-10-14T18:52:37,585][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 533, reason: ApplyCommitRequest{term=2, version=533, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:53:58,784][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 603, reason: ApplyCommitRequest{term=2, version=603, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T18:53:58,932][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T18:53:58,948][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,039][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,137][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,534][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,638][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:53:59,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,137][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,338][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,432][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,446][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,542][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,544][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,645][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,755][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,852][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T18:54:00,866][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T18:54:01,252][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 604, reason: ApplyCommitRequest{term=2, version=604, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:52:19,984][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 673, reason: ApplyCommitRequest{term=2, version=673, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:52:21,335][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T19:52:21,340][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 673, reason: becoming candidate: onLeaderFailure
[2022-10-14T19:52:22,033][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 674, reason: ApplyCommitRequest{term=2, version=674, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:52:22,175][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 675, reason: ApplyCommitRequest{term=2, version=675, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:52:40,050][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T19:52:40,063][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 675, reason: becoming candidate: onLeaderFailure
[2022-10-14T19:52:40,772][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 677, reason: ApplyCommitRequest{term=2, version=677, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:02,187][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 745, reason: ApplyCommitRequest{term=2, version=745, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:02,340][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,442][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,540][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,547][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,642][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,655][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,755][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,942][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:02,957][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,141][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,145][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,333][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,536][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,632][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T19:54:03,932][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T19:54:04,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T19:54:04,141][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T19:54:04,332][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T19:54:04,338][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T19:54:04,357][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 746, reason: ApplyCommitRequest{term=2, version=746, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:04,654][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 747, reason: ApplyCommitRequest{term=2, version=747, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:04,854][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 748, reason: ApplyCommitRequest{term=2, version=748, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:05,166][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 749, reason: ApplyCommitRequest{term=2, version=749, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T19:54:05,649][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 750, reason: ApplyCommitRequest{term=2, version=750, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:52:21,549][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T20:52:21,560][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 820, reason: becoming candidate: onLeaderFailure
[2022-10-14T20:52:21,975][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 821, reason: ApplyCommitRequest{term=2, version=821, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:52:22,438][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 822, reason: ApplyCommitRequest{term=2, version=822, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:52:25,493][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 823, reason: ApplyCommitRequest{term=2, version=823, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:52:43,032][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T20:52:43,053][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 823, reason: becoming candidate: onLeaderFailure
[2022-10-14T20:52:44,379][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 825, reason: ApplyCommitRequest{term=2, version=825, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:54:06,689][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 895, reason: ApplyCommitRequest{term=2, version=895, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T20:54:06,750][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:06,836][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T20:54:06,937][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,045][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,337][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,437][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,532][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,536][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,637][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,640][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,741][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,833][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:07,946][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:08,037][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T20:54:08,045][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T20:54:08,139][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T20:54:08,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T20:54:08,432][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T20:54:09,003][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 896, reason: ApplyCommitRequest{term=2, version=896, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:52:21,644][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T21:52:21,651][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 967, reason: becoming candidate: onLeaderFailure
[2022-10-14T21:52:22,165][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 968, reason: ApplyCommitRequest{term=2, version=968, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:52:25,854][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 969, reason: ApplyCommitRequest{term=2, version=969, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:52:28,763][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 970, reason: ApplyCommitRequest{term=2, version=970, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:52:46,235][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T21:52:46,247][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 970, reason: becoming candidate: onLeaderFailure
[2022-10-14T21:52:46,872][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 972, reason: ApplyCommitRequest{term=2, version=972, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:54:10,516][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1041, reason: ApplyCommitRequest{term=2, version=1041, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T21:54:10,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T21:54:10,738][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:10,837][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,036][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,139][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,243][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,438][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,557][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,656][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,738][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,751][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,837][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T21:54:11,942][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:12,038][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T21:54:12,046][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T21:54:12,112][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1042, reason: ApplyCommitRequest{term=2, version=1042, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:52:21,861][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T22:52:21,867][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1114, reason: becoming candidate: onLeaderFailure
[2022-10-14T22:52:22,341][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1115, reason: ApplyCommitRequest{term=2, version=1115, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:52:29,066][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1116, reason: ApplyCommitRequest{term=2, version=1116, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:52:32,015][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1117, reason: ApplyCommitRequest{term=2, version=1117, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:52:49,366][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T22:52:49,381][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1117, reason: becoming candidate: onLeaderFailure
[2022-10-14T22:52:50,157][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1119, reason: ApplyCommitRequest{term=2, version=1119, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:54:13,400][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1188, reason: ApplyCommitRequest{term=2, version=1188, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:54:13,445][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,532][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,657][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,737][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,947][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T22:54:13,952][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,044][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,341][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,441][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,635][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,734][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,852][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,949][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T22:54:14,961][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:15,038][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T22:54:15,133][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T22:54:15,142][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1189, reason: ApplyCommitRequest{term=2, version=1189, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:54:15,264][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1190, reason: ApplyCommitRequest{term=2, version=1190, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T22:54:15,632][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1191, reason: ApplyCommitRequest{term=2, version=1191, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:52:21,959][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-14T23:52:22,032][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1262, reason: becoming candidate: onLeaderFailure
[2022-10-14T23:52:22,467][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1263, reason: ApplyCommitRequest{term=2, version=1263, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:52:32,286][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1264, reason: ApplyCommitRequest{term=2, version=1264, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:52:35,372][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1265, reason: ApplyCommitRequest{term=2, version=1265, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:52:52,438][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-14T23:52:52,449][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1265, reason: becoming candidate: onLeaderFailure
[2022-10-14T23:52:53,177][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1267, reason: ApplyCommitRequest{term=2, version=1267, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:54:16,720][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1337, reason: ApplyCommitRequest{term=2, version=1337, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-14T23:54:16,932][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:16,943][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,140][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,337][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,346][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,636][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,641][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,833][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,835][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,941][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-14T23:54:17,958][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,047][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,150][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,249][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,340][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,352][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,445][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-14T23:54:18,984][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1338, reason: ApplyCommitRequest{term=2, version=1338, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:52:22,065][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T00:52:22,736][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1406, reason: becoming candidate: onLeaderFailure
[2022-10-15T00:52:23,651][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1407, reason: ApplyCommitRequest{term=2, version=1407, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:52:36,012][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1408, reason: ApplyCommitRequest{term=2, version=1408, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:52:38,706][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1409, reason: ApplyCommitRequest{term=2, version=1409, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:52:54,797][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T00:52:54,809][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1409, reason: becoming candidate: onLeaderFailure
[2022-10-15T00:52:55,395][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1411, reason: ApplyCommitRequest{term=2, version=1411, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:53:25,233][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][36046] overhead, spent [314ms] collecting in the last [1s]
[2022-10-15T00:54:20,211][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1479, reason: ApplyCommitRequest{term=2, version=1479, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T00:54:20,348][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,442][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,638][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,648][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,842][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T00:54:20,937][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,051][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,157][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,251][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,352][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,450][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,552][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,635][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T00:54:21,710][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1480, reason: ApplyCommitRequest{term=2, version=1480, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:52:22,875][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T01:52:22,881][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1553, reason: becoming candidate: onLeaderFailure
[2022-10-15T01:52:23,255][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1554, reason: ApplyCommitRequest{term=2, version=1554, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:52:38,979][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1555, reason: ApplyCommitRequest{term=2, version=1555, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:52:42,052][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1556, reason: ApplyCommitRequest{term=2, version=1556, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:52:57,346][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T01:52:57,435][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1556, reason: becoming candidate: onLeaderFailure
[2022-10-15T01:52:58,073][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1558, reason: ApplyCommitRequest{term=2, version=1558, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:54:22,740][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1627, reason: ApplyCommitRequest{term=2, version=1627, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T01:54:23,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,036][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,135][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,246][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,337][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,559][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,633][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,844][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:23,939][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,036][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,134][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,159][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,247][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,249][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,341][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,342][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,362][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,442][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,540][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T01:54:24,873][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1628, reason: ApplyCommitRequest{term=2, version=1628, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:52:22,954][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T02:52:22,967][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1698, reason: becoming candidate: onLeaderFailure
[2022-10-15T02:52:23,433][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1699, reason: ApplyCommitRequest{term=2, version=1699, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:52:42,374][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1700, reason: ApplyCommitRequest{term=2, version=1700, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:52:44,475][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1701, reason: ApplyCommitRequest{term=2, version=1701, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:53:00,508][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T02:53:00,521][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1701, reason: becoming candidate: onLeaderFailure
[2022-10-15T02:53:01,087][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1703, reason: ApplyCommitRequest{term=2, version=1703, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:53:25,842][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][43242] overhead, spent [307ms] collecting in the last [1.2s]
[2022-10-15T02:54:25,791][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1772, reason: ApplyCommitRequest{term=2, version=1772, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T02:54:25,943][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,044][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,142][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,346][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,446][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,544][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,643][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,737][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,750][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:26,947][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,049][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,134][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,145][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,248][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,341][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T02:54:27,350][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T02:54:28,274][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1773, reason: ApplyCommitRequest{term=2, version=1773, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:52:22,991][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T03:52:23,000][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1844, reason: becoming candidate: onLeaderFailure
[2022-10-15T03:52:23,351][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1845, reason: ApplyCommitRequest{term=2, version=1845, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:52:44,796][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1846, reason: ApplyCommitRequest{term=2, version=1846, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:52:47,593][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1847, reason: ApplyCommitRequest{term=2, version=1847, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:53:03,460][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T03:53:03,479][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1847, reason: becoming candidate: onLeaderFailure
[2022-10-15T03:53:03,993][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1849, reason: ApplyCommitRequest{term=2, version=1849, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:54:29,858][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1917, reason: ApplyCommitRequest{term=2, version=1917, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:54:30,048][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,138][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,340][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,735][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,839][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:30,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,135][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,146][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,248][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,333][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,359][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,446][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,456][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,544][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,734][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,743][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T03:54:31,756][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1918, reason: ApplyCommitRequest{term=2, version=1918, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:54:32,092][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1919, reason: ApplyCommitRequest{term=2, version=1919, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T03:54:32,514][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 1920, reason: ApplyCommitRequest{term=2, version=1920, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:52:23,015][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T04:52:23,039][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1992, reason: becoming candidate: onLeaderFailure
[2022-10-15T04:52:23,662][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1993, reason: ApplyCommitRequest{term=2, version=1993, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:52:48,006][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1994, reason: ApplyCommitRequest{term=2, version=1994, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:52:50,943][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 1995, reason: ApplyCommitRequest{term=2, version=1995, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:53:05,813][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T04:53:05,830][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 1995, reason: becoming candidate: onLeaderFailure
[2022-10-15T04:53:06,173][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 1997, reason: ApplyCommitRequest{term=2, version=1997, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:54:33,629][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2066, reason: ApplyCommitRequest{term=2, version=2066, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:54:33,750][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:33,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:33,851][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T04:54:33,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,132][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,533][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,832][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T04:54:34,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,132][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,340][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,537][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,636][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:35,739][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T04:54:36,037][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:36,037][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T04:54:36,139][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T04:54:36,333][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2067, reason: ApplyCommitRequest{term=2, version=2067, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:54:36,636][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2068, reason: ApplyCommitRequest{term=2, version=2068, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T04:54:37,139][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2069, reason: ApplyCommitRequest{term=2, version=2069, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:52:23,151][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T05:52:23,237][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2140, reason: becoming candidate: onLeaderFailure
[2022-10-15T05:52:23,640][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2141, reason: ApplyCommitRequest{term=2, version=2141, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:52:51,335][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2142, reason: ApplyCommitRequest{term=2, version=2142, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:52:53,492][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2143, reason: ApplyCommitRequest{term=2, version=2143, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:53:08,783][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T05:53:08,798][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2143, reason: becoming candidate: onLeaderFailure
[2022-10-15T05:53:09,632][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2145, reason: ApplyCommitRequest{term=2, version=2145, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:54:38,160][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2214, reason: ApplyCommitRequest{term=2, version=2214, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T05:54:38,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:38,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T05:54:38,534][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:38,542][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:38,643][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T05:54:38,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,138][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,141][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,349][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,437][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,445][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,547][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,555][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,638][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,653][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,737][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,756][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,855][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:39,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T05:54:40,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T05:54:40,566][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2215, reason: ApplyCommitRequest{term=2, version=2215, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:52:23,271][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T06:52:23,286][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2285, reason: becoming candidate: onLeaderFailure
[2022-10-15T06:52:23,953][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2286, reason: ApplyCommitRequest{term=2, version=2286, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:52:53,780][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2287, reason: ApplyCommitRequest{term=2, version=2287, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:52:56,597][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2288, reason: ApplyCommitRequest{term=2, version=2288, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:53:11,088][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T06:53:11,104][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2288, reason: becoming candidate: onLeaderFailure
[2022-10-15T06:53:11,674][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2290, reason: ApplyCommitRequest{term=2, version=2290, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:54:41,569][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2359, reason: ApplyCommitRequest{term=2, version=2359, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T06:54:41,646][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:41,739][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T06:54:41,749][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T06:54:41,840][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T06:54:41,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:41,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,040][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,339][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,349][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,464][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,553][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,635][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,663][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,740][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,766][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,835][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,846][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,935][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,957][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T06:54:42,993][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T06:54:43,776][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2360, reason: ApplyCommitRequest{term=2, version=2360, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:52:23,363][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T07:52:23,369][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2430, reason: becoming candidate: onLeaderFailure
[2022-10-15T07:52:23,497][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2431, reason: ApplyCommitRequest{term=2, version=2431, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:52:56,916][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2432, reason: ApplyCommitRequest{term=2, version=2432, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:52:59,930][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2433, reason: ApplyCommitRequest{term=2, version=2433, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:53:13,629][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T07:53:13,649][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2433, reason: becoming candidate: onLeaderFailure
[2022-10-15T07:53:14,376][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2435, reason: ApplyCommitRequest{term=2, version=2435, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:54:44,700][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2504, reason: ApplyCommitRequest{term=2, version=2504, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T07:54:44,833][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:44,844][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T07:54:44,935][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T07:54:44,950][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,049][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,141][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,143][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,533][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,632][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,648][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,665][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,745][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,762][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,857][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:45,959][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T07:54:46,040][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T07:54:46,148][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T07:54:46,159][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T07:54:47,024][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2505, reason: ApplyCommitRequest{term=2, version=2505, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:52:23,408][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T08:52:23,434][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2577, reason: becoming candidate: onLeaderFailure
[2022-10-15T08:52:23,932][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2578, reason: ApplyCommitRequest{term=2, version=2578, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:53:00,239][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2579, reason: ApplyCommitRequest{term=2, version=2579, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:53:03,277][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2580, reason: ApplyCommitRequest{term=2, version=2580, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:53:16,029][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T08:53:16,042][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2580, reason: becoming candidate: onLeaderFailure
[2022-10-15T08:53:17,244][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2582, reason: ApplyCommitRequest{term=2, version=2582, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:53:36,435][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][64839] overhead, spent [298ms] collecting in the last [1s]
[2022-10-15T08:54:47,758][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2652, reason: ApplyCommitRequest{term=2, version=2652, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T08:54:47,838][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:47,852][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:47,935][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,049][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,147][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,243][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,451][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,637][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,649][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,742][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,755][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T08:54:48,833][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,042][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,239][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,250][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,439][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T08:54:49,450][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T08:54:50,238][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2653, reason: ApplyCommitRequest{term=2, version=2653, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:52:23,451][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T09:52:23,470][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2724, reason: becoming candidate: onLeaderFailure
[2022-10-15T09:52:23,688][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2725, reason: ApplyCommitRequest{term=2, version=2725, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:53:03,560][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2726, reason: ApplyCommitRequest{term=2, version=2726, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:53:06,579][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2727, reason: ApplyCommitRequest{term=2, version=2727, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:53:18,845][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T09:53:18,932][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2727, reason: becoming candidate: onLeaderFailure
[2022-10-15T09:53:19,598][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2729, reason: ApplyCommitRequest{term=2, version=2729, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:54:51,066][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2799, reason: ApplyCommitRequest{term=2, version=2799, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:54:51,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,643][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,744][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,845][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,936][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:51,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,136][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,147][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,157][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,250][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,334][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,442][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,538][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,545][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,554][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,648][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,661][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T09:54:52,737][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2800, reason: ApplyCommitRequest{term=2, version=2800, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:54:52,870][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2801, reason: ApplyCommitRequest{term=2, version=2801, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T09:54:53,569][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2802, reason: ApplyCommitRequest{term=2, version=2802, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:52:23,494][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T10:52:23,501][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2873, reason: becoming candidate: onLeaderFailure
[2022-10-15T10:52:24,052][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2874, reason: ApplyCommitRequest{term=2, version=2874, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:53:06,900][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2875, reason: ApplyCommitRequest{term=2, version=2875, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:53:09,936][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 2876, reason: ApplyCommitRequest{term=2, version=2876, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:53:21,200][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T10:53:21,213][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 2876, reason: becoming candidate: onLeaderFailure
[2022-10-15T10:53:21,760][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 2878, reason: ApplyCommitRequest{term=2, version=2878, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:54:54,247][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2947, reason: ApplyCommitRequest{term=2, version=2947, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T10:54:54,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,344][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,432][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,442][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,534][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,547][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,636][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,643][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,737][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,749][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,834][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:54,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,039][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,048][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,133][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,143][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,238][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,335][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,347][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,435][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T10:54:55,444][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T10:54:56,928][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 2948, reason: ApplyCommitRequest{term=2, version=2948, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:52:23,667][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T11:52:23,697][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3018, reason: becoming candidate: onLeaderFailure
[2022-10-15T11:52:23,839][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3019, reason: ApplyCommitRequest{term=2, version=3019, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:53:10,224][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3020, reason: ApplyCommitRequest{term=2, version=3020, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:53:13,218][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3021, reason: ApplyCommitRequest{term=2, version=3021, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:53:24,009][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T11:53:24,021][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3021, reason: becoming candidate: onLeaderFailure
[2022-10-15T11:53:24,572][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3023, reason: ApplyCommitRequest{term=2, version=3023, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:54:57,890][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3093, reason: ApplyCommitRequest{term=2, version=3093, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T11:54:57,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T11:54:57,943][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,038][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,051][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,144][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,233][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,245][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,334][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,358][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,534][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,552][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,563][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,647][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,657][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,741][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,756][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:58,936][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T11:54:59,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T11:54:59,043][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T11:55:00,232][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3094, reason: ApplyCommitRequest{term=2, version=3094, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:52:23,710][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T12:52:23,723][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3163, reason: becoming candidate: onLeaderFailure
[2022-10-15T12:52:23,970][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3164, reason: ApplyCommitRequest{term=2, version=3164, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:53:13,501][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3165, reason: ApplyCommitRequest{term=2, version=3165, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:53:16,574][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3166, reason: ApplyCommitRequest{term=2, version=3166, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:53:27,148][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T12:53:27,161][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3166, reason: becoming candidate: onLeaderFailure
[2022-10-15T12:53:27,694][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3168, reason: ApplyCommitRequest{term=2, version=3168, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:55:01,073][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3238, reason: ApplyCommitRequest{term=2, version=3238, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T12:55:01,156][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,251][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,337][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,533][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,542][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,551][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,651][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,654][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,835][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,837][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:01,944][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,051][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,240][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,340][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,438][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,651][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,741][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,932][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T12:55:02,949][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T12:55:03,540][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3239, reason: ApplyCommitRequest{term=2, version=3239, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:52:23,760][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T13:52:23,778][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3309, reason: becoming candidate: onLeaderFailure
[2022-10-15T13:52:23,995][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3310, reason: ApplyCommitRequest{term=2, version=3310, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:53:16,863][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3311, reason: ApplyCommitRequest{term=2, version=3311, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:53:20,008][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3312, reason: ApplyCommitRequest{term=2, version=3312, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:53:30,187][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T13:53:30,233][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3312, reason: becoming candidate: onLeaderFailure
[2022-10-15T13:53:31,639][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3314, reason: ApplyCommitRequest{term=2, version=3314, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:55:05,232][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3382, reason: ApplyCommitRequest{term=2, version=3382, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T13:55:05,354][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,542][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,648][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,760][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,763][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,856][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:05,944][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,041][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,443][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,455][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,538][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,551][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,644][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,733][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,745][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T13:55:06,800][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3383, reason: ApplyCommitRequest{term=2, version=3383, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:52:23,862][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T14:52:23,948][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3451, reason: becoming candidate: onLeaderFailure
[2022-10-15T14:52:24,556][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3452, reason: ApplyCommitRequest{term=2, version=3452, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:53:20,216][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3453, reason: ApplyCommitRequest{term=2, version=3453, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:53:22,385][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3454, reason: ApplyCommitRequest{term=2, version=3454, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:53:32,738][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T14:53:32,758][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3454, reason: becoming candidate: onLeaderFailure
[2022-10-15T14:53:33,244][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3456, reason: ApplyCommitRequest{term=2, version=3456, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:53:56,842][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][86446] overhead, spent [294ms] collecting in the last [1s]
[2022-10-15T14:55:07,991][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3525, reason: ApplyCommitRequest{term=2, version=3525, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:55:08,133][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,338][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,432][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,459][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,540][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,645][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,737][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,744][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,836][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,853][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,941][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:08,950][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,036][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,045][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,053][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,146][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,157][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,336][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T14:55:09,538][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3526, reason: ApplyCommitRequest{term=2, version=3526, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:55:09,710][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3527, reason: ApplyCommitRequest{term=2, version=3527, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T14:55:10,183][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3528, reason: ApplyCommitRequest{term=2, version=3528, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:52:23,962][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T15:52:23,977][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3598, reason: becoming candidate: onLeaderFailure
[2022-10-15T15:52:24,352][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3599, reason: ApplyCommitRequest{term=2, version=3599, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:53:22,697][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3600, reason: ApplyCommitRequest{term=2, version=3600, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:53:25,585][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3601, reason: ApplyCommitRequest{term=2, version=3601, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:53:35,563][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T15:53:35,576][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3601, reason: becoming candidate: onLeaderFailure
[2022-10-15T15:53:36,738][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3603, reason: ApplyCommitRequest{term=2, version=3603, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:55:10,960][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3671, reason: ApplyCommitRequest{term=2, version=3671, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T15:55:11,046][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,134][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,145][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,245][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,252][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,343][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,534][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,640][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,846][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:11,953][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,146][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,239][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,254][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,337][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,347][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,440][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,458][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,540][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T15:55:12,838][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T15:55:13,330][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3672, reason: ApplyCommitRequest{term=2, version=3672, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:52:23,992][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T16:52:24,002][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3744, reason: becoming candidate: onLeaderFailure
[2022-10-15T16:52:24,135][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3745, reason: ApplyCommitRequest{term=2, version=3745, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:53:25,873][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3746, reason: ApplyCommitRequest{term=2, version=3746, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:53:28,982][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3747, reason: ApplyCommitRequest{term=2, version=3747, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:53:38,347][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T16:53:38,362][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3747, reason: becoming candidate: onLeaderFailure
[2022-10-15T16:53:40,790][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3751, reason: ApplyCommitRequest{term=2, version=3751, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:54:05,346][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][93650] overhead, spent [382ms] collecting in the last [1s]
[2022-10-15T16:55:14,021][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3822, reason: ApplyCommitRequest{term=2, version=3822, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T16:55:14,134][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,136][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,150][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,243][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,342][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,445][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,537][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,546][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,633][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,646][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,733][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,738][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,754][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,841][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,847][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,861][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,940][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:14,948][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T16:55:15,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T16:55:15,043][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T16:55:15,132][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T16:55:16,721][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3823, reason: ApplyCommitRequest{term=2, version=3823, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:52:24,016][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T17:52:24,021][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3893, reason: becoming candidate: onLeaderFailure
[2022-10-15T17:52:24,358][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3894, reason: ApplyCommitRequest{term=2, version=3894, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:53:29,154][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3895, reason: ApplyCommitRequest{term=2, version=3895, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:53:32,272][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 3896, reason: ApplyCommitRequest{term=2, version=3896, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:53:42,584][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T17:53:42,595][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 3896, reason: becoming candidate: onLeaderFailure
[2022-10-15T17:53:42,946][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 3898, reason: ApplyCommitRequest{term=2, version=3898, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:55:17,493][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3965, reason: ApplyCommitRequest{term=2, version=3965, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T17:55:17,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:17,649][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T17:55:17,733][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T17:55:17,838][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T17:55:17,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:17,946][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,047][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,139][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,147][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,232][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,242][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,256][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,343][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,355][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,438][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,446][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,734][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,832][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,840][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T17:55:18,856][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T17:55:19,424][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 3966, reason: ApplyCommitRequest{term=2, version=3966, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:52:24,038][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T18:52:24,048][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4036, reason: becoming candidate: onLeaderFailure
[2022-10-15T18:52:24,410][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4037, reason: ApplyCommitRequest{term=2, version=4037, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:53:32,520][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4038, reason: ApplyCommitRequest{term=2, version=4038, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:53:35,720][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4039, reason: ApplyCommitRequest{term=2, version=4039, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:53:45,670][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T18:53:45,684][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4039, reason: becoming candidate: onLeaderFailure
[2022-10-15T18:53:47,138][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4041, reason: ApplyCommitRequest{term=2, version=4041, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:55:20,162][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4110, reason: ApplyCommitRequest{term=2, version=4110, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T18:55:20,232][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,241][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,333][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,453][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,542][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,554][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,645][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,733][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,745][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,836][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T18:55:20,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,047][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,141][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,244][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,333][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,535][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,539][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:21,550][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T18:55:22,425][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4111, reason: ApplyCommitRequest{term=2, version=4111, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:52:24,150][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T19:52:24,168][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4182, reason: becoming candidate: onLeaderFailure
[2022-10-15T19:52:24,436][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4183, reason: ApplyCommitRequest{term=2, version=4183, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:53:35,852][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4184, reason: ApplyCommitRequest{term=2, version=4184, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:53:38,852][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4185, reason: ApplyCommitRequest{term=2, version=4185, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:53:48,671][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T19:53:48,737][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4185, reason: becoming candidate: onLeaderFailure
[2022-10-15T19:53:49,555][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4187, reason: ApplyCommitRequest{term=2, version=4187, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:55:23,158][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4256, reason: ApplyCommitRequest{term=2, version=4256, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T19:55:23,251][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,445][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,461][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,553][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,561][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,649][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,774][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,835][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:23,865][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,033][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,035][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,150][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,153][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,160][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,234][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,253][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,345][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,352][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,434][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,469][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T19:55:24,472][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T19:55:25,724][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4257, reason: ApplyCommitRequest{term=2, version=4257, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:52:24,251][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T20:52:24,261][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4325, reason: becoming candidate: onLeaderFailure
[2022-10-15T20:52:24,624][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4326, reason: ApplyCommitRequest{term=2, version=4326, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:53:39,180][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4327, reason: ApplyCommitRequest{term=2, version=4327, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:53:42,192][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4328, reason: ApplyCommitRequest{term=2, version=4328, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:53:51,856][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T20:53:51,868][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4328, reason: becoming candidate: onLeaderFailure
[2022-10-15T20:53:52,290][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4330, reason: ApplyCommitRequest{term=2, version=4330, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:55:26,364][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4399, reason: ApplyCommitRequest{term=2, version=4399, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T20:55:26,538][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T20:55:26,634][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T20:55:26,839][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:26,934][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T20:55:26,941][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,032][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,045][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,135][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,152][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,241][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,252][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,347][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,354][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,443][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,449][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,460][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,543][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,647][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,734][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,745][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:27,752][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T20:55:29,028][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4400, reason: ApplyCommitRequest{term=2, version=4400, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:52:24,282][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T21:52:24,294][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4470, reason: becoming candidate: onLeaderFailure
[2022-10-15T21:52:24,681][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4471, reason: ApplyCommitRequest{term=2, version=4471, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:53:42,597][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4472, reason: ApplyCommitRequest{term=2, version=4472, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:53:45,525][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4473, reason: ApplyCommitRequest{term=2, version=4473, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:53:54,920][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T21:53:54,937][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4473, reason: becoming candidate: onLeaderFailure
[2022-10-15T21:53:55,459][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4475, reason: ApplyCommitRequest{term=2, version=4475, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:55:29,792][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4543, reason: ApplyCommitRequest{term=2, version=4543, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T21:55:29,936][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T21:55:29,945][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,036][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,048][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,147][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,238][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,333][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,436][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,639][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,738][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,743][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,835][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,850][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,943][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,944][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T21:55:30,963][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,040][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,049][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,134][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,140][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,152][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,159][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T21:55:31,562][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4544, reason: ApplyCommitRequest{term=2, version=4544, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:52:24,351][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.65.93:9300][disconnected] disconnected
[2022-10-15T22:52:24,360][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4615, reason: becoming candidate: onLeaderFailure
[2022-10-15T22:52:24,942][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4616, reason: ApplyCommitRequest{term=2, version=4616, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:53:45,826][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4617, reason: ApplyCommitRequest{term=2, version=4617, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:53:48,927][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-7c4fc6d7fd-sbh8z}{hsjBrGCOSEWNqrJ6xZUj2g}{pmlpNj8xQ9yXNfJbD7un3Q}{10.233.69.201}{10.233.69.201:9300}{r}}, term: 2, version: 4618, reason: ApplyCommitRequest{term=2, version=4618, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:53:57,231][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-master-0] master node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.93:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-master-0}{7um74vIcSyewFiD9B7M30w}{zXhrBUK1RSq1Cqgv69Keng}{10.233.72.39}{10.233.72.39:9300}{dmr}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-10-15T22:53:57,334][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}], current []}, term: 2, version: 4618, reason: becoming candidate: onLeaderFailure
[2022-10-15T22:53:57,919][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}]}, term: 2, version: 4620, reason: ApplyCommitRequest{term=2, version=4620, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:54:18,646][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][115250] overhead, spent [305ms] collecting in the last [1s]
[2022-10-15T22:55:32,928][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{No94EsJOTM-kk9Fq8fSO9g}{SdvrZ8yeRMS2hmVy4XKRmQ}{10.233.69.23}{10.233.69.23:9300}{dmr}}, term: 2, version: 4689, reason: ApplyCommitRequest{term=2, version=4689, sourceNode={onap-sdnrdb-master-1}{mPUdSz6LTq-eI9TuIEvEqg}{oqdZdTJjTA-QwvfQxp9DCw}{10.233.65.93}{10.233.65.93:9300}{dmr}}
[2022-10-15T22:55:32,962][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,043][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,133][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,151][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,235][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,252][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [maintenancemode-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,332][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [cmlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,348][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,433][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,541][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [connectionlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,554][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,647][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,662][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,736][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,747][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,755][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,843][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,851][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,933][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations
[2022-10-15T22:55:33,936][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:34,034][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [faultlog-v7][0] primary-replica resync completed with 0 operations
[2022-10-15T22:55:34,046][INFO ][o.e.i.s.IndexShard       ] [onap-sdnrdb-master-0] [mediator-server-v7][2] primary-replica resync completed with 0 operations
[2022-10-15T22:55:34,819][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-ma