Results

By type

raw version 
           15:21:24.04 
 15:21:24.05 Welcome to the Bitnami elasticsearch container
 15:21:24.05 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-elasticsearch
 15:21:24.05 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-elasticsearch/issues
 15:21:24.06 
 15:21:24.06 INFO  ==> ** Starting Elasticsearch setup **
 15:21:24.11 INFO  ==> Configuring/Initializing Elasticsearch...
 15:21:24.18 INFO  ==> Setting default configuration
 15:21:24.20 INFO  ==> Configuring Elasticsearch cluster settings...
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

 15:21:27.23 INFO  ==> ** Elasticsearch setup finished! **
 15:21:27.26 INFO  ==> ** Starting Elasticsearch **
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-06-24T15:21:32,673][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/5.4.0-96-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS]
[2022-06-24T15:21:32,676][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] JVM home [/opt/bitnami/java]
[2022-06-24T15:21:32,677][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-17360857507498043886, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2022-06-24T15:21:34,582][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [aggs-matrix-stats]
[2022-06-24T15:21:34,582][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [analysis-common]
[2022-06-24T15:21:34,583][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [geo]
[2022-06-24T15:21:34,583][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [ingest-common]
[2022-06-24T15:21:34,584][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [ingest-geoip]
[2022-06-24T15:21:34,584][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [ingest-user-agent]
[2022-06-24T15:21:34,584][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [kibana]
[2022-06-24T15:21:34,585][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [lang-expression]
[2022-06-24T15:21:34,585][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [lang-mustache]
[2022-06-24T15:21:34,586][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [lang-painless]
[2022-06-24T15:21:34,586][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [mapper-extras]
[2022-06-24T15:21:34,587][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [parent-join]
[2022-06-24T15:21:34,587][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [percolator]
[2022-06-24T15:21:34,587][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [rank-eval]
[2022-06-24T15:21:34,588][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [reindex]
[2022-06-24T15:21:34,588][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [repository-url]
[2022-06-24T15:21:34,589][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [tasks]
[2022-06-24T15:21:34,589][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded module [transport-netty4]
[2022-06-24T15:21:34,590][INFO ][o.e.p.PluginsService     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] loaded plugin [repository-s3]
[2022-06-24T15:21:34,653][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] using [1] data paths, mounts [[/bitnami/elasticsearch/data (/dev/vda1)]], net usable_space [152gb], net total_space [154.8gb], types [ext4]
[2022-06-24T15:21:34,654][INFO ][o.e.e.NodeEnvironment    ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] heap size [123.7mb], compressed ordinary object pointers [true]
[2022-06-24T15:21:34,713][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] node name [onap-sdnrdb-coordinating-only-99f8cd997-nftr4], node ID [igbnV7qcScCRkekKvjF7_w], cluster name [sdnrdb-cluster]
[2022-06-24T15:21:39,898][INFO ][o.e.t.NettyAllocator     ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}]
[2022-06-24T15:21:40,005][INFO ][o.e.d.DiscoveryModule    ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] using discovery type [zen] and seed hosts providers [settings]
[2022-06-24T15:21:40,400][WARN ][o.e.g.DanglingIndicesState] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-06-24T15:21:40,681][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] initialized
[2022-06-24T15:21:40,681][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] starting ...
[2022-06-24T15:21:40,861][INFO ][o.e.t.TransportService   ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] publish_address {10.233.65.153:9300}, bound_addresses {0.0.0.0:9300}
[2022-06-24T15:21:41,084][INFO ][o.e.b.BootstrapChecks    ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-06-24T15:24:39,745][INFO ][o.e.c.c.JoinHelper       ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] failed to join {onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}, minimumTerm=1, optionalJoin=Optional[Join{term=1, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}, targetNode={onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.72:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 1 does not match current term 2
	at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:1013) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:532) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture$1.doRun(ListenableFuture.java:112) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.lambda$done$0(ListenableFuture.java:98) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:98) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:144) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:127) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.ClusterConnectionManager.lambda$connectToNode$1(ClusterConnectionManager.java:164) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$4.onResponse(ActionListener.java:163) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:474) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:464) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:54) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1162) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
[2022-06-24T15:24:41,122][INFO ][o.e.c.c.JoinHelper       ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] last failed join attempt was 1.3s ago, failed to join {onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}, minimumTerm=1, optionalJoin=Optional[Join{term=1, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}, targetNode={onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.65.72:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 1 does not match current term 2
	at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:1013) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:532) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture$1.doRun(ListenableFuture.java:112) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.lambda$done$0(ListenableFuture.java:98) ~[elasticsearch-7.9.3.jar:7.9.3]
	at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:98) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:144) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:127) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.ClusterConnectionManager.lambda$connectToNode$1(ClusterConnectionManager.java:164) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener$4.onResponse(ActionListener.java:163) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:474) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$5.onResponse(TransportService.java:464) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:54) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1162) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
[2022-06-24T15:24:45,927][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, added {{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr},{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2, reason: ApplyCommitRequest{term=2, version=2, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T15:24:45,950][INFO ][o.e.h.AbstractHttpServerTransport] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] publish_address {10.233.65.153:9000}, bound_addresses {0.0.0.0:9000}
[2022-06-24T15:24:45,951][INFO ][o.e.n.Node               ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] started
[2022-06-24T15:24:52,927][INFO ][o.e.c.s.ClusterSettings  ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] updating [action.auto_create_index] from [true] to [false]
[2022-06-24T15:25:42,744][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 38, reason: ApplyCommitRequest{term=2, version=38, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:24:36,262][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T16:24:36,272][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 131, reason: becoming candidate: onLeaderFailure
[2022-06-24T16:24:38,829][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 132, reason: ApplyCommitRequest{term=2, version=132, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:24:46,879][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T16:24:46,890][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 132, reason: becoming candidate: onLeaderFailure
[2022-06-24T16:24:47,626][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 138, reason: ApplyCommitRequest{term=2, version=138, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:39,952][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 205, reason: ApplyCommitRequest{term=2, version=205, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:44,166][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 206, reason: ApplyCommitRequest{term=2, version=206, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:44,590][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 207, reason: ApplyCommitRequest{term=2, version=207, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:44,972][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 208, reason: ApplyCommitRequest{term=2, version=208, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:45,353][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 209, reason: ApplyCommitRequest{term=2, version=209, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T16:25:45,959][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 210, reason: ApplyCommitRequest{term=2, version=210, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:36,107][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T17:24:36,117][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 280, reason: becoming candidate: onLeaderFailure
[2022-06-24T17:24:36,353][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 281, reason: ApplyCommitRequest{term=2, version=281, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:46,304][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 282, reason: ApplyCommitRequest{term=2, version=282, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:51,411][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T17:24:51,424][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 282, reason: becoming candidate: onLeaderFailure
[2022-06-24T17:24:52,917][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 292, reason: ApplyCommitRequest{term=2, version=292, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:53,628][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 293, reason: ApplyCommitRequest{term=2, version=293, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:54,013][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 294, reason: ApplyCommitRequest{term=2, version=294, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:54,600][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 295, reason: ApplyCommitRequest{term=2, version=295, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:54,909][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 296, reason: ApplyCommitRequest{term=2, version=296, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:55,305][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 297, reason: ApplyCommitRequest{term=2, version=297, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:24:55,718][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 298, reason: ApplyCommitRequest{term=2, version=298, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:25:46,571][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 366, reason: ApplyCommitRequest{term=2, version=366, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T17:25:48,719][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 367, reason: ApplyCommitRequest{term=2, version=367, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:24:35,800][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T18:24:35,808][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 437, reason: becoming candidate: onLeaderFailure
[2022-06-24T18:24:35,914][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 438, reason: ApplyCommitRequest{term=2, version=438, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:24:53,927][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T18:24:53,938][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 438, reason: becoming candidate: onLeaderFailure
[2022-06-24T18:24:54,189][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 440, reason: ApplyCommitRequest{term=2, version=440, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:24:57,101][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 441, reason: ApplyCommitRequest{term=2, version=441, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:24:59,363][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 442, reason: ApplyCommitRequest{term=2, version=442, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:24:59,746][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 443, reason: ApplyCommitRequest{term=2, version=443, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:25:00,281][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 444, reason: ApplyCommitRequest{term=2, version=444, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:25:49,111][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 511, reason: ApplyCommitRequest{term=2, version=511, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:25:51,956][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 512, reason: ApplyCommitRequest{term=2, version=512, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:25:52,460][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 513, reason: ApplyCommitRequest{term=2, version=513, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T18:25:52,970][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 514, reason: ApplyCommitRequest{term=2, version=514, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:24:35,479][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T19:24:35,488][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 583, reason: becoming candidate: onLeaderFailure
[2022-06-24T19:24:35,601][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 584, reason: ApplyCommitRequest{term=2, version=584, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:24:56,601][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T19:24:56,611][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 584, reason: becoming candidate: onLeaderFailure
[2022-06-24T19:24:57,000][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 586, reason: ApplyCommitRequest{term=2, version=586, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:01,447][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 587, reason: ApplyCommitRequest{term=2, version=587, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:05,653][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 588, reason: ApplyCommitRequest{term=2, version=588, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:06,163][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 589, reason: ApplyCommitRequest{term=2, version=589, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:06,643][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 590, reason: ApplyCommitRequest{term=2, version=590, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:06,945][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 591, reason: ApplyCommitRequest{term=2, version=591, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:07,113][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 592, reason: ApplyCommitRequest{term=2, version=592, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:53,430][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 661, reason: ApplyCommitRequest{term=2, version=661, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T19:25:55,781][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 662, reason: ApplyCommitRequest{term=2, version=662, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:24:35,163][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T20:24:35,169][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 731, reason: becoming candidate: onLeaderFailure
[2022-06-24T20:24:35,276][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 732, reason: ApplyCommitRequest{term=2, version=732, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:24:59,300][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T20:24:59,319][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 732, reason: becoming candidate: onLeaderFailure
[2022-06-24T20:24:59,577][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 734, reason: ApplyCommitRequest{term=2, version=734, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:07,776][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 735, reason: ApplyCommitRequest{term=2, version=735, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:11,772][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 736, reason: ApplyCommitRequest{term=2, version=736, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:12,243][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 737, reason: ApplyCommitRequest{term=2, version=737, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:12,946][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 738, reason: ApplyCommitRequest{term=2, version=738, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:56,157][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 806, reason: ApplyCommitRequest{term=2, version=806, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T20:25:57,789][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 807, reason: ApplyCommitRequest{term=2, version=807, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:24:34,849][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T21:24:34,864][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 876, reason: becoming candidate: onLeaderFailure
[2022-06-24T21:24:34,972][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 877, reason: ApplyCommitRequest{term=2, version=877, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:02,027][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T21:25:02,039][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 877, reason: becoming candidate: onLeaderFailure
[2022-06-24T21:25:02,220][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 879, reason: ApplyCommitRequest{term=2, version=879, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:14,175][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 880, reason: ApplyCommitRequest{term=2, version=880, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:16,980][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 881, reason: ApplyCommitRequest{term=2, version=881, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:17,404][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 882, reason: ApplyCommitRequest{term=2, version=882, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:18,191][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 883, reason: ApplyCommitRequest{term=2, version=883, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:25:58,268][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 952, reason: ApplyCommitRequest{term=2, version=952, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T21:26:00,300][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 953, reason: ApplyCommitRequest{term=2, version=953, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:24:34,564][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T22:24:34,572][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1022, reason: becoming candidate: onLeaderFailure
[2022-06-24T22:24:34,701][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1023, reason: ApplyCommitRequest{term=2, version=1023, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:25:04,750][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T22:25:04,762][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1023, reason: becoming candidate: onLeaderFailure
[2022-06-24T22:25:04,871][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1025, reason: ApplyCommitRequest{term=2, version=1025, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:25:18,969][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1026, reason: ApplyCommitRequest{term=2, version=1026, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:25:21,029][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1027, reason: ApplyCommitRequest{term=2, version=1027, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:25:21,451][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1028, reason: ApplyCommitRequest{term=2, version=1028, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:25:21,765][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1029, reason: ApplyCommitRequest{term=2, version=1029, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:26:00,947][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1097, reason: ApplyCommitRequest{term=2, version=1097, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T22:26:02,825][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1098, reason: ApplyCommitRequest{term=2, version=1098, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:24:34,246][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-24T23:24:34,255][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1168, reason: becoming candidate: onLeaderFailure
[2022-06-24T23:24:34,396][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1169, reason: ApplyCommitRequest{term=2, version=1169, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:25:07,466][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-24T23:25:07,477][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1170, reason: becoming candidate: onLeaderFailure
[2022-06-24T23:25:07,678][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1172, reason: ApplyCommitRequest{term=2, version=1172, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:25:22,886][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1173, reason: ApplyCommitRequest{term=2, version=1173, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:25:24,703][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1174, reason: ApplyCommitRequest{term=2, version=1174, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:25:25,075][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1175, reason: ApplyCommitRequest{term=2, version=1175, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:25:25,876][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1176, reason: ApplyCommitRequest{term=2, version=1176, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:26:02,812][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1244, reason: ApplyCommitRequest{term=2, version=1244, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-24T23:26:05,548][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1245, reason: ApplyCommitRequest{term=2, version=1245, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:24:33,950][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T00:24:33,984][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1314, reason: becoming candidate: onLeaderFailure
[2022-06-25T00:24:34,188][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1315, reason: ApplyCommitRequest{term=2, version=1315, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:25:10,262][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T00:25:10,273][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1316, reason: becoming candidate: onLeaderFailure
[2022-06-25T00:25:10,411][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1318, reason: ApplyCommitRequest{term=2, version=1318, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:25:26,484][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1319, reason: ApplyCommitRequest{term=2, version=1319, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:25:28,123][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1320, reason: ApplyCommitRequest{term=2, version=1320, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:25:28,444][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1321, reason: ApplyCommitRequest{term=2, version=1321, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:25:28,678][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1322, reason: ApplyCommitRequest{term=2, version=1322, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:26:05,726][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1393, reason: ApplyCommitRequest{term=2, version=1393, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T00:26:08,270][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1394, reason: ApplyCommitRequest{term=2, version=1394, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:24:33,669][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T01:24:33,684][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1464, reason: becoming candidate: onLeaderFailure
[2022-06-25T01:24:34,087][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1465, reason: ApplyCommitRequest{term=2, version=1465, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:25:12,931][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T01:25:12,946][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1466, reason: becoming candidate: onLeaderFailure
[2022-06-25T01:25:13,108][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1468, reason: ApplyCommitRequest{term=2, version=1468, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:25:29,557][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1469, reason: ApplyCommitRequest{term=2, version=1469, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:25:31,367][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1470, reason: ApplyCommitRequest{term=2, version=1470, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:25:31,699][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1471, reason: ApplyCommitRequest{term=2, version=1471, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:25:32,602][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1472, reason: ApplyCommitRequest{term=2, version=1472, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:26:08,713][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1540, reason: ApplyCommitRequest{term=2, version=1540, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T01:26:10,971][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1541, reason: ApplyCommitRequest{term=2, version=1541, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:24:33,409][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T02:24:33,417][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1611, reason: becoming candidate: onLeaderFailure
[2022-06-25T02:24:33,576][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1612, reason: ApplyCommitRequest{term=2, version=1612, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:25:14,679][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T02:25:14,691][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1613, reason: becoming candidate: onLeaderFailure
[2022-06-25T02:25:14,835][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1615, reason: ApplyCommitRequest{term=2, version=1615, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:25:33,193][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1616, reason: ApplyCommitRequest{term=2, version=1616, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:25:35,833][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1618, reason: ApplyCommitRequest{term=2, version=1618, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:26:11,057][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1686, reason: ApplyCommitRequest{term=2, version=1686, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T02:26:13,215][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1687, reason: ApplyCommitRequest{term=2, version=1687, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:24:33,097][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T03:24:33,110][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1755, reason: becoming candidate: onLeaderFailure
[2022-06-25T03:24:33,317][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1756, reason: ApplyCommitRequest{term=2, version=1756, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:25:16,405][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T03:25:16,416][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1757, reason: becoming candidate: onLeaderFailure
[2022-06-25T03:25:16,544][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1759, reason: ApplyCommitRequest{term=2, version=1759, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:25:36,086][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1761, reason: ApplyCommitRequest{term=2, version=1761, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:25:38,345][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1762, reason: ApplyCommitRequest{term=2, version=1762, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:26:13,398][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1833, reason: ApplyCommitRequest{term=2, version=1833, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T03:26:15,572][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1834, reason: ApplyCommitRequest{term=2, version=1834, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:24:32,819][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T04:24:32,832][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1903, reason: becoming candidate: onLeaderFailure
[2022-06-25T04:24:33,324][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1904, reason: ApplyCommitRequest{term=2, version=1904, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:25:18,166][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T04:25:18,181][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 1905, reason: becoming candidate: onLeaderFailure
[2022-06-25T04:25:18,620][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 1907, reason: ApplyCommitRequest{term=2, version=1907, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:25:38,770][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1909, reason: ApplyCommitRequest{term=2, version=1909, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:25:41,013][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 1910, reason: ApplyCommitRequest{term=2, version=1910, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:26:15,941][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1977, reason: ApplyCommitRequest{term=2, version=1977, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T04:26:17,928][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 1978, reason: ApplyCommitRequest{term=2, version=1978, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:24:32,534][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T05:24:32,544][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2046, reason: becoming candidate: onLeaderFailure
[2022-06-25T05:24:32,776][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2047, reason: ApplyCommitRequest{term=2, version=2047, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:25:20,811][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T05:25:20,822][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2048, reason: becoming candidate: onLeaderFailure
[2022-06-25T05:25:20,942][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2050, reason: ApplyCommitRequest{term=2, version=2050, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:25:42,123][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2052, reason: ApplyCommitRequest{term=2, version=2052, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:25:45,948][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2053, reason: ApplyCommitRequest{term=2, version=2053, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:25:46,203][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2054, reason: ApplyCommitRequest{term=2, version=2054, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:25:46,450][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2055, reason: ApplyCommitRequest{term=2, version=2055, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:26:17,992][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2119, reason: ApplyCommitRequest{term=2, version=2119, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T05:26:20,552][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2121, reason: ApplyCommitRequest{term=2, version=2121, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:24:32,237][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T06:24:32,249][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2189, reason: becoming candidate: onLeaderFailure
[2022-06-25T06:24:32,555][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2190, reason: ApplyCommitRequest{term=2, version=2190, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:25:22,711][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T06:25:22,723][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2191, reason: becoming candidate: onLeaderFailure
[2022-06-25T06:25:22,860][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2193, reason: ApplyCommitRequest{term=2, version=2193, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:25:47,290][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2195, reason: ApplyCommitRequest{term=2, version=2195, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:25:51,963][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2196, reason: ApplyCommitRequest{term=2, version=2196, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:25:52,660][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2197, reason: ApplyCommitRequest{term=2, version=2197, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:25:53,492][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2198, reason: ApplyCommitRequest{term=2, version=2198, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:26:21,341][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2229, reason: ApplyCommitRequest{term=2, version=2229, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T06:26:23,424][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2232, reason: ApplyCommitRequest{term=2, version=2232, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:24:31,962][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T07:24:31,976][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2303, reason: becoming candidate: onLeaderFailure
[2022-06-25T07:24:32,212][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2304, reason: ApplyCommitRequest{term=2, version=2304, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:25:25,247][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T07:25:25,263][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2305, reason: becoming candidate: onLeaderFailure
[2022-06-25T07:25:25,392][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2307, reason: ApplyCommitRequest{term=2, version=2307, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:25:54,359][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2309, reason: ApplyCommitRequest{term=2, version=2309, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:25:56,550][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2310, reason: ApplyCommitRequest{term=2, version=2310, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:25:56,840][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2311, reason: ApplyCommitRequest{term=2, version=2311, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:25:57,069][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2312, reason: ApplyCommitRequest{term=2, version=2312, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:26:24,037][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2365, reason: ApplyCommitRequest{term=2, version=2365, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T07:26:26,475][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2369, reason: ApplyCommitRequest{term=2, version=2369, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:24:31,668][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T08:24:31,676][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2441, reason: becoming candidate: onLeaderFailure
[2022-06-25T08:24:31,903][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2442, reason: ApplyCommitRequest{term=2, version=2442, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:25:27,935][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T08:25:27,950][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2443, reason: becoming candidate: onLeaderFailure
[2022-06-25T08:25:28,104][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2445, reason: ApplyCommitRequest{term=2, version=2445, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:25:58,200][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2447, reason: ApplyCommitRequest{term=2, version=2447, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:26:00,093][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2448, reason: ApplyCommitRequest{term=2, version=2448, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:26:00,349][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2449, reason: ApplyCommitRequest{term=2, version=2449, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:26:00,703][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2450, reason: ApplyCommitRequest{term=2, version=2450, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:26:25,968][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2506, reason: ApplyCommitRequest{term=2, version=2506, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T08:26:29,446][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2510, reason: ApplyCommitRequest{term=2, version=2510, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:24:31,364][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T09:24:31,374][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2581, reason: becoming candidate: onLeaderFailure
[2022-06-25T09:24:32,157][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2582, reason: ApplyCommitRequest{term=2, version=2582, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:25:29,818][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T09:25:29,834][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2583, reason: becoming candidate: onLeaderFailure
[2022-06-25T09:25:29,946][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2585, reason: ApplyCommitRequest{term=2, version=2585, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:01,535][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2587, reason: ApplyCommitRequest{term=2, version=2587, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:03,165][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2588, reason: ApplyCommitRequest{term=2, version=2588, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:03,531][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2589, reason: ApplyCommitRequest{term=2, version=2589, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:03,833][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2590, reason: ApplyCommitRequest{term=2, version=2590, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:30,032][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2655, reason: ApplyCommitRequest{term=2, version=2655, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T09:26:32,660][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2658, reason: ApplyCommitRequest{term=2, version=2658, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:24:31,055][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T10:24:31,062][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2726, reason: becoming candidate: onLeaderFailure
[2022-06-25T10:24:31,187][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2727, reason: ApplyCommitRequest{term=2, version=2727, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:25:32,319][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T10:25:32,331][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2728, reason: becoming candidate: onLeaderFailure
[2022-06-25T10:25:32,439][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2731, reason: ApplyCommitRequest{term=2, version=2731, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:04,967][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2732, reason: ApplyCommitRequest{term=2, version=2732, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:07,380][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2733, reason: ApplyCommitRequest{term=2, version=2733, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:07,879][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2734, reason: ApplyCommitRequest{term=2, version=2734, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:08,389][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2735, reason: ApplyCommitRequest{term=2, version=2735, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:32,243][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2782, reason: ApplyCommitRequest{term=2, version=2782, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:34,974][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2783, reason: ApplyCommitRequest{term=2, version=2783, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:36,072][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2784, reason: ApplyCommitRequest{term=2, version=2784, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:37,950][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2786, reason: ApplyCommitRequest{term=2, version=2786, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:38,166][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2787, reason: ApplyCommitRequest{term=2, version=2787, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T10:26:39,251][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2790, reason: ApplyCommitRequest{term=2, version=2790, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:24:30,736][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T11:24:30,749][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2864, reason: becoming candidate: onLeaderFailure
[2022-06-25T11:24:30,896][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2865, reason: ApplyCommitRequest{term=2, version=2865, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:25:35,055][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T11:25:35,067][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2867, reason: becoming candidate: onLeaderFailure
[2022-06-25T11:25:35,188][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2869, reason: ApplyCommitRequest{term=2, version=2869, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:26:08,590][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2870, reason: ApplyCommitRequest{term=2, version=2870, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:26:11,043][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2871, reason: ApplyCommitRequest{term=2, version=2871, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:26:39,416][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2915, reason: ApplyCommitRequest{term=2, version=2915, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T11:26:42,140][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 2922, reason: ApplyCommitRequest{term=2, version=2922, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:24:30,455][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T12:24:30,464][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2992, reason: becoming candidate: onLeaderFailure
[2022-06-25T12:24:30,624][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2993, reason: ApplyCommitRequest{term=2, version=2993, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:25:36,764][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T12:25:36,777][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 2995, reason: becoming candidate: onLeaderFailure
[2022-06-25T12:25:37,000][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 2997, reason: ApplyCommitRequest{term=2, version=2997, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:11,887][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2998, reason: ApplyCommitRequest{term=2, version=2998, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:13,831][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 2999, reason: ApplyCommitRequest{term=2, version=2999, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:14,167][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3000, reason: ApplyCommitRequest{term=2, version=3000, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:14,787][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3001, reason: ApplyCommitRequest{term=2, version=3001, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:42,689][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3070, reason: ApplyCommitRequest{term=2, version=3070, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T12:26:43,779][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3071, reason: ApplyCommitRequest{term=2, version=3071, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:24:30,154][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T13:24:30,170][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3140, reason: becoming candidate: onLeaderFailure
[2022-06-25T13:24:30,303][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3141, reason: ApplyCommitRequest{term=2, version=3141, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:25:39,465][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T13:25:39,478][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3143, reason: becoming candidate: onLeaderFailure
[2022-06-25T13:25:39,619][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3145, reason: ApplyCommitRequest{term=2, version=3145, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:15,657][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3146, reason: ApplyCommitRequest{term=2, version=3146, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:17,829][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3147, reason: ApplyCommitRequest{term=2, version=3147, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:18,225][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3148, reason: ApplyCommitRequest{term=2, version=3148, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:18,530][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3149, reason: ApplyCommitRequest{term=2, version=3149, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:44,131][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3217, reason: ApplyCommitRequest{term=2, version=3217, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T13:26:45,913][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3218, reason: ApplyCommitRequest{term=2, version=3218, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:24:29,852][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T14:24:29,859][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3285, reason: becoming candidate: onLeaderFailure
[2022-06-25T14:24:30,014][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3286, reason: ApplyCommitRequest{term=2, version=3286, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:25:41,217][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T14:25:41,229][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3288, reason: becoming candidate: onLeaderFailure
[2022-06-25T14:25:41,358][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3290, reason: ApplyCommitRequest{term=2, version=3290, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:19,601][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3291, reason: ApplyCommitRequest{term=2, version=3291, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:23,710][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3292, reason: ApplyCommitRequest{term=2, version=3292, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:24,222][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3293, reason: ApplyCommitRequest{term=2, version=3293, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:25,211][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3294, reason: ApplyCommitRequest{term=2, version=3294, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:46,452][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3338, reason: ApplyCommitRequest{term=2, version=3338, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T14:26:49,874][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3341, reason: ApplyCommitRequest{term=2, version=3341, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:24:29,544][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T15:24:29,553][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3412, reason: becoming candidate: onLeaderFailure
[2022-06-25T15:24:29,743][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3413, reason: ApplyCommitRequest{term=2, version=3413, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:25:43,890][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T15:25:43,899][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3415, reason: becoming candidate: onLeaderFailure
[2022-06-25T15:25:44,062][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3417, reason: ApplyCommitRequest{term=2, version=3417, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:25,782][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3418, reason: ApplyCommitRequest{term=2, version=3418, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:27,846][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3419, reason: ApplyCommitRequest{term=2, version=3419, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:28,214][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3420, reason: ApplyCommitRequest{term=2, version=3420, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:28,365][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3421, reason: ApplyCommitRequest{term=2, version=3421, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:28,545][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3422, reason: ApplyCommitRequest{term=2, version=3422, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:29,203][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3423, reason: ApplyCommitRequest{term=2, version=3423, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:49,094][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3467, reason: ApplyCommitRequest{term=2, version=3467, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T15:26:52,049][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3472, reason: ApplyCommitRequest{term=2, version=3472, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:24:29,244][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T16:24:29,251][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3543, reason: becoming candidate: onLeaderFailure
[2022-06-25T16:24:29,377][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3544, reason: ApplyCommitRequest{term=2, version=3544, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:25:45,631][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T16:25:45,644][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3546, reason: becoming candidate: onLeaderFailure
[2022-06-25T16:25:45,765][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3548, reason: ApplyCommitRequest{term=2, version=3548, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:30,229][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3549, reason: ApplyCommitRequest{term=2, version=3549, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:32,739][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3550, reason: ApplyCommitRequest{term=2, version=3550, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:33,166][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3551, reason: ApplyCommitRequest{term=2, version=3551, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:34,012][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3552, reason: ApplyCommitRequest{term=2, version=3552, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:51,440][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3595, reason: ApplyCommitRequest{term=2, version=3595, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T16:26:54,241][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3600, reason: ApplyCommitRequest{term=2, version=3600, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:24:28,928][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T17:24:28,941][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3671, reason: becoming candidate: onLeaderFailure
[2022-06-25T17:24:29,172][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3672, reason: ApplyCommitRequest{term=2, version=3672, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:25:48,264][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T17:25:48,279][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3674, reason: becoming candidate: onLeaderFailure
[2022-06-25T17:25:48,432][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3676, reason: ApplyCommitRequest{term=2, version=3676, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:35,236][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3677, reason: ApplyCommitRequest{term=2, version=3677, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:36,997][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3678, reason: ApplyCommitRequest{term=2, version=3678, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:37,315][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3679, reason: ApplyCommitRequest{term=2, version=3679, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:37,458][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3680, reason: ApplyCommitRequest{term=2, version=3680, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:37,629][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3681, reason: ApplyCommitRequest{term=2, version=3681, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:38,284][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3682, reason: ApplyCommitRequest{term=2, version=3682, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:54,972][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3726, reason: ApplyCommitRequest{term=2, version=3726, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T17:26:57,200][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3730, reason: ApplyCommitRequest{term=2, version=3730, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:24:28,633][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T18:24:28,641][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3801, reason: becoming candidate: onLeaderFailure
[2022-06-25T18:24:28,808][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3802, reason: ApplyCommitRequest{term=2, version=3802, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:25:50,023][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T18:25:50,034][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3804, reason: becoming candidate: onLeaderFailure
[2022-06-25T18:25:50,197][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3806, reason: ApplyCommitRequest{term=2, version=3806, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:26:38,843][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3807, reason: ApplyCommitRequest{term=2, version=3807, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:26:41,049][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3808, reason: ApplyCommitRequest{term=2, version=3808, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:26:41,376][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3809, reason: ApplyCommitRequest{term=2, version=3809, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:26:41,657][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3810, reason: ApplyCommitRequest{term=2, version=3810, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:26:57,058][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3856, reason: ApplyCommitRequest{term=2, version=3856, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T18:27:00,055][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3861, reason: ApplyCommitRequest{term=2, version=3861, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:24:28,334][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T19:24:28,342][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3933, reason: becoming candidate: onLeaderFailure
[2022-06-25T19:24:28,507][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3934, reason: ApplyCommitRequest{term=2, version=3934, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:25:52,689][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T19:25:52,699][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 3936, reason: becoming candidate: onLeaderFailure
[2022-06-25T19:25:52,828][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 3938, reason: ApplyCommitRequest{term=2, version=3938, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:26:42,098][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3939, reason: ApplyCommitRequest{term=2, version=3939, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:26:43,977][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 3940, reason: ApplyCommitRequest{term=2, version=3940, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:26:59,674][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3989, reason: ApplyCommitRequest{term=2, version=3989, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T19:27:01,834][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 3993, reason: ApplyCommitRequest{term=2, version=3993, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:24:28,022][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T20:24:28,032][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4063, reason: becoming candidate: onLeaderFailure
[2022-06-25T20:24:28,207][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4064, reason: ApplyCommitRequest{term=2, version=4064, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:25:55,378][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T20:25:55,389][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4066, reason: becoming candidate: onLeaderFailure
[2022-06-25T20:25:55,538][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4068, reason: ApplyCommitRequest{term=2, version=4068, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:26:44,339][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4069, reason: ApplyCommitRequest{term=2, version=4069, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:26:45,995][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4070, reason: ApplyCommitRequest{term=2, version=4070, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:27:01,885][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4110, reason: ApplyCommitRequest{term=2, version=4110, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T20:27:04,740][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4116, reason: ApplyCommitRequest{term=2, version=4116, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:24:27,723][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T21:24:27,735][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4190, reason: becoming candidate: onLeaderFailure
[2022-06-25T21:24:27,906][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4191, reason: ApplyCommitRequest{term=2, version=4191, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:25:58,131][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T21:25:58,147][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4193, reason: becoming candidate: onLeaderFailure
[2022-06-25T21:25:58,297][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4195, reason: ApplyCommitRequest{term=2, version=4195, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:26:46,560][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4196, reason: ApplyCommitRequest{term=2, version=4196, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:26:49,209][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4197, reason: ApplyCommitRequest{term=2, version=4197, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:26:49,997][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4198, reason: ApplyCommitRequest{term=2, version=4198, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:26:50,263][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4199, reason: ApplyCommitRequest{term=2, version=4199, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:27:04,689][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4238, reason: ApplyCommitRequest{term=2, version=4238, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T21:27:08,284][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4241, reason: ApplyCommitRequest{term=2, version=4241, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:24:27,426][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T22:24:27,435][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4314, reason: becoming candidate: onLeaderFailure
[2022-06-25T22:24:27,643][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4315, reason: ApplyCommitRequest{term=2, version=4315, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:26:00,813][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T22:26:00,824][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4317, reason: becoming candidate: onLeaderFailure
[2022-06-25T22:26:00,991][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4319, reason: ApplyCommitRequest{term=2, version=4319, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:26:51,073][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4320, reason: ApplyCommitRequest{term=2, version=4320, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:26:52,728][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4321, reason: ApplyCommitRequest{term=2, version=4321, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:26:52,950][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4322, reason: ApplyCommitRequest{term=2, version=4322, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:26:53,346][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4323, reason: ApplyCommitRequest{term=2, version=4323, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:27:07,090][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4354, reason: ApplyCommitRequest{term=2, version=4354, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T22:27:09,151][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4358, reason: ApplyCommitRequest{term=2, version=4358, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:24:27,136][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-25T23:24:27,150][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4430, reason: becoming candidate: onLeaderFailure
[2022-06-25T23:24:27,413][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4431, reason: ApplyCommitRequest{term=2, version=4431, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:26:03,543][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-25T23:26:03,554][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4433, reason: becoming candidate: onLeaderFailure
[2022-06-25T23:26:03,676][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4435, reason: ApplyCommitRequest{term=2, version=4435, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:26:53,881][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4436, reason: ApplyCommitRequest{term=2, version=4436, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:26:56,289][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4437, reason: ApplyCommitRequest{term=2, version=4437, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:27:09,469][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4474, reason: ApplyCommitRequest{term=2, version=4474, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-25T23:27:12,239][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4479, reason: ApplyCommitRequest{term=2, version=4479, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:24:26,833][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T00:24:26,868][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4554, reason: becoming candidate: onLeaderFailure
[2022-06-26T00:24:27,021][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4555, reason: ApplyCommitRequest{term=2, version=4555, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:26:06,341][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-26T00:26:06,351][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4557, reason: becoming candidate: onLeaderFailure
[2022-06-26T00:26:06,571][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4559, reason: ApplyCommitRequest{term=2, version=4559, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:26:56,596][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4560, reason: ApplyCommitRequest{term=2, version=4560, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:26:58,318][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4561, reason: ApplyCommitRequest{term=2, version=4561, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:13,722][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4591, reason: ApplyCommitRequest{term=2, version=4591, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:14,820][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4592, reason: ApplyCommitRequest{term=2, version=4592, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:15,314][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4593, reason: ApplyCommitRequest{term=2, version=4593, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:16,039][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4595, reason: ApplyCommitRequest{term=2, version=4595, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:16,190][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4596, reason: ApplyCommitRequest{term=2, version=4596, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T00:27:17,549][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4599, reason: ApplyCommitRequest{term=2, version=4599, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:24:26,579][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T01:24:26,590][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4676, reason: becoming candidate: onLeaderFailure
[2022-06-26T01:24:26,743][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4677, reason: ApplyCommitRequest{term=2, version=4677, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:26:09,004][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-26T01:26:09,022][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4679, reason: becoming candidate: onLeaderFailure
[2022-06-26T01:26:09,161][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4681, reason: ApplyCommitRequest{term=2, version=4681, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:26:58,758][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4682, reason: ApplyCommitRequest{term=2, version=4682, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:27:00,831][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4683, reason: ApplyCommitRequest{term=2, version=4683, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:27:01,074][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4684, reason: ApplyCommitRequest{term=2, version=4684, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:27:01,378][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4685, reason: ApplyCommitRequest{term=2, version=4685, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:27:16,889][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4733, reason: ApplyCommitRequest{term=2, version=4733, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T01:27:18,996][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4736, reason: ApplyCommitRequest{term=2, version=4736, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:24:26,272][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T02:24:26,283][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4811, reason: becoming candidate: onLeaderFailure
[2022-06-26T02:24:26,431][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4812, reason: ApplyCommitRequest{term=2, version=4812, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:26:10,743][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-26T02:26:10,757][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4814, reason: becoming candidate: onLeaderFailure
[2022-06-26T02:26:10,895][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4816, reason: ApplyCommitRequest{term=2, version=4816, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:02,619][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4817, reason: ApplyCommitRequest{term=2, version=4817, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:05,247][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4818, reason: ApplyCommitRequest{term=2, version=4818, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:05,550][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4819, reason: ApplyCommitRequest{term=2, version=4819, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:06,136][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4820, reason: ApplyCommitRequest{term=2, version=4820, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:18,940][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4860, reason: ApplyCommitRequest{term=2, version=4860, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T02:27:21,114][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4864, reason: ApplyCommitRequest{term=2, version=4864, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:24:25,956][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T03:24:25,968][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4937, reason: becoming candidate: onLeaderFailure
[2022-06-26T03:24:26,160][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4938, reason: ApplyCommitRequest{term=2, version=4938, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:26:13,468][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-26T03:26:13,485][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 4940, reason: becoming candidate: onLeaderFailure
[2022-06-26T03:26:13,887][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 4942, reason: ApplyCommitRequest{term=2, version=4942, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:06,658][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4943, reason: ApplyCommitRequest{term=2, version=4943, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:08,827][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4944, reason: ApplyCommitRequest{term=2, version=4944, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:09,221][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4945, reason: ApplyCommitRequest{term=2, version=4945, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:10,063][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 4946, reason: ApplyCommitRequest{term=2, version=4946, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:20,982][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4974, reason: ApplyCommitRequest{term=2, version=4974, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T03:27:23,554][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 4980, reason: ApplyCommitRequest{term=2, version=4980, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:24:25,661][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T04:24:25,686][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 5058, reason: becoming candidate: onLeaderFailure
[2022-06-26T04:24:26,238][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 5059, reason: ApplyCommitRequest{term=2, version=5059, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:26:16,187][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
[2022-06-26T04:26:16,202][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 5061, reason: becoming candidate: onLeaderFailure
[2022-06-26T04:26:16,331][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 5063, reason: ApplyCommitRequest{term=2, version=5063, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:27:10,287][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 5064, reason: ApplyCommitRequest{term=2, version=5064, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:27:12,213][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-1}{GGccGa1dSbO4lH8NH0whtw}{CxdHGEiFTduhVpCryZtAoA}{10.233.65.72}{10.233.65.72:9300}{dmr}}, term: 2, version: 5065, reason: ApplyCommitRequest{term=2, version=5065, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:27:23,998][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] removed {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 5097, reason: ApplyCommitRequest{term=2, version=5097, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T04:27:27,004][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] added {{onap-sdnrdb-master-2}{1lEmZG9WRqOxtUS8cghAtA}{Jx9zUJPpQX-0W-Cfz5TnDw}{10.233.66.124}{10.233.66.124:9300}{dmr}}, term: 2, version: 5100, reason: ApplyCommitRequest{term=2, version=5100, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T05:24:25,398][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-0][10.233.67.247:9300][disconnected] disconnected
[2022-06-26T05:24:25,410][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}], current []}, term: 2, version: 5173, reason: becoming candidate: onLeaderFailure
[2022-06-26T05:24:25,531][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node changed {previous [], current [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}]}, term: 2, version: 5174, reason: ApplyCommitRequest{term=2, version=5174, sourceNode={onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}}
[2022-06-26T05:26:18,859][INFO ][o.e.c.c.Coordinator      ] [onap-sdnrdb-coordinating-only-99f8cd997-nftr4] master node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{onap-sdnrdb-master-0}{PuWbKEJXS7i-1HsSfeiHCg}{660dxkwCRYaujnXBwP8TpA}{10.233.67.247}{10.233.67.247:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:233) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:225) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:115) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.67.247:9300][internal:coordination/fault_detection/leader_check]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: rejecting leader check since [{onap-sdnrdb-coordinating-only-99f8cd997-nftr4}{igbnV7qcScCRkekKvjF7_w}{bN3GVq68QFi7DV6SUQRIVA}{10.233.65.153}{10.233.65.153:9300}{r}] has been removed from the cluster
	at org.elasticsearch.cluster.coordination.LeaderChecker.handleLeaderCheck(LeaderChecker.java:192) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.LeaderChecker.lambda$new$0(LeaderChecker.java:113) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:176) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractC