By type
[38;5;6m [38;5;5m00:02:59.92 [0m
[38;5;6m [38;5;5m00:02:59.93 [0m[1mWelcome to the Bitnami elasticsearch container[0m
[38;5;6m [38;5;5m00:03:00.01 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch[0m
[38;5;6m [38;5;5m00:03:00.02 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch/issues[0m
[38;5;6m [38;5;5m00:03:00.11 [0m
[38;5;6m [38;5;5m00:03:00.13 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch setup **
[38;5;6m [38;5;5m00:03:00.53 [0m[38;5;2mINFO [0m ==> Configuring/Initializing Elasticsearch...
[38;5;6m [38;5;5m00:03:01.01 [0m[38;5;2mINFO [0m ==> Setting default configuration
[38;5;6m [38;5;5m00:03:01.13 [0m[38;5;2mINFO [0m ==> Configuring Elasticsearch cluster settings...
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[38;5;6m [38;5;5m00:03:23.13 [0m[38;5;2mINFO [0m ==> ** Elasticsearch setup finished! **
[38;5;6m [38;5;5m00:03:23.41 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch **
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2023-03-24T00:04:15,715][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/5.4.0-96-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS]
[2023-03-24T00:04:15,817][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] JVM home [/opt/bitnami/java]
[2023-03-24T00:04:15,818][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-5511041984722313532, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2023-03-24T00:04:35,618][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [aggs-matrix-stats]
[2023-03-24T00:04:35,619][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [analysis-common]
[2023-03-24T00:04:35,620][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [geo]
[2023-03-24T00:04:35,621][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-common]
[2023-03-24T00:04:35,622][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-geoip]
[2023-03-24T00:04:35,622][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-user-agent]
[2023-03-24T00:04:35,623][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [kibana]
[2023-03-24T00:04:35,624][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-expression]
[2023-03-24T00:04:35,624][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-mustache]
[2023-03-24T00:04:35,714][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-painless]
[2023-03-24T00:04:35,716][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [mapper-extras]
[2023-03-24T00:04:35,716][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [parent-join]
[2023-03-24T00:04:35,718][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [percolator]
[2023-03-24T00:04:35,719][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [rank-eval]
[2023-03-24T00:04:35,719][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [reindex]
[2023-03-24T00:04:35,721][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [repository-url]
[2023-03-24T00:04:35,721][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [tasks]
[2023-03-24T00:04:35,722][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [transport-netty4]
[2023-03-24T00:04:35,724][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded plugin [repository-s3]
[2023-03-24T00:04:36,823][INFO ][o.e.e.NodeEnvironment ] [onap-sdnrdb-master-0] using [1] data paths, mounts [[/bitnami/elasticsearch/data (192.168.13.54:/dockerdata-nfs/onap/elastic-master-0)]], net usable_space [95.3gb], net total_space [99.9gb], types [nfs4]
[2023-03-24T00:04:36,824][INFO ][o.e.e.NodeEnvironment ] [onap-sdnrdb-master-0] heap size [123.7mb], compressed ordinary object pointers [true]
[2023-03-24T00:04:37,824][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] node name [onap-sdnrdb-master-0], node ID [ScUJPCxfTXGOqGEpsgqiPQ], cluster name [sdnrdb-cluster]
[2023-03-24T00:05:33,517][INFO ][o.e.t.NettyAllocator ] [onap-sdnrdb-master-0] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}]
[2023-03-24T00:05:34,920][INFO ][o.e.d.DiscoveryModule ] [onap-sdnrdb-master-0] using discovery type [zen] and seed hosts providers [settings]
[2023-03-24T00:05:40,216][WARN ][o.e.g.DanglingIndicesState] [onap-sdnrdb-master-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2023-03-24T00:05:43,726][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] initialized
[2023-03-24T00:05:43,814][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] starting ...
[2023-03-24T00:05:46,315][INFO ][o.e.t.TransportService ] [onap-sdnrdb-master-0] publish_address {10.233.66.171:9300}, bound_addresses {0.0.0.0:9300}
[2023-03-24T00:05:49,615][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.66.171:9300, remoteAddress=/127.0.0.6:37081}], closing connection
java.lang.IllegalStateException: transport not ready yet to handle incoming requests
at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:05:50,213][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.66.171:9300, remoteAddress=/127.0.0.6:45871}], closing connection
java.lang.IllegalStateException: transport not ready yet to handle incoming requests
at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:05:51,422][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][7] overhead, spent [305ms] collecting in the last [1s]
[2023-03-24T00:05:51,514][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.66.171:9300, remoteAddress=/127.0.0.6:56307}], closing connection
java.lang.IllegalStateException: transport not ready yet to handle incoming requests
at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:05:52,017][INFO ][o.e.b.BootstrapChecks ] [onap-sdnrdb-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2023-03-24T00:05:52,115][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.66.171:9300, remoteAddress=/127.0.0.6:42829}], closing connection
java.lang.IllegalStateException: transport not ready yet to handle incoming requests
at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:06:02,226][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]; discovery will continue using [10.233.76.228:9300, 10.233.65.132:9300, 10.233.73.197:9300] from hosts providers and [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2023-03-24T00:06:12,232][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]; discovery will continue using [10.233.76.228:9300, 10.233.65.132:9300, 10.233.73.197:9300] from hosts providers and [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2023-03-24T00:06:22,237][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]; discovery will continue using [10.233.76.228:9300, 10.233.65.132:9300, 10.233.73.197:9300] from hosts providers and [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2023-03-24T00:06:32,242][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]; discovery will continue using [10.233.76.228:9300, 10.233.65.132:9300, 10.233.73.197:9300] from hosts providers and [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2023-03-24T00:06:38,158][INFO ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] setting initial configuration to VotingConfiguration{UC3-LrdpSSutVC5sXkOr5Q,{bootstrap-placeholder}-onap-sdnrdb-master-2,ScUJPCxfTXGOqGEpsgqiPQ}
[2023-03-24T00:06:41,092][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, minimumTerm=0, optionalJoin=Optional[Join{term=1, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.66.171:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: received a newer join from {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.handleJoinRequest(JoinHelper.java:459) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:533) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:06:41,493][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, minimumTerm=1, optionalJoin=Optional[Join{term=2, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.66.171:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: received a newer join from {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.handleJoinRequest(JoinHelper.java:459) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:533) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:06:42,246][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] last failed join attempt was 748ms ago, failed to join {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, minimumTerm=1, optionalJoin=Optional[Join{term=2, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.66.171:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: received a newer join from {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.handleJoinRequest(JoinHelper.java:459) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:533) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) ~[elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:06:42,252][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered or elected yet, an election requires 2 nodes with ids [UC3-LrdpSSutVC5sXkOr5Q, ScUJPCxfTXGOqGEpsgqiPQ], have discovered [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, {onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}] which is a quorum; discovery will continue using [10.233.76.228:9300, 10.233.65.132:9300, 10.233.73.197:9300] from hosts providers and [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}] from last-known cluster state; node term 3, last-accepted version 0 in term 0
[2023-03-24T00:06:43,450][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, minimumTerm=2, optionalJoin=Optional[Join{term=3, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.66.171:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: received a newer join from {onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.handleJoinRequest(JoinHelper.java:459) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:533) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
[2023-03-24T00:06:43,935][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] elected-as-master ([2] nodes joined)[{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr} elect leader, {onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 4, version: 1, delta: master node changed {previous [], current [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]}, added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T00:06:45,109][INFO ][o.e.c.c.CoordinationState] [onap-sdnrdb-master-0] cluster UUID set to [N-1RybaMQKeVAs6bbg91_w]
[2023-03-24T00:06:45,770][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-0}{ScUJPCxfTXGOqGEpsgqiPQ}{b1prO5JRQkKpXH2q3EwT6A}{10.233.66.171}{10.233.66.171:9300}{dmr}]}, added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 1, reason: Publication{term=4, version=1}
[2023-03-24T00:06:46,016][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 2, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T00:06:46,218][INFO ][o.e.h.AbstractHttpServerTransport] [onap-sdnrdb-master-0] publish_address {10.233.66.171:9200}, bound_addresses {0.0.0.0:9200}
[2023-03-24T00:06:46,314][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] started
[2023-03-24T00:06:46,857][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 2, reason: Publication{term=4, version=2}
[2023-03-24T00:06:47,316][INFO ][o.e.g.GatewayService ] [onap-sdnrdb-master-0] recovered [0] indices into cluster_state
[2023-03-24T00:07:02,924][INFO ][o.e.c.s.ClusterSettings ] [onap-sdnrdb-master-0] updating [action.auto_create_index] from [true] to [false]
[2023-03-24T00:07:06,621][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [guicutthrough-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:07:14,020][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 6, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T00:07:16,432][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 6, reason: Publication{term=4, version=6}
[2023-03-24T00:07:27,620][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [eventlog-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:07:37,625][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [inventoryequipment-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:07:45,522][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [userdata-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:07:53,817][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [cmlog-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:00,115][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [faultlog-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:09,420][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [historicalperformance15min-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:16,622][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [maintenancemode-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:24,216][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [mediator-server-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:32,220][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [historicalperformance24h-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:37,123][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [networkelement-connection-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:44,316][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [connectionlog-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:52,425][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [faultcurrent-v7] creating index, cause [api], templates [], shards [5]/[1]
[2023-03-24T00:08:59,761][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultcurrent-v7][4], [faultcurrent-v7][2]]]).
[2023-03-24T01:06:38,322][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 90, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T01:06:42,529][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 90, reason: Publication{term=4, version=90}
[2023-03-24T01:06:43,020][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [mediator-server-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:06:43,323][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [mediator-server-v7][4] primary-replica resync completed with 0 operations
[2023-03-24T01:06:43,521][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [userdata-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:06:43,619][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [userdata-v7][4] primary-replica resync completed with 0 operations
[2023-03-24T01:06:43,833][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [inventoryequipment-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,114][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [networkelement-connection-v7][2] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,216][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultcurrent-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,237][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [maintenancemode-v7][2] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,432][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [cmlog-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,529][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,614][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultlog-v7][2] primary-replica resync completed with 0 operations
[2023-03-24T01:06:44,723][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [guicutthrough-v7][4] primary-replica resync completed with 0 operations
[2023-03-24T01:06:45,014][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [guicutthrough-v7][2] primary-replica resync completed with 0 operations
[2023-03-24T01:06:45,116][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:06:45,137][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [52.8s] (44 delayed shards)
[2023-03-24T01:06:45,218][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [connectionlog-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:06:45,514][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 91, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T01:06:45,616][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [connectionlog-v7][4] primary-replica resync completed with 0 operations
[2023-03-24T01:06:45,689][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 91, reason: Publication{term=4, version=91}
[2023-03-24T01:06:45,716][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 92, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T01:06:46,131][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 92, reason: Publication{term=4, version=92}
[2023-03-24T01:06:46,519][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 93, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T01:06:46,831][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 93, reason: Publication{term=4, version=93}
[2023-03-24T01:06:46,835][WARN ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] failed to validate incoming join request from node [{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}]
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.76.228:9300][internal:cluster/coordination/join/validate] disconnected
[2023-03-24T01:06:47,714][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 94, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T01:06:48,231][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 94, reason: Publication{term=4, version=94}
[2023-03-24T01:06:48,317][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 95, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T01:06:50,383][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 95, reason: Publication{term=4, version=95}
[2023-03-24T01:07:13,816][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 118, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:16,283][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 118, reason: Publication{term=4, version=118}
[2023-03-24T01:07:16,423][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [cmlog-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:07:16,621][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:07:16,817][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [eventlog-v7][3] primary-replica resync completed with 0 operations
[2023-03-24T01:07:16,913][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations
[2023-03-24T01:07:17,015][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [inventoryequipment-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:07:17,314][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:07:17,523][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][1] primary-replica resync completed with 0 operations
[2023-03-24T01:07:18,151][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 120, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:18,452][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 120, reason: Publication{term=4, version=120}
[2023-03-24T01:07:18,721][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 121, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:20,047][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 121, reason: Publication{term=4, version=121}
[2023-03-24T01:07:20,050][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 122, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:20,759][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 122, reason: Publication{term=4, version=122}
[2023-03-24T01:07:20,823][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 123, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:21,152][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 123, reason: Publication{term=4, version=123}
[2023-03-24T01:07:23,198][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 126, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T01:07:25,264][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 126, reason: Publication{term=4, version=126}
[2023-03-24T01:07:30,225][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[guicutthrough-v7][0], [inventoryequipment-v7][4], [userdata-v7][2], [inventoryequipment-v7][3], [eventlog-v7][4], [eventlog-v7][1], [cmlog-v7][3]]]).
[2023-03-24T01:07:43,364][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [29.9s] (22 delayed shards)
[2023-03-24T01:08:34,580][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]).
[2023-03-24T02:06:47,618][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 197, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:52,052][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 197, reason: Publication{term=4, version=197}
[2023-03-24T02:06:52,057][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.5s] (44 delayed shards)
[2023-03-24T02:06:52,059][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 198, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T02:06:52,731][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 198, reason: Publication{term=4, version=198}
[2023-03-24T02:06:52,735][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 199, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:53,429][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 199, reason: Publication{term=4, version=199}
[2023-03-24T02:06:53,626][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 200, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:54,128][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 200, reason: Publication{term=4, version=200}
[2023-03-24T02:06:54,133][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 201, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:54,614][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 201, reason: Publication{term=4, version=201}
[2023-03-24T02:06:54,722][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 202, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:54,859][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 202, reason: Publication{term=4, version=202}
[2023-03-24T02:06:54,914][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 203, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T02:06:55,335][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 203, reason: Publication{term=4, version=203}
[2023-03-24T02:06:55,340][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 204, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T02:06:59,054][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 204, reason: Publication{term=4, version=204}
[2023-03-24T02:07:21,614][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 233, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T02:07:24,446][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 233, reason: Publication{term=4, version=233}
[2023-03-24T02:07:25,818][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 236, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T02:07:27,538][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 236, reason: Publication{term=4, version=236}
[2023-03-24T02:07:36,738][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[faultcurrent-v7][1], [inventoryequipment-v7][3], [eventlog-v7][4], [eventlog-v7][1]]]).
[2023-03-24T02:07:48,473][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [33s] (19 delayed shards)
[2023-03-24T02:08:21,328][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]).
[2023-03-24T03:06:54,443][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 304, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T03:06:54,600][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 304, reason: Publication{term=4, version=304}
[2023-03-24T03:06:55,069][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 305, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T03:06:58,234][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 305, reason: Publication{term=4, version=305}
[2023-03-24T03:06:58,237][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [56.8s] (44 delayed shards)
[2023-03-24T03:06:58,240][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 306, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T03:06:58,627][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 306, reason: Publication{term=4, version=306}
[2023-03-24T03:06:58,631][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 307, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T03:07:00,535][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 307, reason: Publication{term=4, version=307}
[2023-03-24T03:07:24,926][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 339, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T03:07:27,458][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 339, reason: Publication{term=4, version=339}
[2023-03-24T03:07:27,462][WARN ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] failed to validate incoming join request from node [{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}]
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-2][10.233.73.197:9300][internal:cluster/coordination/join/validate] disconnected
[2023-03-24T03:07:29,761][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 343, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T03:07:32,397][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 343, reason: Publication{term=4, version=343}
[2023-03-24T03:07:38,132][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[eventlog-v7][4], [eventlog-v7][1], [guicutthrough-v7][0], [userdata-v7][2], [inventoryequipment-v7][4]]]).
[2023-03-24T03:07:57,615][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [27.2s] (15 delayed shards)
[2023-03-24T03:08:21,218][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]).
[2023-03-24T04:06:57,461][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 413, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T04:06:57,662][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 413, reason: Publication{term=4, version=413}
[2023-03-24T04:06:58,263][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 414, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T04:07:00,427][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 414, reason: Publication{term=4, version=414}
[2023-03-24T04:07:00,431][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [57.8s] (44 delayed shards)
[2023-03-24T04:07:00,632][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 415, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T04:07:01,027][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 415, reason: Publication{term=4, version=415}
[2023-03-24T04:07:01,415][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 416, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T04:07:03,440][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 416, reason: Publication{term=4, version=416}
[2023-03-24T04:07:28,718][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 460, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:34,201][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 460, reason: Publication{term=4, version=460}
[2023-03-24T04:07:34,206][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 461, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:34,484][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 461, reason: Publication{term=4, version=461}
[2023-03-24T04:07:34,527][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 462, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:35,749][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 462, reason: Publication{term=4, version=462}
[2023-03-24T04:07:36,421][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 464, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:36,581][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 464, reason: Publication{term=4, version=464}
[2023-03-24T04:07:36,610][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 465, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:36,746][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 465, reason: Publication{term=4, version=465}
[2023-03-24T04:07:37,992][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 468, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T04:07:39,600][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 468, reason: Publication{term=4, version=468}
[2023-03-24T04:07:39,604][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[guicutthrough-v7][0], [eventlog-v7][1]]]).
[2023-03-24T04:07:59,178][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [29.5s] (19 delayed shards)
[2023-03-24T04:08:23,878][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]).
[2023-03-24T05:07:00,601][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 539, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T05:07:00,768][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 539, reason: Publication{term=4, version=539}
[2023-03-24T05:07:01,337][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 540, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:04,431][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 540, reason: Publication{term=4, version=540}
[2023-03-24T05:07:04,436][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [56.8s] (44 delayed shards)
[2023-03-24T05:07:04,438][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader, {onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 541, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr},{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T05:07:04,769][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr},{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 541, reason: Publication{term=4, version=541}
[2023-03-24T05:07:05,016][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 542, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:05,325][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 542, reason: Publication{term=4, version=542}
[2023-03-24T05:07:05,422][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 543, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:05,933][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 543, reason: Publication{term=4, version=543}
[2023-03-24T05:07:06,024][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 544, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:06,337][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 544, reason: Publication{term=4, version=544}
[2023-03-24T05:07:06,416][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 545, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:06,833][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 545, reason: Publication{term=4, version=545}
[2023-03-24T05:07:07,021][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 546, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:07,327][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 546, reason: Publication{term=4, version=546}
[2023-03-24T05:07:07,416][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 547, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:07,836][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 547, reason: Publication{term=4, version=547}
[2023-03-24T05:07:07,882][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 548, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:08,236][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 548, reason: Publication{term=4, version=548}
[2023-03-24T05:07:08,241][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 549, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:08,632][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 549, reason: Publication{term=4, version=549}
[2023-03-24T05:07:08,672][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 550, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:08,943][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 550, reason: Publication{term=4, version=550}
[2023-03-24T05:07:09,155][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 551, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T05:07:11,533][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 551, reason: Publication{term=4, version=551}
[2023-03-24T05:07:37,132][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 588, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:39,851][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 588, reason: Publication{term=4, version=588}
[2023-03-24T05:07:39,856][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 589, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:40,650][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 589, reason: Publication{term=4, version=589}
[2023-03-24T05:07:40,688][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 590, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:41,846][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 590, reason: Publication{term=4, version=590}
[2023-03-24T05:07:41,850][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 591, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:42,251][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 591, reason: Publication{term=4, version=591}
[2023-03-24T05:07:42,414][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 592, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:42,522][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 592, reason: Publication{term=4, version=592}
[2023-03-24T05:07:43,107][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 594, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T05:07:44,751][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 594, reason: Publication{term=4, version=594}
[2023-03-24T05:07:51,351][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[userdata-v7][2], [inventoryequipment-v7][3], [inventoryequipment-v7][4], [eventlog-v7][1]]]).
[2023-03-24T05:08:02,007][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [35.1s] (24 delayed shards)
[2023-03-24T05:08:36,174][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]).
[2023-03-24T06:07:03,688][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 665, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T06:07:03,848][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 665, reason: Publication{term=4, version=665}
[2023-03-24T06:07:06,664][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 666, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T06:07:06,928][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 666, reason: Publication{term=4, version=666}
[2023-03-24T06:07:09,080][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 667, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T06:07:12,341][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 667, reason: Publication{term=4, version=667}
[2023-03-24T06:07:12,344][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [56.7s] (44 delayed shards)
[2023-03-24T06:07:12,346][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 668, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T06:07:12,508][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 668, reason: Publication{term=4, version=668}
[2023-03-24T06:07:12,536][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 669, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T06:07:12,734][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 669, reason: Publication{term=4, version=669}
[2023-03-24T06:07:13,488][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 670, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T06:07:15,398][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 670, reason: Publication{term=4, version=670}
[2023-03-24T06:07:42,916][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 713, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:45,955][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 713, reason: Publication{term=4, version=713}
[2023-03-24T06:07:46,872][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 715, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:47,236][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 715, reason: Publication{term=4, version=715}
[2023-03-24T06:07:47,422][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 716, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:48,746][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 716, reason: Publication{term=4, version=716}
[2023-03-24T06:07:49,800][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 718, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:49,988][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 718, reason: Publication{term=4, version=718}
[2023-03-24T06:07:50,123][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 719, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:50,338][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 719, reason: Publication{term=4, version=719}
[2023-03-24T06:07:50,748][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 721, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:50,949][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 721, reason: Publication{term=4, version=721}
[2023-03-24T06:07:50,988][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 722, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:53,052][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 722, reason: Publication{term=4, version=722}
[2023-03-24T06:07:53,057][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 723, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:53,551][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 723, reason: Publication{term=4, version=723}
[2023-03-24T06:07:53,615][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 724, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:53,971][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 724, reason: Publication{term=4, version=724}
[2023-03-24T06:07:53,976][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[inventoryequipment-v7][3], [guicutthrough-v7][0], [eventlog-v7][4], [eventlog-v7][1]]]).
[2023-03-24T06:07:54,372][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 726, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T06:07:56,341][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 726, reason: Publication{term=4, version=726}
[2023-03-24T06:08:10,355][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [32.5s] (20 delayed shards)
[2023-03-24T06:08:39,204][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]).
[2023-03-24T07:07:06,646][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 796, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T07:07:07,027][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 796, reason: Publication{term=4, version=796}
[2023-03-24T07:07:09,718][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 797, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T07:07:09,960][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 797, reason: Publication{term=4, version=797}
[2023-03-24T07:07:13,519][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 798, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T07:07:15,839][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 798, reason: Publication{term=4, version=798}
[2023-03-24T07:07:15,842][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [57.6s] (44 delayed shards)
[2023-03-24T07:07:16,257][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 799, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T07:07:17,866][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 799, reason: Publication{term=4, version=799}
[2023-03-24T07:07:54,081][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][2]]]).
[2023-03-24T07:07:54,531][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 868, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T07:07:57,086][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 868, reason: Publication{term=4, version=868}
[2023-03-24T07:07:57,090][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [57.4s] (43 delayed shards)
[2023-03-24T07:07:57,326][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 869, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T07:07:58,779][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 869, reason: Publication{term=4, version=869}
[2023-03-24T07:08:31,914][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]).
[2023-03-24T08:07:09,695][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} reason: disconnected], term: 4, version: 939, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T08:07:09,839][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 939, reason: Publication{term=4, version=939}
[2023-03-24T08:07:12,746][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r} join existing leader], term: 4, version: 940, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}
[2023-03-24T08:07:12,939][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-dssg5}{wXIVmhgSScuUUEF-1cmwCQ}{pfzZzIwpTHCAT6uZPmvRRg}{10.233.65.132}{10.233.65.132:9300}{r}}, term: 4, version: 940, reason: Publication{term=4, version=940}
[2023-03-24T08:07:16,222][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} reason: disconnected], term: 4, version: 941, delta: removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T08:07:18,782][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 941, reason: Publication{term=4, version=941}
[2023-03-24T08:07:18,786][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [57.4s] (44 delayed shards)
[2023-03-24T08:07:19,258][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr} join existing leader], term: 4, version: 942, delta: added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}
[2023-03-24T08:07:21,047][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{UC3-LrdpSSutVC5sXkOr5Q}{D6bvyRXyTbqkUrOzv_P7zw}{10.233.76.228}{10.233.76.228:9300}{dmr}}, term: 4, version: 942, reason: Publication{term=4, version=942}
[2023-03-24T08:07:57,815][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 995, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:00,785][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 995, reason: Publication{term=4, version=995}
[2023-03-24T08:08:01,462][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 997, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:01,847][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 997, reason: Publication{term=4, version=997}
[2023-03-24T08:08:02,022][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 998, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:03,664][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 998, reason: Publication{term=4, version=998}
[2023-03-24T08:08:04,437][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 1000, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:04,659][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 1000, reason: Publication{term=4, version=1000}
[2023-03-24T08:08:04,714][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} reason: disconnected], term: 4, version: 1001, delta: removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:05,002][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 1001, reason: Publication{term=4, version=1001}
[2023-03-24T08:08:05,004][WARN ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] failed to validate incoming join request from node [{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}]
org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-2][10.233.73.197:9300][internal:cluster/coordination/join/validate] disconnected
[2023-03-24T08:08:05,006][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[inventoryequipment-v7][4], [guicutthrough-v7][0], [eventlog-v7][1], [eventlog-v7][4]]]).
[2023-03-24T08:08:05,990][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr} join existing leader], term: 4, version: 1004, delta: added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}
[2023-03-24T08:08:07,698][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{IPl93Pm3TFetr2CUjxMngA}{8JfDAOXKRGi8S0H51uMXQQ}{10.233.73.197}{10.233.73.197:9300}{dmr}}, term: 4, version: 1004, reason: Publication{term=4, version=1004}
[2023-03-24T08:08:18,138][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [39.6s] (27 delayed shards)
[2023-03-24T08:08:52,368][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]).