By type
[38;5;6m [38;5;5m00:02:32.74 [0m [38;5;6m [38;5;5m00:02:32.75 [0m[1mWelcome to the Bitnami elasticsearch container[0m [38;5;6m [38;5;5m00:02:32.76 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch[0m [38;5;6m [38;5;5m00:02:32.84 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch/issues[0m [38;5;6m [38;5;5m00:02:32.85 [0m [38;5;6m [38;5;5m00:02:32.86 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch setup ** [38;5;6m [38;5;5m00:02:33.27 [0m[38;5;2mINFO [0m ==> Configuring/Initializing Elasticsearch... [38;5;6m [38;5;5m00:02:33.77 [0m[38;5;2mINFO [0m ==> Setting default configuration [38;5;6m [38;5;5m00:02:33.95 [0m[38;5;2mINFO [0m ==> Configuring Elasticsearch cluster settings... OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. [38;5;6m [38;5;5m00:03:02.54 [0m[38;5;2mINFO [0m ==> ** Elasticsearch setup finished! ** [38;5;6m [38;5;5m00:03:02.65 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch ** OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. [2023-03-05T00:03:53,938][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/5.4.0-96-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS] [2023-03-05T00:03:53,943][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] JVM home [/opt/bitnami/java] [2023-03-05T00:03:53,944][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-2808006915926717683, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true] [2023-03-05T00:04:18,642][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [aggs-matrix-stats] [2023-03-05T00:04:18,644][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [analysis-common] [2023-03-05T00:04:18,644][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [geo] [2023-03-05T00:04:18,645][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-common] [2023-03-05T00:04:18,645][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-geoip] [2023-03-05T00:04:18,646][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [ingest-user-agent] [2023-03-05T00:04:18,739][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [kibana] [2023-03-05T00:04:18,740][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-expression] [2023-03-05T00:04:18,740][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-mustache] [2023-03-05T00:04:18,741][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [lang-painless] [2023-03-05T00:04:18,742][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [mapper-extras] [2023-03-05T00:04:18,742][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [parent-join] [2023-03-05T00:04:18,743][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [percolator] [2023-03-05T00:04:18,744][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [rank-eval] [2023-03-05T00:04:18,744][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [reindex] [2023-03-05T00:04:18,745][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [repository-url] [2023-03-05T00:04:18,745][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [tasks] [2023-03-05T00:04:18,746][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded module [transport-netty4] [2023-03-05T00:04:18,747][INFO ][o.e.p.PluginsService ] [onap-sdnrdb-master-0] loaded plugin [repository-s3] [2023-03-05T00:04:19,851][INFO ][o.e.e.NodeEnvironment ] [onap-sdnrdb-master-0] using [1] data paths, mounts [[/bitnami/elasticsearch/data (192.168.13.90:/dockerdata-nfs/onap/elastic-master-1)]], net usable_space [95.7gb], net total_space [99.9gb], types [nfs4] [2023-03-05T00:04:19,940][INFO ][o.e.e.NodeEnvironment ] [onap-sdnrdb-master-0] heap size [123.7mb], compressed ordinary object pointers [true] [2023-03-05T00:04:20,945][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] node name [onap-sdnrdb-master-0], node ID [wbKfu8TrTwWlVr90bhJALQ], cluster name [sdnrdb-cluster] [2023-03-05T00:05:30,339][INFO ][o.e.t.NettyAllocator ] [onap-sdnrdb-master-0] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}] [2023-03-05T00:05:31,639][INFO ][o.e.d.DiscoveryModule ] [onap-sdnrdb-master-0] using discovery type [zen] and seed hosts providers [settings] [2023-03-05T00:05:36,946][WARN ][o.e.g.DanglingIndicesState] [onap-sdnrdb-master-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually [2023-03-05T00:05:40,042][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] initialized [2023-03-05T00:05:40,043][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] starting ... [2023-03-05T00:05:42,842][INFO ][o.e.t.TransportService ] [onap-sdnrdb-master-0] publish_address {10.233.68.194:9300}, bound_addresses {0.0.0.0:9300} [2023-03-05T00:05:46,144][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.68.194:9300, remoteAddress=/127.0.0.6:46055}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:05:46,551][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.68.194:9300, remoteAddress=/127.0.0.6:47583}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:05:47,539][WARN ][o.e.t.TcpTransport ] [onap-sdnrdb-master-0] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.233.68.194:9300, remoteAddress=/127.0.0.6:57617}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.3.jar:7.9.3] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:05:48,143][INFO ][o.e.b.BootstrapChecks ] [onap-sdnrdb-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks [2023-03-05T00:05:51,645][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][11] overhead, spent [398ms] collecting in the last [1.3s] [2023-03-05T00:05:58,352][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:08,356][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:18,360][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:28,367][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:38,371][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:48,376][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:06:58,379][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:08,383][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:18,385][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:28,392][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:38,396][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:48,399][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.79:9300, 10.233.70.171:9300, 10.233.72.148:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:07:58,403][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:08:08,408][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:08:18,413][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:08:28,417][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:08:38,421][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [onap-sdnrdb-master-0, onap-sdnrdb-master-1, onap-sdnrdb-master-2] to bootstrap a cluster: have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0 [2023-03-05T00:08:43,542][INFO ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] setting initial configuration to VotingConfiguration{{bootstrap-placeholder}-onap-sdnrdb-master-1,mdKELhvYRwCh4n9elxK_gw,wbKfu8TrTwWlVr90bhJALQ} [2023-03-05T00:08:48,425][WARN ][o.e.c.c.ClusterFormationFailureHelper] [onap-sdnrdb-master-0] master not discovered or elected yet, an election requires 2 nodes with ids [mdKELhvYRwCh4n9elxK_gw, wbKfu8TrTwWlVr90bhJALQ], have discovered [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, {onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}] which is a quorum; discovery will continue using [10.233.70.171:9300, 10.233.72.148:9300, 10.233.70.79:9300] from hosts providers and [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}] from last-known cluster state; node term 2, last-accepted version 0 in term 0 [2023-03-05T00:08:48,879][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, minimumTerm=0, optionalJoin=Optional[Join{term=1, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}}]} org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.68.194:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: received a newer join from {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.handleJoinRequest(JoinHelper.java:459) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:533) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:375) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:800) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:08:50,139][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] elected-as-master ([2] nodes joined)[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} elect leader, {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 3, version: 1, delta: master node changed {previous [], current [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]}, added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T00:08:51,046][WARN ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] failing [elected-as-master ([2] nodes joined)[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} elect leader, {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_]]: failed to commit cluster state version [1] org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: publication failed at org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication$4.onFailure(Coordinator.java:1467) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.action.ActionRunnable.onFailure(ActionRunnable.java:88) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:68) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication.onCompletion(Coordinator.java:1390) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.onPossibleCompletion(Publication.java:125) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.onPossibleCommitFailure(Publication.java:173) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.access$500(Publication.java:42) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication$PublicationTarget$PublishResponseHandler.onFailure(Publication.java:368) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator$5.onFailure(Coordinator.java:1151) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext.lambda$sendClusterState$2(PublicationTransportHandler.java:412) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext$3.handleException(PublicationTransportHandler.java:430) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: non-failed nodes do not form a quorum at org.elasticsearch.cluster.coordination.Publication.onPossibleCommitFailure(Publication.java:171) ~[elasticsearch-7.9.3.jar:7.9.3] ... 11 more [2023-03-05T00:08:51,049][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, minimumTerm=1, optionalJoin=Optional[Join{term=2, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, targetNode={onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}]} org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-2][10.233.70.79:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: node is no longer master for term 3 while handling publication at org.elasticsearch.cluster.coordination.Coordinator.publish(Coordinator.java:1083) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.MasterService.publish(MasterService.java:268) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:250) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:08:51,140][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, minimumTerm=2, optionalJoin=Optional[Join{term=3, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, targetNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}}]} org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-0][10.233.68.194:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: publication failed at org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication$4.onFailure(Coordinator.java:1467) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.action.ActionRunnable.onFailure(ActionRunnable.java:88) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:68) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication.onCompletion(Coordinator.java:1390) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.onPossibleCompletion(Publication.java:125) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.onPossibleCommitFailure(Publication.java:173) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.access$500(Publication.java:42) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication$PublicationTarget$PublishResponseHandler.onFailure(Publication.java:368) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator$5.onFailure(Coordinator.java:1151) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext.lambda$sendClusterState$2(PublicationTransportHandler.java:412) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext$3.handleException(PublicationTransportHandler.java:430) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: non-failed nodes do not form a quorum at org.elasticsearch.cluster.coordination.Publication.onPossibleCommitFailure(Publication.java:171) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication.access$500(Publication.java:42) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Publication$PublicationTarget$PublishResponseHandler.onFailure(Publication.java:368) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator$5.onFailure(Coordinator.java:1151) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext.lambda$sendClusterState$2(PublicationTransportHandler.java:412) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.PublicationTransportHandler$PublicationContext$3.handleException(PublicationTransportHandler.java:430) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:235) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) ~[?:?] [2023-03-05T00:08:51,642][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] elected-as-master ([3] nodes joined)[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} elect leader, {onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} elect leader, {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 5, version: 2, delta: master node changed {previous [], current [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]}, added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T00:08:51,742][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, minimumTerm=3, optionalJoin=Optional.empty} org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-2][10.233.70.79:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: became follower at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.lambda$close$3(JoinHelper.java:484) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.HashMap$Values.forEach(HashMap.java:976) ~[?:?] at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.close(JoinHelper.java:484) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.becomeFollower(Coordinator.java:621) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.onFollowerCheckRequest(Coordinator.java:257) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.FollowersChecker$2.doRun(FollowersChecker.java:198) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T00:08:52,533][INFO ][o.e.c.c.CoordinationState] [onap-sdnrdb-master-0] cluster UUID set to [CGEGSk9hRbapJhVSaVGnKQ] [2023-03-05T00:08:53,644][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]}, added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r},{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 2, reason: Publication{term=5, version=2} [2023-03-05T00:08:54,345][INFO ][o.e.h.AbstractHttpServerTransport] [onap-sdnrdb-master-0] publish_address {10.233.68.194:9200}, bound_addresses {0.0.0.0:9200} [2023-03-05T00:08:54,441][INFO ][o.e.n.Node ] [onap-sdnrdb-master-0] started [2023-03-05T00:08:54,745][INFO ][o.e.g.GatewayService ] [onap-sdnrdb-master-0] recovered [0] indices into cluster_state [2023-03-05T00:09:22,145][INFO ][o.e.c.s.ClusterSettings ] [onap-sdnrdb-master-0] updating [action.auto_create_index] from [true] to [false] [2023-03-05T00:09:25,039][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [guicutthrough-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:09:42,043][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [eventlog-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:09:53,741][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [inventoryequipment-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:10:01,743][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [userdata-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:10:12,553][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [cmlog-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:10:20,445][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 34, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T00:10:24,401][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 34, reason: Publication{term=5, version=34} [2023-03-05T00:10:24,449][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [faultlog-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:10:34,645][INFO ][o.e.c.c.C.CoordinatorPublication] [onap-sdnrdb-master-0] after [10s] publication of cluster state version [35] is still waiting for {onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} [SENT_APPLY_COMMIT] [2023-03-05T00:11:09,550][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [historicalperformance15min-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:11:28,738][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [mediator-server-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:11:43,946][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [networkelement-connection-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:12:00,040][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [connectionlog-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:12:13,441][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [faultcurrent-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:12:26,239][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [maintenancemode-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:12:38,020][INFO ][o.e.c.m.MetadataCreateIndexService] [onap-sdnrdb-master-0] [historicalperformance24h-v7] creating index, cause [api], templates [], shards [5]/[1] [2023-03-05T00:13:03,190][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[mediator-server-v7][4]]]). [2023-03-05T01:08:41,650][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 129, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T01:08:48,096][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 129, reason: Publication{term=5, version=129} [2023-03-05T01:08:48,639][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [guicutthrough-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:08:48,739][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][4] primary-replica resync completed with 0 operations [2023-03-05T01:08:48,741][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [maintenancemode-v7][2] primary-replica resync completed with 0 operations [2023-03-05T01:08:48,744][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance24h-v7][2] primary-replica resync completed with 0 operations [2023-03-05T01:08:48,950][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [cmlog-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,342][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [userdata-v7][2] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,344][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [eventlog-v7][0] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,542][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [userdata-v7][0] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,558][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [networkelement-connection-v7][4] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,650][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [connectionlog-v7][3] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,744][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [51.2s] (43 delayed shards) [2023-03-05T01:08:49,747][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 130, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T01:08:49,844][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [inventoryequipment-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:08:49,855][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [mediator-server-v7][3] primary-replica resync completed with 0 operations [2023-03-05T01:08:50,590][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 130, reason: Publication{term=5, version=130} [2023-03-05T01:08:50,943][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 131, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T01:08:51,528][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 131, reason: Publication{term=5, version=131} [2023-03-05T01:08:51,540][WARN ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] failed to validate incoming join request from node [{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}] org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-2][10.233.70.79:9300][internal:cluster/coordination/join/validate] disconnected [2023-03-05T01:08:51,642][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 132, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T01:08:51,996][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 132, reason: Publication{term=5, version=132} [2023-03-05T01:08:52,710][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 133, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T01:08:57,017][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 133, reason: Publication{term=5, version=133} [2023-03-05T01:08:57,023][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 134, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T01:08:57,589][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 134, reason: Publication{term=5, version=134} [2023-03-05T01:10:17,744][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 195, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:26,917][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 195, reason: Publication{term=5, version=195} [2023-03-05T01:10:27,039][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [mediator-server-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,146][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][2] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,243][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [historicalperformance15min-v7][0] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,447][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [networkelement-connection-v7][3] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,639][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [networkelement-connection-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,750][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [connectionlog-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,849][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultcurrent-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:10:27,956][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [49.4s] (44 delayed shards) [2023-03-05T01:10:28,042][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultcurrent-v7][4] primary-replica resync completed with 0 operations [2023-03-05T01:10:28,043][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 196, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:28,240][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultlog-v7][3] primary-replica resync completed with 0 operations [2023-03-05T01:10:28,246][INFO ][o.e.i.s.IndexShard ] [onap-sdnrdb-master-0] [faultlog-v7][1] primary-replica resync completed with 0 operations [2023-03-05T01:10:28,721][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 196, reason: Publication{term=5, version=196} [2023-03-05T01:10:28,843][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 197, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:32,118][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 197, reason: Publication{term=5, version=197} [2023-03-05T01:10:33,977][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 199, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:34,617][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 199, reason: Publication{term=5, version=199} [2023-03-05T01:10:34,792][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 200, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:37,115][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 200, reason: Publication{term=5, version=200} [2023-03-05T01:10:37,121][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T01:10:38,230][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 202, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:38,748][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 202, reason: Publication{term=5, version=202} [2023-03-05T01:10:38,944][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 203, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:39,344][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 203, reason: Publication{term=5, version=203} [2023-03-05T01:10:39,350][WARN ][o.e.c.c.Coordinator ] [onap-sdnrdb-master-0] failed to validate incoming join request from node [{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}] org.elasticsearch.transport.NodeDisconnectedException: [onap-sdnrdb-master-1][10.233.72.148:9300][internal:cluster/coordination/join/validate] disconnected [2023-03-05T01:10:40,444][WARN ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] [guicutthrough-v7][3] marking unavailable shards as stale: [F3OL3eHaQ7yviTDE4duVug] [2023-03-05T01:10:41,063][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 207, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T01:10:49,752][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 207, reason: Publication{term=5, version=207} [2023-03-05T01:12:20,501][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][4]]]). [2023-03-05T02:08:52,445][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 280, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:08:58,681][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 280, reason: Publication{term=5, version=280} [2023-03-05T02:08:58,692][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.7s] (43 delayed shards) [2023-03-05T02:08:58,743][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 281, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T02:08:59,380][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 281, reason: Publication{term=5, version=281} [2023-03-05T02:08:59,384][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 282, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:00,273][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 282, reason: Publication{term=5, version=282} [2023-03-05T02:09:00,398][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 283, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:00,881][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 283, reason: Publication{term=5, version=283} [2023-03-05T02:09:00,888][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 284, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:01,572][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 284, reason: Publication{term=5, version=284} [2023-03-05T02:09:01,638][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 285, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:02,086][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 285, reason: Publication{term=5, version=285} [2023-03-05T02:09:02,091][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader, {onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 286, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r},{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:02,786][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r},{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 286, reason: Publication{term=5, version=286} [2023-03-05T02:09:02,868][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 287, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:03,376][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 287, reason: Publication{term=5, version=287} [2023-03-05T02:09:04,186][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 288, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T02:09:07,817][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 288, reason: Publication{term=5, version=288} [2023-03-05T02:10:27,197][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T02:10:40,541][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 355, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T02:10:45,740][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 355, reason: Publication{term=5, version=355} [2023-03-05T02:10:45,749][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.7s] (44 delayed shards) [2023-03-05T02:10:45,751][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 356, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T02:10:46,238][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 356, reason: Publication{term=5, version=356} [2023-03-05T02:10:46,539][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 357, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T02:10:46,699][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 357, reason: Publication{term=5, version=357} [2023-03-05T02:10:47,493][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 358, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T02:10:51,408][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 358, reason: Publication{term=5, version=358} [2023-03-05T02:11:59,318][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][4]]]). [2023-03-05T03:09:01,874][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 428, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T03:09:02,273][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 428, reason: Publication{term=5, version=428} [2023-03-05T03:09:03,810][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 429, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:09,192][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 429, reason: Publication{term=5, version=429} [2023-03-05T03:09:09,204][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.5s] (43 delayed shards) [2023-03-05T03:09:09,207][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader, {onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 430, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r},{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:10,067][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r},{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 430, reason: Publication{term=5, version=430} [2023-03-05T03:09:10,143][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 431, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:10,482][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 431, reason: Publication{term=5, version=431} [2023-03-05T03:09:10,539][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 432, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:10,930][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 432, reason: Publication{term=5, version=432} [2023-03-05T03:09:11,018][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 433, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:11,080][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 433, reason: Publication{term=5, version=433} [2023-03-05T03:09:11,485][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 434, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T03:09:14,768][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 434, reason: Publication{term=5, version=434} [2023-03-05T03:09:48,044][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][11038] overhead, spent [614ms] collecting in the last [1.2s] [2023-03-05T03:10:17,815][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]). [2023-03-05T03:10:47,428][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 503, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T03:10:53,081][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 503, reason: Publication{term=5, version=503} [2023-03-05T03:10:53,089][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.3s] (44 delayed shards) [2023-03-05T03:10:53,139][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 504, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T03:10:54,515][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 504, reason: Publication{term=5, version=504} [2023-03-05T03:10:54,608][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 505, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T03:10:54,912][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 505, reason: Publication{term=5, version=505} [2023-03-05T03:10:55,276][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 506, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T03:10:58,466][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 506, reason: Publication{term=5, version=506} [2023-03-05T03:12:06,800][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][2]]]). [2023-03-05T04:09:04,771][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 576, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T04:09:04,878][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 576, reason: Publication{term=5, version=576} [2023-03-05T04:09:07,803][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 577, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T04:09:08,648][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 577, reason: Publication{term=5, version=577} [2023-03-05T04:09:11,441][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 578, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T04:09:16,456][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 578, reason: Publication{term=5, version=578} [2023-03-05T04:09:16,462][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.9s] (43 delayed shards) [2023-03-05T04:09:16,465][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 579, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T04:09:16,938][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 579, reason: Publication{term=5, version=579} [2023-03-05T04:09:17,050][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 580, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T04:09:17,238][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 580, reason: Publication{term=5, version=580} [2023-03-05T04:09:17,561][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 581, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T04:09:21,084][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 581, reason: Publication{term=5, version=581} [2023-03-05T04:10:23,516][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]). [2023-03-05T04:10:55,260][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 649, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:00,455][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 649, reason: Publication{term=5, version=649} [2023-03-05T04:11:00,460][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.7s] (44 delayed shards) [2023-03-05T04:11:00,462][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 650, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:01,255][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 650, reason: Publication{term=5, version=650} [2023-03-05T04:11:01,335][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 651, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:01,787][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 651, reason: Publication{term=5, version=651} [2023-03-05T04:11:01,792][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 652, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:02,143][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 652, reason: Publication{term=5, version=652} [2023-03-05T04:11:02,256][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 653, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:02,653][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 653, reason: Publication{term=5, version=653} [2023-03-05T04:11:02,864][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 654, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T04:11:05,310][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 654, reason: Publication{term=5, version=654} [2023-03-05T04:12:18,634][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][4]]]). [2023-03-05T05:09:07,743][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 724, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T05:09:07,849][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 724, reason: Publication{term=5, version=724} [2023-03-05T05:09:10,003][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 725, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T05:09:10,365][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 725, reason: Publication{term=5, version=725} [2023-03-05T05:09:17,440][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 726, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T05:09:22,839][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 726, reason: Publication{term=5, version=726} [2023-03-05T05:09:22,845][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.5s] (43 delayed shards) [2023-03-05T05:09:22,848][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 727, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T05:09:23,239][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 727, reason: Publication{term=5, version=727} [2023-03-05T05:09:23,441][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 728, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T05:09:23,582][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 728, reason: Publication{term=5, version=728} [2023-03-05T05:09:23,961][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 729, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T05:09:27,625][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 729, reason: Publication{term=5, version=729} [2023-03-05T05:10:30,011][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T05:11:02,804][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 798, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T05:11:08,825][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 798, reason: Publication{term=5, version=798} [2023-03-05T05:11:08,829][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.9s] (44 delayed shards) [2023-03-05T05:11:08,832][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 799, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T05:11:09,438][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 799, reason: Publication{term=5, version=799} [2023-03-05T05:11:09,743][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 800, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T05:11:09,892][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 800, reason: Publication{term=5, version=800} [2023-03-05T05:11:10,439][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 801, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T05:11:13,577][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 801, reason: Publication{term=5, version=801} [2023-03-05T05:12:27,044][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T06:09:09,910][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 872, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T06:09:10,070][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 872, reason: Publication{term=5, version=872} [2023-03-05T06:09:12,233][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 873, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T06:09:12,769][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 873, reason: Publication{term=5, version=873} [2023-03-05T06:09:23,897][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 874, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T06:09:27,833][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 874, reason: Publication{term=5, version=874} [2023-03-05T06:09:27,838][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [56s] (43 delayed shards) [2023-03-05T06:09:27,840][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 875, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T06:09:28,827][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 875, reason: Publication{term=5, version=875} [2023-03-05T06:09:29,023][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 876, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T06:09:29,204][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 876, reason: Publication{term=5, version=876} [2023-03-05T06:09:29,609][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 877, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T06:09:33,105][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 877, reason: Publication{term=5, version=877} [2023-03-05T06:10:51,917][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]). [2023-03-05T06:11:10,443][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 945, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T06:11:16,545][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 945, reason: Publication{term=5, version=945} [2023-03-05T06:11:16,551][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.7s] (44 delayed shards) [2023-03-05T06:11:16,553][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 946, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T06:11:16,987][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 946, reason: Publication{term=5, version=946} [2023-03-05T06:11:17,083][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 947, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T06:11:17,259][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 947, reason: Publication{term=5, version=947} [2023-03-05T06:11:17,453][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 948, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T06:11:21,073][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 948, reason: Publication{term=5, version=948} [2023-03-05T06:12:27,522][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][2]]]). [2023-03-05T07:09:12,149][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1019, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T07:09:12,441][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1019, reason: Publication{term=5, version=1019} [2023-03-05T07:09:14,658][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1020, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T07:09:15,093][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1020, reason: Publication{term=5, version=1020} [2023-03-05T07:09:29,379][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1021, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T07:09:35,527][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1021, reason: Publication{term=5, version=1021} [2023-03-05T07:09:35,532][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.8s] (43 delayed shards) [2023-03-05T07:09:35,536][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1022, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T07:09:36,001][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1022, reason: Publication{term=5, version=1022} [2023-03-05T07:09:36,145][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1023, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T07:09:36,372][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1023, reason: Publication{term=5, version=1023} [2023-03-05T07:09:37,007][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1024, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T07:09:39,109][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1024, reason: Publication{term=5, version=1024} [2023-03-05T07:10:45,958][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]). [2023-03-05T07:11:17,446][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1092, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T07:11:22,515][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1092, reason: Publication{term=5, version=1092} [2023-03-05T07:11:22,523][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.8s] (44 delayed shards) [2023-03-05T07:11:22,528][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1093, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T07:11:23,322][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1093, reason: Publication{term=5, version=1093} [2023-03-05T07:11:23,375][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1094, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T07:11:23,707][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1094, reason: Publication{term=5, version=1094} [2023-03-05T07:11:24,234][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1095, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T07:11:27,145][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1095, reason: Publication{term=5, version=1095} [2023-03-05T07:12:36,917][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][4]]]). [2023-03-05T08:09:14,590][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1166, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T08:09:14,872][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1166, reason: Publication{term=5, version=1166} [2023-03-05T08:09:17,541][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1167, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T08:09:18,012][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1167, reason: Publication{term=5, version=1167} [2023-03-05T08:09:36,935][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1168, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T08:09:42,811][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1168, reason: Publication{term=5, version=1168} [2023-03-05T08:09:42,817][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.1s] (43 delayed shards) [2023-03-05T08:09:42,818][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1169, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T08:09:43,372][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1169, reason: Publication{term=5, version=1169} [2023-03-05T08:09:43,466][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1170, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T08:09:43,570][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1170, reason: Publication{term=5, version=1170} [2023-03-05T08:09:44,576][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1171, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T08:09:47,358][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1171, reason: Publication{term=5, version=1171} [2023-03-05T08:10:40,789][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]). [2023-03-05T08:11:24,114][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1239, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T08:11:28,472][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1239, reason: Publication{term=5, version=1239} [2023-03-05T08:11:28,475][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.6s] (44 delayed shards) [2023-03-05T08:11:28,477][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1240, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T08:11:28,966][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1240, reason: Publication{term=5, version=1240} [2023-03-05T08:11:29,063][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1241, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T08:11:29,301][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1241, reason: Publication{term=5, version=1241} [2023-03-05T08:11:29,813][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1242, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T08:11:32,204][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1242, reason: Publication{term=5, version=1242} [2023-03-05T08:12:35,650][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T09:09:17,483][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1312, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T09:09:17,791][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1312, reason: Publication{term=5, version=1312} [2023-03-05T09:09:20,614][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1313, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T09:09:20,843][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1313, reason: Publication{term=5, version=1313} [2023-03-05T09:09:44,362][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1314, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T09:09:49,101][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1314, reason: Publication{term=5, version=1314} [2023-03-05T09:09:49,106][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.2s] (43 delayed shards) [2023-03-05T09:09:49,108][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1315, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T09:09:49,839][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1315, reason: Publication{term=5, version=1315} [2023-03-05T09:09:50,044][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1316, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T09:09:50,270][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1316, reason: Publication{term=5, version=1316} [2023-03-05T09:09:50,513][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1317, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T09:09:54,699][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1317, reason: Publication{term=5, version=1317} [2023-03-05T09:10:58,211][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]). [2023-03-05T09:11:29,803][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1384, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T09:11:33,849][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1384, reason: Publication{term=5, version=1384} [2023-03-05T09:11:33,855][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.9s] (44 delayed shards) [2023-03-05T09:11:33,860][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1385, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T09:11:34,371][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1385, reason: Publication{term=5, version=1385} [2023-03-05T09:11:34,442][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1386, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T09:11:34,645][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1386, reason: Publication{term=5, version=1386} [2023-03-05T09:11:35,497][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1387, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T09:11:41,241][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1387, reason: Publication{term=5, version=1387} [2023-03-05T09:12:51,859][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T10:09:20,557][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1458, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T10:09:20,791][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1458, reason: Publication{term=5, version=1458} [2023-03-05T10:09:23,173][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1459, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T10:09:23,829][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1459, reason: Publication{term=5, version=1459} [2023-03-05T10:09:50,480][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1460, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T10:09:57,181][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1460, reason: Publication{term=5, version=1460} [2023-03-05T10:09:57,184][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.2s] (43 delayed shards) [2023-03-05T10:09:57,186][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1461, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T10:09:57,677][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1461, reason: Publication{term=5, version=1461} [2023-03-05T10:09:57,763][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1462, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T10:09:57,929][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1462, reason: Publication{term=5, version=1462} [2023-03-05T10:09:58,330][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1463, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T10:10:00,835][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1463, reason: Publication{term=5, version=1463} [2023-03-05T10:11:00,953][INFO ][o.e.m.j.JvmGcMonitorService] [onap-sdnrdb-master-0] [gc][36294] overhead, spent [675ms] collecting in the last [1.5s] [2023-03-05T10:11:15,786][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]). [2023-03-05T10:11:35,448][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1531, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T10:11:41,042][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1531, reason: Publication{term=5, version=1531} [2023-03-05T10:11:41,050][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.3s] (44 delayed shards) [2023-03-05T10:11:41,054][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1532, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T10:11:41,438][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1532, reason: Publication{term=5, version=1532} [2023-03-05T10:11:41,520][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1533, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T10:11:41,632][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1533, reason: Publication{term=5, version=1533} [2023-03-05T10:11:42,435][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1534, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T10:11:44,918][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1534, reason: Publication{term=5, version=1534} [2023-03-05T10:12:42,853][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T11:09:23,100][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1605, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T11:09:23,334][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1605, reason: Publication{term=5, version=1605} [2023-03-05T11:09:26,372][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1606, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T11:09:27,174][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1606, reason: Publication{term=5, version=1606} [2023-03-05T11:09:58,344][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1607, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T11:10:03,681][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1607, reason: Publication{term=5, version=1607} [2023-03-05T11:10:03,686][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [54.5s] (43 delayed shards) [2023-03-05T11:10:03,688][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1608, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T11:10:04,266][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1608, reason: Publication{term=5, version=1608} [2023-03-05T11:10:04,608][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1609, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T11:10:04,761][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1609, reason: Publication{term=5, version=1609} [2023-03-05T11:10:05,697][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1610, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T11:10:09,428][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1610, reason: Publication{term=5, version=1610} [2023-03-05T11:11:09,495][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][1]]]). [2023-03-05T11:11:42,381][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1678, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:47,200][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1678, reason: Publication{term=5, version=1678} [2023-03-05T11:11:47,205][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.1s] (44 delayed shards) [2023-03-05T11:11:47,207][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1679, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:47,601][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1679, reason: Publication{term=5, version=1679} [2023-03-05T11:11:47,649][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1680, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:47,838][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1680, reason: Publication{term=5, version=1680} [2023-03-05T11:11:47,844][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1681, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:48,228][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1681, reason: Publication{term=5, version=1681} [2023-03-05T11:11:48,323][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1682, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:48,419][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1682, reason: Publication{term=5, version=1682} [2023-03-05T11:11:48,897][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1683, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T11:11:51,479][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1683, reason: Publication{term=5, version=1683} [2023-03-05T11:12:56,396][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T12:09:26,232][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1755, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T12:09:26,556][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1755, reason: Publication{term=5, version=1755} [2023-03-05T12:09:28,699][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1756, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T12:09:29,200][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1756, reason: Publication{term=5, version=1756} [2023-03-05T12:10:05,563][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1757, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T12:10:10,158][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1757, reason: Publication{term=5, version=1757} [2023-03-05T12:10:10,164][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.3s] (43 delayed shards) [2023-03-05T12:10:10,166][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1758, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T12:10:10,616][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1758, reason: Publication{term=5, version=1758} [2023-03-05T12:10:10,724][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1759, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T12:10:10,858][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1759, reason: Publication{term=5, version=1759} [2023-03-05T12:10:11,363][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} join existing leader], term: 5, version: 1760, delta: added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T12:10:17,370][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1760, reason: Publication{term=5, version=1760} [2023-03-05T12:11:27,980][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]). [2023-03-05T12:11:48,867][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1827, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T12:11:52,884][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1827, reason: Publication{term=5, version=1827} [2023-03-05T12:11:52,887][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.9s] (44 delayed shards) [2023-03-05T12:11:52,889][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1828, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T12:11:53,343][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1828, reason: Publication{term=5, version=1828} [2023-03-05T12:11:53,432][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 5, version: 1829, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T12:11:53,656][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1829, reason: Publication{term=5, version=1829} [2023-03-05T12:11:53,998][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 5, version: 1830, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T12:11:57,089][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 5, version: 1830, reason: Publication{term=5, version=1830} [2023-03-05T12:13:08,586][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][2]]]). [2023-03-05T13:09:28,586][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} reason: disconnected], term: 5, version: 1901, delta: removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T13:09:28,836][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1901, reason: Publication{term=5, version=1901} [2023-03-05T13:09:31,273][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r} join existing leader], term: 5, version: 1902, delta: added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}} [2023-03-05T13:09:31,510][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-coordinating-only-54c85fc557-tzv5n}{f-O_jYjsTLK_36x0HkIPKw}{2tFqMISDSAWaLdWpn9bWlw}{10.233.70.171}{10.233.70.171:9300}{r}}, term: 5, version: 1902, reason: Publication{term=5, version=1902} [2023-03-05T13:10:11,277][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} reason: disconnected], term: 5, version: 1903, delta: removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T13:10:17,291][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 5, version: 1903, reason: Publication{term=5, version=1903} [2023-03-05T13:10:17,343][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [53.9s] (43 delayed shards) [2023-03-05T13:10:17,344][WARN ][o.e.c.c.C.CoordinatorPublication] [onap-sdnrdb-master-0] after [6.2s] publication of cluster state version [1903] is still waiting for {onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} [SENT_APPLY_COMMIT] [2023-03-05T13:10:17,345][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}], current []}, term: 5, version: 1903, reason: becoming candidate: joinLeaderInTerm [2023-03-05T13:10:17,918][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] elected-as-master ([2] nodes joined)[{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr} elect leader, {onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 7, version: 1904, delta: master node changed {previous [], current [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]}, added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}} [2023-03-05T13:10:18,538][INFO ][o.e.c.c.JoinHelper ] [onap-sdnrdb-master-0] failed to join {onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} with JoinRequest{sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, minimumTerm=5, optionalJoin=Optional[Join{term=6, lastAcceptedTerm=5, lastAcceptedVersion=1903, sourceNode={onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}, targetNode={onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}]} org.elasticsearch.transport.RemoteTransportException: [onap-sdnrdb-master-1][10.233.72.148:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 6 does not match current term 7 at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:225) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:1013) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.Optional.ifPresent(Optional.java:183) ~[?:?] at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:532) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:496) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.ClusterConnectionManager.connectToNode(ClusterConnectionManager.java:120) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:378) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:362) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:483) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:136) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] [2023-03-05T13:10:21,209][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] master node changed {previous [], current [{onap-sdnrdb-master-0}{wbKfu8TrTwWlVr90bhJALQ}{SkesZ9qlQe6BmvT7u_tURQ}{10.233.68.194}{10.233.68.194:9300}{dmr}]}, added {{onap-sdnrdb-master-2}{mdKELhvYRwCh4n9elxK_gw}{37ZwFbQSTJCdSygLCKtbpQ}{10.233.70.79}{10.233.70.79:9300}{dmr}}, term: 7, version: 1904, reason: Publication{term=7, version=1904} [2023-03-05T13:11:32,564][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][0]]]). [2023-03-05T13:11:54,042][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 7, version: 1973, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T13:11:58,175][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 7, version: 1973, reason: Publication{term=7, version=1973} [2023-03-05T13:11:58,179][INFO ][o.e.c.r.DelayedAllocationService] [onap-sdnrdb-master-0] scheduling reroute for delayed shards in [55.7s] (44 delayed shards) [2023-03-05T13:11:58,180][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 7, version: 1974, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T13:11:58,552][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 7, version: 1974, reason: Publication{term=7, version=1974} [2023-03-05T13:11:58,642][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-left[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} reason: disconnected], term: 7, version: 1975, delta: removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T13:11:58,863][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] removed {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 7, version: 1975, reason: Publication{term=7, version=1975} [2023-03-05T13:11:59,261][INFO ][o.e.c.s.MasterService ] [onap-sdnrdb-master-0] node-join[{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr} join existing leader], term: 7, version: 1976, delta: added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}} [2023-03-05T13:12:01,766][INFO ][o.e.c.s.ClusterApplierService] [onap-sdnrdb-master-0] added {{onap-sdnrdb-master-1}{_-FghAlqSeq4AjsN_5Vy3Q}{vAViE3gCQ-eig6LAEK92ew}{10.233.72.148}{10.233.72.148:9300}{dmr}}, term: 7, version: 1976, reason: Publication{term=7, version=1976} [2023-03-05T13:13:12,390][INFO ][o.e.c.r.a.AllocationService] [onap-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[guicutthrough-v7][3]]]).