[38;5;6m [38;5;5m10:51:53.05 [0m
[38;5;6m [38;5;5m10:51:53.06 [0m[1mWelcome to the Bitnami elasticsearch container[0m
[38;5;6m [38;5;5m10:51:53.14 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch[0m
[38;5;6m [38;5;5m10:51:53.16 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/bitnami-docker-elasticsearch/issues[0m
[38;5;6m [38;5;5m10:51:53.16 [0m
[38;5;6m [38;5;5m10:51:53.17 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch setup **
[38;5;6m [38;5;5m10:51:53.55 [0m[38;5;2mINFO [0m ==> Configuring/Initializing Elasticsearch...
[38;5;6m [38;5;5m10:51:53.87 [0m[38;5;2mINFO [0m ==> Setting default configuration
[38;5;6m [38;5;5m10:51:53.96 [0m[38;5;2mINFO [0m ==> Configuring Elasticsearch cluster settings...
[38;5;6m [38;5;5m10:51:54.16 [0m[38;5;3mWARN [0m ==> Found more than one IP address associated to hostname dev-sdnrdb-master-0: fd00:100::e55 10.242.14.85, will use fd00:100::e55
[38;5;6m [38;5;5m10:51:54.35 [0m[38;5;3mWARN [0m ==> Found more than one IP address associated to hostname dev-sdnrdb-master-0: fd00:100::e55 10.242.14.85, will use fd00:100::e55
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

[38;5;6m [38;5;5m10:52:13.86 [0m[38;5;2mINFO [0m ==> ** Elasticsearch setup finished! **
[38;5;6m [38;5;5m10:52:14.06 [0m[38;5;2mINFO [0m ==> ** Starting Elasticsearch **
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2021-06-02T10:52:50,349][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/4.15.0-117-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS]
[2021-06-02T10:52:50,351][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] JVM home [/opt/bitnami/java]
[2021-06-02T10:52:50,351][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-12280390948884804340, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2021-06-02T10:53:04,347][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [aggs-matrix-stats]
[2021-06-02T10:53:04,348][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [analysis-common]
[2021-06-02T10:53:04,348][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [geo]
[2021-06-02T10:53:04,349][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [ingest-common]
[2021-06-02T10:53:04,349][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [ingest-geoip]
[2021-06-02T10:53:04,350][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [ingest-user-agent]
[2021-06-02T10:53:04,350][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [kibana]
[2021-06-02T10:53:04,351][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [lang-expression]
[2021-06-02T10:53:04,351][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [lang-mustache]
[2021-06-02T10:53:04,352][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [lang-painless]
[2021-06-02T10:53:04,352][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [mapper-extras]
[2021-06-02T10:53:04,353][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [parent-join]
[2021-06-02T10:53:04,354][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [percolator]
[2021-06-02T10:53:04,354][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [rank-eval]
[2021-06-02T10:53:04,355][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [reindex]
[2021-06-02T10:53:04,355][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [repository-url]
[2021-06-02T10:53:04,356][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [tasks]
[2021-06-02T10:53:04,356][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded module [transport-netty4]
[2021-06-02T10:53:04,445][INFO ][o.e.p.PluginsService     ] [dev-sdnrdb-master-0] loaded plugin [repository-s3]
[2021-06-02T10:53:05,047][INFO ][o.e.e.NodeEnvironment    ] [dev-sdnrdb-master-0] using [1] data paths, mounts [[/bitnami/elasticsearch/data (172.16.10.41:/dockerdata-nfs/dev/elastic-master-2)]], net usable_space [179.8gb], net total_space [195.8gb], types [nfs4]
[2021-06-02T10:53:05,047][INFO ][o.e.e.NodeEnvironment    ] [dev-sdnrdb-master-0] heap size [123.7mb], compressed ordinary object pointers [true]
[2021-06-02T10:53:05,450][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] node name [dev-sdnrdb-master-0], node ID [r6QlQ5LKTi6O14WJkzRg6w], cluster name [sdnrdb-cluster]
[2021-06-02T10:53:46,456][INFO ][o.e.t.NettyAllocator     ] [dev-sdnrdb-master-0] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}]
[2021-06-02T10:53:47,350][INFO ][o.e.d.DiscoveryModule    ] [dev-sdnrdb-master-0] using discovery type [zen] and seed hosts providers [settings]
[2021-06-02T10:53:50,648][WARN ][o.e.g.DanglingIndicesState] [dev-sdnrdb-master-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2021-06-02T10:53:52,647][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] initialized
[2021-06-02T10:53:52,648][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] starting ...
[2021-06-02T10:53:54,351][INFO ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] publish_address {[fd00:100::e55]:9300}, bound_addresses {[::]:9300}
[2021-06-02T10:53:57,546][INFO ][o.e.b.BootstrapChecks    ] [dev-sdnrdb-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2021-06-02T10:54:07,753][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:54:17,757][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:54:27,760][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:54:37,764][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:54:47,768][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:54:57,770][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:07,773][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:17,775][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:27,777][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:37,781][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:47,784][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:55:57,786][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:07,789][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:17,790][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:27,793][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:37,796][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:47,800][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:56:57,804][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:07,807][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:17,809][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:27,846][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:37,848][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:47,850][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:57:57,853][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.35:9300, 10.242.0.37:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:58:07,857][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:58:17,871][WARN ][o.e.c.c.ClusterFormationFailureHelper] [dev-sdnrdb-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [dev-sdnrdb-master-0, dev-sdnrdb-master-1, dev-sdnrdb-master-2] to bootstrap a cluster: have discovered [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]; discovery will continue using [10.242.14.85:9300, 10.242.128.204:9300, 10.242.0.37:9300, 10.242.0.35:9300] from hosts providers and [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-06-02T10:58:24,752][INFO ][o.e.c.c.Coordinator      ] [dev-sdnrdb-master-0] setting initial configuration to VotingConfiguration{tkbY1npwSkibDdI75JBqoQ,r6QlQ5LKTi6O14WJkzRg6w,{bootstrap-placeholder}-dev-sdnrdb-master-2}
[2021-06-02T10:58:27,761][INFO ][o.e.c.c.JoinHelper       ] [dev-sdnrdb-master-0] failed to join {dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr} with JoinRequest{sourceNode={dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}, minimumTerm=0, optionalJoin=Optional[Join{term=1, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}, targetNode={dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}}]}
org.elasticsearch.transport.RemoteTransportException: [dev-sdnrdb-master-0][[fd00:100::e55]:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: became follower
	at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.lambda$close$3(JoinHelper.java:484) [elasticsearch-7.9.3.jar:7.9.3]
	at java.util.HashMap$Values.forEach(HashMap.java:976) [?:?]
	at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.close(JoinHelper.java:484) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.becomeFollower(Coordinator.java:621) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.Coordinator.onFollowerCheckRequest(Coordinator.java:257) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.cluster.coordination.FollowersChecker$2.doRun(FollowersChecker.java:198) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.3.jar:7.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
[2021-06-02T10:58:28,659][INFO ][o.e.c.c.CoordinationState] [dev-sdnrdb-master-0] cluster UUID set to [WyP306zTTleE7qPjj5ZMvA]
[2021-06-02T10:58:28,875][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] master node changed {previous [], current [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}]}, added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 2, version: 1, reason: ApplyCommitRequest{term=2, version=1, sourceNode={dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T10:58:28,962][INFO ][o.e.h.AbstractHttpServerTransport] [dev-sdnrdb-master-0] publish_address {[fd00:100::e55]:9200}, bound_addresses {[::]:9200}
[2021-06-02T10:58:28,962][INFO ][o.e.n.Node               ] [dev-sdnrdb-master-0] started
[2021-06-02T10:58:30,846][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 2, version: 2, reason: ApplyCommitRequest{term=2, version=2, sourceNode={dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T10:58:58,447][INFO ][o.e.c.s.ClusterSettings  ] [dev-sdnrdb-master-0] updating [action.auto_create_index] from [true] to [false]
[2021-06-02T10:59:15,367][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-2}{V-4sjr6NTay3ZTl53xmLgQ}{pc_2mQnrTK66yMFu0ANe1g}{fd00:100:0:0:0:0:0:80cc}{[fd00:100::80cc]:9300}{dmr}}, term: 2, version: 12, reason: ApplyCommitRequest{term=2, version=12, sourceNode={dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:29:45,056][INFO ][o.e.c.c.Coordinator      ] [dev-sdnrdb-master-0] master node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}] failed, restarting discovery
org.elasticsearch.ElasticsearchException: node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}] failed [3] consecutive checks
	at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:293) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1172) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:1073) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) [elasticsearch-7.9.3.jar:7.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.transport.ReceiveTimeoutTransportException: [dev-sdnrdb-master-1][[fd00:100::25]:9300][internal:coordination/fault_detection/leader_check] request_id [5462] timed out after [10008ms]
	at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:1074) ~[elasticsearch-7.9.3.jar:7.9.3]
	... 4 more
[2021-06-02T11:29:45,072][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] master node changed {previous [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], current []}, term: 2, version: 81, reason: becoming candidate: onLeaderFailure
[2021-06-02T11:29:45,850][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] elected-as-master ([2] nodes joined)[{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr} elect leader, {dev-sdnrdb-master-2}{V-4sjr6NTay3ZTl53xmLgQ}{pc_2mQnrTK66yMFu0ANe1g}{fd00:100:0:0:0:0:0:80cc}{[fd00:100::80cc]:9300}{dmr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 3, version: 82, delta: master node changed {previous [], current [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]}
[2021-06-02T11:29:55,870][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [82] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST], {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:30:15,880][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] master node changed {previous [], current [{dev-sdnrdb-master-0}{r6QlQ5LKTi6O14WJkzRg6w}{kWiC9H0BRp2YIAN6anJGJQ}{fd00:100:0:0:0:0:0:e55}{[fd00:100::e55]:9300}{dmr}]}, term: 3, version: 82, reason: Publication{term=3, version=82}
[2021-06-02T11:30:16,049][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [82] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST], {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:30:17,875][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 83, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:30:18,068][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 83, reason: Publication{term=3, version=83}
[2021-06-02T11:30:18,451][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 84, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:30:20,571][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 84, reason: Publication{term=3, version=84}
[2021-06-02T11:30:20,753][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:30:20,849][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:30:20,852][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T11:30:20,953][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][0] primary-replica resync completed with 0 operations
[2021-06-02T11:30:21,046][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:30:21,056][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:30:21,148][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:30:21,152][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [56.9s] (37 delayed shards)
[2021-06-02T11:30:21,350][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T11:30:21,375][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:31:19,368][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][1] marking unavailable shards as stale: [nOFFHqPuSzi1BUwp3WyPwQ]
[2021-06-02T11:31:20,390][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [m5BrlpV5ThSJ0xFqnwh3jg]
[2021-06-02T11:31:20,391][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [BxREcepHSB6dmDVKevWUxw]
[2021-06-02T11:31:20,391][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][0] marking unavailable shards as stale: [vGt_YURsT22Z31qsi-tHHA]
[2021-06-02T11:31:24,075][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][0] marking unavailable shards as stale: [JmFczVUKROycHpFPSHpoOg]
[2021-06-02T11:31:24,505][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][1] marking unavailable shards as stale: [QpzowI0cR_-ZofprfRa-og]
[2021-06-02T11:31:24,506][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [38UiKpk8QcCTkXkWWZ9f8g]
[2021-06-02T11:31:24,506][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [Sor3HBb3ToC1La7bu4pplA]
[2021-06-02T11:31:27,246][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [Se7c_n6KRm2FNTz4ZdIYkA]
[2021-06-02T11:31:27,846][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [qnWyaeKdRoOM6vj8EuTQ6Q]
[2021-06-02T11:31:27,847][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [fUE1Y2E8TiKuSnXjz6YnAg]
[2021-06-02T11:31:27,847][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][1] marking unavailable shards as stale: [29Hul1ClS-uylBOITCYuXg]
[2021-06-02T11:31:30,367][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [gqnx40lLSbGAFeJZsYkhSA]
[2021-06-02T11:31:30,619][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][0] marking unavailable shards as stale: [kfSl81ZfTMminWSPIrSnPg]
[2021-06-02T11:31:31,061][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][1] marking unavailable shards as stale: [XBxTvgbORLC9kHCepZNSlw]
[2021-06-02T11:31:31,062][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [dzP0ahFXR8GpwRb4lzZ2lA]
[2021-06-02T11:31:32,748][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [mlNZtqCFQgmuNzXMzR7dBA]
[2021-06-02T11:31:33,070][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][0] marking unavailable shards as stale: [OgOF1ZbyQDSI17JX9qcdUQ]
[2021-06-02T11:31:33,070][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [0QOzjLNuTA-lDSqSzggMRA]
[2021-06-02T11:31:34,749][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [C3VTYVfZQWumGzW4XsHagw]
[2021-06-02T11:31:35,988][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [IUiLJTobQuKwiSMIUSz31A]
[2021-06-02T11:31:36,462][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [JiICQrDcRFW1LDwghZrQSQ]
[2021-06-02T11:31:37,072][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [tqDYPKIRSOaQNZfmIiLuLQ]
[2021-06-02T11:31:38,095][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][0] marking unavailable shards as stale: [OojX9GjNQsSe4OaJwJhXLQ]
[2021-06-02T11:31:38,523][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][0] marking unavailable shards as stale: [e3gaOHwQRymjj9ynbaFkjQ]
[2021-06-02T11:31:38,948][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [4O0MWQKzTxG1xi8AgZDRfQ]
[2021-06-02T11:31:40,447][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [4olNt1CqSYuj1wRPzdTZ-g]
[2021-06-02T11:31:41,579][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [7vfnt_SuTq-29QaLPsvKag]
[2021-06-02T11:31:42,395][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][1] marking unavailable shards as stale: [suGcZCPBSuuHJHj_9kGEAQ]
[2021-06-02T11:31:42,395][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [b9qGMJHrQ0WR3xjmgQgBlA]
[2021-06-02T11:31:44,146][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [gagCDfsXQB2CyzXtU8pDAQ]
[2021-06-02T11:31:45,072][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][0] marking unavailable shards as stale: [ZCF73j2eRzuv0fwxCQ8clQ]
[2021-06-02T11:31:46,714][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [ug58bKYAT1eKrBqKuYg5tw]
[2021-06-02T11:31:46,715][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][1] marking unavailable shards as stale: [qpa9-UbrQoSh2dP54UmA5w]
[2021-06-02T11:31:49,270][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][1] marking unavailable shards as stale: [b3I09QVdSyeRkxgPRrpZYQ]
[2021-06-02T11:31:49,581][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [r7ngvETlRGGqymJX3iwnWg]
[2021-06-02T11:31:51,444][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [9nz7UGerTCS_3AvLOuMK2A]
[2021-06-02T11:31:52,055][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][2]]]).
[2021-06-02T11:33:30,377][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 145, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:33:30,605][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 145, reason: Publication{term=3, version=145}
[2021-06-02T11:33:33,673][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 146, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:33:37,350][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 146, reason: Publication{term=3, version=146}
[2021-06-02T11:38:03,698][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [16813ms] ago, timed out [1803ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [8740]
[2021-06-02T11:38:03,746][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [16813ms] ago, timed out [6806ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [8744]
[2021-06-02T11:38:44,345][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [29426ms] ago, timed out [19418ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [8858]
[2021-06-02T11:38:44,346][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [18417ms] ago, timed out [8406ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [8889]
[2021-06-02T11:42:16,350][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 211, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:42:21,764][WARN ][o.e.c.InternalClusterInfoService] [dev-sdnrdb-master-0] Failed to update shard information for ClusterInfoUpdateJob within 15s timeout
[2021-06-02T11:42:26,452][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [9.8s] publication of cluster state version [211] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:42:46,452][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 211, reason: Publication{term=3, version=211}
[2021-06-02T11:42:46,555][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:42:46,563][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:42:46,653][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [29.6s] (37 delayed shards)
[2021-06-02T11:42:46,654][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [211] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T11:42:46,752][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:42:46,957][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:43:19,691][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [16487ms] ago, timed out [6471ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [10159]
[2021-06-02T11:43:26,313][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [212] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:43:46,371][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [212] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T11:43:46,446][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [AY5KysaLRKmw6t-2g_dFxQ]
[2021-06-02T11:43:56,453][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [213] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T11:44:16,458][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [213] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T11:44:16,459][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [sm8-oXg6Rhav4J-yzTLyVg]
[2021-06-02T11:44:16,460][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [2gXEHTx-SUWut-m1jNaUsA]
[2021-06-02T11:44:16,460][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][1] marking unavailable shards as stale: [5m-i36huRKWxlTqZluRLTg]
[2021-06-02T11:44:16,655][WARN ][o.e.c.c.LagDetector      ] [dev-sdnrdb-master-0] node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}] is lagging at cluster state version [210], although publication of cluster state version [211] completed [1.5m] ago
[2021-06-02T11:44:26,546][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [214] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:44:46,548][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [214] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:44:46,554][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: lagging, {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 215, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:44:46,769][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 215, reason: Publication{term=3, version=215}
[2021-06-02T11:44:48,090][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [Fbmhdf7-SR2i2dhShHC3tw]
[2021-06-02T11:44:48,365][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [wdmIKchuT-uZWbVd0ZuCKg]
[2021-06-02T11:44:48,770][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][1] marking unavailable shards as stale: [8NxOo9Q9RZyj02VoRkeAEw]
[2021-06-02T11:44:48,770][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [BMEPv9QrSLubP99e2AYyOw]
[2021-06-02T11:44:50,446][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [zejqwxMzSUiLB5_r2AnZ7Q]
[2021-06-02T11:44:50,669][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][1] marking unavailable shards as stale: [wqz6BgJ7SMuApR2phhQWRg]
[2021-06-02T11:44:50,669][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [V5DVc82OQg-Il_aaoP0DWQ]
[2021-06-02T11:44:51,548][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [8axWEC5IQN2IvocA3-TYlg]
[2021-06-02T11:44:52,570][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [R7v6pGOiTyy8drAQizgKvQ]
[2021-06-02T11:44:54,078][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [7-zoWLfWS4ifPbMonjdFMw]
[2021-06-02T11:44:54,079][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [CQWA_H7sTR2GagSNlGxtqg]
[2021-06-02T11:44:54,321][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [6ILsNu-LSUOAWUCEa8O50g]
[2021-06-02T11:44:55,176][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][1] marking unavailable shards as stale: [aqVA6ougTlWW3h8QMDqV0w]
[2021-06-02T11:44:56,577][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [KRLeywmpSYue7hQd35jKhA]
[2021-06-02T11:44:56,577][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [FN4cfC3ZR8O2g3HnYAk4GA]
[2021-06-02T11:44:57,547][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [ofnppdJaRAmsCbQpfCgz8w]
[2021-06-02T11:44:58,772][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [eAHSsbDhTMKOCj5PRB4Z6w]
[2021-06-02T11:44:59,668][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [L98m38CdS-2Keth3uJuiyA]
[2021-06-02T11:44:59,668][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [xQArhXHYS2qNQRYhtKumvg]
[2021-06-02T11:45:01,185][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [FKFz5zu5TqeCdM2LSUtlJg]
[2021-06-02T11:45:01,593][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [pW7H2zWRScSm-WSVV5sU_Q]
[2021-06-02T11:45:02,549][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][1] marking unavailable shards as stale: [pyosVgLjTdmlrryCZuvp3Q]
[2021-06-02T11:45:02,550][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [L23x2iy5TBu0pIvZ1epqLw]
[2021-06-02T11:45:04,367][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [7r8IePgtSVSb8dLqspqoAA]
[2021-06-02T11:45:04,980][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [v8Z7feIuSJ6XttM3aBGfbg]
[2021-06-02T11:45:05,972][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [Vu4f2nHpQ8i_aMC6Zbl8FA]
[2021-06-02T11:45:05,972][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [W9RWyP05SZKgPwIVV1s3yA]
[2021-06-02T11:45:06,307][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [ALcbF7_bTe2c4qnbTO10-w]
[2021-06-02T11:45:07,048][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [dMb5BmjxR16E_QDUWSThIA]
[2021-06-02T11:45:08,148][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [wxTFL6c4SzeOCa1GPSkFtw]
[2021-06-02T11:45:09,370][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [Y0jK1MWjS2eGU7eJ87NU6w]
[2021-06-02T11:45:11,116][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [7X-Y_uJITQa1i7fFEXrD7w]
[2021-06-02T11:45:11,835][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [6ShPdX_kRhOm8-jLh4Bejg]
[2021-06-02T11:45:12,266][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][3]]]).
[2021-06-02T11:46:19,836][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 274, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:46:20,002][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 274, reason: Publication{term=3, version=274}
[2021-06-02T11:46:22,360][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 275, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:46:25,445][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 275, reason: Publication{term=3, version=275}
[2021-06-02T11:48:22,700][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [26823ms] ago, timed out [16812ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [12333]
[2021-06-02T11:48:22,702][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [15812ms] ago, timed out [5803ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [12373]
[2021-06-02T11:49:11,186][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 337, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:49:15,012][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [20015ms] ago, timed out [10007ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [12542]
[2021-06-02T11:49:19,013][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [23417ms] ago, timed out [8406ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [12544]
[2021-06-02T11:49:20,102][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [40829ms] ago, timed out [30821ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [12496]
[2021-06-02T11:49:20,705][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [30422ms] ago, timed out [20415ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [12528]
[2021-06-02T11:49:20,706][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [19414ms] ago, timed out [9407ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [12559]
[2021-06-02T11:49:21,191][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [337] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:49:41,191][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 337, reason: Publication{term=3, version=337}
[2021-06-02T11:49:41,197][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [337] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:49:55,402][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 338, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:49:56,676][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 338, reason: Publication{term=3, version=338}
[2021-06-02T11:49:56,781][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T11:49:56,854][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:49:56,859][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:49:56,955][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:49:57,054][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [eventlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:49:57,159][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [58.2s] (37 delayed shards)
[2021-06-02T11:49:57,164][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:49:57,260][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T11:49:57,263][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:50:55,749][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [ZJmg7HCmSAqgo7uKL9OfGw]
[2021-06-02T11:50:56,186][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [kHwVtUeKSAmQfiwfrgrx_A]
[2021-06-02T11:50:56,186][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [m-RAW51-Qm-oRdYGkLXa-w]
[2021-06-02T11:50:56,187][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [lSpl52diSEa-MLftjFhDhw]
[2021-06-02T11:50:58,405][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [xVyFHhWDSdK7NT4ZSFHVmg]
[2021-06-02T11:50:58,702][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [-JO_Zpz-QROJyxF1dtcDhQ]
[2021-06-02T11:50:59,187][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [6kKXJBGITliZV3KZmxMziA]
[2021-06-02T11:50:59,188][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [xkUGrZt3SliVarWaG9e_IA]
[2021-06-02T11:50:59,993][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [ErnSJ2GZTLu_K6MmUfdpbA]
[2021-06-02T11:51:01,047][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [TPhP9wC8QTu9fXQKdGrWtQ]
[2021-06-02T11:51:01,417][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][1] marking unavailable shards as stale: [fH8bjdt1RHuKAubPVdfiQA]
[2021-06-02T11:51:01,671][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [NGjJ03epQ-2T9NGhMe8nWQ]
[2021-06-02T11:51:02,254][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [Rbcyrnc4TEKVvRRqmBAsJQ]
[2021-06-02T11:51:02,998][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [TQ4kB8ssS6mKzS4e0o2ncA]
[2021-06-02T11:51:03,881][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][1] marking unavailable shards as stale: [Ha0xMv59TkO-AYwj8XX4NA]
[2021-06-02T11:51:03,881][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [cU4LnydLSfW8A9K7d7u8qQ]
[2021-06-02T11:51:04,192][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [7ybqMxb_QwuNGoM4vufUiA]
[2021-06-02T11:51:05,732][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [wBAZ-5dZTqqlj2Gjhc5YfQ]
[2021-06-02T11:51:06,149][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [TvIUUEr-QNy6sNwQFYytvw]
[2021-06-02T11:51:07,174][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [W9jpRRGXSkaeeOADuF36ZQ]
[2021-06-02T11:51:07,175][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [6wY0jqvfTiGEYEtLDad8EQ]
[2021-06-02T11:51:08,346][WARN ][o.e.m.j.JvmGcMonitorService] [dev-sdnrdb-master-0] [gc][3432] overhead, spent [616ms] collecting in the last [1.1s]
[2021-06-02T11:51:09,299][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [LgboV6F1RaSG4ndwC33cdw]
[2021-06-02T11:51:09,597][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [eRGIEdznToSq5Lm0HQ28hg]
[2021-06-02T11:51:10,591][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [pJeYeczzSGiDhzztRlyUYg]
[2021-06-02T11:51:10,591][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [1drquXN1TFiYBa0E9c79EA]
[2021-06-02T11:51:10,592][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][1] marking unavailable shards as stale: [GZtdQO-mRr6j0pDr2ZtWbA]
[2021-06-02T11:51:11,196][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [m2cgI2AiRaeARqziKdQ7tg]
[2021-06-02T11:51:12,705][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [zsHPeKo9Qb-GSMrTdWcWng]
[2021-06-02T11:51:12,977][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [JZjAwg-RTpC2_Lcm0XnLug]
[2021-06-02T11:51:13,747][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [kJuxsMDkTgiilWOUdv75xw]
[2021-06-02T11:51:13,747][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [29cMrxguRiy8w4T8gLtiGw]
[2021-06-02T11:51:14,469][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [ZYDxwz8aSxeS3PVF8Ljq9w]
[2021-06-02T11:51:15,157][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [A3q_bc4ESlSA7gGYLG1F2w]
[2021-06-02T11:51:16,264][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][1] marking unavailable shards as stale: [rS0j_RFdSISYwf2eyPpNrw]
[2021-06-02T11:51:16,265][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [ubjnH3o9S1CiWXusV605XQ]
[2021-06-02T11:51:18,127][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [dRpwiZEVR9-kDeR8jaICyA]
[2021-06-02T11:51:18,612][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [s9bIrSHlRyq5RWOjopEnvQ]
[2021-06-02T11:51:19,577][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][3]]]).
[2021-06-02T11:51:57,775][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 401, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:51:57,871][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 401, reason: Publication{term=3, version=401}
[2021-06-02T11:51:59,463][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 402, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:52:01,146][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 402, reason: Publication{term=3, version=402}
[2021-06-02T11:54:04,674][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [22240ms] ago, timed out [12234ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14761]
[2021-06-02T11:54:04,677][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [11233ms] ago, timed out [1223ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14794]
[2021-06-02T11:54:32,820][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [21423ms] ago, timed out [11408ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14861]
[2021-06-02T11:54:32,821][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [10408ms] ago, timed out [400ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14894]
[2021-06-02T11:54:32,878][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [34864ms] ago, timed out [19822ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14808]
[2021-06-02T11:54:43,358][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 466, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:54:49,446][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [16016ms] ago, timed out [6009ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [14921]
[2021-06-02T11:54:50,786][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 466, reason: Publication{term=3, version=466}
[2021-06-02T11:54:50,968][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:54:50,974][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,061][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,073][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,163][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,250][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,264][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,264][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,361][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,363][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [51.9s] (36 delayed shards)
[2021-06-02T11:54:51,449][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T11:54:51,554][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T11:55:45,277][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [b31TfevjQiO9Ut55dcDiOA]
[2021-06-02T11:55:51,417][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [4uP2V1LHRlyHoYZZF4ecqg]
[2021-06-02T11:55:51,417][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [AWPTvfwLQva_szR_G-CzOw]
[2021-06-02T11:55:51,418][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [VHlVaXOjSYOWvf6jEiypvg]
[2021-06-02T11:55:53,846][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [lhAThTGFR5iAci54ydNx8Q]
[2021-06-02T11:55:55,641][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [2ptQH4okS3aBeK0BZlWo1A]
[2021-06-02T11:55:55,642][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [1nvvfCH-QKOC78GOz4qVVA]
[2021-06-02T11:55:55,642][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [Ey9fH2UrS2GVxQSk8ckYXA]
[2021-06-02T11:55:57,146][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [7QwPf6eDRHW3w2G6Yak8jw]
[2021-06-02T11:55:57,442][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [y2hHSv2wRraoOY27Cbxu_A]
[2021-06-02T11:55:57,442][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [xvFmc3c_RiC05x9k-BXRJg]
[2021-06-02T11:56:04,620][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [v6sEmg7lQkmFAK-11yivnA]
[2021-06-02T11:56:14,625][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [482] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:56:34,676][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [482] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T11:56:34,747][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [w81-ACFfQLWntvLiHn6eAA]
[2021-06-02T11:56:43,984][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [37293ms] ago, timed out [27283ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [15522]
[2021-06-02T11:56:43,988][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [26282ms] ago, timed out [16275ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [15551]
[2021-06-02T11:56:43,989][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [15269ms] ago, timed out [5204ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [15585]
[2021-06-02T11:56:44,158][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 484, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:56:44,360][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 484, reason: Publication{term=3, version=484}
[2021-06-02T11:56:44,956][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [7vrNNkdpQcmX2mESAU0KJg]
[2021-06-02T11:56:44,957][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [XjlrHKj8RZ2KvxyLNvQk-w]
[2021-06-02T11:56:45,446][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [7qy-erH8QyaE245gNl8gDw]
[2021-06-02T11:56:46,343][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [RnnWoIq3RTqBG-EFKU7NsA]
[2021-06-02T11:56:46,645][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [XaIr2zhcTr2HvQ4ZjzSMxQ]
[2021-06-02T11:56:47,453][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [glfH9sVgShCgY0f24hQe-w]
[2021-06-02T11:56:47,453][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [obIEhnU3SRG7I-7wXtpP8Q]
[2021-06-02T11:56:47,780][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [FDXszYjBRZimQrI85SQH-w]
[2021-06-02T11:56:49,598][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [W2OaaUFTRaySLXehZW4B8g]
[2021-06-02T11:56:50,198][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [Hs6Kz_vhTWKq1M2PUyum8A]
[2021-06-02T11:56:52,799][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][1] marking unavailable shards as stale: [rNgJtBKWT2q-F_z5J0pN8Q]
[2021-06-02T11:56:52,800][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [XTVp4QsRRumGnsCD9I8GHg]
[2021-06-02T11:56:54,077][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [2ts7haFMRfWYNHq7M3m_2w]
[2021-06-02T11:56:55,769][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [rkKjo-pfSaKAplwpLTRqBA]
[2021-06-02T11:56:56,791][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [9qH-b4W_SjGAJxrusOzG-A]
[2021-06-02T11:56:57,719][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [e9IvrcDoT4Cpz0TBbCeI6Q]
[2021-06-02T11:56:57,720][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [NmzS432dQA6q8EVPKeg8EA]
[2021-06-02T11:56:58,353][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][1] marking unavailable shards as stale: [e-35qeX9Rlac9YdiBZba9w]
[2021-06-02T11:56:59,160][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][1] marking unavailable shards as stale: [vnMWwukJQQ2hAbe4GS1Rsg]
[2021-06-02T11:57:00,112][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [5A2ygmz8Q1icskBzKpLG-A]
[2021-06-02T11:57:00,113][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [W9PlNkWcSqqiFxTgX-N-OQ]
[2021-06-02T11:57:00,395][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [LUe7zeYcSNWeDLQfquwhmw]
[2021-06-02T11:57:01,626][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [GuaoOSKjSv6kFIOIR2B8uQ]
[2021-06-02T11:57:02,186][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][4]]]).
[2021-06-02T11:58:42,362][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 524, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T11:58:42,698][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 524, reason: Publication{term=3, version=524}
[2021-06-02T11:58:43,048][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 525, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T11:58:46,554][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 525, reason: Publication{term=3, version=525}
[2021-06-02T12:00:45,803][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [23624ms] ago, timed out [13618ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17475]
[2021-06-02T12:00:45,839][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [12617ms] ago, timed out [2605ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17507]
[2021-06-02T12:00:59,462][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [37836ms] ago, timed out [22825ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17468]
[2021-06-02T12:01:40,075][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 591, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:01:41,662][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [33634ms] ago, timed out [23626ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [17624]
[2021-06-02T12:01:41,663][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [22626ms] ago, timed out [12616ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [17662]
[2021-06-02T12:01:41,664][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [11616ms] ago, timed out [1601ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [17681]
[2021-06-02T12:01:41,679][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [27829ms] ago, timed out [17822ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17646]
[2021-06-02T12:01:41,893][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [17023ms] ago, timed out [7005ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17673]
[2021-06-02T12:01:42,077][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [26429ms] ago, timed out [11416ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [17652]
[2021-06-02T12:01:50,088][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [591] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:02:10,083][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 591, reason: Publication{term=3, version=591}
[2021-06-02T12:02:10,086][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [591] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:02:42,079][WARN ][o.e.c.InternalClusterInfoService] [dev-sdnrdb-master-0] Failed to update shard information for ClusterInfoUpdateJob within 15s timeout
[2021-06-02T12:02:46,049][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 592, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:02:47,488][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 592, reason: Publication{term=3, version=592}
[2021-06-02T12:02:47,557][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,577][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,649][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,746][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,758][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,861][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:02:47,954][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:02:48,146][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:02:48,156][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:02:48,173][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [57.7s] (37 delayed shards)
[2021-06-02T12:02:48,174][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:02:48,247][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:02:48,351][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:03:46,446][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [3uBIbzV0SHKy4hB61H_rnA]
[2021-06-02T12:03:46,447][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [n22K9UHQQa6YNEt-9fHJRQ]
[2021-06-02T12:03:46,779][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [DfGp_ffORcOd6kLdHJEQNA]
[2021-06-02T12:03:47,278][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [j_DmgaV4TjeFov9MALf8BQ]
[2021-06-02T12:03:48,578][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [f4fnZXWDRvym3v8ygexODQ]
[2021-06-02T12:03:48,803][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [rpqcIIrkQkK-TNGbbp6LuQ]
[2021-06-02T12:03:49,803][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [INsAUlx7TWiMGzBQMv9sjA]
[2021-06-02T12:03:49,803][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][1] marking unavailable shards as stale: [2e_vCDXYSdKaVvkZcBWcbQ]
[2021-06-02T12:03:50,280][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [Ixqc3jOoSCSPfYUuqlLXlg]
[2021-06-02T12:03:51,291][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [QGhITOtcQ7CckNqCDwTIFQ]
[2021-06-02T12:03:52,077][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [ffx-z2LGSHOMWYZgSNA1Pw]
[2021-06-02T12:03:52,646][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][1] marking unavailable shards as stale: [9FAHMpdxTRGoP7sXnnwugg]
[2021-06-02T12:03:52,646][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [H4Osds6KTWq5hyHACTpyqw]
[2021-06-02T12:03:54,007][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [W51uixLOSnehjpmH7BZLYA]
[2021-06-02T12:03:54,268][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [zWaScS7FQU2qRbLK_kYDoA]
[2021-06-02T12:03:54,969][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][1] marking unavailable shards as stale: [6-7-8IPSQFOli2t7h99o7A]
[2021-06-02T12:03:54,970][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [RxdCwRn5RwWATbcAdJLJFA]
[2021-06-02T12:03:55,269][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [cbtHkMDaQPa04vRfMEF1xw]
[2021-06-02T12:03:56,565][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [_icX7D2IREGwYKPvAYw-Lg]
[2021-06-02T12:03:57,402][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][1] marking unavailable shards as stale: [8Vp_h472T3GD5Z5WqhwNbA]
[2021-06-02T12:03:58,193][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [U8VoczbVS-msuB9B_1F4aw]
[2021-06-02T12:03:58,194][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [yssM8M_XQuqYqNlSaFBZdw]
[2021-06-02T12:03:59,128][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [O8J616TAQUOrAjmQpxG-Kg]
[2021-06-02T12:03:59,543][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [8aTnyymOSyq9j3GbMYsFxg]
[2021-06-02T12:04:00,077][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [1aBmg3j6QCO61U6xCnXRvg]
[2021-06-02T12:04:00,805][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [kuf-bOaASaunldoR-Gmz2A]
[2021-06-02T12:04:00,806][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][1] marking unavailable shards as stale: [g3PvWOC6QEuEpm0sv4NJJA]
[2021-06-02T12:04:01,037][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [XAS-HlF5RL2BTSG4ZIAnZQ]
[2021-06-02T12:04:01,346][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [FvSaHRNrQfi37D3PI5-_yQ]
[2021-06-02T12:04:02,673][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [i8XK8_ucT5ytWX6Xx4ACbw]
[2021-06-02T12:04:03,012][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [gv1qi_oDTMS41abExDQW3A]
[2021-06-02T12:04:03,012][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [5lqtBkdITJqnsuGBxkXnXw]
[2021-06-02T12:04:03,369][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [XtF-RQDSR3eHbvxhvqZbHQ]
[2021-06-02T12:04:04,915][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [qwFL8WJhTn67CzY7Kme2wQ]
[2021-06-02T12:04:05,474][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [YquB0JMzRo2Yrg2eUIdVhQ]
[2021-06-02T12:04:05,697][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [DyrAFbMKTIGQd3t73Vf0jA]
[2021-06-02T12:04:07,027][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [Xul14jVLReeoxGkOhKI5Cw]
[2021-06-02T12:04:07,343][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][3]]]).
[2021-06-02T12:04:36,359][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 656, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:04:36,776][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 656, reason: Publication{term=3, version=656}
[2021-06-02T12:04:39,503][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 657, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:04:41,686][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 657, reason: Publication{term=3, version=657}
[2021-06-02T12:06:51,048][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 721, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:07:01,247][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [9.8s] publication of cluster state version [721] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:07:04,866][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [52647ms] ago, timed out [37637ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [19903]
[2021-06-02T12:07:12,269][WARN ][o.e.c.InternalClusterInfoService] [dev-sdnrdb-master-0] Failed to update node information for ClusterInfoUpdateJob within 15s timeout
[2021-06-02T12:07:12,372][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [21225ms] ago, timed out [11208ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [20102]
[2021-06-02T12:07:12,372][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [10207ms] ago, timed out [201ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}], id [20133]
[2021-06-02T12:07:13,489][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [54648ms] ago, timed out [44641ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [19935]
[2021-06-02T12:07:13,489][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [43640ms] ago, timed out [33633ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [19963]
[2021-06-02T12:07:13,489][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [32632ms] ago, timed out [22625ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [20000]
[2021-06-02T12:07:21,248][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 721, reason: Publication{term=3, version=721}
[2021-06-02T12:07:21,548][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,548][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [eventlog-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,646][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [eventlog-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,647][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,653][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,683][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,755][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,758][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,761][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [29.2s] (37 delayed shards)
[2021-06-02T12:07:21,846][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][1] primary-replica resync completed with 0 operations
[2021-06-02T12:07:21,847][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30.5s] publication of cluster state version [721] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:07:21,870][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:07:47,411][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 722, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:07:47,463][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 722, reason: Publication{term=3, version=722}
[2021-06-02T12:07:51,347][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [Y6sRVQiDS-mqJyU0FWD5dA]
[2021-06-02T12:07:51,542][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [dIcOkn8NQQSbHT6Vp0QJLQ]
[2021-06-02T12:07:51,542][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [O5MFFVjpTrKhZUT3jow1NA]
[2021-06-02T12:07:51,883][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [-7JwAvMpSUmhUhTqcqWNXw]
[2021-06-02T12:07:53,464][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][1] marking unavailable shards as stale: [sm0oZO-tTGSjF0SjDjHgqQ]
[2021-06-02T12:07:53,622][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [shLZ13iSR76e7gVJWw3pDA]
[2021-06-02T12:07:54,078][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [E1qcm0q5SD-2HHatEMZSgA]
[2021-06-02T12:07:54,078][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][1] marking unavailable shards as stale: [gN7CpT9TTiKxN-jznpaRpg]
[2021-06-02T12:07:55,470][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [Aho0SNHyR6KrVn3QJKiOnQ]
[2021-06-02T12:07:55,627][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [Azr4v76lRter1ri2UWuzoQ]
[2021-06-02T12:07:56,465][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [t-sT0XbvR9u6w-_gmkqV0w]
[2021-06-02T12:07:56,465][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [oriIkiACRrKG3LbGsy42aA]
[2021-06-02T12:07:57,394][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [5QErjudQQPOJzq7cnTpc2Q]
[2021-06-02T12:07:57,646][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [U9z5btmwQya-4dElfx6hVw]
[2021-06-02T12:07:58,787][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [2UNqokXRSu6rK2R3wggutA]
[2021-06-02T12:07:58,788][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [scseq47ESjyCXGvSTkWaIw]
[2021-06-02T12:07:59,366][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [uIM0SzgJTr6eFN0k4aH2Yw]
[2021-06-02T12:08:00,564][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [5OxPP8BcT7CbHlXuEYVuQw]
[2021-06-02T12:08:01,192][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][1] marking unavailable shards as stale: [r4Nl8qP5R1CYNyff_bwzTg]
[2021-06-02T12:08:01,666][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [gCy7MADNSKSpWQsnmytRfQ]
[2021-06-02T12:08:02,367][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [7_Li284fRBqNI8zyPqEj1g]
[2021-06-02T12:08:02,368][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [s8pi16AlQh6V-IlYE4Z6yg]
[2021-06-02T12:08:03,664][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [BO3sON_VROyRCPWd7koiLg]
[2021-06-02T12:08:03,992][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [ikZo3l2lQb6cqYV6Z9YJ8w]
[2021-06-02T12:08:04,848][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][1] marking unavailable shards as stale: [bV58NolDRt23GD4KRcDz0Q]
[2021-06-02T12:08:04,848][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [JOHYGTjvSJOUXg1VE_TjCg]
[2021-06-02T12:08:05,082][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [gnmgkaLGS0qTrf_SSVvyjQ]
[2021-06-02T12:08:05,850][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [n8_qLntbRmSEnH0spZulQA]
[2021-06-02T12:08:07,064][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [tUXH1iDiS-KH52Gu6s-vEA]
[2021-06-02T12:08:07,064][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [enE1yaUiS2CND0azV8meTw]
[2021-06-02T12:08:07,445][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [sJC05GBORo26vLnz143sxw]
[2021-06-02T12:08:08,906][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [puus_jtaS9SU3-LDmVpENQ]
[2021-06-02T12:08:09,151][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [56bOokDkSUmMMnAtffjfUw]
[2021-06-02T12:08:09,799][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][1] marking unavailable shards as stale: [Eib_F6szSPORcmjvf7rHWQ]
[2021-06-02T12:08:09,800][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [P2F5CU4QRV6cqb5fD5-Eqg]
[2021-06-02T12:08:10,003][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [jeJZFm9TQ6WkR5BwsPIRtA]
[2021-06-02T12:08:10,997][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [InaJlVveQiSFI0LiqticEw]
[2021-06-02T12:08:11,398][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][4]]]).
[2021-06-02T12:09:01,247][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 783, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:09:01,369][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 783, reason: Publication{term=3, version=783}
[2021-06-02T12:09:03,647][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 784, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:09:04,946][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 784, reason: Publication{term=3, version=784}
[2021-06-02T12:15:49,314][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [14410ms] ago, timed out [4403ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23422]
[2021-06-02T12:16:07,409][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [17014ms] ago, timed out [7208ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23474]
[2021-06-02T12:16:12,512][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [28425ms] ago, timed out [13412ms] ago, action [cluster:monitor/nodes/stats[n]], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23453]
[2021-06-02T12:17:06,749][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [23021ms] ago, timed out [13010ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23674]
[2021-06-02T12:17:08,171][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [13411ms] ago, timed out [3402ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23696]
[2021-06-02T12:17:28,782][WARN ][o.e.t.TransportService   ] [dev-sdnrdb-master-0] Received response for a request that has timed out, sent [15212ms] ago, timed out [5204ms] ago, action [internal:coordination/fault_detection/follower_check], node [{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}], id [23769]
[2021-06-02T12:17:42,262][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 843, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:17:52,266][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [843] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:18:09,394][WARN ][o.e.c.InternalClusterInfoService] [dev-sdnrdb-master-0] Failed to update shard information for ClusterInfoUpdateJob within 15s timeout
[2021-06-02T12:18:12,266][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 843, reason: Publication{term=3, version=843}
[2021-06-02T12:18:12,272][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30s] publication of cluster state version [843] is still waiting for {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} [SENT_PUBLISH_REQUEST]
[2021-06-02T12:18:12,447][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 844, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:18:13,574][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 844, reason: Publication{term=3, version=844}
[2021-06-02T12:18:13,748][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,749][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,755][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [connectionlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,847][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,859][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [58.4s] (36 delayed shards)
[2021-06-02T12:18:13,861][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultcurrent-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,951][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:18:13,964][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [eventlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:19:13,088][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [SSx_WpQfRYuh34T0CaSkaQ]
[2021-06-02T12:19:13,553][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [KdQUngFxSGqWfuJJej8LUQ]
[2021-06-02T12:19:13,553][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [NjXjTrz4QVWPWZ-2EtzYJQ]
[2021-06-02T12:19:13,554][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [JYMOSdxUQCWczC6xkiCZRg]
[2021-06-02T12:19:14,968][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [fxWzjT-aSaKiztwy5nq2Vw]
[2021-06-02T12:19:15,131][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [QZQ4j_wAQk2aPnwwmVC8Qg]
[2021-06-02T12:19:15,858][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [Sq7pcr46SG-0oim2CPn-nQ]
[2021-06-02T12:19:15,858][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [osqOZd5GRtOUBam3BDLJPQ]
[2021-06-02T12:19:16,287][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [Dt5Yu1wqS5iewDT8DomB1Q]
[2021-06-02T12:19:17,223][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [qwyl78VNR--8kEZM1j99Dw]
[2021-06-02T12:19:17,573][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [533WREZ2TRqqLTYoBMhOOA]
[2021-06-02T12:19:19,057][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [qscnWUqWRaSI37333NjQtg]
[2021-06-02T12:19:19,057][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [dtwGgP0QRqKMCJ6YG0fHDg]
[2021-06-02T12:19:19,348][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [zzZXEIjuRUyWB1GVOTorDA]
[2021-06-02T12:19:21,090][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [MzXDR-imSa-6-NM5I8291Q]
[2021-06-02T12:19:21,502][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [Sjhv9y-jRMqX_rM6oWRSgg]
[2021-06-02T12:19:22,347][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [jKyCPoiFQGikGyP3-Tit5Q]
[2021-06-02T12:19:22,348][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [2-kDU8SCQOK6NG_DjdLCKQ]
[2021-06-02T12:19:22,650][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][1] marking unavailable shards as stale: [vo47vKL-SKKRMvnB9BwsHA]
[2021-06-02T12:19:23,768][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [TLqE5yhVSe2RuxgJJKds0Q]
[2021-06-02T12:19:24,358][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [N4qqxv6UTJ-eHxF1wzt17w]
[2021-06-02T12:19:25,447][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [PQDuXCNYRaCTg9e7cqvQ9Q]
[2021-06-02T12:19:25,448][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [OrbPjcD6R-el0roVQuS5hw]
[2021-06-02T12:19:28,316][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [wqBHqf2CRpagKS7aj5yz7g]
[2021-06-02T12:19:28,698][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][1] marking unavailable shards as stale: [3CaL1XINRxCFgMJ9GmAk9Q]
[2021-06-02T12:19:28,699][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [jhEP3oFeRpCLpccmOAbjOQ]
[2021-06-02T12:19:29,513][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [wfFv2ataTHSY04JX-A_MqA]
[2021-06-02T12:19:29,786][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [j2lkc8l1QsOSLikDsOBCfA]
[2021-06-02T12:19:30,517][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [ip4bPBBsQJ6UELYx3QfLcA]
[2021-06-02T12:19:30,517][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [eqX44bwkT2ml_AvFzqVf3w]
[2021-06-02T12:19:30,742][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [J-Ls1iRlQXeI6-aUJjWedw]
[2021-06-02T12:19:31,951][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [1bn81CJJThC-Fvjs3ZBUgg]
[2021-06-02T12:19:32,221][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [4DWB2n2YS3OGhjcY_rjzMw]
[2021-06-02T12:19:32,929][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [i5nZGfZxQPOwezVCv5Q0Nw]
[2021-06-02T12:19:33,131][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][1] marking unavailable shards as stale: [I_25NKMBR56zL83d6slIMQ]
[2021-06-02T12:19:33,911][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [2T9CbWH6SC-UL5Fd4QOnZg]
[2021-06-02T12:19:34,267][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][3]]]).
[2021-06-02T12:20:07,762][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 905, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:20:07,975][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 905, reason: Publication{term=3, version=905}
[2021-06-02T12:20:09,703][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 906, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:20:11,152][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 906, reason: Publication{term=3, version=906}
[2021-06-02T12:22:30,479][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: followers check retry count exceeded], term: 3, version: 969, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:22:40,551][INFO ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [10s] publication of cluster state version [969] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T12:22:48,772][WARN ][o.e.c.InternalClusterInfoService] [dev-sdnrdb-master-0] Failed to update shard information for ClusterInfoUpdateJob within 15s timeout
[2021-06-02T12:23:00,552][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 969, reason: Publication{term=3, version=969}
[2021-06-02T12:23:00,670][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [guicutthrough-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,680][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,761][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,852][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,867][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [maintenancemode-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,946][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,957][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [mediator-server-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:23:00,972][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][4] primary-replica resync completed with 0 operations
[2021-06-02T12:23:01,046][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [29.4s] (37 delayed shards)
[2021-06-02T12:23:01,047][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:23:01,049][WARN ][o.e.c.c.C.CoordinatorPublication] [dev-sdnrdb-master-0] after [30.4s] publication of cluster state version [969] is still waiting for {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} [SENT_APPLY_COMMIT]
[2021-06-02T12:23:01,147][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] primary-replica resync completed with 0 operations
[2021-06-02T12:23:17,935][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: followers check retry count exceeded], term: 3, version: 970, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:23:17,986][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 970, reason: Publication{term=3, version=970}
[2021-06-02T12:23:30,966][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][2] marking unavailable shards as stale: [ZpyovMw4QoSv3qJYrVc3wA]
[2021-06-02T12:23:31,275][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][3] marking unavailable shards as stale: [At0oso3KRBmQ_RxaxRAfdA]
[2021-06-02T12:23:31,276][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][1] marking unavailable shards as stale: [w5zFjhsIR8iLtrtp4fyXVQ]
[2021-06-02T12:23:31,569][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [guicutthrough-v5][4] marking unavailable shards as stale: [Ee2aACwHQQOOQymiavGlVw]
[2021-06-02T12:23:33,291][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][3] marking unavailable shards as stale: [PgMKpIm7RfCmKIMfOvZVlg]
[2021-06-02T12:23:33,969][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][2] marking unavailable shards as stale: [VzCU-5J8Tpe4JkKxjE4kXg]
[2021-06-02T12:23:33,969][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [connectionlog-v5][4] marking unavailable shards as stale: [S2Z_Oi_8Qzi8rv76JR_yJQ]
[2021-06-02T12:23:34,046][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][3] marking unavailable shards as stale: [iALI5b-1SDy1jJnkerACoQ]
[2021-06-02T12:23:35,495][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][2] marking unavailable shards as stale: [aXCw5-EHS4KbUg4QkZ-lTQ]
[2021-06-02T12:23:35,852][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][3] marking unavailable shards as stale: [XVh50fprSV2_mNyZivaGjw]
[2021-06-02T12:23:36,768][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][4] marking unavailable shards as stale: [q-agy8jAR6eVQEpsRGcA7A]
[2021-06-02T12:23:36,768][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [inventoryequipment-v5][2] marking unavailable shards as stale: [5BNUWj8_Tr2lHYKXlLRrxw]
[2021-06-02T12:23:37,258][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][2] marking unavailable shards as stale: [fIivsLydTByI77bLwlhzrQ]
[2021-06-02T12:23:40,146][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][3] marking unavailable shards as stale: [c6Oh1K3vQHaKGDcYVcuByQ]
[2021-06-02T12:23:40,407][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [mediator-server-v5][4] marking unavailable shards as stale: [8Tbj0Y0bTBK0NfDKGeVhSQ]
[2021-06-02T12:23:41,098][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultcurrent-v5][4] marking unavailable shards as stale: [Dlr6BbK6Sn638VjxT7wS8g]
[2021-06-02T12:23:41,098][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][2] marking unavailable shards as stale: [js_5QW8eT7yxmrN2pMCR2A]
[2021-06-02T12:23:41,348][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][3] marking unavailable shards as stale: [hHiGEMN1Sseyu8Ltw6uzfQ]
[2021-06-02T12:23:43,094][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][1] marking unavailable shards as stale: [Dt-5WLhYT5228Vq-iKG8OA]
[2021-06-02T12:23:43,390][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance15min-v5][4] marking unavailable shards as stale: [Ys40wjh8Toy4QgMJ7_xPgg]
[2021-06-02T12:23:44,110][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][2] marking unavailable shards as stale: [xp29RY1mQaGHvlSRByy18w]
[2021-06-02T12:23:44,111][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][3] marking unavailable shards as stale: [K0ADyoLKQwKZelLXK5-g7A]
[2021-06-02T12:23:44,394][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][2] marking unavailable shards as stale: [bCmZtBcOQi2uLuwliPYBwQ]
[2021-06-02T12:23:45,350][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [eventlog-v5][4] marking unavailable shards as stale: [-oEN4DwvRDGJlESe4VmPwQ]
[2021-06-02T12:23:46,127][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][2] marking unavailable shards as stale: [Ya3d6hgtRfq2q25iNTZpSA]
[2021-06-02T12:23:46,127][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][4] marking unavailable shards as stale: [PbCbqXvgRaSB5rpvmaDB2A]
[2021-06-02T12:23:46,679][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][1] marking unavailable shards as stale: [JCeSGfMPTMOQXVeKvsHWjg]
[2021-06-02T12:23:47,967][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [networkelement-connection-v5][3] marking unavailable shards as stale: [PBaZBE5UQ7ygh63ppw8hZQ]
[2021-06-02T12:23:48,946][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][3] marking unavailable shards as stale: [XzocaKHGQ6uBXMgZWNHIqA]
[2021-06-02T12:23:50,466][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [historicalperformance24h-v5][4] marking unavailable shards as stale: [T1zU4V7kR5SKFin2ryFgag]
[2021-06-02T12:23:50,466][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][2] marking unavailable shards as stale: [LPlYFk9UReym-xOkyy2C9A]
[2021-06-02T12:23:51,451][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][3] marking unavailable shards as stale: [pcumQ9KJQX605Sa6kYjJrw]
[2021-06-02T12:23:52,383][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][2] marking unavailable shards as stale: [X4buj6CpQdOcbSMpCq2OoA]
[2021-06-02T12:23:53,852][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [maintenancemode-v5][4] marking unavailable shards as stale: [CCz_h26MTPGhKF9H4SdfHg]
[2021-06-02T12:23:53,853][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][1] marking unavailable shards as stale: [U5fFCOULSICWK0fZQFOd-g]
[2021-06-02T12:23:54,092][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [kEg7v7WQRZ68OVoWAoRSzQ]
[2021-06-02T12:23:54,984][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [PhSE7Y1xQnu5faiL8k4IWw]
[2021-06-02T12:23:55,410][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][4]]]).
[2021-06-02T12:31:11,043][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} join existing leader], term: 3, version: 1031, delta: added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:31:11,175][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 1031, reason: Publication{term=3, version=1031}
[2021-06-02T12:31:14,714][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-join[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} join existing leader], term: 3, version: 1032, delta: added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:31:16,444][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] added {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 1032, reason: Publication{term=3, version=1032}
[2021-06-02T12:31:21,384][WARN ][o.e.c.NodeConnectionsService] [dev-sdnrdb-master-0] failed to connect to {dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} (tried [1] times)
org.elasticsearch.transport.ConnectTransportException: [dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z][[fd00:100::23]:9300] connect_exception
	at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.onFailure(TcpTransport.java:966) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener.lambda$toBiConsumer$2(ActionListener.java:198) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.concurrent.CompletableContext.lambda$addListener$0(CompletableContext.java:42) ~[elasticsearch-core-7.9.3.jar:7.9.3]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
	at org.elasticsearch.common.concurrent.CompletableContext.completeExceptionally(CompletableContext.java:57) ~[elasticsearch-core-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4TcpChannel.lambda$addListener$0(Netty4TcpChannel.java:68) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:570) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:549) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:321) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:337) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: fd00:100:0:0:0:0:0:23/fd00:100:0:0:0:0:0:23:9300
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) ~[?:?]
	... 7 more
[2021-06-02T12:31:21,395][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r} reason: disconnected], term: 3, version: 1037, delta: removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}
[2021-06-02T12:31:21,568][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-coordinating-only-65cbf8cbbd-5rz5z}{ydc165mrRGq2h52UCOK3Mw}{4Pv4uUtdSqm5zBMTIJfw5Q}{fd00:100:0:0:0:0:0:23}{[fd00:100::23]:9300}{r}}, term: 3, version: 1037, reason: Publication{term=3, version=1037}
[2021-06-02T12:31:21,575][INFO ][o.e.c.s.MasterService    ] [dev-sdnrdb-master-0] node-left[{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} reason: disconnected], term: 3, version: 1038, delta: removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}
[2021-06-02T12:31:21,578][WARN ][o.e.c.NodeConnectionsService] [dev-sdnrdb-master-0] failed to connect to {dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr} (tried [1] times)
org.elasticsearch.transport.ConnectTransportException: [dev-sdnrdb-master-1][[fd00:100::25]:9300] connect_exception
	at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.onFailure(TcpTransport.java:966) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.action.ActionListener.lambda$toBiConsumer$2(ActionListener.java:198) ~[elasticsearch-7.9.3.jar:7.9.3]
	at org.elasticsearch.common.concurrent.CompletableContext.lambda$addListener$0(CompletableContext.java:42) ~[elasticsearch-core-7.9.3.jar:7.9.3]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
	at org.elasticsearch.common.concurrent.CompletableContext.completeExceptionally(CompletableContext.java:57) ~[elasticsearch-core-7.9.3.jar:7.9.3]
	at org.elasticsearch.transport.netty4.Netty4TcpChannel.lambda$addListener$0(Netty4TcpChannel.java:68) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:570) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:549) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608) ~[?:?]
	at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:321) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:337) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: fd00:100:0:0:0:0:0:25/fd00:100:0:0:0:0:0:25:9300
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) ~[?:?]
	... 7 more
[2021-06-02T12:31:22,138][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-master-0] removed {{dev-sdnrdb-master-1}{tkbY1npwSkibDdI75JBqoQ}{wvVS1SeERE679yBUmO9OQg}{fd00:100:0:0:0:0:0:25}{[fd00:100::25]:9300}{dmr}}, term: 3, version: 1038, reason: Publication{term=3, version=1038}
[2021-06-02T12:31:22,177][INFO ][o.e.c.r.DelayedAllocationService] [dev-sdnrdb-master-0] scheduling reroute for delayed shards in [59.3s] (2 delayed shards)
[2021-06-02T12:31:22,188][INFO ][o.e.i.s.IndexShard       ] [dev-sdnrdb-master-0] [faultlog-v5][3] primary-replica resync completed with 0 operations
[2021-06-02T12:32:21,765][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][3] marking unavailable shards as stale: [FtCFHQuyRM2o_cSQmQeQ1g]
[2021-06-02T12:32:21,975][WARN ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] [faultlog-v5][4] marking unavailable shards as stale: [mAxaR4v8TfOqBEcntjp3iA]
[2021-06-02T12:32:22,760][INFO ][o.e.c.r.a.AllocationService] [dev-sdnrdb-master-0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[faultlog-v5][4]]]).