10:51:05.83   10:51:05.84 Welcome to the Bitnami elasticsearch container  10:51:05.85 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-elasticsearch  10:51:05.86 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-elasticsearch/issues  10:51:05.88   10:51:05.89 INFO  ==> ** Starting Elasticsearch setup **  10:51:06.01 INFO  ==> Configuring/Initializing Elasticsearch...  10:51:06.11 INFO  ==> Setting default configuration  10:51:06.15 INFO  ==> Configuring Elasticsearch cluster settings...  10:51:06.18 WARN  ==> Found more than one IP address associated to hostname dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh: fd00:100::e4b0 10.242.228.176, will use fd00:100::e4b0  10:51:06.24 WARN  ==> Found more than one IP address associated to hostname dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh: fd00:100::e4b0 10.242.228.176, will use fd00:100::e4b0 OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.  10:51:10.21 INFO  ==> ** Elasticsearch setup finished! **  10:51:10.29 INFO  ==> ** Starting Elasticsearch ** OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. [2021-04-14T10:51:14,593][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] version[7.9.3], pid[1], build[oss/tar/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/4.15.0-117-generic/amd64], JVM[BellSoft/OpenJDK 64-Bit Server VM/11.0.9/11.0.9+11-LTS] [2021-04-14T10:51:14,595][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] JVM home [/opt/bitnami/java] [2021-04-14T10:51:14,595][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-14667826429283919297, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/opt/bitnami/elasticsearch, -Des.path.conf=/opt/bitnami/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar, -Des.bundled_jdk=true] [2021-04-14T10:51:16,076][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [aggs-matrix-stats] [2021-04-14T10:51:16,078][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [analysis-common] [2021-04-14T10:51:16,079][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [geo] [2021-04-14T10:51:16,079][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [ingest-common] [2021-04-14T10:51:16,080][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [ingest-geoip] [2021-04-14T10:51:16,080][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [ingest-user-agent] [2021-04-14T10:51:16,081][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [kibana] [2021-04-14T10:51:16,081][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [lang-expression] [2021-04-14T10:51:16,082][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [lang-mustache] [2021-04-14T10:51:16,082][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [lang-painless] [2021-04-14T10:51:16,083][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [mapper-extras] [2021-04-14T10:51:16,083][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [parent-join] [2021-04-14T10:51:16,084][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [percolator] [2021-04-14T10:51:16,084][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [rank-eval] [2021-04-14T10:51:16,084][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [reindex] [2021-04-14T10:51:16,085][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [repository-url] [2021-04-14T10:51:16,085][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [tasks] [2021-04-14T10:51:16,086][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded module [transport-netty4] [2021-04-14T10:51:16,087][INFO ][o.e.p.PluginsService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] loaded plugin [repository-s3] [2021-04-14T10:51:16,151][INFO ][o.e.e.NodeEnvironment ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] using [1] data paths, mounts [[/bitnami/elasticsearch/data (/dev/vda1)]], net usable_space [139.9gb], net total_space [154.8gb], types [ext4] [2021-04-14T10:51:16,153][INFO ][o.e.e.NodeEnvironment ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] heap size [123.7mb], compressed ordinary object pointers [true] [2021-04-14T10:51:16,222][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] node name [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh], node ID [Q833XYo9Tk2UYlX9_IUP5Q], cluster name [sdnrdb-cluster] [2021-04-14T10:51:20,333][INFO ][o.e.t.NettyAllocator ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=false, g1gc_region_size=0b, heap_size=123.7mb}] [2021-04-14T10:51:20,413][INFO ][o.e.d.DiscoveryModule ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] using discovery type [zen] and seed hosts providers [settings] [2021-04-14T10:51:20,813][WARN ][o.e.g.DanglingIndicesState] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually [2021-04-14T10:51:21,034][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] initialized [2021-04-14T10:51:21,034][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] starting ... [2021-04-14T10:51:21,258][INFO ][o.e.t.TransportService ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] publish_address {[fd00:100::e4b0]:9300}, bound_addresses {[::]:9300} [2021-04-14T10:51:21,481][INFO ][o.e.b.BootstrapChecks ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] bound or publishing to a non-loopback address, enforcing bootstrap checks [2021-04-14T10:53:09,532][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] master node changed {previous [], current [{dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}]}, added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr},{dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}}, term: 2, version: 2, reason: ApplyCommitRequest{term=2, version=2, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T10:53:09,570][INFO ][o.e.h.AbstractHttpServerTransport] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] publish_address {[fd00:100::e4b0]:9000}, bound_addresses {[::]:9000} [2021-04-14T10:53:09,571][INFO ][o.e.n.Node ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] started [2021-04-14T10:53:14,700][INFO ][o.e.c.s.ClusterSettings ] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] updating [action.auto_create_index] from [true] to [false] [2021-04-14T10:54:30,284][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-2}{y-7BEwEpSeOr9O1O2w03Rw}{Ye72TLwpQumEMWHlPufznQ}{fd00:100:0:0:0:0:0:c69e}{[fd00:100::c69e]:9300}{dmr}}, term: 2, version: 75, reason: ApplyCommitRequest{term=2, version=75, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:01:53,622][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 135, reason: ApplyCommitRequest{term=2, version=135, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:04:38,666][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 195, reason: ApplyCommitRequest{term=2, version=195, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:09:32,861][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 258, reason: ApplyCommitRequest{term=2, version=258, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:12:38,502][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 313, reason: ApplyCommitRequest{term=2, version=313, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:13:38,707][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 315, reason: ApplyCommitRequest{term=2, version=315, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:15:18,033][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 316, reason: ApplyCommitRequest{term=2, version=316, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:25:05,992][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 379, reason: ApplyCommitRequest{term=2, version=379, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:26:04,487][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 380, reason: ApplyCommitRequest{term=2, version=380, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:27:04,505][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 382, reason: ApplyCommitRequest{term=2, version=382, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:27:13,582][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 404, reason: ApplyCommitRequest{term=2, version=404, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:36:17,753][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 467, reason: ApplyCommitRequest{term=2, version=467, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:38:24,121][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 527, reason: ApplyCommitRequest{term=2, version=527, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:41:48,768][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 561, reason: ApplyCommitRequest{term=2, version=561, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:48:16,710][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 596, reason: ApplyCommitRequest{term=2, version=596, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:49:16,829][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 598, reason: ApplyCommitRequest{term=2, version=598, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:53:11,016][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 599, reason: ApplyCommitRequest{term=2, version=599, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T11:55:06,088][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 602, reason: ApplyCommitRequest{term=2, version=602, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:07:09,205][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 603, reason: ApplyCommitRequest{term=2, version=603, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:07:39,231][WARN ][o.e.c.NodeConnectionsService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] failed to connect to {dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr} (tried [1] times) org.elasticsearch.transport.ConnectTransportException: [dev-sdnrdb-master-1][[fd00:100::52c]:9300] handshake_timeout[30s] at org.elasticsearch.transport.TransportHandshaker.lambda$sendHandshake$1(TransportHandshaker.java:73) ~[elasticsearch-7.9.3.jar:7.9.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) ~[elasticsearch-7.9.3.jar:7.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] [2021-04-14T12:07:39,285][WARN ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] cluster state applier task [ApplyCommitRequest{term=2, version=603, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}}] took [30s] which is above the warn threshold of [30s]: [running task [ApplyCommitRequest{term=2, version=603, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}}]] took [0ms], [connecting to new nodes] took [30070ms], [applying settings] took [0ms], [running applier [org.elasticsearch.script.ScriptService@2fc90a80]] took [0ms], [running applier [org.elasticsearch.snapshots.RestoreService@5f0b4fd]] took [0ms], [running applier [org.elasticsearch.ingest.IngestService@7f247ff2]] took [0ms], [running applier [org.elasticsearch.action.ingest.IngestActionForwarder@3f08c038]] took [0ms], [running applier [org.elasticsearch.action.admin.cluster.repositories.cleanup.TransportCleanupRepositoryAction$$Lambda$2394/0x0000000100928440@58f25053]] took [0ms], [running applier [org.elasticsearch.tasks.TaskManager@582550ec]] took [0ms], [notifying listener [org.elasticsearch.cluster.InternalClusterInfoService@45b84559]] took [0ms], [notifying listener [org.elasticsearch.cluster.metadata.TemplateUpgradeService@527247cc]] took [0ms], [notifying listener [org.elasticsearch.node.ResponseCollectorService@3c7c93f5]] took [0ms], [notifying listener [org.elasticsearch.persistent.PersistentTasksClusterService@4d6e6603]] took [0ms], [notifying listener [org.elasticsearch.cluster.routing.DelayedAllocationService@4a7b707a]] took [0ms], [notifying listener [org.elasticsearch.persistent.PersistentTasksNodeService@af702de]] took [0ms], [notifying listener [org.elasticsearch.gateway.GatewayService@524cdb4e]] took [0ms], [notifying listener [org.elasticsearch.indices.recovery.PeerRecoverySourceService@6c2cf17a]] took [0ms], [notifying listener [org.elasticsearch.cluster.service.ClusterApplierService$LocalNodeMasterListeners@7dc5a425]] took [0ms] [2021-04-14T12:08:09,312][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 605, reason: ApplyCommitRequest{term=2, version=605, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:25:17,198][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 606, reason: ApplyCommitRequest{term=2, version=606, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:33:43,308][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 665, reason: ApplyCommitRequest{term=2, version=665, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:33:43,320][WARN ][o.e.t.TaskCancellationService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] Cannot send ban for tasks with the parent [Q833XYo9Tk2UYlX9_IUP5Q:7661] to the node [{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}] [2021-04-14T12:33:43,326][INFO ][o.e.t.TaskCancellationService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] failed to remove the parent ban for task Q833XYo9Tk2UYlX9_IUP5Q:7661 on node {dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr} [2021-04-14T12:34:26,781][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 666, reason: ApplyCommitRequest{term=2, version=666, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:35:26,591][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 668, reason: ApplyCommitRequest{term=2, version=668, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:38:18,465][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 726, reason: ApplyCommitRequest{term=2, version=726, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:55:29,756][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 786, reason: ApplyCommitRequest{term=2, version=786, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:58:48,828][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 848, reason: ApplyCommitRequest{term=2, version=848, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T12:59:48,869][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 850, reason: ApplyCommitRequest{term=2, version=850, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:17:53,994][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 851, reason: ApplyCommitRequest{term=2, version=851, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:23:21,096][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 910, reason: ApplyCommitRequest{term=2, version=910, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:24:30,881][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 934, reason: ApplyCommitRequest{term=2, version=934, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:28:42,007][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 995, reason: ApplyCommitRequest{term=2, version=995, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:33:56,198][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1059, reason: ApplyCommitRequest{term=2, version=1059, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:34:56,185][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1061, reason: ApplyCommitRequest{term=2, version=1061, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:35:29,505][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1062, reason: ApplyCommitRequest{term=2, version=1062, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:36:36,342][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1064, reason: ApplyCommitRequest{term=2, version=1064, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:43:47,193][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1065, reason: ApplyCommitRequest{term=2, version=1065, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T13:45:00,790][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] removed {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1067, reason: ApplyCommitRequest{term=2, version=1067, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}} [2021-04-14T14:05:27,791][INFO ][o.e.c.s.ClusterApplierService] [dev-sdnrdb-coordinating-only-65cbf8cbbd-hp5gh] added {{dev-sdnrdb-master-1}{GxlOGSMxRDyO9ww8Zdsfag}{FojDL4ZOSUO-dtDRcxMTBA}{fd00:100:0:0:0:0:0:52c}{[fd00:100::52c]:9300}{dmr}}, term: 2, version: 1068, reason: ApplyCommitRequest{term=2, version=1068, sourceNode={dev-sdnrdb-master-0}{ZxsDM5oETU2XXRgTLIIDtA}{9JT9cB29Rcyo13oOG9Jx1Q}{fd00:100:0:0:0:0:0:3e94}{[fd00:100::3e94]:9300}{dmr}}