By type
*** retrieve Truststore and Keystore password
*** actual launch of AAI Resources
2021-05-21 11:01:23.104 INFO --- [ main] org.onap.aai.util.AAIConfig : Initializing AAIConfig
2021-05-21 11:01:23.110 DEBUG --- [ main] org.onap.aai.util.AAIConfig : Reloading config from /opt/app/aai-traversal/./resources/etc/appprops/aaiconfig.properties
2021-05-21 11:01:23.110 INFO --- [ main] org.onap.aai.util.AAIConfig : Found the aaiconfig.properties in the following location: /opt/app/aai-traversal/./resources/etc/appprops/aaiconfig.properties
2021-05-21 11:01:23.113 INFO --- [ main] org.onap.aai.util.AAIConfig : A&AI Server Node Name = aai.config.nodename
2021-05-21 11:01:24.919 WARN 11 --- [ main] o.s.b.l.logback.LogbackLoggingSystem : Ignoring 'logback.configurationFile' system property. Please use 'logging.config' instead.
2021-05-21 11:01:25.007 DEBUG 11 --- [kground-preinit] .h.v.m.ResourceBundleMessageInterpolator : Loaded expression factory via original TCCL
2021-05-21 11:01:25.023 DEBUG 11 --- [kground-preinit] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.messageinterpolation.ResourceBundleMessageInterpolator as ValidatorFactory-scoped message interpolator.
2021-05-21 11:01:25.023 DEBUG 11 --- [kground-preinit] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.resolver.JPATraversableResolver as ValidatorFactory-scoped traversable resolver.
2021-05-21 11:01:25.023 DEBUG 11 --- [kground-preinit] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.util.ExecutableParameterNameProvider as ValidatorFactory-scoped parameter name provider.
2021-05-21 11:01:25.023 DEBUG 11 --- [kground-preinit] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.DefaultClockProvider as ValidatorFactory-scoped clock provider.
2021-05-21 11:01:25.023 DEBUG 11 --- [kground-preinit] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.scripting.DefaultScriptEvaluatorFactory as ValidatorFactory-scoped script evaluator factory.
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.12.RELEASE)
11:01:29,697 |-INFO in LogbackRequestLog - Will use configuration resource [/localhost-access-logback.xml]
11:01:29,703 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - debug attribute not set
11:01:29,703 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
11:01:29,703 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS]
11:01:29,704 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@739973450 - No compression will be used
11:01:29,704 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@739973450 - Will use the pattern /opt/app/aai-traversal/logs/ajsc-jetty/localhost_access.log.%d{yyyy-MM-dd} for the active file
11:01:29,704 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/app/aai-traversal/logs/ajsc-jetty/localhost_access.log.%d{yyyy-MM-dd}'.
11:01:29,704 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
11:01:29,705 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Fri May 21 11:01:29 GMT 2021
11:01:29,720 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: /opt/app/aai-traversal/logs/ajsc-jetty/localhost_access.log
11:01:29,720 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [/opt/app/aai-traversal/logs/ajsc-jetty/localhost_access.log]
11:01:29,720 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to null
11:01:29,720 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - End of configuration.
11:01:29,720 |-INFO in ch.qos.logback.access.joran.JoranConfigurator@4dbb42b7 - Registering current configuration as safe fallback point
2021-05-21 11:01:29.907 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED CLASSES [class org.onap.aai.rest.QueryConsumer, class org.onap.aai.rest.DslConsumer, class org.onap.aai.rest.search.SearchProvider, class org.onap.aai.rest.search.ModelAndNamedQueryRestProvider, class org.onap.aai.rest.RecentAPIConsumer, class org.onap.aai.rest.CQ2Gremlin, class org.onap.aai.rest.util.EchoResponse, class org.onap.aai.rest.CQ2GremlinTest]
2021-05-21 11:01:29.911 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.VersionLatestInterceptor result=true
2021-05-21 11:01:29.912 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RetiredInterceptor result=true
2021-05-21 11:01:29.913 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestModification result=true
2021-05-21 11:01:29.913 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.InvalidResponseStatus result=true
2021-05-21 11:01:29.914 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.OneWaySslAuthorization result=false
2021-05-21 11:01:29.915 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.HttpHeaderInterceptor result=true
2021-05-21 11:01:29.915 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.ResponseTransactionLogging result=true
2021-05-21 11:01:29.916 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.ResponseHeaderManipulation result=true
2021-05-21 11:01:29.916 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestHeaderManipulation result=true
2021-05-21 11:01:29.917 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.HeaderValidation result=true
2021-05-21 11:01:29.918 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.VersionInterceptor result=true
2021-05-21 11:01:29.918 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestTransactionLogging result=true
2021-05-21 11:01:29.922 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.VersionLatestInterceptor result=true
2021-05-21 11:01:29.922 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RetiredInterceptor result=true
2021-05-21 11:01:29.923 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestModification result=true
2021-05-21 11:01:29.923 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.InvalidResponseStatus result=true
2021-05-21 11:01:29.924 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.OneWaySslAuthorization result=false
2021-05-21 11:01:29.924 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.HttpHeaderInterceptor result=true
2021-05-21 11:01:29.925 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.ResponseTransactionLogging result=true
2021-05-21 11:01:29.926 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.post.ResponseHeaderManipulation result=true
2021-05-21 11:01:29.926 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestHeaderManipulation result=true
2021-05-21 11:01:29.927 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.HeaderValidation result=true
2021-05-21 11:01:29.928 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.VersionInterceptor result=true
2021-05-21 11:01:29.928 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : isEnabledByActiveProfiles: annotatedElement: class org.onap.aai.interceptors.pre.RequestTransactionLogging result=true
2021-05-21 11:01:29.929 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.post.InvalidResponseStatus
2021-05-21 11:01:29.929 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.RequestTransactionLogging
2021-05-21 11:01:29.930 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.HeaderValidation
2021-05-21 11:01:29.930 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.post.ResponseTransactionLogging
2021-05-21 11:01:29.931 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.HttpHeaderInterceptor
2021-05-21 11:01:29.931 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.post.ResponseHeaderManipulation
2021-05-21 11:01:29.932 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.VersionLatestInterceptor
2021-05-21 11:01:29.932 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.RetiredInterceptor
2021-05-21 11:01:29.933 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.VersionInterceptor
2021-05-21 11:01:29.933 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.RequestHeaderManipulation
2021-05-21 11:01:29.935 DEBUG 11 --- [ main] org.onap.aai.web.JerseyConfiguration : REGISTERED FILTERS org.onap.aai.interceptors.pre.RequestModification
2021-05-21 11:01:30.353 INFO 11 --- [ main] org.onap.aai.config.NodesConfiguration : Translator is SchemaServiceTranslator
2021-05-21 11:01:30.744 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:30.744 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:30.744 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:30.745 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:01:30.753 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/versions, BODY=
2021-05-21 11:01:30.873 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:30.873 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:30.873 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:30.874 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:31.327 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:31.425 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Processing the translator
2021-05-21 11:01:31.426 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv11
2021-05-21 11:01:31.426 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:31.513 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:31.513 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:31.513 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:31.513 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:31.545 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:31.596 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:31.597 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:31.597 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:34.985 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 222
2021-05-21 11:01:35.012 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv12
2021-05-21 11:01:35.012 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:35.014 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:35.014 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:35.014 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:35.014 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:35.045 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:35.104 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:35.105 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:35.105 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:36.915 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 235
2021-05-21 11:01:36.929 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv13
2021-05-21 11:01:36.929 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:36.932 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:36.932 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:36.932 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:36.932 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:36.950 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:36.966 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:36.967 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:36.967 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:38.471 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 237
2021-05-21 11:01:38.485 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv14
2021-05-21 11:01:38.486 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:38.488 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:38.488 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:38.488 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:38.488 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:38.515 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:38.532 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:38.533 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:38.533 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:40.121 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 272
2021-05-21 11:01:40.184 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv15
2021-05-21 11:01:40.184 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:40.186 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:40.186 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:40.186 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:40.186 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:40.207 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:40.224 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:40.225 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:40.225 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:41.901 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 279
2021-05-21 11:01:41.911 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv16
2021-05-21 11:01:41.911 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:41.913 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:41.913 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:41.913 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:41.913 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:41.948 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:41.964 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:41.965 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:41.965 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:43.319 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281
2021-05-21 11:01:43.328 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv17
2021-05-21 11:01:43.328 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:43.330 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:43.330 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:43.330 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:43.330 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:43.350 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:43.369 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:43.370 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:43.370 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:45.285 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281
2021-05-21 11:01:45.296 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv18
2021-05-21 11:01:45.297 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:45.300 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:45.300 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:45.301 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:45.301 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:45.364 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:45.383 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:45.383 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:45.383 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:46.938 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281
2021-05-21 11:01:46.948 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv19
2021-05-21 11:01:46.948 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:46.951 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:46.951 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:46.951 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:46.951 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:46.974 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:46.992 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:46.993 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:46.993 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:48.576 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 301
2021-05-21 11:01:48.595 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv20
2021-05-21 11:01:48.596 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:48.600 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:48.600 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:48.600 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:48.600 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:48.627 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:48.651 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:48.685 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:48.686 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:51.213 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 305
2021-05-21 11:01:51.223 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv21
2021-05-21 11:01:51.223 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:51.225 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:51.225 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:51.226 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:51.226 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:51.246 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:51.268 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:51.269 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:51.269 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:53.210 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 371
2021-05-21 11:01:53.222 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv22
2021-05-21 11:01:53.223 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:53.225 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:53.225 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:53.225 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:53.225 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:53.253 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:53.284 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:53.284 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:53.284 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:55.189 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 378
2021-05-21 11:01:55.203 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv23
2021-05-21 11:01:55.204 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml}
2021-05-21 11:01:55.206 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:01:55.206 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:01:55.207 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:01:55.207 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:01:55.230 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:01:55.284 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:01:55.284 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService
2021-05-21 11:01:55.284 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream
2021-05-21 11:01:57.455 DEBUG 11 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 378
2021-05-21 11:01:57.498 DEBUG 11 --- [ main] org.onap.aai.TraversalApp : AAI Server initialization started...
2021-05-21 11:01:57.498 DEBUG 11 --- [ main] org.onap.aai.TraversalApp : Starting AAIGraph connections and the NodeInjestor
2021-05-21 11:01:57.499 INFO 11 --- [ main] org.onap.aai.util.AAIConfig : Initializing AAIConfig
2021-05-21 11:01:57.499 DEBUG 11 --- [ main] org.onap.aai.util.AAIConfig : Reloading config from /opt/app/aai-traversal/./resources/etc/appprops/aaiconfig.properties
2021-05-21 11:01:57.499 INFO 11 --- [ main] org.onap.aai.util.AAIConfig : Found the aaiconfig.properties in the following location: /opt/app/aai-traversal/./resources/etc/appprops/aaiconfig.properties
2021-05-21 11:01:57.499 INFO 11 --- [ main] org.onap.aai.util.AAIConfig : A&AI Server Node Name = aai.config.nodename
2021-05-21 11:01:57.866 INFO 11 --- [ main] com.datastax.driver.core : DataStax Java driver 3.6.0 for Apache Cassandra
2021-05-21 11:01:57.875 INFO 11 --- [ main] c.d.driver.core.GuavaCompatibility : Detected Guava >= 19 in the classpath, using modern compatibility layer
2021-05-21 11:01:57.876 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NEW_NODE_DELAY_SECONDS is undefined, using default value 1
2021-05-21 11:01:57.876 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NOTIF_LOCK_TIMEOUT_SECONDS is undefined, using default value 60
2021-05-21 11:01:57.895 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true
2021-05-21 11:01:58.181 INFO 11 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps.
2021-05-21 11:01:58.190 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NON_BLOCKING_EXECUTOR_SIZE is undefined, using default value 2
2021-05-21 11:01:58.271 DEBUG 11 --- [ main] com.datastax.driver.core.Cluster : Starting new cluster with contact points [cassandra/10.242.184.18:9042]
2021-05-21 11:01:58.308 DEBUG 11 --- [ main] i.n.u.i.logging.InternalLoggerFactory : Using SLF4J as the default logging framework
2021-05-21 11:01:58.329 DEBUG 11 --- [ main] i.n.u.internal.InternalThreadLocalMap : -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
2021-05-21 11:01:58.329 DEBUG 11 --- [ main] i.n.u.internal.InternalThreadLocalMap : -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
2021-05-21 11:01:58.369 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : -Dio.netty.noUnsafe: false
2021-05-21 11:01:58.369 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : Java version: 8
2021-05-21 11:01:58.371 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : sun.misc.Unsafe.theUnsafe: available
2021-05-21 11:01:58.372 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : sun.misc.Unsafe.copyMemory: available
2021-05-21 11:01:58.373 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.Buffer.address: available
2021-05-21 11:01:58.374 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : direct buffer constructor: available
2021-05-21 11:01:58.375 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.Bits.unaligned: available, true
2021-05-21 11:01:58.375 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9
2021-05-21 11:01:58.375 DEBUG 11 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.DirectByteBuffer.(long, int): available
2021-05-21 11:01:58.375 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : sun.misc.Unsafe: available
2021-05-21 11:01:58.376 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
2021-05-21 11:01:58.376 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.bitMode: 64 (sun.arch.data.model)
2021-05-21 11:01:58.378 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.maxDirectMemory: 1020067840 bytes
2021-05-21 11:01:58.378 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.uninitializedArrayAllocationThreshold: -1
2021-05-21 11:01:58.380 DEBUG 11 --- [ main] io.netty.util.internal.CleanerJava6 : java.nio.ByteBuffer.cleaner(): available
2021-05-21 11:01:58.380 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.noPreferDirect: false
2021-05-21 11:01:58.386 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FORCE_NIO is undefined, using default value false
2021-05-21 11:01:58.393 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
2021-05-21 11:01:58.393 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : -Dio.netty.native.deleteLibAfterLoading: true
2021-05-21 11:01:58.393 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : -Dio.netty.native.tryPatchShadedId: true
2021-05-21 11:01:58.397 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : Unable to load the library 'netty_transport_native_epoll_x86_64', trying other loading mechanism.
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
2021-05-21 11:01:58.398 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : netty_transport_native_epoll_x86_64 cannot be loaded from java.library.path, now trying export to -Dio.netty.native.workdir: /tmp
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
... 64 common frames omitted
2021-05-21 11:01:58.422 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : Unable to load the library '/tmp/libnetty_transport_native_epoll_x86_648351059140850565236.so', trying other loading mechanism.
java.lang.UnsatisfiedLinkError: null
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:382)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:203)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
... 65 common frames omitted
Caused by: java.lang.RuntimeException: failed to get field ID: DefaultFileRegion.transfered
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:36)
... 72 common frames omitted
2021-05-21 11:01:58.424 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : Unable to load the library 'netty_transport_native_epoll', trying other loading mechanism.
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:290)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
2021-05-21 11:01:58.425 DEBUG 11 --- [ main] i.n.util.internal.NativeLibraryLoader : netty_transport_native_epoll cannot be loaded from java.library.path, now trying export to -Dio.netty.native.workdir: /tmp
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:290)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
... 64 common frames omitted
2021-05-21 11:01:58.427 WARN 11 --- [ main] com.datastax.driver.core.NettyUtil : Found Netty's native epoll transport in the classpath, but epoll is not available. Using NIO instead.
java.lang.UnsatisfiedLinkError: could not load a native library: netty_transport_native_epoll_x86_64
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287)
at io.netty.channel.epoll.Native.(Native.java:69)
at io.netty.channel.epoll.Epoll.(Epoll.java:33)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.datastax.driver.core.NettyUtil.(NettyUtil.java:57)
at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95)
at com.datastax.driver.core.Connection$Factory.(Connection.java:926)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272)
at org.janusgraph.diskstorage.cql.CQLStoreManager.(CQLStoreManager.java:165)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1382)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:73)
at org.onap.aai.dbmap.AAIGraph.(AAIGraph.java:54)
at org.onap.aai.dbmap.AAIGraph$Helper.(AAIGraph.java:80)
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94)
at org.onap.aai.TraversalApp.init(TraversalApp.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.onap.aai.TraversalApp.main(TraversalApp.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:578)
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
... 63 common frames omitted
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
... 64 common frames omitted
Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_transport_native_epoll
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:290)
... 62 common frames omitted
Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll.so
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:173)
... 63 common frames omitted
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
... 63 common frames omitted
Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
... 64 common frames omitted
Caused by: java.lang.RuntimeException: failed to get field ID: DefaultFileRegion.transfered
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:36)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:203)
... 63 common frames omitted
2021-05-21 11:01:58.437 DEBUG 11 --- [ main] i.n.channel.MultithreadEventLoopGroup : -Dio.netty.eventLoopThreads: 4
2021-05-21 11:01:58.454 DEBUG 11 --- [ main] io.netty.channel.nio.NioEventLoop : -Dio.netty.noKeySetOptimization: false
2021-05-21 11:01:58.454 DEBUG 11 --- [ main] io.netty.channel.nio.NioEventLoop : -Dio.netty.selectorAutoRebuildThreshold: 512
2021-05-21 11:01:58.473 DEBUG 11 --- [ main] i.netty.util.internal.PlatformDependent : org.jctools-core.MpscChunkedArrayQueue: available
2021-05-21 11:01:58.501 DEBUG 11 --- [ main] io.netty.util.ResourceLeakDetector : -Dio.netty.leakDetection.level: simple
2021-05-21 11:01:58.501 DEBUG 11 --- [ main] io.netty.util.ResourceLeakDetector : -Dio.netty.leakDetection.targetRecords: 4
2021-05-21 11:01:58.501 DEBUG 11 --- [ main] i.n.util.ResourceLeakDetectorFactory : Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@7fd3197b
2021-05-21 11:01:58.510 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.EXTENDED_PEER_CHECK is undefined, using default value true
2021-05-21 11:01:58.564 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:01:58.569 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.DISABLE_COALESCING is undefined, using default value false
2021-05-21 11:01:58.569 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FLUSHER_SCHEDULE_PERIOD_NS is undefined, using default value 10000
2021-05-21 11:01:58.569 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FLUSHER_RUN_WITHOUT_WORK_TIMES is undefined, using default value 5
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.numHeapArenas: 4
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.numDirectArenas: 4
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.pageSize: 8192
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.maxOrder: 11
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.chunkSize: 16777216
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.tinyCacheSize: 512
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.smallCacheSize: 256
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.normalCacheSize: 64
2021-05-21 11:01:58.632 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2021-05-21 11:01:58.633 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.cacheTrimInterval: 8192
2021-05-21 11:01:58.633 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2021-05-21 11:01:58.633 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.useCacheForAllThreads: true
2021-05-21 11:01:58.633 DEBUG 11 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2021-05-21 11:01:58.668 DEBUG 11 --- [ main] io.netty.channel.DefaultChannelId : -Dio.netty.processId: 11 (auto-detected)
2021-05-21 11:01:58.672 DEBUG 11 --- [ main] io.netty.util.NetUtil : -Djava.net.preferIPv4Stack: false
2021-05-21 11:01:58.672 DEBUG 11 --- [ main] io.netty.util.NetUtil : -Djava.net.preferIPv6Addresses: false
2021-05-21 11:01:58.674 DEBUG 11 --- [ main] io.netty.util.NetUtil : Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo)
2021-05-21 11:01:58.675 DEBUG 11 --- [ main] io.netty.util.NetUtil : /proc/sys/net/core/somaxconn: 128
2021-05-21 11:01:58.676 DEBUG 11 --- [ main] io.netty.channel.DefaultChannelId : -Dio.netty.machineId: fe:93:85:ff:fe:2c:6f:e2 (auto-detected)
2021-05-21 11:01:58.721 DEBUG 11 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.allocator.type: pooled
2021-05-21 11:01:58.721 DEBUG 11 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.threadLocalDirectBufferSize: 0
2021-05-21 11:01:58.721 DEBUG 11 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.maxThreadLocalCharBufferSize: 16384
2021-05-21 11:01:58.778 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:01:58.819 DEBUG 11 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.maxCapacityPerThread: 4096
2021-05-21 11:01:58.819 DEBUG 11 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.maxSharedCapacityFactor: 2
2021-05-21 11:01:58.819 DEBUG 11 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.linkCapacity: 16
2021-05-21 11:01:58.819 DEBUG 11 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.ratio: 8
2021-05-21 11:01:58.834 DEBUG 11 --- [er-nio-worker-0] io.netty.buffer.AbstractByteBuf : -Dio.netty.buffer.checkAccessible: true
2021-05-21 11:01:58.834 DEBUG 11 --- [er-nio-worker-0] io.netty.buffer.AbstractByteBuf : -Dio.netty.buffer.checkBounds: true
2021-05-21 11:01:58.834 DEBUG 11 --- [er-nio-worker-0] i.n.util.ResourceLeakDetectorFactory : Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@158bd21d
2021-05-21 11:01:58.902 DEBUG 11 --- [er-nio-worker-0] c.datastax.driver.core.SystemProperties : com.datastax.driver.NATIVE_TRANSPORT_MAX_FRAME_SIZE_IN_MB is undefined, using default value 256
2021-05-21 11:01:59.487 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:01:59.490 DEBUG 11 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:01:59.648 DEBUG 11 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 11:01:59.795 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.184.18:9042
2021-05-21 11:01:59.795 INFO 11 --- [ main] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'Pod' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2021-05-21 11:01:59.797 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.131.197:9042 added
2021-05-21 11:01:59.797 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host cassandra/10.242.184.18:9042 added
2021-05-21 11:01:59.797 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.129.86:9042 added
2021-05-21 11:01:59.801 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.CHECK_IO_DEADLOCKS is undefined, using default value true
2021-05-21 11:01:59.806 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:01:59.809 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:01:59.813 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:01:59.813 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:01:59.814 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:01:59.814 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:02:00.019 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:00.022 DEBUG 11 --- [er-nio-worker-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.131.197:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:00.022 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Session : Added connection pool for /10.242.131.197:9042
2021-05-21 11:02:00.099 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:00.099 DEBUG 11 --- [er-nio-worker-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:00.099 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:02:00.100 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:00.100 DEBUG 11 --- [er-nio-worker-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.129.86:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:00.101 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Session : Added connection pool for /10.242.129.86:9042
2021-05-21 11:02:00.358 DEBUG 11 --- [ main] org.reflections.Reflections : going to scan these urls:
file:/opt/app/aai-traversal/resources/
2021-05-21 11:02:00.368 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file application-keycloak.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.369 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file application-keycloak.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.369 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ipsec-configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.370 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ipsec-configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.370 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.370 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.370 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/reserved-prop-names-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.370 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/reserved-prop-names-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.371 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tagged-inventory-item-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.371 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tagged-inventory-item-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.371 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpe-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.371 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpe-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.372 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/constrained-element-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.372 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/constrained-element-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.372 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tenant-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.372 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tenant-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.372 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vce-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.373 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vce-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.373 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/lag-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.373 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/lag-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.373 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vf-module-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.373 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vf-module-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.374 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/newvce-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.374 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/newvce-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.374 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/customer-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.374 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/customer-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.374 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vig-server-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.375 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vig-server-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.375 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vnf-image-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.375 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vnf-image-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.375 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/snapshot-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.375 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/snapshot-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.376 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-interface-ipv4-address-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.376 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-interface-ipv4-address-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.376 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.376 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.377 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.377 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.377 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/oam-network-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.377 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/oam-network-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.378 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/entitlement-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.378 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/entitlement-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.378 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/site-pair-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.378 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/site-pair-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.378 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/lag-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.379 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/lag-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.379 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/collection-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.379 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/collection-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.379 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/result-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.379 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/result-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.380 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/allotted-resource-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.380 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/allotted-resource-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.380 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/group-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.380 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/group-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.381 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/pnf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.381 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/pnf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.382 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/license-key-resource-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.382 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/license-key-resource-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.383 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/update-node-key-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.383 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/update-node-key-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.384 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/cloud-region-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.384 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/cloud-region-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.384 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/zone-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.385 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/zone-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.385 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/route-table-reference-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.385 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/route-table-reference-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.386 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/cvlan-tag-entry-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.386 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/cvlan-tag-entry-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.386 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-capability-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.386 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-capability-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.387 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/pserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.387 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/pserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.388 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/connector-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.388 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/connector-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.388 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/segmentation-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.389 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/segmentation-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.389 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/property-constraint-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.389 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/property-constraint-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.390 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/esr-system-info-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.390 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/esr-system-info-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.390 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/routing-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.390 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/routing-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.391 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-element-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.391 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-element-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.391 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/related-lookup-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.391 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/related-lookup-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.392 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/class-of-service-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.392 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/class-of-service-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.392 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/key-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.392 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/key-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.393 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/network-policy-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.393 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/network-policy-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.393 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/named-query-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.393 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/named-query-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.394 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/license-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.394 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/license-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.394 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/logical-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.394 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/logical-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.395 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpn-binding-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.395 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpn-binding-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.395 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.395 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.395 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/instance-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.396 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/instance-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.396 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/start-node-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.396 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/start-node-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.397 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/volume-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.397 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/volume-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.397 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-constraint-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.397 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-constraint-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.397 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/inventory-item-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.398 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/inventory-item-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.398 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/image-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.398 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/image-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.398 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/network-profile-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.399 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/network-profile-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.399 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ctag-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.399 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ctag-assignment-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.399 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/port-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.400 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/port-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.400 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/multicast-configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.401 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/multicast-configuration-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.401 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/p-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.401 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/p-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.402 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/physical-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.402 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/physical-link-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.402 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/generic-vnf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.402 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/generic-vnf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.403 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.403 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l-interface-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.403 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/az-and-dvs-switches-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.403 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/az-and-dvs-switches-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.404 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/virtual-data-center-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.404 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/virtual-data-center-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.404 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-interface-ipv6-address-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.404 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-interface-ipv6-address-list-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.405 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/secondary-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.405 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/secondary-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.405 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vlan-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.405 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vlan-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.406 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/site-pair-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.406 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/site-pair-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.406 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/edge-prop-names-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.407 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/edge-prop-names-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.407 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/metadatum-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.407 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/metadatum-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.407 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/availability-zone-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.408 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/availability-zone-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.408 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/dvs-switch-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.408 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/dvs-switch-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.409 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/action-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.409 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/action-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.409 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/action-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.410 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/action-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.410 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.410 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.411 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/named-query-element-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.411 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/named-query-element-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.411 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/subnet-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.411 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/subnet-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.412 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/include-node-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.412 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/include-node-filter-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.412 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vnfc-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.413 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vnfc-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.413 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/flavor-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.413 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/flavor-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.414 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/volume-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.414 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/volume-group-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.414 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/complex-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.414 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/complex-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.414 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ctag-pool-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.415 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/ctag-pool-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.415 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/element-choice-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.415 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/element-choice-set-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.416 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-network-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.416 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/l3-network-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.416 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpls-pe-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.416 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/vpls-pe-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.416 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-subscription-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/service-subscription-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/inventory-item-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/inventory-item-data-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/sriov-vf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/sriov-vf-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tunnel-xconnect-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/tunnel-xconnect-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.417 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-ver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.418 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/widget-model-json/model-ver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.418 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/service-model-json/000-README in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.418 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/service-model-json/000-README in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.418 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/resource-model-json/000-README in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.418 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/resource-model-json/000-README in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getServiceInstanceSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getServiceInstanceSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology-1.1.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology-1.1.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology2-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology2-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getVserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.419 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getVserver-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/closed-loop-named-query-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/closed-loop-named-query-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getTenantInfoAtSvcInstance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getTenantInfoAtSvcInstance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.420 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVServiceTopology-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getTenantInfoAtSvcSubscription-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getTenantInfoAtSvcSubscription-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.2.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.2.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getServiceInstanceModel-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getServiceInstanceModel-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.421 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getWlBundleId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getWlBundleId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVLogicalLinkByCircuitId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVLogicalLinkByCircuitId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVComplexLocationId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVComplexLocationId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getRouterRoadmTailSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.422 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getRouterRoadmTailSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/vnf-to-service-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/vnf-to-service-instance-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getLogicalLinkByCloudRegionId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getLogicalLinkByCloudRegionId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.1.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.1.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getSvcSubscriberModelInfo-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.423 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getSvcSubscriberModelInfo-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getL3networkCloudRegionByNetworkRole-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getL3networkCloudRegionByNetworkRole-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getClfiRoadmTailSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getClfiRoadmTailSummary-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getVnfVlanByCircuitId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.424 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getVnfVlanByCircuitId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVComplexHostname-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVComplexHostname-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getPinterfacePhysicalLinkBySvcInstId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getPinterfacePhysicalLinkBySvcInstId-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVLogicalLink-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getDHVLogicalLink-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/vnf-to-esr-system-info-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.425 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/vnf-to-esr-system-info-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComponentList-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComplexByPnfName-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/named-query-json/getComplexByPnfName-1.0.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/auth/aai_keystore in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/auth/aai_keystore in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.426 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/auth/realm.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/auth/realm.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/logging.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/logging.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiEventDMaaPPublisher.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiEventDMaaPPublisher.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-realtime.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.427 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-realtime.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-cached.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-cached.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/error.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/error.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiconfig.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiconfig.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.428 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/query/stored-queries.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.429 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/query/stored-queries.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.429 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v9/aai_oxm_v9.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.429 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v9/aai_oxm_v9.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.429 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v14/aai_oxm_v14.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.429 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v14/aai_oxm_v14.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v19/aai_oxm_v19.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v19/aai_oxm_v19.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v15/aai_oxm_v15.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v15/aai_oxm_v15.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v21/aai_oxm_v21.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.430 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v21/aai_oxm_v21.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v17/aai_oxm_v17.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v17/aai_oxm_v17.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v13/aai_oxm_v13.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v13/aai_oxm_v13.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v8/aai_oxm_v8.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.431 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v8/aai_oxm_v8.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.432 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v18/aai_oxm_v18.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.432 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v18/aai_oxm_v18.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.432 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v11/aai_oxm_v11.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.432 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v11/aai_oxm_v11.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.432 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v22/aai_oxm_v22.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.433 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v22/aai_oxm_v22.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.433 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v23/aai_oxm_v23.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.433 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v23/aai_oxm_v23.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.433 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v16/aai_oxm_v16.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.433 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v16/aai_oxm_v16.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.434 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v20/aai_oxm_v20.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.434 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v20/aai_oxm_v20.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.434 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v10/aai_oxm_v10.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.434 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v10/aai_oxm_v10.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v12/aai_oxm_v12.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v12/aai_oxm_v12.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v9/DbEdgeRules_v9.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v9/DbEdgeRules_v9.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_esr_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.435 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_esr_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_hpa_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_hpa_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_ccvpn_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_ccvpn_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.436 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_pnp_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_pnp_v14.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_esr_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_esr_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_hpa_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_hpa_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.437 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_ccvpn_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_ccvpn_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_bbs_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_bbs_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_pnp_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_pnp_v19.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.438 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_hpa_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_hpa_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_pnp_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_pnp_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_ccvpn_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_ccvpn_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.439 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_esr_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_esr_v15.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_esr_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_esr_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_hpa_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.440 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_hpa_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_pnp_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_pnp_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_ccvpn_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_ccvpn_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_bbs_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.441 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_bbs_v21.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_hpa_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_hpa_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_ccvpn_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_ccvpn_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_bbs_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_bbs_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_pnp_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.442 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_pnp_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_esr_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_esr_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_v17.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_esr_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_esr_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.443 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_hpa_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_hpa_v13.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v8/DbEdgeRules_v8.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v8/DbEdgeRules_v8.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_hpa_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.444 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_hpa_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_ccvpn_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_ccvpn_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_pnp_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_pnp_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_bbs_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_bbs_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_esr_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_esr_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.445 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_v18.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_esr_v11.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_esr_v11.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_v11.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_v11.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_pnp_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.446 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_pnp_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_ccvpn_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_ccvpn_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_bbs_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_bbs_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_hpa_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.447 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_hpa_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_esr_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_esr_v22.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_ccvpn_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_ccvpn_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_hpa_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_hpa_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.448 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_bbs_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_bbs_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_pnp_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_pnp_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_esr_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_esr_v23.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_pnp_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.449 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_pnp_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_ccvpn_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_ccvpn_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_hpa_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_hpa_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.450 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_esr_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_esr_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_bbs_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_bbs_v16.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_bbs_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_bbs_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_ccvpn_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_ccvpn_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.451 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_hpa_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_hpa_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_pnp_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_pnp_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_esr_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_esr_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_v20.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.452 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v10/DbEdgeRules_v10.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v10/DbEdgeRules_v10.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_hpa_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_hpa_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_esr_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.453 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_esr_v12.json in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.osaaf.location.props in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.osaaf.location.props in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/bath_config.csv in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/bath_config.csv in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/permissions.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/permissions.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.454 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.osaaf.location.props in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.osaaf.location.props in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.keyfile in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.keyfile in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.p12 in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.p12 in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.props in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/org.onap.aai.props in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.455 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/truststoreONAPall.jks in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/onap/truststoreONAPall.jks in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.keyfile in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.keyfile in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.p12 in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.p12 in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.props in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/org.onap.aai.props in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/truststoreONAPall.jks in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file aaf/truststoreONAPall.jks in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.456 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file localhost-access-logback.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.457 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file localhost-access-logback.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.457 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.457 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.457 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/dsl/v1/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/dsl/v1/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/dsl/v2/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/dsl/v2/AAIDsl.g4 in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file logback.xml in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file logback.xml in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file retired.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file retired.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file cadi.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.458 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file cadi.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.459 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file application.properties in url file:/opt/app/aai-traversal/resources/ with scanner TypeAnnotationsScanner
2021-05-21 11:02:00.459 DEBUG 11 --- [ main] org.reflections.Reflections : could not scan file application.properties in url file:/opt/app/aai-traversal/resources/ with scanner SubTypesScanner
2021-05-21 11:02:00.460 INFO 11 --- [ main] org.reflections.Reflections : Reflections took 101 ms to scan 1 urls, producing 0 keys and 0 values
2021-05-21 11:02:00.591 DEBUG 11 --- [ main] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 11:02:00.592 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-1, inFlight=0, closed=true] closed, remaining = 0
2021-05-21 11:02:00.594 DEBUG 11 --- [ main] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 11:02:00.594 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=true] closed, remaining = 1
2021-05-21 11:02:00.595 DEBUG 11 --- [ main] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 11:02:00.595 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-1, inFlight=0, closed=true] closed, remaining = 0
2021-05-21 11:02:00.595 DEBUG 11 --- [ main] com.datastax.driver.core.Cluster : Shutting down
2021-05-21 11:02:00.596 DEBUG 11 --- [ main] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 11:02:00.596 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=true] closed, remaining = 0
2021-05-21 11:02:02.611 DEBUG 11 --- [er-nio-worker-0] io.netty.buffer.PoolThreadCache : Freed 35 thread-local buffer(s) from thread: JanusGraph Cluster-nio-worker-0
2021-05-21 11:02:02.611 DEBUG 11 --- [er-nio-worker-1] io.netty.buffer.PoolThreadCache : Freed 21 thread-local buffer(s) from thread: JanusGraph Cluster-nio-worker-1
2021-05-21 11:02:02.629 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true
2021-05-21 11:02:02.629 INFO 11 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps.
2021-05-21 11:02:02.629 DEBUG 11 --- [ main] com.datastax.driver.core.Cluster : Starting new cluster with contact points [cassandra/10.242.184.18:9042]
2021-05-21 11:02:02.632 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:02:02.635 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:02:02.749 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:02.750 DEBUG 11 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:02:02.780 DEBUG 11 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 11:02:02.826 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.184.18:9042
2021-05-21 11:02:02.826 INFO 11 --- [ main] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'Pod' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2021-05-21 11:02:02.827 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.131.197:9042 added
2021-05-21 11:02:02.827 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host cassandra/10.242.184.18:9042 added
2021-05-21 11:02:02.827 INFO 11 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.129.86:9042 added
2021-05-21 11:02:02.827 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:02:02.828 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:02:02.828 DEBUG 11 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:02:02.829 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:02:02.831 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:02:02.832 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:02:03.010 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:03.011 DEBUG 11 --- [er-nio-worker-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.131.197:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:03.011 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Session : Added connection pool for /10.242.131.197:9042
2021-05-21 11:02:03.015 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:03.016 DEBUG 11 --- [er-nio-worker-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:03.016 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:02:03.475 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:02:03.475 DEBUG 11 --- [er-nio-worker-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.129.86:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:02:03.475 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Session : Added connection pool for /10.242.129.86:9042
2021-05-21 11:02:04.600 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Processing the translator
2021-05-21 11:02:04.601 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv11
2021-05-21 11:02:04.602 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.602 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v11, BODY=
2021-05-21 11:02:04.604 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.604 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.604 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.604 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.618 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.619 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.620 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.620 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv12
2021-05-21 11:02:04.620 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.621 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v12, BODY=
2021-05-21 11:02:04.622 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.622 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.622 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.622 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.631 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.634 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.635 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.635 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv13
2021-05-21 11:02:04.635 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.636 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v13, BODY=
2021-05-21 11:02:04.638 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.638 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.638 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.639 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.649 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.652 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.653 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.653 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv14
2021-05-21 11:02:04.654 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.655 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v14, BODY=
2021-05-21 11:02:04.657 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.657 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.658 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.658 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.667 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.669 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.670 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.670 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv15
2021-05-21 11:02:04.670 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.671 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v15, BODY=
2021-05-21 11:02:04.672 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.672 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.672 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.672 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.680 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.682 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.683 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.683 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv16
2021-05-21 11:02:04.683 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.684 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v16, BODY=
2021-05-21 11:02:04.684 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.685 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.685 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.685 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.693 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.695 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.696 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.696 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv17
2021-05-21 11:02:04.696 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.697 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v17, BODY=
2021-05-21 11:02:04.698 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.698 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.698 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.698 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.706 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.710 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.711 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.711 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv18
2021-05-21 11:02:04.711 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.711 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v18, BODY=
2021-05-21 11:02:04.712 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.713 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.713 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.713 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.721 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.723 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.723 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.724 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv19
2021-05-21 11:02:04.724 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.724 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v19, BODY=
2021-05-21 11:02:04.726 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.726 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.726 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.726 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.734 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.737 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.737 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.737 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv20
2021-05-21 11:02:04.737 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.738 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v20, BODY=
2021-05-21 11:02:04.739 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.739 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.739 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.739 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.748 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.753 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.753 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.754 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv21
2021-05-21 11:02:04.754 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.755 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v21, BODY=
2021-05-21 11:02:04.756 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.756 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.756 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.756 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.768 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.775 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.776 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.776 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv22
2021-05-21 11:02:04.777 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.777 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v22, BODY=
2021-05-21 11:02:04.779 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.779 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.779 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.779 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.791 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.800 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.800 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.801 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv23
2021-05-21 11:02:04.801 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.802 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v23, BODY=
2021-05-21 11:02:04.803 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.803 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.803 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.804 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.819 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:04.826 DEBUG 11 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK
2021-05-21 11:02:04.827 DEBUG 11 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService
2021-05-21 11:02:04.942 INFO 11 --- [ main] o.o.a.rest.search.SchemaServiceCQConfig : Calling the SchemaService to retrieve stored queries
2021-05-21 11:02:04.944 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {}
2021-05-21 11:02:04.944 DEBUG 11 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/stored-queries, BODY=
2021-05-21 11:02:04.946 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName]
2021-05-21 11:02:04.946 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName
2021-05-21 11:02:04.946 INFO 11 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty
2021-05-21 11:02:04.946 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke
2021-05-21 11:02:04.957 INFO 11 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn
2021-05-21 11:02:05.518 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Found javax.persistence.Persistence on classpath containing 'getPersistenceUtil'. Assuming JPA 2 environment. Trying to instantiate JPA aware TraversableResolver
2021-05-21 11:02:05.519 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Instantiated JPA aware TraversableResolver of type org.hibernate.validator.internal.engine.resolver.JPATraversableResolver.
2021-05-21 11:02:05.519 DEBUG 11 --- [ main] .h.v.m.ResourceBundleMessageInterpolator : Loaded expression factory via original TCCL
2021-05-21 11:02:05.521 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Found javax.persistence.Persistence on classpath containing 'getPersistenceUtil'. Assuming JPA 2 environment. Trying to instantiate JPA aware TraversableResolver
2021-05-21 11:02:05.521 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Instantiated JPA aware TraversableResolver of type org.hibernate.validator.internal.engine.resolver.JPATraversableResolver.
2021-05-21 11:02:05.523 DEBUG 11 --- [ main] o.h.v.internal.engine.ConfigurationImpl : Setting custom MessageInterpolator of type org.springframework.validation.beanvalidation.LocaleContextMessageInterpolator
2021-05-21 11:02:05.523 DEBUG 11 --- [ main] o.h.v.internal.engine.ConfigurationImpl : Setting custom ConstraintValidatorFactory of type org.springframework.validation.beanvalidation.SpringConstraintValidatorFactory
2021-05-21 11:02:05.524 DEBUG 11 --- [ main] o.h.v.internal.engine.ConfigurationImpl : Setting custom ParameterNameProvider of type org.springframework.validation.beanvalidation.LocalValidatorFactoryBean$1
2021-05-21 11:02:05.527 DEBUG 11 --- [ main] o.h.v.i.xml.config.ValidationXmlParser : Trying to load META-INF/validation.xml for XML based Validator configuration.
2021-05-21 11:02:05.527 DEBUG 11 --- [ main] o.h.v.i.xml.config.ResourceLoaderHelper : Trying to load META-INF/validation.xml via user class loader
2021-05-21 11:02:05.528 DEBUG 11 --- [ main] o.h.v.i.xml.config.ResourceLoaderHelper : Trying to load META-INF/validation.xml via TCCL
2021-05-21 11:02:05.528 DEBUG 11 --- [ main] o.h.v.i.xml.config.ResourceLoaderHelper : Trying to load META-INF/validation.xml via Hibernate Validator's class loader
2021-05-21 11:02:05.529 DEBUG 11 --- [ main] o.h.v.i.xml.config.ValidationXmlParser : No META-INF/validation.xml found. Using annotation based configuration only.
2021-05-21 11:02:05.543 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.springframework.validation.beanvalidation.LocaleContextMessageInterpolator as ValidatorFactory-scoped message interpolator.
2021-05-21 11:02:05.543 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.resolver.JPATraversableResolver as ValidatorFactory-scoped traversable resolver.
2021-05-21 11:02:05.543 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.util.ExecutableParameterNameProvider as ValidatorFactory-scoped parameter name provider.
2021-05-21 11:02:05.543 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.DefaultClockProvider as ValidatorFactory-scoped clock provider.
2021-05-21 11:02:05.543 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.scripting.DefaultScriptEvaluatorFactory as ValidatorFactory-scoped script evaluator factory.
2021-05-21 11:02:06.238 DEBUG 11 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true
2021-05-21 11:02:06.238 INFO 11 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps.
2021-05-21 11:02:06.248 WARN 11 --- [ main] .b.a.g.t.GroovyTemplateAutoConfiguration : Cannot find template location: classpath:/templates/ (please add some templates, check your Groovy configuration, or set spring.groovy.template.check-template-location=false)
2021-05-21 11:02:08.509 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Found javax.persistence.Persistence on classpath containing 'getPersistenceUtil'. Assuming JPA 2 environment. Trying to instantiate JPA aware TraversableResolver
2021-05-21 11:02:08.515 DEBUG 11 --- [ main] o.h.v.i.e.resolver.TraversableResolvers : Instantiated JPA aware TraversableResolver of type org.hibernate.validator.internal.engine.resolver.JPATraversableResolver.
2021-05-21 11:02:08.517 DEBUG 11 --- [ main] o.h.v.i.xml.config.ValidationXmlParser : Trying to load META-INF/validation.xml for XML based Validator configuration.
2021-05-21 11:02:08.517 DEBUG 11 --- [ main] o.h.v.i.xml.config.ResourceLoaderHelper : Trying to load META-INF/validation.xml via TCCL
2021-05-21 11:02:08.518 DEBUG 11 --- [ main] o.h.v.i.xml.config.ResourceLoaderHelper : Trying to load META-INF/validation.xml via Hibernate Validator's class loader
2021-05-21 11:02:08.518 DEBUG 11 --- [ main] o.h.v.i.xml.config.ValidationXmlParser : No META-INF/validation.xml found. Using annotation based configuration only.
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] .h.v.m.ResourceBundleMessageInterpolator : Loaded expression factory via original TCCL
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.messageinterpolation.ResourceBundleMessageInterpolator as ValidatorFactory-scoped message interpolator.
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.resolver.JPATraversableResolver as ValidatorFactory-scoped traversable resolver.
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.util.ExecutableParameterNameProvider as ValidatorFactory-scoped parameter name provider.
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.DefaultClockProvider as ValidatorFactory-scoped clock provider.
2021-05-21 11:02:08.532 DEBUG 11 --- [ main] o.h.v.i.engine.ValidatorFactoryImpl : HV000234: Using org.hibernate.validator.internal.engine.scripting.DefaultScriptEvaluatorFactory as ValidatorFactory-scoped script evaluator factory.
2021-05-21 11:02:08.686 WARN 11 --- [ main] org.glassfish.jersey.internal.Errors : The following warnings have been detected: WARNING: The (sub)resource method getC2Qgremlin in org.onap.aai.rest.CQ2Gremlin contains empty path annotation.
WARNING: The (sub)resource method getC2Qgremlin in org.onap.aai.rest.CQ2GremlinTest contains empty path annotation.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.onap.aai.rest.RecentAPIConsumer.getRecentData(java.lang.String,java.lang.String,java.lang.String,javax.ws.rs.core.HttpHeaders,javax.servlet.http.HttpServletRequest,javax.ws.rs.core.UriInfo), should not consume any entity.
2021-05-21 11:02:08.695 DEBUG 11 --- [ main] org.onap.aai.config.ErrorHandler : Filter 'errorHandler' configured for use
2021-05-21 11:02:09.637 INFO 11 --- [ main] org.onap.aai.TraversalApp : Application 'aai-traversal' is running on 8446!
2021-05-21 11:02:09.637 DEBUG 11 --- [ main] org.onap.aai.TraversalApp : Traversal MicroService Started
Traversal Microservice Started
2021-05-21 11:02:32.797 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:02:32.800 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:02:44.616 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:02:44.625 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:02:53.249 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE sdcartifact, scheduling delivery
2021-05-21 11:02:54.264 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcartifact
2021-05-21 11:02:54.274 DEBUG 11 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdcartifact.
2021-05-21 11:02:54.285 DEBUG 11 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdcartifact completed in 11 milliseconds
2021-05-21 11:02:54.627 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:02:54.630 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:02:55.077 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcartifact.resources, scheduling delivery
2021-05-21 11:02:56.079 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcartifact.resources (TABLE)
2021-05-21 11:02:59.987 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcartifact.sdcschemafiles, scheduling delivery
2021-05-21 11:03:00.988 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcartifact.sdcschemafiles (TABLE)
2021-05-21 11:03:01.689 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE sdcrepository, scheduling delivery
2021-05-21 11:03:02.690 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcrepository
2021-05-21 11:03:02.781 DEBUG 11 --- [luster-worker-3] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdcartifact.
2021-05-21 11:03:02.797 DEBUG 11 --- [luster-worker-3] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdcartifact completed in 15 milliseconds
2021-05-21 11:03:04.485 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcrepository.migrationtasks, scheduling delivery
2021-05-21 11:03:05.486 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcrepository.migrationtasks (TABLE)
2021-05-21 11:03:06.795 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcrepository.operationalenvironment, scheduling delivery
2021-05-21 11:03:07.795 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcrepository.operationalenvironment (TABLE)
2021-05-21 11:03:09.882 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcrepository.operationalenvironment, scheduling delivery
2021-05-21 11:03:10.883 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcrepository.operationalenvironment (TABLE)
2021-05-21 11:03:12.283 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcrepository.featuretogglestate, scheduling delivery
2021-05-21 11:03:13.283 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcrepository.featuretogglestate (TABLE)
2021-05-21 11:03:13.887 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE sdcaudit, scheduling delivery
2021-05-21 11:03:14.888 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit
2021-05-21 11:03:14.978 DEBUG 11 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdcartifact.
2021-05-21 11:03:14.992 DEBUG 11 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdcartifact completed in 14 milliseconds
2021-05-21 11:03:16.584 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.useradminevent, scheduling delivery
2021-05-21 11:03:17.585 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.useradminevent (TABLE)
2021-05-21 11:03:19.188 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.useradminevent, scheduling delivery
2021-05-21 11:03:20.189 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.useradminevent (TABLE)
2021-05-21 11:03:22.884 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.useraccessevent, scheduling delivery
2021-05-21 11:03:23.885 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.useraccessevent (TABLE)
2021-05-21 11:03:25.277 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.useraccessevent, scheduling delivery
2021-05-21 11:03:26.278 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.useraccessevent (TABLE)
2021-05-21 11:03:27.395 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:28.396 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:30.093 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:31.094 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:32.391 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:33.392 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:34.289 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:35.289 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:36.585 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:37.586 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:38.788 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:39.789 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:40.897 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:41.898 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:44.988 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.resourceadminevent, scheduling delivery
2021-05-21 11:03:45.279 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:03:45.280 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:03:45.988 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.resourceadminevent (TABLE)
2021-05-21 11:03:47.102 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.distributiondownloadevent, scheduling delivery
2021-05-21 11:03:48.103 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondownloadevent (TABLE)
2021-05-21 11:03:48.485 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributiondownloadevent, scheduling delivery
2021-05-21 11:03:49.485 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondownloadevent (TABLE)
2021-05-21 11:03:53.380 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.distributionengineevent, scheduling delivery
2021-05-21 11:03:54.381 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionengineevent (TABLE)
2021-05-21 11:03:55.185 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionengineevent, scheduling delivery
2021-05-21 11:03:56.186 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionengineevent (TABLE)
2021-05-21 11:03:56.685 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionengineevent, scheduling delivery
2021-05-21 11:03:57.685 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionengineevent (TABLE)
2021-05-21 11:04:00.340 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.distributionnotificationevent, scheduling delivery
2021-05-21 11:04:01.343 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionnotificationevent (TABLE)
2021-05-21 11:04:01.878 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionnotificationevent, scheduling delivery
2021-05-21 11:04:02.879 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionnotificationevent (TABLE)
2021-05-21 11:04:04.691 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionnotificationevent, scheduling delivery
2021-05-21 11:04:05.692 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionnotificationevent (TABLE)
2021-05-21 11:04:08.778 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionnotificationevent, scheduling delivery
2021-05-21 11:04:09.778 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionnotificationevent (TABLE)
2021-05-21 11:04:11.081 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.distributionstatusevent, scheduling delivery
2021-05-21 11:04:12.082 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionstatusevent (TABLE)
2021-05-21 11:04:12.497 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionstatusevent, scheduling delivery
2021-05-21 11:04:13.498 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionstatusevent (TABLE)
2021-05-21 11:04:14.194 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributionstatusevent, scheduling delivery
2021-05-21 11:04:15.195 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributionstatusevent (TABLE)
2021-05-21 11:04:16.677 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.distributiondeployevent, scheduling delivery
2021-05-21 11:04:17.677 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondeployevent (TABLE)
2021-05-21 11:04:18.976 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributiondeployevent, scheduling delivery
2021-05-21 11:04:19.977 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondeployevent (TABLE)
2021-05-21 11:04:21.306 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributiondeployevent, scheduling delivery
2021-05-21 11:04:22.307 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondeployevent (TABLE)
2021-05-21 11:04:23.194 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributiondeployevent, scheduling delivery
2021-05-21 11:04:24.194 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondeployevent (TABLE)
2021-05-21 11:04:25.589 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:04:25.591 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:04:30.585 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.distributiondeployevent, scheduling delivery
2021-05-21 11:04:31.586 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.distributiondeployevent (TABLE)
2021-05-21 11:04:33.812 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.auditinggetuebclusterevent, scheduling delivery
2021-05-21 11:04:34.414 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.auditinggetuebclusterevent, scheduling delivery
2021-05-21 11:04:35.415 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.auditinggetuebclusterevent (TABLE)
2021-05-21 11:04:41.092 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.authevent, scheduling delivery
2021-05-21 11:04:42.093 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.authevent (TABLE)
2021-05-21 11:04:44.193 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.authevent, scheduling delivery
2021-05-21 11:04:45.194 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.authevent (TABLE)
2021-05-21 11:04:52.193 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.authevent, scheduling delivery
2021-05-21 11:04:53.194 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.authevent (TABLE)
2021-05-21 11:04:55.189 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.categoryevent, scheduling delivery
2021-05-21 11:04:56.191 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.categoryevent (TABLE)
2021-05-21 11:04:58.289 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.categoryevent, scheduling delivery
2021-05-21 11:04:59.290 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.categoryevent (TABLE)
2021-05-21 11:05:00.579 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.getuserslistevent, scheduling delivery
2021-05-21 11:05:01.580 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.getuserslistevent (TABLE)
2021-05-21 11:05:04.995 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.getuserslistevent, scheduling delivery
2021-05-21 11:05:05.995 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.getuserslistevent (TABLE)
2021-05-21 11:05:07.077 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.getcategoryhierarchyevent, scheduling delivery
2021-05-21 11:05:08.077 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.getcategoryhierarchyevent (TABLE)
2021-05-21 11:05:09.981 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.getcategoryhierarchyevent, scheduling delivery
2021-05-21 11:05:10.982 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.getcategoryhierarchyevent (TABLE)
2021-05-21 11:05:11.167 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:05:11.169 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:05:12.393 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.externalapievent, scheduling delivery
2021-05-21 11:05:13.393 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.externalapievent (TABLE)
2021-05-21 11:05:14.696 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.externalapievent, scheduling delivery
2021-05-21 11:05:15.697 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.externalapievent (TABLE)
2021-05-21 11:05:16.190 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.externalapievent, scheduling delivery
2021-05-21 11:05:17.191 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.externalapievent (TABLE)
2021-05-21 11:05:20.284 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE sdcaudit.externalapievent, scheduling delivery
2021-05-21 11:05:21.284 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.externalapievent (TABLE)
2021-05-21 11:05:22.581 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdcaudit.ecompopenvironmentevent, scheduling delivery
2021-05-21 11:05:23.582 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdcaudit.ecompopenvironmentevent (TABLE)
2021-05-21 11:05:33.998 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE sdctitan, scheduling delivery
2021-05-21 11:05:34.999 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan
2021-05-21 11:05:35.080 DEBUG 11 --- [luster-worker-3] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:05:35.087 DEBUG 11 --- [luster-worker-3] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 7 milliseconds
2021-05-21 11:05:37.881 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.system_properties, scheduling delivery
2021-05-21 11:05:38.882 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.system_properties (TABLE)
2021-05-21 11:05:44.697 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.janusgraph_ids, scheduling delivery
2021-05-21 11:05:45.697 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.janusgraph_ids (TABLE)
2021-05-21 11:05:46.483 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:05:46.485 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:05:46.589 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.edgestore, scheduling delivery
2021-05-21 11:05:47.590 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.edgestore (TABLE)
2021-05-21 11:05:49.689 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.edgestore_lock_, scheduling delivery
2021-05-21 11:05:50.690 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.edgestore_lock_ (TABLE)
2021-05-21 11:05:52.289 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.graphindex, scheduling delivery
2021-05-21 11:05:53.290 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.graphindex (TABLE)
2021-05-21 11:05:55.343 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.graphindex_lock_, scheduling delivery
2021-05-21 11:05:56.344 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.graphindex_lock_ (TABLE)
2021-05-21 11:05:57.020 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.txlog, scheduling delivery
2021-05-21 11:05:58.020 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.txlog (TABLE)
2021-05-21 11:06:00.780 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.systemlog, scheduling delivery
2021-05-21 11:06:01.781 DEBUG 11 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.systemlog (TABLE)
2021-05-21 11:06:04.077 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE sdctitan.system_properties_lock_, scheduling delivery
2021-05-21 11:06:05.078 DEBUG 11 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for sdctitan.system_properties_lock_ (TABLE)
2021-05-21 11:06:35.085 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:06:35.087 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:06:42.213 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:06:42.215 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:05.088 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:05.090 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:09.741 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1219056031-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@611235ec at consistency LOCAL_QUORUM
2021-05-21 11:07:09.742 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1219056031-0] Error querying /10.242.129.86:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.129.86:9042] Timed out waiting for server response
2021-05-21 11:07:12.479 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:14.265 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] Response received on stream 5248 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 3 responses.
2021-05-21 11:07:14.286 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:17.567 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:17.580 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:26.836 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [803881372-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6fd5d7be at consistency LOCAL_QUORUM
2021-05-21 11:07:26.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [803881372-0] Error querying /10.242.129.86:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.129.86:9042] Timed out waiting for server response
2021-05-21 11:07:35.096 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:35.098 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:38.654 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=1, closed=false] Response received on stream 5376 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 11:07:47.581 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:47.583 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:07:56.894 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:07:56.901 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:08:05.109 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:08:05.177 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:08:33.113 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:08:33.116 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:08:35.178 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:08:35.180 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:05.181 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:09:05.185 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:07.882 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:09:07.885 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:35.186 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:09:35.192 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:48.321 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:09:48.324 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:55.984 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE workflow, scheduling delivery
2021-05-21 11:09:56.986 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for workflow
2021-05-21 11:09:56.994 DEBUG 11 --- [luster-worker-4] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:09:57.003 DEBUG 11 --- [luster-worker-4] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 9 milliseconds
2021-05-21 11:09:57.483 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE zusammen_workflow, scheduling delivery
2021-05-21 11:09:58.416 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:09:58.417 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:09:58.484 DEBUG 11 --- [luster-worker-5] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow
2021-05-21 11:09:58.492 DEBUG 11 --- [luster-worker-5] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:09:58.500 DEBUG 11 --- [luster-worker-5] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 8 milliseconds
2021-05-21 11:10:15.518 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE workflow.unique_value, scheduling delivery
2021-05-21 11:10:16.519 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for workflow.unique_value (TABLE)
2021-05-21 11:10:17.595 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.item, scheduling delivery
2021-05-21 11:10:18.603 DEBUG 11 --- [luster-worker-5] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.item (TABLE)
2021-05-21 11:10:21.709 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version, scheduling delivery
2021-05-21 11:10:22.710 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version (TABLE)
2021-05-21 11:10:30.178 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version_elements, scheduling delivery
2021-05-21 11:10:31.179 DEBUG 11 --- [luster-worker-5] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version_elements (TABLE)
2021-05-21 11:10:35.254 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_namespace, scheduling delivery
2021-05-21 11:10:36.255 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element_namespace (TABLE)
2021-05-21 11:10:54.495 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element, scheduling delivery
2021-05-21 11:10:55.495 DEBUG 11 --- [luster-worker-5] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element (TABLE)
2021-05-21 11:10:59.384 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:10:59.477 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:11:03.288 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE dox, scheduling delivery
2021-05-21 11:11:04.289 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox
2021-05-21 11:11:04.294 DEBUG 11 --- [luster-worker-4] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:11:04.302 DEBUG 11 --- [luster-worker-4] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 8 milliseconds
2021-05-21 11:11:13.386 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE zusammen_dox, scheduling delivery
2021-05-21 11:11:14.389 DEBUG 11 --- [luster-worker-5] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox
2021-05-21 11:11:14.392 DEBUG 11 --- [luster-worker-5] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:11:14.409 DEBUG 11 --- [luster-worker-5] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 16 milliseconds
2021-05-21 11:11:21.107 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:11:21.115 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:11:21.387 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_synchronization_state, scheduling delivery
2021-05-21 11:11:22.388 DEBUG 11 --- [luster-worker-4] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element_synchronization_state (TABLE)
2021-05-21 11:11:52.396 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:11:52.477 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:11:53.285 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_stage, scheduling delivery
2021-05-21 11:11:54.286 DEBUG 11 --- [luster-worker-6] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element_stage (TABLE)
2021-05-21 11:12:10.881 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version_stage, scheduling delivery
2021-05-21 11:12:11.882 DEBUG 11 --- [luster-worker-7] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version_stage (TABLE)
2021-05-21 11:12:39.229 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:12:39.302 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:12:41.890 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:12:41.893 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:13:11.905 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:13:11.907 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:13:23.910 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:13:23.919 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:13:38.966 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:13:38.969 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:13:41.908 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:13:41.983 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:13:53.481 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TYPE dox.version, scheduling delivery
2021-05-21 11:13:54.484 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.version (TYPE)
2021-05-21 11:13:57.079 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TYPE dox.user_candidate_version, scheduling delivery
2021-05-21 11:13:58.080 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.user_candidate_version (TYPE)
2021-05-21 11:13:59.304 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.version_info, scheduling delivery
2021-05-21 11:14:00.305 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.version_info (TABLE)
2021-05-21 11:14:00.659 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.version_info_deleted, scheduling delivery
2021-05-21 11:14:01.660 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.version_info_deleted (TABLE)
2021-05-21 11:14:04.710 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.unique_value, scheduling delivery
2021-05-21 11:14:05.711 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.unique_value (TABLE)
2021-05-21 11:14:06.583 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.package_details, scheduling delivery
2021-05-21 11:14:07.583 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.package_details (TABLE)
2021-05-21 11:14:08.131 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.vsp_enriched_service_template, scheduling delivery
2021-05-21 11:14:09.132 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.vsp_enriched_service_template (TABLE)
2021-05-21 11:14:10.324 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.vsp_enriched_service_artifact, scheduling delivery
2021-05-21 11:14:11.324 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.vsp_enriched_service_artifact (TABLE)
2021-05-21 11:14:12.891 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.application_config, scheduling delivery
2021-05-21 11:14:13.891 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.application_config (TABLE)
2021-05-21 11:14:15.696 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:16.697 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:17.789 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:18.789 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:19.505 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:14:19.507 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:14:20.272 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:21.272 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:22.459 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:23.459 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:26.484 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:27.485 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:29.776 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:30.777 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.action (TABLE)
2021-05-21 11:14:31.482 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.ecompcomponent, scheduling delivery
2021-05-21 11:14:32.483 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.ecompcomponent (TABLE)
2021-05-21 11:14:33.310 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UPDATED TABLE dox.action, scheduling delivery
2021-05-21 11:14:34.198 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.action_artifact, scheduling delivery
2021-05-21 11:14:35.199 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox
2021-05-21 11:14:35.216 DEBUG 11 --- [luster-worker-8] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:14:35.225 DEBUG 11 --- [luster-worker-8] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 9 milliseconds
2021-05-21 11:14:35.701 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.activity_log, scheduling delivery
2021-05-21 11:14:36.702 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.activity_log (TABLE)
2021-05-21 11:14:41.089 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.healing, scheduling delivery
2021-05-21 11:14:42.090 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.healing (TABLE)
2021-05-21 11:14:42.981 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.migration, scheduling delivery
2021-05-21 11:14:43.982 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.migration (TABLE)
2021-05-21 11:14:45.178 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.item_permissions, scheduling delivery
2021-05-21 11:14:46.180 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.item_permissions (TABLE)
2021-05-21 11:14:47.542 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.user_permission_items, scheduling delivery
2021-05-21 11:14:48.543 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.user_permission_items (TABLE)
2021-05-21 11:14:49.807 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:14:49.810 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:14:50.796 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.notification_subscribers, scheduling delivery
2021-05-21 11:14:51.796 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.notification_subscribers (TABLE)
2021-05-21 11:14:54.182 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.last_notification, scheduling delivery
2021-05-21 11:14:55.182 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.last_notification (TABLE)
2021-05-21 11:14:56.184 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.notifications, scheduling delivery
2021-05-21 11:14:57.185 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.notifications (TABLE)
2021-05-21 11:14:58.204 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE dox.vsp_merge_hint, scheduling delivery
2021-05-21 11:14:59.205 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for dox.vsp_merge_hint (TABLE)
2021-05-21 11:15:02.181 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.item, scheduling delivery
2021-05-21 11:15:03.181 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.item (TABLE)
2021-05-21 11:15:04.811 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.version, scheduling delivery
2021-05-21 11:15:05.813 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.version (TABLE)
2021-05-21 11:15:10.992 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.version_elements, scheduling delivery
2021-05-21 11:15:11.994 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.version_elements (TABLE)
2021-05-21 11:15:13.795 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.element_namespace, scheduling delivery
2021-05-21 11:15:14.796 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.element_namespace (TABLE)
2021-05-21 11:15:18.394 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.element, scheduling delivery
2021-05-21 11:15:19.394 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.element (TABLE)
2021-05-21 11:15:21.084 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.element_synchronization_state, scheduling delivery
2021-05-21 11:15:22.084 DEBUG 11 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.element_synchronization_state (TABLE)
2021-05-21 11:15:25.908 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.element_stage, scheduling delivery
2021-05-21 11:15:26.909 DEBUG 11 --- [luster-worker-8] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.element_stage (TABLE)
2021-05-21 11:15:45.702 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:15:46.862 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:15:52.683 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.version_stage, scheduling delivery
2021-05-21 11:15:53.684 DEBUG 11 --- [uster-worker-10] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.version_stage (TABLE)
2021-05-21 11:16:23.689 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:16:23.691 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:16:36.912 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:16:36.915 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:16:53.702 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:16:53.704 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:17:14.274 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:17:14.277 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:17:23.706 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:17:23.777 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:17:24.469 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:17:24.470 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:17:53.777 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:17:53.780 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:18:23.781 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:18:23.783 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:18:30.405 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:18:30.407 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:18:53.784 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:18:53.791 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:18:56.605 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:18:57.447 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 11:19:06.807 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:19:06.810 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:19:23.792 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:19:23.798 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:19:53.798 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:19:53.800 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:19:53.881 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:20:02.675 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:20:03.909 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:20:03.911 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:20:23.804 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:20:23.877 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:20:32.681 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:20:36.545 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:20:53.885 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:20:53.887 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:21:19.604 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:21:19.606 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:21:23.893 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:21:23.977 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:21:39.934 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:21:39.937 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:21:53.978 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:21:53.984 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:22:23.985 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:22:23.991 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:22:34.485 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:22:34.501 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:22:49.714 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:22:49.716 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:22:53.992 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:22:53.994 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:23:23.997 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:23:23.999 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:23:32.210 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:23:32.221 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:23:54.000 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:23:54.002 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:24:24.003 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:24:24.006 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:24:34.973 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:24:35.066 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:24:54.008 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:24:54.077 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:24:55.000 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] connection error
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 11:24:55.001 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] because: [/10.242.129.86:9042] Unexpected exception triggered (java.io.IOException: Connection reset by peer)
2021-05-21 11:24:55.005 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preventing new connections for the next 1000 ms
2021-05-21 11:24:55.005 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-1, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:24:55.007 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 11:24:55.008 DEBUG 11 --- [uster-worker-11] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] marking host DOWN
2021-05-21 11:24:55.017 DEBUG 11 --- [uster-worker-11] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 11:24:55.017 DEBUG 11 --- [uster-worker-11] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] starting reconnection attempt
2021-05-21 11:24:55.082 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.129.86:9042, scheduling delivery
2021-05-21 11:24:55.083 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] received event DOWN
2021-05-21 11:24:56.017 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:24:56.024 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-2, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 11:24:56.026 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-2, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:24:56.026 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-2, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:24:56.026 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 11:24:56.027 DEBUG 11 --- [uster-worker-12] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:24:56.027 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 2000 milliseconds
2021-05-21 11:24:56.027 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 2000 ms
2021-05-21 11:24:56.085 DEBUG 11 --- [uster-worker-11] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:24:58.028 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:24:58.030 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-3, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 11:24:58.030 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-3, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:24:58.031 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-3, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:24:58.031 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-3, inFlight=0, closed=true] closing connection
2021-05-21 11:24:58.031 DEBUG 11 --- [uster-worker-12] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:24:58.031 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 4000 milliseconds
2021-05-21 11:24:58.031 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 4000 ms
2021-05-21 11:25:02.032 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:25:02.036 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-4, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 11:25:02.036 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-4, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:25:02.037 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-4, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:25:02.037 DEBUG 11 --- [uster-worker-11] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:25:02.037 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-4, inFlight=0, closed=true] closing connection
2021-05-21 11:25:02.038 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 8000 milliseconds
2021-05-21 11:25:02.038 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 8000 ms
2021-05-21 11:25:10.038 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:25:15.045 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-5, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:25:15.045 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-5, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:25:15.046 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-5, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:25:15.046 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-5, inFlight=0, closed=true] closing connection
2021-05-21 11:25:15.046 DEBUG 11 --- [uster-worker-12] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:25:15.046 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 16000 milliseconds
2021-05-21 11:25:15.047 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 16000 ms
2021-05-21 11:25:15.702 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:25:15.704 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:25:25.086 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:25:25.088 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:25:31.047 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:25:36.053 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-6, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:25:36.053 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-6, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:25:36.054 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-6, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:25:36.054 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-6, inFlight=0, closed=true] closing connection
2021-05-21 11:25:36.054 DEBUG 11 --- [uster-worker-13] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:25:36.054 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 32000 milliseconds
2021-05-21 11:25:36.054 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 32000 ms
2021-05-21 11:25:55.089 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:25:55.091 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:26:08.055 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:26:11.237 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:26:11.301 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:26:13.058 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-7, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:26:13.059 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-7, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:26:13.059 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] Connection[/10.242.129.86:9042-7, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:26:13.060 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-7, inFlight=0, closed=true] closing connection
2021-05-21 11:26:13.061 DEBUG 11 --- [uster-worker-14] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.86:9042
2021-05-21 11:26:13.062 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 64000 milliseconds
2021-05-21 11:26:13.062 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 64000 ms
2021-05-21 11:26:25.093 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:26:25.096 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:26:44.597 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT NEW_NODE /10.242.129.101:9042, scheduling delivery
2021-05-21 11:26:44.598 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event NEW_NODE
2021-05-21 11:26:44.598 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.129.101:9042, scheduling delivery
2021-05-21 11:26:44.598 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event UP
2021-05-21 11:26:45.600 DEBUG 11 --- [uster-worker-15] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.129.101:9042
2021-05-21 11:26:45.610 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] adding host
2021-05-21 11:26:45.611 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.129.101:9042
2021-05-21 11:26:45.611 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:26:45.614 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:26:49.562 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:26:52.568 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 0 new connections, total = 1
2021-05-21 11:26:52.568 DEBUG 11 --- [uster-worker-15] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.129.101:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:26:52.569 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Session : Added connection pool for /10.242.129.101:9042
2021-05-21 11:26:53.573 DEBUG 11 --- [uster-worker-16] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:26:53.583 INFO 11 --- [uster-worker-16] com.datastax.driver.core.Cluster : Cassandra host /10.242.129.86:9042 removed
2021-05-21 11:26:53.584 DEBUG 11 --- [uster-worker-15] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] removing host
2021-05-21 11:26:53.586 DEBUG 11 --- [uster-worker-16] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:26:53.598 DEBUG 11 --- [uster-worker-16] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 12 milliseconds
2021-05-21 11:27:17.062 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:27:22.069 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-8, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:27:22.070 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-8, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:27:22.070 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-8, inFlight=0, closed=true] closing connection
2021-05-21 11:27:22.070 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 128000 milliseconds
2021-05-21 11:27:22.070 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 128000 ms
2021-05-21 11:27:23.592 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:27:23.600 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:27:53.604 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:27:53.608 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:28:02.923 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:28:02.925 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:28:18.385 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:28:18.462 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:28:23.609 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:28:23.679 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:28:53.679 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:28:53.682 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:29:15.534 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:29:15.537 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:29:23.683 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:29:23.693 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:29:30.071 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:29:35.077 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-9, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:29:35.078 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-9, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:29:35.078 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-9, inFlight=0, closed=true] closing connection
2021-05-21 11:29:35.079 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 256000 milliseconds
2021-05-21 11:29:35.079 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 256000 ms
2021-05-21 11:29:40.918 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:29:40.920 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:29:53.693 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:29:53.709 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:30:23.710 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:30:23.712 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:30:53.713 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:30:53.979 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:30:55.781 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:30:55.783 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:31:05.981 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:31:06.062 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:31:23.980 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:31:36.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:31:36.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:36.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:36.055 DEBUG 11 --- [ter-timeouter-0] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 0ms
2021-05-21 11:31:36.055 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [Control connection] starting reconnection attempt
2021-05-21 11:31:36.055 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 11:31:36.055 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:31:36.056 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:31:36.058 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-2, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:31:36.062 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-2, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:36.062 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:36.062 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-2, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:36.062 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 11:31:36.063 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-2, inFlight=0, closed=true] has already terminated
2021-05-21 11:31:36.063 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:36.063 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 1000 ms
2021-05-21 11:31:37.064 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:31:37.066 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-2, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:31:37.068 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-2, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:37.069 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:37.069 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-2, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:37.069 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 11:31:37.069 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-2, inFlight=0, closed=true] has already terminated
2021-05-21 11:31:37.069 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:37.070 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 2000 ms
2021-05-21 11:31:39.070 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:31:39.072 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:31:39.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [229830267-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@2ee8dd85 at consistency LOCAL_QUORUM
2021-05-21 11:31:39.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [229830267-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:31:42.197 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:31:47.273 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:31:47.275 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:31:51.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:31:51.141 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:51.141 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:51.141 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=true] closing connection
2021-05-21 11:31:51.142 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:31:51.142 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] error on cassandra/10.242.184.18:9042 connection, trying next host
com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Operation timed out
at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1496)
at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1575)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 11:31:51.143 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:31:51.144 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-3, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:31:51.209 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-3, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:51.209 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:51.209 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-3, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:51.209 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-3, inFlight=0, closed=true] closing connection
2021-05-21 11:31:51.209 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-3, inFlight=0, closed=true] has already terminated
2021-05-21 11:31:51.209 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:51.210 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 4000 ms
2021-05-21 11:31:53.094 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 11:31:55.211 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=1, closed=false] Response received on stream 12544 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ROWS [4 columns]
---
2021-05-21 11:31:55.211 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:31:55.215 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-4, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:31:55.219 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-4, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:55.219 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:31:55.219 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-4, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:31:55.219 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-4, inFlight=0, closed=true] closing connection
2021-05-21 11:31:55.220 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-4, inFlight=0, closed=true] has already terminated
2021-05-21 11:31:55.220 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:31:55.220 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 8000 ms
2021-05-21 11:32:02.660 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-1, inFlight=0, closed=true] has already terminated
2021-05-21 11:32:03.221 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:32:03.223 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-3, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:32:03.264 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-3, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:32:03.264 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:32:03.265 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-3, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:32:03.265 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-3, inFlight=0, closed=true] closing connection
2021-05-21 11:32:03.265 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-3, inFlight=0, closed=true] has already terminated
2021-05-21 11:32:03.265 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:32:03.266 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 16000 ms
2021-05-21 11:32:09.409 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:32:09.410 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:32:17.660 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-3, inFlight=0, closed=true] has already terminated
2021-05-21 11:32:19.266 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:32:19.269 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:32:20.705 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:32:20.705 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:32:21.284 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:32:21.292 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 6 milliseconds
2021-05-21 11:32:21.299 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 11:32:21.644 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:32:21.648 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 4 milliseconds
2021-05-21 11:32:21.655 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.184.18:9042
2021-05-21 11:32:21.655 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:32:51.618 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:32:51.681 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:32:55.576 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:32:55.579 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:33:21.518 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:33:21.522 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:33:21.682 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:33:21.689 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:33:51.079 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 3
2021-05-21 11:33:51.698 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:33:51.777 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:33:52.417 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:33:52.419 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:33:56.085 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-10, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:33:56.086 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-10, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:33:56.086 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-10, inFlight=0, closed=true] closing connection
2021-05-21 11:33:56.087 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 512000 milliseconds
2021-05-21 11:33:56.087 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 512000 ms
2021-05-21 11:34:09.717 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:34:09.777 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:34:21.778 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:34:21.780 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:34:36.113 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:34:36.115 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:34:51.787 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:34:52.584 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:35:10.281 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:35:10.362 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:35:22.585 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:35:22.649 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:35:52.649 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:35:52.685 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:36:12.904 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:36:12.908 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:36:22.686 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:36:22.688 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:36:42.933 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:36:43.001 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:36:52.710 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:37:04.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 0ms
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [Control connection] starting reconnection attempt
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=true] closing connection
2021-05-21 11:37:04.738 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:04.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:37:04.741 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-5, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:04.805 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-5, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:04.805 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:04.805 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-5, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:04.805 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-5, inFlight=0, closed=true] closing connection
2021-05-21 11:37:04.806 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-5, inFlight=0, closed=true] has already terminated
2021-05-21 11:37:04.806 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:04.806 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 1000 ms
2021-05-21 11:37:05.807 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:05.809 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-4, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:09.721 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:37:10.871 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-4, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 11:37:10.871 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:10.871 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-4, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:10.871 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-4, inFlight=0, closed=true] closing connection
2021-05-21 11:37:10.872 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-4, inFlight=0, closed=true] has already terminated
2021-05-21 11:37:10.872 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 11:37:10.872 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 2000 ms
2021-05-21 11:37:12.872 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:12.874 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:16.881 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 11:37:22.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [466261783-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@26abc76f at consistency LOCAL_QUORUM
2021-05-21 11:37:22.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [466261783-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:37:23.377 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] Response received on stream 13888 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 11:37:28.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:37:28.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:28.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:28.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=true] closing connection
2021-05-21 11:37:28.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:37:28.938 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] error on cassandra/10.242.184.18:9042 connection, trying next host
com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Operation timed out
at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1496)
at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1575)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 11:37:28.938 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:28.940 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-6, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:29.002 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-6, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:29.002 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:29.003 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-6, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:29.003 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-6, inFlight=0, closed=true] closing connection
2021-05-21 11:37:29.003 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-6, inFlight=0, closed=true] has already terminated
2021-05-21 11:37:29.003 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:29.004 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 4000 ms
2021-05-21 11:37:30.106 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:37:30.161 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:37:32.666 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-4, inFlight=0, closed=true] has already terminated
2021-05-21 11:37:33.004 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:33.008 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-7, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:33.012 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-7, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:33.012 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:37:33.012 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-7, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:37:33.012 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-7, inFlight=0, closed=true] closing connection
2021-05-21 11:37:33.013 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-7, inFlight=0, closed=true] has already terminated
2021-05-21 11:37:33.013 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:37:33.013 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 8000 ms
2021-05-21 11:37:41.014 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:37:41.016 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:37:45.970 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:37:45.970 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:37:45.992 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:37:45.997 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 5 milliseconds
2021-05-21 11:37:46.005 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 11:37:46.127 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:37:46.133 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 6 milliseconds
2021-05-21 11:37:46.142 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] established to /10.242.129.101:9042
2021-05-21 11:37:46.142 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:38:02.667 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-5, inFlight=0, closed=true] has already terminated
2021-05-21 11:38:03.963 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 11:38:03.963 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 11:38:04.964 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 11:38:15.981 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:38:23.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1817333664-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@3912a1cb at consistency LOCAL_QUORUM
2021-05-21 11:38:23.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1817333664-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:38:28.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:38:28.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 2000 ms
2021-05-21 11:38:28.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:38:28.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 11:38:28.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:38:28.038 DEBUG 11 --- [uster-worker-17] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 11:38:28.039 DEBUG 11 --- [uster-worker-17] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 11:38:28.039 DEBUG 11 --- [uster-worker-17] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 11:38:29.039 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:38:29.041 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:38:31.389 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:38:31.393 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:38:33.966 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:38:33.969 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:38:44.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:38:44.439 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:38:44.440 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=true] closing connection
2021-05-21 11:38:44.440 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:38:44.441 DEBUG 11 --- [uster-worker-18] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:38:44.441 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 11:38:44.441 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 11:38:46.442 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:38:46.445 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:39:00.136 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:39:00.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:39:00.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=true] closing connection
2021-05-21 11:39:00.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:39:00.138 DEBUG 11 --- [uster-worker-19] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:39:00.139 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 11:39:00.139 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 4000 ms
2021-05-21 11:39:02.668 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-2, inFlight=0, closed=true] has already terminated
2021-05-21 11:39:03.970 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:39:03.972 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:39:04.139 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:39:04.141 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:39:16.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:39:16.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:39:16.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=true] closing connection
2021-05-21 11:39:16.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:39:16.238 DEBUG 11 --- [uster-worker-20] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:39:16.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 11:39:16.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 8000 ms
2021-05-21 11:39:17.669 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-6, inFlight=0, closed=true] has already terminated
2021-05-21 11:39:24.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:39:24.241 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:39:32.669 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-7, inFlight=0, closed=true] has already terminated
2021-05-21 11:39:33.973 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:39:33.975 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:39:36.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:39:36.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:39:36.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=true] closing connection
2021-05-21 11:39:36.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:39:36.338 DEBUG 11 --- [uster-worker-21] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:39:36.338 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 16000 milliseconds
2021-05-21 11:39:36.338 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 16000 ms
2021-05-21 11:39:47.669 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-8, inFlight=0, closed=true] has already terminated
2021-05-21 11:39:52.339 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:39:52.341 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:40:02.670 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-9, inFlight=0, closed=true] has already terminated
2021-05-21 11:40:03.814 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:40:03.815 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 11:40:03.865 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 11:40:03.865 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 11:40:03.865 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 11:40:03.866 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 11:40:13.378 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 11:40:13.379 DEBUG 11 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:40:13.384 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:40:13.385 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:40:20.890 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1544868584-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@3e2efda0 at consistency null
2021-05-21 11:40:20.890 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1544868584-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 11:40:31.357 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1390160037-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1505d1c8 at consistency null
2021-05-21 11:40:31.358 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1390160037-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 11:40:33.865 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:40:33.872 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:40:39.166 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 11:40:39.166 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 11:40:40.168 DEBUG 11 --- [uster-worker-22] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 11:41:10.174 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:41:10.176 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:41:40.177 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:41:40.182 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:41:58.190 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:41:58.192 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:42:10.183 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:42:10.270 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:42:28.087 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 4
2021-05-21 11:42:33.096 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-11, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:42:33.097 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-11, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:42:33.097 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-11, inFlight=0, closed=true] closing connection
2021-05-21 11:42:33.098 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 11:42:33.098 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 11:42:37.370 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:42:37.373 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:42:40.271 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:42:40.273 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:43:10.277 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:43:10.362 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:43:40.363 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:43:40.369 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:44:10.389 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:44:10.392 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:44:10.710 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:44:10.712 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:44:20.917 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:44:25.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [356222611-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5170bd1b at consistency LOCAL_QUORUM
2021-05-21 11:44:25.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [356222611-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:44:29.778 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:44:29.778 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] Response received on stream 4608 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ROWS [4 columns]
---
2021-05-21 11:44:40.394 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:44:47.018 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:45:00.981 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:45:00.985 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:45:17.022 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:45:18.563 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:45:38.834 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:45:39.082 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:45:48.565 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:45:48.893 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:46:01.001 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:46:01.067 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:46:18.894 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:46:18.898 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:46:26.197 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:46:26.277 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:46:48.898 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:46:48.963 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:46:56.278 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:46:56.281 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:47:18.964 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:47:21.369 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:47:35.701 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:47:35.703 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:47:51.373 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:47:51.461 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:48:21.462 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:48:21.667 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:48:40.908 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:48:40.911 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:48:51.668 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:48:51.765 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:49:12.389 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:49:12.470 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:49:21.767 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:49:21.768 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:49:33.582 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 11:49:33.583 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 11:49:34.584 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 11:49:36.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [918013560-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7ab1f5a7 at consistency LOCAL_QUORUM
2021-05-21 11:49:36.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [918013560-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:49:42.470 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:49:42.685 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:49:42.844 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:49:54.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:49:54.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:49:54.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:49:54.938 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=true] closing connection
2021-05-21 11:49:54.938 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:49:54.939 DEBUG 11 --- [uster-worker-23] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 11:49:54.942 DEBUG 11 --- [uster-worker-23] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 11:49:54.942 DEBUG 11 --- [uster-worker-23] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 11:49:55.942 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:49:55.945 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:49:59.681 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:49:59.681 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 11:50:00.799 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 11:50:00.799 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 11:50:00.799 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 11:50:00.799 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 11:50:06.407 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 11:50:06.408 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:50:06.408 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:50:06.408 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:50:17.083 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 11:50:17.083 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 11:50:18.084 DEBUG 11 --- [uster-worker-24] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 11:50:32.684 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-10, inFlight=0, closed=true] has already terminated
2021-05-21 11:50:36.413 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:50:36.520 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 11:50:38.496 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:50:26.601Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:50:47.065 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:50:47.100 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:50:48.171 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:50:48.262 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:50:55.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1014164532-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@587d8af at consistency LOCAL_QUORUM
2021-05-21 11:50:55.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1014164532-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:50:57.121 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:50:58.479 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 11:50:58.479 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 11:50:58.563 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 11:50:59.385 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] Response received on stream 3776 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 11:50:59.480 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 11:51:16.412 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:50:42.998Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:51:17.101 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:51:17.106 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:51:28.489 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:51:28.668 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:51:43.523 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:51:26.766Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:51:47.107 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:51:47.109 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:51:57.745 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:51:57.763 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:51:58.670 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:51:58.672 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:52:04.998 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:51:40Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:52:18.605 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:52:18.608 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:52:22.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1899582244-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@70ee1975 at consistency LOCAL_QUORUM
2021-05-21 11:52:22.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1899582244-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:52:22.195 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] Response received on stream 4032 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 11:52:27.764 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:52:28.084 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:52:28.673 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:52:28.693 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:52:33.099 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 5
2021-05-21 11:52:33.152 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:52:09.499Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:52:38.105 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-12, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 11:52:38.106 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-12, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 11:52:38.106 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-12, inFlight=0, closed=true] closing connection
2021-05-21 11:52:38.107 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 11:52:38.107 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 11:52:51.149 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:52:37.655Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:52:58.085 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:52:58.091 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:52:58.695 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:52:58.773 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:53:18.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1472144300-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@759e1bc8 at consistency LOCAL_QUORUM
2021-05-21 11:53:18.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1472144300-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 11:53:21.157 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:53:28.774 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:53:28.777 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:53:33.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:53:33.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:53:33.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:53:33.240 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=true] closing connection
2021-05-21 11:53:33.240 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:53:33.240 DEBUG 11 --- [uster-worker-25] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 11:53:33.242 DEBUG 11 --- [uster-worker-25] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 11:53:33.242 DEBUG 11 --- [uster-worker-25] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 11:53:34.242 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:53:34.245 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:53:46.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:53:46.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:53:46.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=true] closing connection
2021-05-21 11:53:46.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:53:46.338 DEBUG 11 --- [uster-worker-26] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:53:46.339 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 11:53:46.339 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 11:53:48.339 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:53:48.341 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:53:58.778 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:53:58.965 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:54:00.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:54:00.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:54:00.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=true] closing connection
2021-05-21 11:54:00.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:54:00.438 DEBUG 11 --- [uster-worker-25] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:54:00.438 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 11:54:00.438 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 4000 ms
2021-05-21 11:54:02.688 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-11, inFlight=0, closed=true] has already terminated
2021-05-21 11:54:04.439 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:54:04.441 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:54:16.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:54:16.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:54:16.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=true] closing connection
2021-05-21 11:54:16.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:54:16.538 DEBUG 11 --- [uster-worker-27] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:54:16.539 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 11:54:16.539 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 8000 ms
2021-05-21 11:54:17.689 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-12, inFlight=0, closed=true] has already terminated
2021-05-21 11:54:24.539 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:54:24.542 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:54:28.966 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:54:28.976 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:54:32.689 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-13, inFlight=0, closed=true] has already terminated
2021-05-21 11:54:36.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:54:36.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:54:36.638 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=true] closing connection
2021-05-21 11:54:36.638 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:54:36.638 DEBUG 11 --- [uster-worker-28] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:54:36.639 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 16000 milliseconds
2021-05-21 11:54:36.639 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 16000 ms
2021-05-21 11:54:47.690 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-14, inFlight=0, closed=true] has already terminated
2021-05-21 11:54:52.639 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:54:52.642 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:54:58.981 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:55:02.690 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-15, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:06.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [969403628-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@77c49108 at consistency LOCAL_QUORUM
2021-05-21 11:55:06.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [969403628-0] Error querying /10.242.129.101:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.129.101:9042] Timed out waiting for server response
2021-05-21 11:55:11.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] because: [/10.242.129.101:9042] Heartbeat query timed out
2021-05-21 11:55:11.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:11.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-5, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:11.037 DEBUG 11 --- [ter-timeouter-0] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 0ms
2021-05-21 11:55:11.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [Control connection] starting reconnection attempt
2021-05-21 11:55:11.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=true] closing connection
2021-05-21 11:55:11.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-5, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:55:11.037 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:11.040 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-8, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:11.104 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-8, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:11.104 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:11.105 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-8, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:11.105 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-8, inFlight=0, closed=true] closing connection
2021-05-21 11:55:11.105 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-8, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:11.105 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:11.105 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 1000 ms
2021-05-21 11:55:12.106 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:12.108 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-6, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:12.736 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:55:12.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:55:12.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=true] closing connection
2021-05-21 11:55:12.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:55:12.738 DEBUG 11 --- [uster-worker-29] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:55:12.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 32000 milliseconds
2021-05-21 11:55:12.740 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 32000 ms
2021-05-21 11:55:18.573 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=1, closed=false] Response received on stream 11648 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 11:55:23.766 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-6, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:23.767 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:23.767 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-6, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:23.767 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-6, inFlight=0, closed=true] closing connection
2021-05-21 11:55:23.768 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-6, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:23.768 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:23.769 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 2000 ms
2021-05-21 11:55:25.769 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:25.771 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-9, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:25.774 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-9, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:25.774 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:25.774 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-9, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:25.775 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-9, inFlight=0, closed=true] closing connection
2021-05-21 11:55:25.776 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-9, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:25.776 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:25.776 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 4000 ms
2021-05-21 11:55:29.776 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:29.778 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-7, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:32.381 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-7, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:32.381 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:32.381 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-7, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:32.382 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-7, inFlight=0, closed=true] closing connection
2021-05-21 11:55:32.382 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-7, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:32.382 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:32.382 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 8000 ms
2021-05-21 11:55:39.574 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:55:28.111Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:55:40.383 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:40.385 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-10, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:40.403 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-10, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:40.403 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:40.403 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-10, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:40.403 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-10, inFlight=0, closed=true] closing connection
2021-05-21 11:55:40.404 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-10, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:40.404 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:40.404 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 16000 ms
2021-05-21 11:55:44.740 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:55:44.742 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:47.692 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-5, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:47.692 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-16, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:56.404 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:55:56.406 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-8, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:55:56.673 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:55:56.673 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 11:55:56.673 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 11:55:56.673 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 11:55:56.674 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 11:55:59.293 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-8, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:59.294 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:55:59.294 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-8, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:55:59.294 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-8, inFlight=0, closed=true] closing connection
2021-05-21 11:55:59.294 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-8, inFlight=0, closed=true] has already terminated
2021-05-21 11:55:59.294 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:55:59.295 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 32000 ms
2021-05-21 11:56:01.214 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 11:56:01.214 DEBUG 11 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:56:01.214 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:56:01.215 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:56:31.230 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:56:31.295 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:56:31.299 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-11, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:56:31.305 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-11, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:56:31.305 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 11:56:31.305 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-11, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:56:31.305 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-11, inFlight=0, closed=true] closing connection
2021-05-21 11:56:31.306 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-11, inFlight=0, closed=true] has already terminated
2021-05-21 11:56:31.306 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:56:31.307 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 64000 ms
2021-05-21 11:56:35.989 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:56:52.425 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:56:31.402Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:56:56.917 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:56:56.919 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:57:22.444 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:57:34.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 11:57:34.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 11:57:34.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:57:34.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=true] closing connection
2021-05-21 11:57:34.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:57:34.538 DEBUG 11 --- [uster-worker-30] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 11:57:34.539 DEBUG 11 --- [uster-worker-30] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 11:57:34.539 DEBUG 11 --- [uster-worker-30] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 11:57:35.307 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:57:35.309 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-9, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:57:35.539 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:57:35.541 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:57:37.838 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-9, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:57:37.838 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 11:57:37.838 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-9, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 11:57:37.839 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-9, inFlight=0, closed=true] closing connection
2021-05-21 11:57:37.839 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-9, inFlight=0, closed=true] has already terminated
2021-05-21 11:57:37.840 ERROR 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 11:57:37.840 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 128000 ms
2021-05-21 11:57:47.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:57:47.638 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:57:47.638 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=true] closing connection
2021-05-21 11:57:47.639 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:57:47.639 DEBUG 11 --- [uster-worker-31] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:57:47.639 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 11:57:47.639 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 11:57:49.640 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:57:49.643 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:57:55.381 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:57:57.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1243378231-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@32ca6082 at consistency LOCAL_QUORUM
2021-05-21 11:57:57.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1243378231-0] Error querying /10.242.129.101:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.129.101:9042] Timed out waiting for server response
2021-05-21 11:57:58.173 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:57:58.293 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] Response received on stream 12352 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ROWS [4 columns]
---
2021-05-21 11:58:01.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:58:01.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:58:01.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=true] closing connection
2021-05-21 11:58:01.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:58:01.738 DEBUG 11 --- [uster-worker-30] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:58:01.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 11:58:01.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 4000 ms
2021-05-21 11:58:02.694 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-17, inFlight=0, closed=true] has already terminated
2021-05-21 11:58:05.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:58:05.741 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:58:17.695 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-18, inFlight=0, closed=true] has already terminated
2021-05-21 11:58:17.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 11:58:17.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 11:58:17.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=true] closing connection
2021-05-21 11:58:17.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=true]: there are still pending requests
2021-05-21 11:58:17.838 DEBUG 11 --- [uster-worker-32] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 11:58:17.838 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 11:58:17.838 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 8000 ms
2021-05-21 11:58:25.839 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 11:58:25.841 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:58:30.504 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:58:19.870Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:58:32.695 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-19, inFlight=0, closed=true] has already terminated
2021-05-21 11:58:37.908 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:58:37.908 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 11:58:37.909 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 11:58:37.909 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 11:58:37.909 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 11:58:42.291 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 11:58:42.292 DEBUG 11 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 11:58:42.292 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 11:58:42.292 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 11:58:47.696 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-20, inFlight=0, closed=true] has already terminated
2021-05-21 11:58:59.400 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:58:47.202Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:59:11.046 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:58:47.202Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:59:12.713 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:59:12.715 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:59:17.739 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:59:17.761 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:59:39.192 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T11:59:21.079Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 11:59:45.841 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 11:59:45.845 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 11:59:46.582 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 11:59:46.601 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 11:59:48.401 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 11:59:48.402 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 11:59:50.480 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:59:50.484 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 4 milliseconds
2021-05-21 11:59:50.491 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 11:59:50.637 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 11:59:50.642 DEBUG 11 --- [-reconnection-1] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 5 milliseconds
2021-05-21 11:59:50.648 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.184.18:9042
2021-05-21 11:59:50.648 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:00:14.881 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:00:15.657 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:00:20.614 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:00:22.480 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:00:41.286 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:00:41.302 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:00:52.175 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:00:52.179 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:00:52.482 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:00:52.488 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:01:12.436 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:01:12.500 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:01:22.489 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:01:23.426 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:01:30.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2042891552-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@39cb936b at consistency LOCAL_QUORUM
2021-05-21 12:01:30.938 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2042891552-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:01:34.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:01:34.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:01:34.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 12:01:34.539 DEBUG 11 --- [ter-timeouter-0] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 0ms
2021-05-21 12:01:34.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [Control connection] starting reconnection attempt
2021-05-21 12:01:34.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=true] closing connection
2021-05-21 12:01:34.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:01:34.540 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:01:34.546 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-12, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:01:35.436 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:01:35.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:01:35.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=true] closing connection
2021-05-21 12:01:35.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:01:35.439 DEBUG 11 --- [uster-worker-33] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:01:35.439 DEBUG 11 --- [uster-worker-33] c.d.driver.core.ControlConnection : [Control connection] cassandra/10.242.184.18:9042 is down/removed and it was the control host, triggering reconnect
2021-05-21 12:01:35.441 DEBUG 11 --- [uster-worker-33] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:01:35.441 DEBUG 11 --- [uster-worker-33] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:01:36.441 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:01:36.443 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:01:39.605 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-12, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 12:01:39.606 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 12:01:39.606 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-12, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 12:01:39.606 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-12, inFlight=0, closed=true] closing connection
2021-05-21 12:01:39.606 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-12, inFlight=0, closed=true] has already terminated
2021-05-21 12:01:39.606 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 12:01:39.607 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 1000 ms
2021-05-21 12:01:40.607 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:01:40.609 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-13, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:01:40.612 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.131.197:9042-13, inFlight=0, closed=false] because: Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:01:40.612 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preventing new connections for the next 1000 ms
2021-05-21 12:01:40.612 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-13, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 12:01:40.612 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-13, inFlight=0, closed=true] closing connection
2021-05-21 12:01:40.612 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-13, inFlight=0, closed=true] has already terminated
2021-05-21 12:01:40.612 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.131.197:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:01:40.613 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 2000 ms
2021-05-21 12:01:42.613 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:01:42.615 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-10, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:01:46.240 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:01:46.241 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:01:46.241 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:01:46.241 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:01:46.241 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:01:46.960 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:01:46.961 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:01:46.961 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:01:46.961 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:01:47.819 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-10, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 12:01:47.819 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 1000 ms
2021-05-21 12:01:47.819 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-10, inFlight=0, closed=false] failed, remaining = 1
2021-05-21 12:01:47.819 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-10, inFlight=0, closed=true] closing connection
2021-05-21 12:01:47.820 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-10, inFlight=0, closed=true] has already terminated
2021-05-21 12:01:47.820 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 12:01:47.820 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] next reconnection attempt in 4000 ms
2021-05-21 12:01:51.820 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:01:51.823 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:01:52.702 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.131.197:9042] Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:01:52.702 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 12:01:52.708 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 12:01:52.713 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 5 milliseconds
2021-05-21 12:01:52.719 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema
2021-05-21 12:01:52.845 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 12:01:52.852 DEBUG 11 --- [-reconnection-0] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 7 milliseconds
2021-05-21 12:01:52.861 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [Control connection] established to /10.242.131.197:9042
2021-05-21 12:01:52.861 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:02:02.699 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-21, inFlight=0, closed=true] has already terminated
2021-05-21 12:02:02.699 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-22, inFlight=0, closed=true] has already terminated
2021-05-21 12:02:22.813 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:02:22.816 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:02:38.107 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 6
2021-05-21 12:02:43.113 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-13, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (connection timed out: /10.242.129.86:9042)
2021-05-21 12:02:43.114 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-13, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 12:02:43.114 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-13, inFlight=0, closed=true] closing connection
2021-05-21 12:02:43.115 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:02:43.115 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 12:02:52.821 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:02:52.900 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:02:59.139 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:03:09.234 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:03:09.235 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:03:10.236 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:03:11.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:03:11.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:03:11.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:03:11.239 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=true] closing connection
2021-05-21 12:03:11.239 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:03:11.239 DEBUG 11 --- [uster-worker-34] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:03:11.240 DEBUG 11 --- [uster-worker-34] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:03:11.241 DEBUG 11 --- [uster-worker-34] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:03:12.241 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:03:12.243 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:03:15.302 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.129.101:9042, scheduling delivery
2021-05-21 12:03:15.303 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event DOWN
2021-05-21 12:03:16.303 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.129.101:9042 because it still has active connections
2021-05-21 12:03:24.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:03:24.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:03:24.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=true] closing connection
2021-05-21 12:03:24.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:03:24.338 DEBUG 11 --- [uster-worker-35] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:03:24.338 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 12:03:24.338 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 12:03:26.339 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:03:26.341 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:03:29.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [967903208-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5e02477e at consistency LOCAL_QUORUM
2021-05-21 12:03:29.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [967903208-0] Error querying /10.242.129.101:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.129.101:9042] Timed out waiting for server response
2021-05-21 12:03:29.342 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:03:11.613Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:133)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:159)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:79)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:03:29.995 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] Response received on stream 13696 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:03:31.310 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:03:31.311 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:03:31.314 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:03:31.314 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:03:31.315 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:03:31.315 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:03:31.321 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:03:31.322 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:03:31.322 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:03:31.322 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:03:34.804 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:03:34.804 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:03:35.804 DEBUG 11 --- [uster-worker-34] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:03:47.701 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-23, inFlight=0, closed=true] has already terminated
2021-05-21 12:03:56.503 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.129.101:9042, scheduling delivery
2021-05-21 12:03:56.503 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event UP
2021-05-21 12:03:57.504 DEBUG 11 --- [uster-worker-36] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.129.101:9042
2021-05-21 12:04:02.702 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-24, inFlight=0, closed=true] has already terminated
2021-05-21 12:04:06.524 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:04:06.541 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:04:26.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [629300763-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@36536b92 at consistency LOCAL_QUORUM
2021-05-21 12:04:26.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [629300763-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:04:27.531 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:04:27.602 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:04:29.692 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] Response received on stream 3904 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ROWS [4 columns]
---
2021-05-21 12:04:36.554 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:04:41.912 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.129.101:9042, scheduling delivery
2021-05-21 12:04:41.913 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event DOWN
2021-05-21 12:04:42.914 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.129.101:9042 because it still has active connections
2021-05-21 12:04:48.636 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] because: [/10.242.129.101:9042] Heartbeat query timed out
2021-05-21 12:04:48.636 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preventing new connections for the next 2000 ms
2021-05-21 12:04:48.636 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-1, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:04:48.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 12:04:48.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-1, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:04:48.638 DEBUG 11 --- [uster-worker-37] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] marking host DOWN
2021-05-21 12:04:48.639 DEBUG 11 --- [uster-worker-37] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:04:48.639 DEBUG 11 --- [uster-worker-37] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] starting reconnection attempt
2021-05-21 12:04:49.640 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:04:49.643 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-11, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:05:01.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-11, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:05:01.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-11, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:05:01.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-11, inFlight=0, closed=true] closing connection
2021-05-21 12:05:01.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-11, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:05:01.738 DEBUG 11 --- [uster-worker-38] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:05:01.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 12:05:01.739 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 2000 ms
2021-05-21 12:05:03.739 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:05:03.741 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-12, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:05:11.926 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:05:12.001 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:05:15.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-12, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:05:15.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-12, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:05:15.838 DEBUG 11 --- [uster-worker-37] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:05:15.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-12, inFlight=0, closed=true] closing connection
2021-05-21 12:05:15.839 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-12, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:05:15.839 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 12:05:15.839 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 4000 ms
2021-05-21 12:05:17.703 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-1, inFlight=0, closed=true] has already terminated
2021-05-21 12:05:19.840 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:05:19.843 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-13, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:05:31.936 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-13, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:05:31.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-13, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:05:31.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-13, inFlight=0, closed=true] closing connection
2021-05-21 12:05:31.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-13, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:05:31.938 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 12:05:31.938 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 8000 ms
2021-05-21 12:05:31.938 DEBUG 11 --- [uster-worker-39] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:05:32.704 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-11, inFlight=0, closed=true] has already terminated
2021-05-21 12:05:39.938 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:05:39.940 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-14, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:05:40.417 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:05:40.501 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:05:42.002 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:05:42.003 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:05:47.705 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-12, inFlight=0, closed=true] has already terminated
2021-05-21 12:05:52.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-14, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:05:52.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-14, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:05:52.039 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-14, inFlight=0, closed=true] closing connection
2021-05-21 12:05:52.039 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-14, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:05:52.039 DEBUG 11 --- [uster-worker-40] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:05:52.039 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 16000 milliseconds
2021-05-21 12:05:52.039 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 16000 ms
2021-05-21 12:06:02.706 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-13, inFlight=0, closed=true] has already terminated
2021-05-21 12:06:03.106 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:06:03.107 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:06:04.107 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:06:08.040 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:06:08.043 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-15, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:06:08.400 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:06:08.400 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:06:09.402 DEBUG 11 --- [uster-worker-41] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:06:17.707 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-14, inFlight=0, closed=true] has already terminated
2021-05-21 12:06:20.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-15, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:06:20.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-15, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:06:20.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-15, inFlight=0, closed=true] closing connection
2021-05-21 12:06:20.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-15, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:06:20.137 DEBUG 11 --- [uster-worker-40] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:06:20.137 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 32000 milliseconds
2021-05-21 12:06:20.138 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 32000 ms
2021-05-21 12:06:36.810 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:06:23.201Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:06:39.434 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:06:39.501 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:06:47.708 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-15, inFlight=0, closed=true] has already terminated
2021-05-21 12:06:52.138 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:06:52.140 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-16, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:06:56.804 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:06:56.828 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:07:04.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-16, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:07:04.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-16, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:07:04.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-16, inFlight=0, closed=true] closing connection
2021-05-21 12:07:04.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-16, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:07:04.238 DEBUG 11 --- [uster-worker-42] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:07:04.238 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 64000 milliseconds
2021-05-21 12:07:04.238 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 64000 ms
2021-05-21 12:07:09.502 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:07:09.511 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:07:32.709 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-16, inFlight=0, closed=true] has already terminated
2021-05-21 12:07:39.529 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:07:39.601 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:07:55.825 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:07:55.828 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:08:08.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:08:08.242 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-17, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:08:09.602 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:08:09.604 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:08:20.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-17, inFlight=0, closed=false] because: [/10.242.129.101:9042] Operation timed out
2021-05-21 12:08:20.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-17, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:08:20.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-17, inFlight=0, closed=true] closing connection
2021-05-21 12:08:20.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-17, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:08:20.338 DEBUG 11 --- [uster-worker-43] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:08:20.338 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Operation timed out), scheduling retry in 128000 milliseconds
2021-05-21 12:08:20.338 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 128000 ms
2021-05-21 12:08:39.633 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:08:39.701 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:08:47.711 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-17, inFlight=0, closed=true] has already terminated
2021-05-21 12:08:55.103 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.129.101:9042, scheduling delivery
2021-05-21 12:08:55.104 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event UP
2021-05-21 12:08:56.106 DEBUG 11 --- [uster-worker-44] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.129.101:9042
2021-05-21 12:08:56.109 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] marking host UP
2021-05-21 12:08:56.110 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:08:56.110 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.129.101:9042
2021-05-21 12:08:56.110 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:08:56.114 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-18, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:08:57.763 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-18, inFlight=0, closed=false] because: Authentication error on host /10.242.129.101:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
2021-05-21 12:08:57.764 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-18, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:08:57.765 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-18, inFlight=0, closed=true] closing connection
2021-05-21 12:08:57.766 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-18, inFlight=0, closed=true] has already terminated
2021-05-21 12:08:57.766 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:08:57.768 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-19, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:09:00.890 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-19, inFlight=0, closed=false] connection error
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:09:00.890 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-19, inFlight=0, closed=false] because: [/10.242.129.101:9042] Unexpected exception triggered (java.io.IOException: Connection reset by peer)
2021-05-21 12:09:00.890 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-19, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:00.890 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-19, inFlight=0, closed=true] closing connection
2021-05-21 12:09:00.890 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.129.101:9042-19, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:09:00.902 WARN 11 --- [er-nio-worker-3] c.d.driver.core.HostConnectionPool : Error creating connection to /10.242.129.101:9042
com.datastax.driver.core.exceptions.TransportException: [/10.242.129.101:9042] Connection has been closed
at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1397)
at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1378)
at com.datastax.driver.core.Connection.defunct(Connection.java:570)
at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1301)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:300)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:279)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:271)
at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:300)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:279)
at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:174)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:09:00.903 DEBUG 11 --- [er-nio-worker-3] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.129.101:9042 (1 connections needed, 0 successfully opened)
2021-05-21 12:09:00.903 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Session : Added connection pool for /10.242.129.101:9042
2021-05-21 12:09:00.904 DEBUG 11 --- [uster-worker-45] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] marking host DOWN
2021-05-21 12:09:00.904 DEBUG 11 --- [uster-worker-45] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:09:00.904 DEBUG 11 --- [uster-worker-45] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] starting reconnection attempt
2021-05-21 12:09:00.904 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:01.004 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.129.101:9042, scheduling delivery
2021-05-21 12:09:01.005 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] received event DOWN
2021-05-21 12:09:01.904 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:09:01.907 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-20, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (Connection refused: /10.242.129.101:9042)
2021-05-21 12:09:01.907 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-20, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:09:01.907 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-20, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:01.907 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-20, inFlight=0, closed=true] closing connection
2021-05-21 12:09:01.908 DEBUG 11 --- [uster-worker-45] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:01.908 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 2000 milliseconds
2021-05-21 12:09:01.908 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 2000 ms
2021-05-21 12:09:02.005 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:03.908 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:09:03.910 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-21, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (Connection refused: /10.242.129.101:9042)
2021-05-21 12:09:03.911 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-21, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:09:03.911 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-21, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:03.911 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-21, inFlight=0, closed=true] closing connection
2021-05-21 12:09:03.911 DEBUG 11 --- [uster-worker-45] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:03.912 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 4000 milliseconds
2021-05-21 12:09:03.912 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 4000 ms
2021-05-21 12:09:07.912 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:09:07.915 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-22, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (Connection refused: /10.242.129.101:9042)
2021-05-21 12:09:07.915 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-22, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:09:07.915 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-22, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:07.916 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-22, inFlight=0, closed=true] closing connection
2021-05-21 12:09:07.916 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:07.916 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 8000 milliseconds
2021-05-21 12:09:07.916 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 8000 ms
2021-05-21 12:09:15.916 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:09:15.919 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-23, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (Connection refused: /10.242.129.101:9042)
2021-05-21 12:09:15.920 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-23, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:09:15.920 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-23, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:15.920 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-23, inFlight=0, closed=true] closing connection
2021-05-21 12:09:15.920 DEBUG 11 --- [uster-worker-45] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:15.921 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 16000 milliseconds
2021-05-21 12:09:15.921 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 16000 ms
2021-05-21 12:09:31.025 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:09:31.101 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:09:31.921 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:09:32.712 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-19, inFlight=0, closed=true] has already terminated
2021-05-21 12:09:36.924 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-24, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:09:36.925 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-24, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:09:36.925 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-24, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:09:36.925 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-24, inFlight=0, closed=true] closing connection
2021-05-21 12:09:36.926 DEBUG 11 --- [uster-worker-44] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:09:36.926 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 32000 milliseconds
2021-05-21 12:09:36.926 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 32000 ms
2021-05-21 12:09:43.236 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1933856788-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@19e80477 at consistency LOCAL_QUORUM
2021-05-21 12:09:43.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1933856788-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:09:48.305 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:09:25.602Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:09:48.810 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:09:48.811 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:09:49.812 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:09:56.216 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:10:05.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [301173386-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6c3bebb9 at consistency LOCAL_QUORUM
2021-05-21 12:10:05.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [301173386-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:10:05.344 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:09:25.602Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:133)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:159)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:79)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:10:08.236 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:10:08.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:10:08.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:10:08.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=true] closing connection
2021-05-21 12:10:08.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:10:08.237 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:10:08.239 DEBUG 11 --- [uster-worker-46] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:10:08.239 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:10:08.926 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:10:09.239 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:10:09.241 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-26, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:10:10.401 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1914707738-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1c0ff6a8 at consistency null
2021-05-21 12:10:10.401 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1914707738-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:10.501 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [581055403-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5b532b07 at consistency null
2021-05-21 12:10:10.501 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [581055403-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:10.601 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1932989211-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4af7eb26 at consistency null
2021-05-21 12:10:10.601 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1932989211-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:10.705 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [2124247085-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@585b2588 at consistency null
2021-05-21 12:10:10.705 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [2124247085-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:10.808 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [204961835-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@3b82229d at consistency null
2021-05-21 12:10:10.809 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [204961835-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:10.912 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1069235565-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@79cb3df9 at consistency null
2021-05-21 12:10:10.912 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1069235565-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:11.115 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1952407114-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@c90473d at consistency null
2021-05-21 12:10:11.115 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1952407114-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:11.318 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1633047796-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@d664442 at consistency null
2021-05-21 12:10:11.319 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1633047796-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:11.801 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1452773077-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@27da19b9 at consistency null
2021-05-21 12:10:11.802 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1452773077-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:12.605 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [807502143-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7decc4e at consistency null
2021-05-21 12:10:12.605 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [807502143-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:13.933 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-25, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:10:13.933 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-25, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:10:13.933 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-25, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:10:13.933 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-25, inFlight=0, closed=true] closing connection
2021-05-21 12:10:13.934 DEBUG 11 --- [uster-worker-47] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:10:13.934 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 64000 milliseconds
2021-05-21 12:10:13.934 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 64000 ms
2021-05-21 12:10:14.208 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1040331903-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@979ca27 at consistency null
2021-05-21 12:10:14.208 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1040331903-0] Error querying /10.242.131.197:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:10:14.209 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:09:25.602Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.131.197:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)))
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.131.197:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)))
at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:265)
at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:62)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:357)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:539)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:521)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:791)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
2021-05-21 12:10:17.024 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:10:17.024 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:10:18.025 DEBUG 11 --- [uster-worker-46] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:10:18.103 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:10:18.104 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:10:18.104 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:10:18.104 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:10:18.106 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:10:19.600 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:10:19.796 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-26, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:10:19.797 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:10:19.801 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:10:25.916 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 2
2021-05-21 12:10:25.916 DEBUG 11 --- [uster-worker-46] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:10:25.916 DEBUG 11 --- [uster-worker-46] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:10:25.917 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-26, inFlight=0, closed=true] closing connection
2021-05-21 12:10:25.917 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-26, inFlight=0, closed=true] closed, remaining = 1
2021-05-21 12:10:25.917 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:10:32.713 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-25, inFlight=0, closed=true] has already terminated
2021-05-21 12:10:49.803 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:10:49.805 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:11:16.233 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:11:16.237 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:11:17.936 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:19.805 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:11:19.808 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:11:22.942 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-26, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:11:22.943 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-26, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:11:22.943 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] Connection[/10.242.129.101:9042-26, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:22.944 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-26, inFlight=0, closed=true] closing connection
2021-05-21 12:11:22.944 DEBUG 11 --- [uster-worker-48] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.101:9042
2021-05-21 12:11:22.945 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 128000 milliseconds
2021-05-21 12:11:22.945 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 128000 ms
2021-05-21 12:11:34.101 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT NEW_NODE /10.242.129.80:9042, scheduling delivery
2021-05-21 12:11:34.101 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] received event NEW_NODE
2021-05-21 12:11:34.102 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.129.80:9042, scheduling delivery
2021-05-21 12:11:34.102 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] received event UP
2021-05-21 12:11:35.103 DEBUG 11 --- [uster-worker-49] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.129.80:9042
2021-05-21 12:11:35.108 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] adding host
2021-05-21 12:11:35.108 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.129.80:9042
2021-05-21 12:11:35.108 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:35.111 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-1, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:35.378 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-1, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:35.379 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preventing new connections for the next 1000 ms
2021-05-21 12:11:35.379 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-1, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:35.379 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-1, inFlight=0, closed=true] closing connection
2021-05-21 12:11:35.380 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-1, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:35.380 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:35.381 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-2, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:35.474 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-2, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:35.475 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preventing new connections for the next 1000 ms
2021-05-21 12:11:35.475 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-2, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:35.475 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-2, inFlight=0, closed=true] closing connection
2021-05-21 12:11:35.478 WARN 11 --- [er-nio-worker-1] com.datastax.driver.core.Session : Error creating pool to /10.242.129.80:9042
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:11:35.480 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Connection pool cannot be created, not marking /10.242.129.80:9042 UP
2021-05-21 12:11:35.480 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-2, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:35.480 DEBUG 11 --- [uster-worker-48] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] marking host DOWN
2021-05-21 12:11:35.481 DEBUG 11 --- [uster-worker-48] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:11:35.481 DEBUG 11 --- [uster-worker-48] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] starting reconnection attempt
2021-05-21 12:11:35.481 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:11:36.481 DEBUG 11 --- [uster-worker-48] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 12:11:36.481 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:36.482 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-3, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:36.504 INFO 11 --- [uster-worker-48] com.datastax.driver.core.Cluster : Cassandra host /10.242.129.101:9042 removed
2021-05-21 12:11:36.504 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] removing host
2021-05-21 12:11:36.506 DEBUG 11 --- [uster-worker-48] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 12:11:36.514 DEBUG 11 --- [uster-worker-48] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 8 milliseconds
2021-05-21 12:11:36.568 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-3, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:36.568 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-3, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:36.568 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-3, inFlight=0, closed=true] closing connection
2021-05-21 12:11:36.568 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:11:36.569 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-3, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:36.569 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:36.569 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 2000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:11:36.570 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 2000 ms
2021-05-21 12:11:37.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [613290163-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@3b029b22 at consistency LOCAL_QUORUM
2021-05-21 12:11:37.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [613290163-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:11:38.570 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:38.572 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-4, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:38.762 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-4, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:38.763 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-4, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:38.763 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-4, inFlight=0, closed=true] closing connection
2021-05-21 12:11:38.763 DEBUG 11 --- [uster-worker-48] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:11:38.763 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-4, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:38.763 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:38.764 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 4000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:11:38.765 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 4000 ms
2021-05-21 12:11:39.012 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:11:39.012 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:11:40.013 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:11:42.765 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:42.767 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-5, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:42.771 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-5, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:42.771 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-5, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:42.771 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-5, inFlight=0, closed=true] closing connection
2021-05-21 12:11:42.771 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:11:42.772 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-5, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:42.772 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:42.773 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 8000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:11:42.774 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 8000 ms
2021-05-21 12:11:50.774 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:11:50.776 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-6, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:11:50.781 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-6, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:50.781 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-6, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:11:50.782 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-6, inFlight=0, closed=true] closing connection
2021-05-21 12:11:50.782 DEBUG 11 --- [uster-worker-48] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:11:50.782 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-6, inFlight=0, closed=true] has already terminated
2021-05-21 12:11:50.782 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:11:50.783 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 16000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:11:50.785 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 16000 ms
2021-05-21 12:11:50.817 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:11:50.893 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] Response received on stream 4160 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:11:50.932 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:11:50.935 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:11:25.309Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 28 common frames omitted
2021-05-21 12:12:06.785 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:12:06.787 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-7, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:12:06.864 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-7, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:12:06.865 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-7, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:12:06.865 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-7, inFlight=0, closed=true] closing connection
2021-05-21 12:12:06.865 DEBUG 11 --- [uster-worker-49] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:12:06.866 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-7, inFlight=0, closed=true] has already terminated
2021-05-21 12:12:06.866 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:12:06.866 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 32000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:12:06.868 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 32000 ms
2021-05-21 12:12:09.016 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:12:09.017 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:12:38.869 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:12:38.874 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-8, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:12:38.965 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.80:9042-8, inFlight=0, closed=false] because: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:12:38.965 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-8, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:12:38.966 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-8, inFlight=0, closed=true] closing connection
2021-05-21 12:12:38.967 DEBUG 11 --- [uster-worker-50] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.129.80:9042
2021-05-21 12:12:38.967 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-8, inFlight=0, closed=true] has already terminated
2021-05-21 12:12:38.967 ERROR 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
2021-05-21 12:12:38.969 ERROR 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.129.80:9042, scheduling retry in 64000 milliseconds
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.129.80:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at com.datastax.driver.core.Connection$8.apply(Connection.java:503)
at com.datastax.driver.core.Connection$8.apply(Connection.java:465)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442)
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433)
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2021-05-21 12:12:38.970 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] next reconnection attempt in 64000 ms
2021-05-21 12:12:39.040 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:12:39.101 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:12:43.115 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 7
2021-05-21 12:12:43.118 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-14, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 12:12:43.118 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-14, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 12:12:43.118 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-14, inFlight=0, closed=true] closing connection
2021-05-21 12:12:43.118 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:12:43.119 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 12:12:48.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [816969788-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@8d7c9d8 at consistency LOCAL_QUORUM
2021-05-21 12:12:48.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [816969788-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:13:01.714 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:13:02.193 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 12:13:02.193 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=false] Response received on stream 4608 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ROWS [4 columns]
---
2021-05-21 12:13:09.102 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:13:09.104 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:13:10.203 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:13:10.203 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:13:11.204 DEBUG 11 --- [uster-worker-51] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:13:26.800 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:13:11.801Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:13:30.946 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:13:34.505 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:13:34.505 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:13:35.505 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:13:35.949 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-27, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:13:35.949 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-27, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:13:35.949 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-27, inFlight=0, closed=true] closing connection
2021-05-21 12:13:35.950 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 256000 milliseconds
2021-05-21 12:13:35.950 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 256000 ms
2021-05-21 12:13:40.507 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:13:11.801Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:13:40.703 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:13:40.703 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:13:41.704 DEBUG 11 --- [uster-worker-52] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:13:42.405 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:13:42.408 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:13:42.970 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:13:42.972 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:13:45.065 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:13:45.066 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.129.80:9042
2021-05-21 12:13:45.101 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to /10.242.129.80:9042, setting host UP
2021-05-21 12:13:45.101 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] adding host
2021-05-21 12:13:45.102 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.129.80:9042
2021-05-21 12:13:45.168 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.80:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:13:45.168 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.129.80:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:13:45.168 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for /10.242.129.80:9042
2021-05-21 12:13:45.169 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:13:46.169 DEBUG 11 --- [uster-worker-53] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map
2021-05-21 12:13:46.176 DEBUG 11 --- [uster-worker-53] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan.
2021-05-21 12:13:46.180 DEBUG 11 --- [uster-worker-53] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 4 milliseconds
2021-05-21 12:14:10.520 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:14:10.608 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:14:10.609 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:14:11.609 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:14:22.536 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:14:22.536 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:14:22.536 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=false] failed, remaining = 0
2021-05-21 12:14:22.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=true] closing connection
2021-05-21 12:14:22.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-27, inFlight=1, closed=true]: there are still pending requests
2021-05-21 12:14:22.537 DEBUG 11 --- [uster-worker-54] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:14:22.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2006516349-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4d6f3af9 at consistency LOCAL_QUORUM
2021-05-21 12:14:22.538 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2006516349-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.TransportException: [cassandra/10.242.184.18:9042] Connection has been closed
2021-05-21 12:14:22.539 DEBUG 11 --- [uster-worker-54] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:14:22.539 DEBUG 11 --- [uster-worker-54] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:14:23.539 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:14:23.542 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:14:35.636 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:14:35.636 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:14:35.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=true] closing connection
2021-05-21 12:14:35.637 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:14:35.637 DEBUG 11 --- [uster-worker-55] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:14:35.637 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 12:14:35.637 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 12:14:37.637 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:14:37.639 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:14:40.612 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:14:40.707 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:14:47.717 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-27, inFlight=0, closed=true] has already terminated
2021-05-21 12:14:48.380 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:14:48.381 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:14:48.403 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:14:48.403 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:14:48.403 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:14:48.403 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:14:48.429 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:14:48.429 DEBUG 11 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:14:48.429 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:14:48.429 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:15:02.717 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-28, inFlight=0, closed=true] has already terminated
2021-05-21 12:15:05.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1160802259-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1fbf7e8b at consistency LOCAL_QUORUM
2021-05-21 12:15:05.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1160802259-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:15:13.104 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:15:13.106 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:15:18.411 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:15:18.414 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:15:18.433 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:15:22.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [595155572-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@2b333777 at consistency LOCAL_QUORUM
2021-05-21 12:15:22.539 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [595155572-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:15:24.514 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Response received on stream 3648 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:15:24.514 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:15:27.899 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] Response received on stream 3712 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:15:35.482 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:15:35.562 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:15:36.990 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:15:00Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:15:48.418 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:15:48.420 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:16:07.001 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:16:08.187 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:16:13.866 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [548229331-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1b367b15 at consistency null
2021-05-21 12:16:13.866 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [548229331-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:16:18.421 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:16:18.501 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:16:40.812 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:16:40.813 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:16:41.815 DEBUG 11 --- [uster-worker-56] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:16:43.872 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:16:50.595 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:16:54.104 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:16:54.107 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:04.367 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:04.370 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:11.821 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:11.823 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:25.707 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:17:25.707 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:17:26.708 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:17:30.719 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:30.801 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:34.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [559946452-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5591e51f at consistency LOCAL_QUORUM
2021-05-21 12:17:34.937 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [559946452-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:17:35.864 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:35.866 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:47.895 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:51.951 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 2
2021-05-21 12:17:55.094 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] heartbeat query succeeded
2021-05-21 12:17:55.717 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:17:55.719 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:17:56.957 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-28, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:17:56.957 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-28, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:17:56.958 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-28, inFlight=0, closed=true] closing connection
2021-05-21 12:17:56.958 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 512000 milliseconds
2021-05-21 12:17:56.958 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 512000 ms
2021-05-21 12:17:57.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1285355406-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1a0e88b9 at consistency LOCAL_QUORUM
2021-05-21 12:17:57.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1285355406-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:18:03.192 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Response received on stream 4352 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:18:03.193 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Response received on stream 4288 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:18:10.267 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:18:10.269 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:18:25.733 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:18:25.800 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:18:33.198 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:18:33.204 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:18:35.201 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:18:35.202 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:18:36.204 DEBUG 11 --- [uster-worker-57] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:18:53.102 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:18:53.103 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:18:54.103 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:19:00.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [972082635-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7ea1f655 at consistency LOCAL_QUORUM
2021-05-21 12:19:00.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [972082635-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:19:03.206 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:19:07.033 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:19:11.317 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] Response received on stream 4544 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:19:13.403 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:19:13.405 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:19:21.613 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:19:05.069Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:19:23.119 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:19:23.201 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:19:35.673 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:19:35.675 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:19:43.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1256278630-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4c9bfe79 at consistency LOCAL_QUORUM
2021-05-21 12:19:43.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1256278630-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:19:47.386 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] Response received on stream 4736 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses.
2021-05-21 12:19:53.201 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:19:53.203 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:20:17.389 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:20:17.417 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:20:23.203 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:20:23.207 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:20:30.694 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [327027609-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7f96d8a5 at consistency null
2021-05-21 12:20:30.694 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [327027609-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:20:40.908 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [938617812-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@63e0df3d at consistency null
2021-05-21 12:20:40.909 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [938617812-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:20:48.485 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [892558882-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1be576b9 at consistency null
2021-05-21 12:20:48.486 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [892558882-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:20:53.208 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:20:53.218 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:20:53.813 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [300346253-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4b19eeeb at consistency null
2021-05-21 12:20:53.814 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [300346253-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:21:23.219 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:21:23.301 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:21:23.823 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:21:24.490 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:21:34.825 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1416604139-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@571f887f at consistency null
2021-05-21 12:21:34.826 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1416604139-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:21:41.978 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1837178352-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@53f55ac1 at consistency null
2021-05-21 12:21:41.978 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1837178352-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:21:53.302 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:21:53.305 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:21:57.290 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [23049974-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4bc849ce at consistency null
2021-05-21 12:21:57.290 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [23049974-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:22:23.318 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:22:23.320 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:22:25.429 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [971253481-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@57ff2972 at consistency null
2021-05-21 12:22:25.429 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [971253481-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:22:31.786 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [236470566-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7c080f82 at consistency null
2021-05-21 12:22:31.786 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [236470566-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:22:43.119 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 8
2021-05-21 12:22:43.124 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-15, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 12:22:43.124 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-15, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 12:22:43.124 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-15, inFlight=0, closed=true] closing connection
2021-05-21 12:22:43.125 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:22:43.126 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 12:22:53.321 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:22:53.323 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:23:01.791 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:23:01.905 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:23:12.780 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [536268529-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5526190 at consistency null
2021-05-21 12:23:12.780 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [536268529-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:23:23.323 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:23:23.401 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:23:42.333 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1786122835-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@cddcecd at consistency null
2021-05-21 12:23:42.333 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.RequestHandler : [1786122835-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:23:47.807 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:23:47.808 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:23:48.809 DEBUG 11 --- [uster-worker-58] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:23:58.367 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:23:58.370 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:24:01.992 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:23:47.002Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:24:18.904 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:24:18.914 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:24:31.989 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:24:44.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:24:44.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:24:44.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=false] failed, remaining = 0
2021-05-21 12:24:44.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=true] closing connection
2021-05-21 12:24:44.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-29, inFlight=1, closed=true]: there are still pending requests
2021-05-21 12:24:44.038 DEBUG 11 --- [uster-worker-59] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:24:44.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1652675222-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@9bf18ce at consistency LOCAL_QUORUM
2021-05-21 12:24:44.038 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1652675222-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.TransportException: [cassandra/10.242.184.18:9042] Connection has been closed
2021-05-21 12:24:44.042 DEBUG 11 --- [uster-worker-59] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:24:44.042 DEBUG 11 --- [uster-worker-59] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:24:45.042 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:24:45.045 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:24:47.380 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:24:47.462 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:24:48.915 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:24:48.917 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:25:11.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:25:11.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:25:11.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=true] closing connection
2021-05-21 12:25:11.338 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:25:11.339 DEBUG 11 --- [uster-worker-60] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:25:11.339 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 12:25:11.339 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 12:25:13.339 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:25:13.341 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:25:17.727 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-29, inFlight=0, closed=true] has already terminated
2021-05-21 12:25:18.906 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:25:18.906 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:25:19.906 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:25:25.437 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:25:25.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:25:25.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=true] closing connection
2021-05-21 12:25:25.438 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:25:25.439 DEBUG 11 --- [uster-worker-61] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:25:25.440 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 12:25:25.440 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 4000 ms
2021-05-21 12:25:29.441 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:25:29.443 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:25:42.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:25:42.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:25:42.139 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=true] closing connection
2021-05-21 12:25:42.139 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:25:42.139 DEBUG 11 --- [uster-worker-62] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:25:42.146 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 12:25:42.146 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 8000 ms
2021-05-21 12:25:47.728 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-30, inFlight=0, closed=true] has already terminated
2021-05-21 12:25:48.909 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:25:48.911 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:25:50.147 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:25:50.149 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:25:50.317 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:25:50.319 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:26:02.728 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-31, inFlight=0, closed=true] has already terminated
2021-05-21 12:26:07.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:26:07.737 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:26:07.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=true] closing connection
2021-05-21 12:26:07.738 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:26:07.738 DEBUG 11 --- [uster-worker-63] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:26:07.739 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 16000 milliseconds
2021-05-21 12:26:07.739 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 16000 ms
2021-05-21 12:26:17.729 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-32, inFlight=0, closed=true] has already terminated
2021-05-21 12:26:18.913 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:26:18.915 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:26:23.739 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:26:23.741 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:26:28.959 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 3
2021-05-21 12:26:32.729 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-33, inFlight=0, closed=true] has already terminated
2021-05-21 12:26:33.961 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-29, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:26:33.961 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-29, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:26:33.962 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-29, inFlight=0, closed=true] closing connection
2021-05-21 12:26:33.962 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:26:33.962 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 600000 ms
2021-05-21 12:26:43.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:26:43.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:26:43.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=true] closing connection
2021-05-21 12:26:43.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:26:43.337 DEBUG 11 --- [uster-worker-64] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:26:43.338 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 32000 milliseconds
2021-05-21 12:26:43.338 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 32000 ms
2021-05-21 12:26:48.916 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:26:48.918 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:27:15.338 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:27:15.342 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:27:17.731 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-34, inFlight=0, closed=true] has already terminated
2021-05-21 12:27:18.918 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:27:19.003 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:27:26.199 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:27:26.199 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:27:26.204 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:27:26.204 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:27:26.204 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:27:26.204 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:27:33.526 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:27:33.526 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:27:33.526 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:27:33.527 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:27:37.073 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [1840766288-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@29eff35d at consistency null
2021-05-21 12:27:37.073 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [1840766288-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:27:48.926 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [830732116-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@451a467c at consistency null
2021-05-21 12:27:48.926 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [830732116-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:27:55.265 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [720593733-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6c36f91d at consistency null
2021-05-21 12:27:55.265 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [720593733-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:27:56.209 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:27:56.211 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:28:25.278 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:28:25.280 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:28:26.215 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:28:26.217 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:28:30.577 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [806553690-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@296eae4a at consistency null
2021-05-21 12:28:30.577 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [806553690-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:28:56.218 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:28:56.220 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:29:00.581 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:29:06.784 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:29:21.106 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [157630551-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7fcdd521 at consistency null
2021-05-21 12:29:21.106 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [157630551-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:29:26.234 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:29:26.235 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:29:44.037 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [2001235316-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@68f35502 at consistency null
2021-05-21 12:29:44.037 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [2001235316-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:29:56.240 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:29:56.242 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:29:57.305 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [657169221-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@3356b710 at consistency null
2021-05-21 12:29:57.305 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.RequestHandler : [657169221-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive)
2021-05-21 12:30:01.305 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:30:01.305 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:30:01.362 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:30:01.364 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:30:02.306 DEBUG 11 --- [uster-worker-65] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:30:32.312 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:30:32.331 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:30:32.377 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:30:32.401 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:31:02.402 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:31:02.403 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:31:02.978 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:31:02.979 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:31:32.411 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:31:32.412 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:31:53.578 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:31:53.581 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:32:02.414 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:32:02.421 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:32:29.370 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:32:29.372 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:32:32.422 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:32:32.501 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:32:43.126 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] preparing to open 1 new connections, total = 9
2021-05-21 12:32:43.130 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-16, inFlight=0, closed=false] Error connecting to /10.242.129.86:9042 (Connection refused: /10.242.129.86:9042)
2021-05-21 12:32:43.130 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.86:9042-16, inFlight=0, closed=false] because: [/10.242.129.86:9042] Cannot connect
2021-05-21 12:32:43.130 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.129.86:9042-16, inFlight=0, closed=true] closing connection
2021-05-21 12:32:43.131 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.86:9042 ([/10.242.129.86:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:32:43.131 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.129.86:9042] next reconnection attempt in 600000 ms
2021-05-21 12:32:54.914 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:32:54.915 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:33:02.502 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:33:02.505 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:33:21.009 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:33:21.009 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:33:22.010 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:33:24.944 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:33:24.946 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:33:33.786 WARN 11 --- [pool-8-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-21T12:33:18.250Z] (this read will be retried)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 9 common frames omitted
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126)
at io.vavr.API$Match$Case0.apply(API.java:3174)
at io.vavr.API$Match.of(API.java:3137)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123)
at io.vavr.control.Try.getOrElseThrow(Try.java:671)
at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798)
at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)
at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 10 common frames omitted
Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at io.vavr.control.Try.of(Try.java:62)
at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124)
at com.datastax.driver.core.Responses$Error.asException(Responses.java:169)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 common frames omitted
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 26 common frames omitted
2021-05-21 12:33:51.012 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:33:51.014 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:34:03.786 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:34:11.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1162265246-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@4c0772c7 at consistency LOCAL_QUORUM
2021-05-21 12:34:11.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1162265246-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:34:15.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:34:15.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:34:15.837 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:34:15.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=true] closing connection
2021-05-21 12:34:15.838 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:34:15.838 DEBUG 11 --- [uster-worker-66] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:34:15.840 DEBUG 11 --- [uster-worker-66] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:34:15.840 DEBUG 11 --- [uster-worker-66] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:34:16.840 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:34:16.843 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-36, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:34:21.017 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:34:21.101 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:34:24.119 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:34:24.201 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:34:29.493 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-36, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:34:29.493 DEBUG 11 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:34:29.501 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:34:29.502 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:34:29.502 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:34:29.502 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:34:30.838 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:34:30.838 DEBUG 11 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:34:30.838 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:34:30.839 DEBUG 11 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:34:47.738 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-35, inFlight=0, closed=true] has already terminated
2021-05-21 12:34:49.205 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:34:49.205 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:34:50.207 DEBUG 11 --- [uster-worker-67] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:35:01.507 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:35:01.508 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:35:20.214 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:35:20.217 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:35:31.536 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:35:31.601 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-1, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:35:37.089 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-36, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:35:39.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [61955390-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7aa08f62 at consistency LOCAL_QUORUM
2021-05-21 12:35:39.537 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [61955390-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra/10.242.184.18:9042] Timed out waiting for server response
2021-05-21 12:35:44.714 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.184.18:9042, scheduling delivery
2021-05-21 12:35:44.715 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event DOWN
2021-05-21 12:35:45.715 DEBUG 11 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on cassandra/10.242.184.18:9042 because it still has active connections
2021-05-21 12:35:49.136 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-36, inFlight=1, closed=false] because: [cassandra/10.242.184.18:9042] Heartbeat query timed out
2021-05-21 12:35:49.136 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preventing new connections for the next 1000 ms
2021-05-21 12:35:49.137 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-36, inFlight=1, closed=false] failed, remaining = 0
2021-05-21 12:35:49.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-36, inFlight=1, closed=true] closing connection
2021-05-21 12:35:49.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-36, inFlight=1, closed=true]: there are still pending requests
2021-05-21 12:35:49.138 DEBUG 11 --- [uster-worker-68] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host DOWN
2021-05-21 12:35:49.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1575605919-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6e4f406c at consistency LOCAL_QUORUM
2021-05-21 12:35:49.138 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1575605919-0] Error querying cassandra/10.242.184.18:9042 : com.datastax.driver.core.exceptions.TransportException: [cassandra/10.242.184.18:9042] Connection has been closed
2021-05-21 12:35:49.138 DEBUG 11 --- [uster-worker-68] com.datastax.driver.core.RequestHandler : onException triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=1, inProgress=true, cancelled=false), queryStateRef = QueryState(count=1, inProgress=true, cancelled=false))
2021-05-21 12:35:49.139 DEBUG 11 --- [uster-worker-68] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms
2021-05-21 12:35:49.140 DEBUG 11 --- [uster-worker-68] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] starting reconnection attempt
2021-05-21 12:35:50.140 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:35:50.143 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:35:52.463 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:35:52.466 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.129.80:9042-9, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:36:02.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:36:02.237 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:36:02.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=true] closing connection
2021-05-21 12:36:02.238 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:36:02.238 DEBUG 11 --- [uster-worker-69] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:36:02.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 2000 milliseconds
2021-05-21 12:36:02.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 2000 ms
2021-05-21 12:36:04.239 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:36:04.242 DEBUG 11 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:36:14.719 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:36:14.801 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:36:16.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:36:16.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:36:16.336 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=true] closing connection
2021-05-21 12:36:16.337 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:36:16.337 DEBUG 11 --- [uster-worker-68] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:36:16.337 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 4000 milliseconds
2021-05-21 12:36:16.337 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 4000 ms
2021-05-21 12:36:17.740 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-36, inFlight=0, closed=true] has already terminated
2021-05-21 12:36:20.337 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:36:20.339 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-39, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:36:32.740 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-37, inFlight=0, closed=true] has already terminated
2021-05-21 12:36:33.962 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] preparing to open 1 new connections, total = 4
2021-05-21 12:36:37.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[cassandra/10.242.184.18:9042-39, inFlight=0, closed=false] because: [cassandra/10.242.184.18:9042] Operation timed out
2021-05-21 12:36:37.036 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-39, inFlight=0, closed=false] failed, remaining = 0
2021-05-21 12:36:37.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-39, inFlight=0, closed=true] closing connection
2021-05-21 12:36:37.037 DEBUG 11 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[cassandra/10.242.184.18:9042-39, inFlight=0, closed=true]: there are still pending requests
2021-05-21 12:36:37.037 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to cassandra/10.242.184.18:9042 ([cassandra/10.242.184.18:9042] Operation timed out), scheduling retry in 8000 milliseconds
2021-05-21 12:36:37.037 DEBUG 11 --- [uster-worker-70] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host cassandra/10.242.184.18:9042
2021-05-21 12:36:37.037 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] next reconnection attempt in 8000 ms
2021-05-21 12:36:38.967 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-30, inFlight=0, closed=false] Error connecting to /10.242.129.101:9042 (connection timed out: /10.242.129.101:9042)
2021-05-21 12:36:38.967 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.129.101:9042-30, inFlight=0, closed=false] because: [/10.242.129.101:9042] Cannot connect
2021-05-21 12:36:38.967 DEBUG 11 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.129.101:9042-30, inFlight=0, closed=true] closing connection
2021-05-21 12:36:38.967 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.129.101:9042 ([/10.242.129.101:9042] Cannot connect), scheduling retry in 600000 milliseconds
2021-05-21 12:36:38.967 DEBUG 11 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.129.101:9042] next reconnection attempt in 600000 ms
2021-05-21 12:36:44.802 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2021-05-21 12:36:44.803 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.131.197:9042-14, inFlight=0, closed=false] heartbeat query succeeded
2021-05-21 12:36:45.038 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 1 new connections, total = 1
2021-05-21 12:36:45.040 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-40, inFlight=0, closed=false] Connection established, initializing transport
2021-05-21 12:36:47.741 DEBUG 11 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.184.18:9042-38, inFlight=0, closed=true] has already terminated
2021-05-21 12:36:50.204 DEBUG 11 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] Connection[cassandra/10.242.184.18:9042-40, inFlight=0, closed=false] Transport initialized, connection ready
2021-05-21 12:36:50.205 DEBUG 11 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042
2021-05-21 12:36:50.208 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to cassandra/10.242.184.18:9042, setting host UP
2021-05-21 12:36:50.208 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] marking host UP
2021-05-21 12:36:50.208 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP
2021-05-21 12:36:50.208 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node cassandra/10.242.184.18:9042
2021-05-21 12:36:50.286 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.184.18:9042] preparing to open 0 new connections, total = 1
2021-05-21 12:36:50.287 DEBUG 11 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.184.18:9042 (1 connections needed, 1 successfully opened)
2021-05-21 12:36:50.287 DEBUG 11 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.184.18:9042
2021-05-21 12:36:50.287 DEBUG 11 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future
2021-05-21 12:36:50.509 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.184.18:9042, scheduling delivery
2021-05-21 12:36:50.509 DEBUG 11 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.184.18:9042] received event UP
2021-05-21 12:36:51.510 DEBUG 11 --- [uster-worker-71] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on cassandra/10.242.184.18:9042