By type
Project Build Version: 1.0.0 2021-05-02 22:54:53.784 INFO --- [ main] org.onap.aai.util.AAIConfig : Initializing AAIConfig 2021-05-02 22:54:53.795 DEBUG --- [ main] org.onap.aai.util.AAIConfig : Reloading config from /opt/app/aai-graphadmin/./resources/etc/appprops/aaiconfig.properties 2021-05-02 22:54:53.795 INFO --- [ main] org.onap.aai.util.AAIConfig : Found the aaiconfig.properties in the following location: /opt/app/aai-graphadmin/./resources/etc/appprops/aaiconfig.properties 2021-05-02 22:54:53.798 INFO --- [ main] org.onap.aai.util.AAIConfig : A&AI Server Node Name = aai.config.nodename 2021-05-02 22:54:55.730 WARN 1 --- [ main] o.s.b.l.logback.LogbackLoggingSystem : Ignoring 'logback.configurationFile' system property. Please use 'logging.config' instead. . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v2.1.12.RELEASE) 2021-05-02 22:54:56.410 INFO 1 --- [ main] org.onap.aai.GraphAdminApp : Starting GraphAdminApp on aai-graphadmin with PID 1 (/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar started by aaiadmin in /opt/app/aai-graphadmin) 2021-05-02 22:54:56.411 DEBUG 1 --- [ main] org.onap.aai.GraphAdminApp : Running with Spring Boot v2.1.12.RELEASE, Spring v5.1.13.RELEASE 2021-05-02 22:54:56.411 INFO 1 --- [ main] org.onap.aai.GraphAdminApp : The following profiles are active: dmaap,one-way-ssl 22:55:01,907 |-INFO in LogbackRequestLog - Will use configuration resource [/localhost-access-logback.xml] 22:55:01,919 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - debug attribute not set 22:55:01,919 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender] 22:55:01,919 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS] 22:55:01,920 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1138410383 - No compression will be used 22:55:01,920 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1138410383 - Will use the pattern /opt/app/aai-graphadmin/logs/ajsc-jetty/localhost_access.log.%d{yyyy-MM-dd} for the active file 22:55:01,920 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/app/aai-graphadmin/logs/ajsc-jetty/localhost_access.log.%d{yyyy-MM-dd}'. 22:55:01,920 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight. 22:55:01,921 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Sun May 02 22:55:01 GMT 2021 22:55:01,944 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: /opt/app/aai-graphadmin/logs/ajsc-jetty/localhost_access.log 22:55:01,944 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [/opt/app/aai-graphadmin/logs/ajsc-jetty/localhost_access.log] 22:55:01,944 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to null 22:55:01,944 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - End of configuration. 22:55:01,944 |-INFO in ch.qos.logback.access.joran.JoranConfigurator@342c38f8 - Registering current configuration as safe fallback point 2021-05-02 22:55:02.196 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/lib/aai-els-onap-logging-1.8.1.jar!/ 2021-05-02 22:55:02.265 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 67 ms to scan 1 urls, producing 6 keys and 6 values 2021-05-02 22:55:02.270 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/classes!/ 2021-05-02 22:55:02.287 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 16 ms to scan 1 urls, producing 7 keys and 53 values 2021-05-02 22:55:02.322 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/lib/aai-els-onap-logging-1.8.1.jar!/ 2021-05-02 22:55:02.328 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 5 ms to scan 1 urls, producing 6 keys and 6 values 2021-05-02 22:55:02.332 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/classes!/ 2021-05-02 22:55:02.347 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 14 ms to scan 1 urls, producing 7 keys and 53 values 2021-05-02 22:55:02.354 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/lib/aai-els-onap-logging-1.8.1.jar!/ 2021-05-02 22:55:02.359 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 5 ms to scan 1 urls, producing 6 keys and 6 values 2021-05-02 22:55:02.363 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: jar:file:/opt/app/aai-graphadmin/lib/aai-graphadmin-1.8.0.jar!/BOOT-INF/classes!/ 2021-05-02 22:55:02.377 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 14 ms to scan 1 urls, producing 7 keys and 53 values 2021-05-02 22:55:02.812 INFO 1 --- [ main] org.onap.aai.config.NodesConfiguration : Translator is SchemaServiceTranslator 2021-05-02 22:55:03.308 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:03.308 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:03.308 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:03.309 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:03.319 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/versions, BODY= 2021-05-02 22:55:03.433 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:03.433 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:03.434 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:03.434 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:03.838 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:03.996 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Processing the translator 2021-05-02 22:55:03.997 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv11 2021-05-02 22:55:03.997 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:04.111 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:04.111 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:04.111 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:04.111 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:04.145 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:04.218 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:04.218 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:04.219 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:08.293 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 222 2021-05-02 22:55:08.329 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv12 2021-05-02 22:55:08.330 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:08.332 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:08.333 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:08.333 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:08.333 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:08.356 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:08.375 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:08.376 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:08.376 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:10.111 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 235 2021-05-02 22:55:10.127 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv13 2021-05-02 22:55:10.128 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:10.130 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:10.130 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:10.130 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:10.130 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:10.184 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:10.209 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:10.210 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:10.210 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:12.239 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 237 2021-05-02 22:55:12.303 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv14 2021-05-02 22:55:12.304 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:12.306 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:12.306 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:12.306 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:12.306 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:12.555 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:12.604 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:12.604 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:12.605 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:14.390 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 272 2021-05-02 22:55:14.406 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv15 2021-05-02 22:55:14.406 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:14.408 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:14.408 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:14.408 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:14.408 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:14.434 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:14.487 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:14.488 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:14.488 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:16.470 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 279 2021-05-02 22:55:16.490 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv16 2021-05-02 22:55:16.490 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:16.493 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:16.493 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:16.493 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:16.493 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:16.513 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:16.531 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:16.531 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:16.531 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:18.350 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281 2021-05-02 22:55:18.362 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv17 2021-05-02 22:55:18.363 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:18.366 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:18.367 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:18.367 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:18.367 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:18.393 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:18.418 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:18.418 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:18.419 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:20.106 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281 2021-05-02 22:55:20.128 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv18 2021-05-02 22:55:20.129 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:20.132 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:20.132 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:20.133 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:20.133 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:20.157 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:20.185 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:20.185 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:20.186 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:21.830 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 281 2021-05-02 22:55:21.845 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv19 2021-05-02 22:55:21.845 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:21.848 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:21.848 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:21.849 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:21.849 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:21.885 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:21.911 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:21.912 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:21.912 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:24.129 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 301 2021-05-02 22:55:24.141 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv20 2021-05-02 22:55:24.142 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:24.144 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:24.144 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:24.144 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:24.144 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:24.173 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:24.208 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:24.209 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:24.209 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:25.771 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 305 2021-05-02 22:55:25.782 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv21 2021-05-02 22:55:25.782 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:25.785 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:25.785 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:25.785 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:25.785 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:25.830 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:25.864 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:25.865 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:25.865 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:27.970 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 371 2021-05-02 22:55:27.994 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv22 2021-05-02 22:55:27.994 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:27.997 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:27.997 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:27.997 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:27.997 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:28.017 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:28.065 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:28.066 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:28.066 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:30.311 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 378 2021-05-02 22:55:30.331 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Version being processedv23 2021-05-02 22:55:30.331 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Headers: {Accept=application/xml, Content-Type=application/xml} 2021-05-02 22:55:30.334 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:30.334 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:30.334 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:30.334 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:30.354 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:30.389 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:30.389 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Retrieved OXMs from SchemaService 2021-05-02 22:55:30.390 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Ingested the InputStream 2021-05-02 22:55:32.655 DEBUG 1 --- [ main] org.onap.aai.nodes.NodeIngestor : Types size 378 2021-05-02 22:55:32.924 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:32.924 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:32.924 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:32.926 INFO 1 --- [ main] o.onap.aai.rest.client.ApertureService : Successfully initialized the aperture service 2021-05-02 22:55:33.293 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Processing the translator 2021-05-02 22:55:33.296 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv11 2021-05-02 22:55:33.297 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.298 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v11, BODY= 2021-05-02 22:55:33.300 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.300 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.300 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.300 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.316 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.319 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.319 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.320 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv12 2021-05-02 22:55:33.321 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.322 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v12, BODY= 2021-05-02 22:55:33.323 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.324 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.324 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.324 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.337 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.339 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.340 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.341 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv13 2021-05-02 22:55:33.341 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.342 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v13, BODY= 2021-05-02 22:55:33.344 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.344 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.344 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.344 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.383 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.388 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.389 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.390 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv14 2021-05-02 22:55:33.390 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.392 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v14, BODY= 2021-05-02 22:55:33.394 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.394 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.394 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.394 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.410 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.415 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.416 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.416 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv15 2021-05-02 22:55:33.417 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.418 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v15, BODY= 2021-05-02 22:55:33.420 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.421 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.421 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.421 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.431 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.436 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.437 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.438 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv16 2021-05-02 22:55:33.438 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.439 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v16, BODY= 2021-05-02 22:55:33.441 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.441 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.441 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.441 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.455 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.459 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.460 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.483 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv17 2021-05-02 22:55:33.484 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.485 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v17, BODY= 2021-05-02 22:55:33.487 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.487 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.487 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.487 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.498 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.501 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.502 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.503 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv18 2021-05-02 22:55:33.503 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.504 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v18, BODY= 2021-05-02 22:55:33.506 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.506 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.506 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.506 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.519 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.523 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.524 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.524 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv19 2021-05-02 22:55:33.525 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.525 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v19, BODY= 2021-05-02 22:55:33.527 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.527 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.527 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.527 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.543 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.547 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.548 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.548 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv20 2021-05-02 22:55:33.549 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.550 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v20, BODY= 2021-05-02 22:55:33.551 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.552 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.552 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.552 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.562 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.566 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.566 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.567 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv21 2021-05-02 22:55:33.567 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.568 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v21, BODY= 2021-05-02 22:55:33.570 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.570 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.570 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.570 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.594 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.600 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.601 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.601 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv22 2021-05-02 22:55:33.602 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.603 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v22, BODY= 2021-05-02 22:55:33.604 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.604 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.604 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.604 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.620 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.626 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.626 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:33.627 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Version being processedv23 2021-05-02 22:55:33.627 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : Request Headers: {} 2021-05-02 22:55:33.628 DEBUG 1 --- [ main] org.onap.aai.restclient.RestClient : METHOD=GET, URL=https://aai-schema-service.onap:8452/aai/schema-service/v1/edgerules?version=v23, BODY= 2021-05-02 22:55:33.629 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Checking for system property [partnerName] 2021-05-02 22:55:33.629 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : System property was null or empty. Checking environment variable for: partnerName 2021-05-02 22:55:33.629 INFO 1 --- [ main] org.onap.logging.filter.base.MDCSetup : Environment variable: partnerName was null or empty 2021-05-02 22:55:33.630 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : Invoke 2021-05-02 22:55:33.642 INFO 1 --- [ main] o.o.l.f.base.AbstractMetricLogFilter : InvokeReturn 2021-05-02 22:55:33.648 DEBUG 1 --- [ main] o.o.aai.setup.SchemaServiceTranslator : SchemaResponse Status code200 OK 2021-05-02 22:55:33.649 DEBUG 1 --- [ main] org.onap.aai.edges.EdgeIngestor : Retrieved json from SchemaService 2021-05-02 22:55:34.244 WARN 1 --- [ main] o.apache.activemq.broker.BrokerService : Memory Usage for the Broker (1024mb) is more than the maximum available for the JVM: 972 mb - resetting to 70% of maximum available: 680 mb 2021-05-02 22:55:35.691 INFO 1 --- [ main] com.datastax.driver.core : DataStax Java driver 3.6.0 for Apache Cassandra 2021-05-02 22:55:35.708 INFO 1 --- [ main] c.d.driver.core.GuavaCompatibility : Detected Guava >= 19 in the classpath, using modern compatibility layer 2021-05-02 22:55:35.709 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NEW_NODE_DELAY_SECONDS is undefined, using default value 1 2021-05-02 22:55:35.709 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NOTIF_LOCK_TIMEOUT_SECONDS is undefined, using default value 60 2021-05-02 22:55:35.778 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true 2021-05-02 22:55:36.116 INFO 1 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps. 2021-05-02 22:55:36.119 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.NON_BLOCKING_EXECUTOR_SIZE is undefined, using default value 2 2021-05-02 22:55:36.207 WARN 1 --- [ main] .b.a.g.t.GroovyTemplateAutoConfiguration : Cannot find template location: classpath:/templates/ (please add some templates, check your Groovy configuration, or set spring.groovy.template.check-template-location=false) 2021-05-02 22:55:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 22:55:37.289 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 22:55:40.014 WARN 1 --- [ main] org.glassfish.jersey.internal.Errors : The following warnings have been detected: WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.onap.aai.rest.AuditSqlDbConsumer.executeAudit(java.lang.String,java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity. 2021-05-02 22:55:40.394 INFO 1 --- [ main] org.onap.aai.GraphAdminApp : Started GraphAdminApp in 46.2 seconds (JVM running for 48.37) 2021-05-02 22:55:40.398 INFO 1 --- [ main] org.onap.aai.GraphAdminApp : Application 'aai-graphadmin' is running on 8449! 2021-05-02 22:55:40.829 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true 2021-05-02 22:55:40.829 INFO 1 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps. 2021-05-02 22:55:40.830 DEBUG 1 --- [ main] com.datastax.driver.core.Cluster : Starting new cluster with contact points [cassandra/10.242.5.156:9042] 2021-05-02 22:55:40.883 DEBUG 1 --- [ main] i.n.u.i.logging.InternalLoggerFactory : Using SLF4J as the default logging framework 2021-05-02 22:55:40.952 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : -Dio.netty.noUnsafe: false 2021-05-02 22:55:40.953 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : Java version: 8 2021-05-02 22:55:40.955 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : sun.misc.Unsafe.theUnsafe: available 2021-05-02 22:55:40.956 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : sun.misc.Unsafe.copyMemory: available 2021-05-02 22:55:40.958 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.Buffer.address: available 2021-05-02 22:55:40.959 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : direct buffer constructor: available 2021-05-02 22:55:40.960 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.Bits.unaligned: available, true 2021-05-02 22:55:40.961 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 2021-05-02 22:55:40.961 DEBUG 1 --- [ main] i.n.util.internal.PlatformDependent0 : java.nio.DirectByteBuffer.
(long, int): available 2021-05-02 22:55:40.961 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : sun.misc.Unsafe: available 2021-05-02 22:55:40.961 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.tmpdir: /tmp (java.io.tmpdir) 2021-05-02 22:55:40.961 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.bitMode: 64 (sun.arch.data.model) 2021-05-02 22:55:40.964 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.noPreferDirect: false 2021-05-02 22:55:40.965 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.maxDirectMemory: 1020067840 bytes 2021-05-02 22:55:40.965 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : -Dio.netty.uninitializedArrayAllocationThreshold: -1 2021-05-02 22:55:40.966 DEBUG 1 --- [ main] io.netty.util.internal.CleanerJava6 : java.nio.ByteBuffer.cleaner(): available 2021-05-02 22:55:40.982 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FORCE_NIO is undefined, using default value false 2021-05-02 22:55:40.990 DEBUG 1 --- [ main] i.n.util.internal.NativeLibraryLoader : -Dio.netty.native.workdir: /tmp (io.netty.tmpdir) 2021-05-02 22:55:40.997 DEBUG 1 --- [ main] i.n.util.internal.NativeLibraryLoader : Unable to load the library 'netty_transport_native_epoll_x86_64', trying other loading mechanism. java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287) at io.netty.channel.epoll.Native. (Native.java:69) at io.netty.channel.epoll.Epoll. (Epoll.java:33) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at com.datastax.driver.core.NettyUtil. (NettyUtil.java:57) at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95) at com.datastax.driver.core.Connection$Factory. (Connection.java:926) at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499) at com.datastax.driver.core.Cluster.init(Cluster.java:208) at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376) at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355) at com.datastax.driver.core.Cluster.connect(Cluster.java:305) at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272) at org.janusgraph.diskstorage.cql.CQLStoreManager. (CQLStoreManager.java:165) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58) at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477) at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409) at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration. (GraphDatabaseConfiguration.java:1382) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113) at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107) at org.onap.aai.dbmap.AAIGraph. (AAIGraph.java:73) at org.onap.aai.dbmap.AAIGraph. (AAIGraph.java:54) at org.onap.aai.dbmap.AAIGraph$Helper. (AAIGraph.java:80) at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94) at org.onap.aai.GraphAdminApp.main(GraphAdminApp.java:135) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:595) 2021-05-02 22:55:40.998 DEBUG 1 --- [ main] i.n.util.internal.NativeLibraryLoader : netty_transport_native_epoll_x86_64 cannot be loaded from java.libary.path, now trying export to -Dio.netty.native.workdir: /tmp java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:287) at io.netty.channel.epoll.Native. (Native.java:69) at io.netty.channel.epoll.Epoll. (Epoll.java:33) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at com.datastax.driver.core.NettyUtil. (NettyUtil.java:57) at com.datastax.driver.core.NettyOptions.eventLoopGroup(NettyOptions.java:95) at com.datastax.driver.core.Connection$Factory. (Connection.java:926) at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499) at com.datastax.driver.core.Cluster.init(Cluster.java:208) at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376) at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355) at com.datastax.driver.core.Cluster.connect(Cluster.java:305) at org.janusgraph.diskstorage.cql.CQLStoreManager.initializeSession(CQLStoreManager.java:272) at org.janusgraph.diskstorage.cql.CQLStoreManager. (CQLStoreManager.java:165) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58) at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477) at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409) at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration. (GraphDatabaseConfiguration.java:1382) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113) at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:107) at org.onap.aai.dbmap.AAIGraph. (AAIGraph.java:73) at org.onap.aai.dbmap.AAIGraph. (AAIGraph.java:54) at org.onap.aai.dbmap.AAIGraph$Helper. (AAIGraph.java:80) at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:94) at org.onap.aai.GraphAdminApp.main(GraphAdminApp.java:135) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:595) Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) ... 41 common frames omitted 2021-05-02 22:55:41.025 DEBUG 1 --- [ main] i.n.util.internal.NativeLibraryLoader : Successfully loaded the library /tmp/libnetty_transport_native_epoll_x86_647881549874230491671.so 2021-05-02 22:55:41.032 DEBUG 1 --- [ main] io.netty.util.NetUtil : Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo) 2021-05-02 22:55:41.033 DEBUG 1 --- [ main] io.netty.util.NetUtil : /proc/sys/net/core/somaxconn: 128 2021-05-02 22:55:41.034 INFO 1 --- [ main] com.datastax.driver.core.NettyUtil : Found Netty's native epoll transport in the classpath, using it 2021-05-02 22:55:41.131 DEBUG 1 --- [ main] i.n.channel.MultithreadEventLoopGroup : -Dio.netty.eventLoopThreads: 4 2021-05-02 22:55:41.209 DEBUG 1 --- [ main] i.netty.util.internal.PlatformDependent : org.jctools-core.MpscChunkedArrayQueue: available 2021-05-02 22:55:41.229 DEBUG 1 --- [ main] io.netty.util.ResourceLeakDetector : -Dio.netty.leakDetection.level: simple 2021-05-02 22:55:41.229 DEBUG 1 --- [ main] io.netty.util.ResourceLeakDetector : -Dio.netty.leakDetection.targetRecords: 4 2021-05-02 22:55:41.229 DEBUG 1 --- [ main] i.n.util.ResourceLeakDetectorFactory : Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@383d2e2d 2021-05-02 22:55:41.243 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.EXTENDED_PEER_CHECK is undefined, using default value true 2021-05-02 22:55:41.321 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:41.330 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.DISABLE_COALESCING is undefined, using default value false 2021-05-02 22:55:41.330 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FLUSHER_SCHEDULE_PERIOD_NS is undefined, using default value 10000 2021-05-02 22:55:41.330 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.FLUSHER_RUN_WITHOUT_WORK_TIMES is undefined, using default value 5 2021-05-02 22:55:41.351 DEBUG 1 --- [ main] i.n.u.internal.InternalThreadLocalMap : -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 2021-05-02 22:55:41.352 DEBUG 1 --- [ main] i.n.u.internal.InternalThreadLocalMap : -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 2021-05-02 22:55:41.408 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.numHeapArenas: 4 2021-05-02 22:55:41.408 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.numDirectArenas: 4 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.pageSize: 8192 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.maxOrder: 11 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.chunkSize: 16777216 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.tinyCacheSize: 512 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.smallCacheSize: 256 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.normalCacheSize: 64 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.maxCachedBufferCapacity: 32768 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.cacheTrimInterval: 8192 2021-05-02 22:55:41.409 DEBUG 1 --- [ main] io.netty.buffer.PooledByteBufAllocator : -Dio.netty.allocator.useCacheForAllThreads: true 2021-05-02 22:55:41.518 DEBUG 1 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.allocator.type: unpooled 2021-05-02 22:55:41.518 DEBUG 1 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.threadLocalDirectBufferSize: 65536 2021-05-02 22:55:41.518 DEBUG 1 --- [ main] io.netty.buffer.ByteBufUtil : -Dio.netty.maxThreadLocalCharBufferSize: 16384 2021-05-02 22:55:41.605 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:41.660 DEBUG 1 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.maxCapacity.default: 32768 2021-05-02 22:55:41.660 DEBUG 1 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.maxSharedCapacityFactor: 2 2021-05-02 22:55:41.660 DEBUG 1 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.linkCapacity: 16 2021-05-02 22:55:41.660 DEBUG 1 --- [er-nio-worker-0] io.netty.util.Recycler : -Dio.netty.recycler.ratio: 8 2021-05-02 22:55:41.677 DEBUG 1 --- [er-nio-worker-0] io.netty.buffer.AbstractByteBuf : -Dio.netty.buffer.bytebuf.checkAccessible: true 2021-05-02 22:55:41.677 DEBUG 1 --- [er-nio-worker-0] i.n.util.ResourceLeakDetectorFactory : Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@5eabd31 2021-05-02 22:55:41.767 DEBUG 1 --- [er-nio-worker-0] c.datastax.driver.core.SystemProperties : com.datastax.driver.NATIVE_TRANSPORT_MAX_FRAME_SIZE_IN_MB is undefined, using default value 256 2021-05-02 22:55:42.465 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:42.467 DEBUG 1 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map 2021-05-02 22:55:42.644 DEBUG 1 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema 2021-05-02 22:55:42.818 DEBUG 1 --- [ main] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan. 2021-05-02 22:55:42.831 DEBUG 1 --- [ main] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 12 milliseconds 2021-05-02 22:55:42.846 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.5.156:9042 2021-05-02 22:55:42.847 INFO 1 --- [ main] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'Pod' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) 2021-05-02 22:55:42.849 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.232.3:9042 added 2021-05-02 22:55:42.850 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host cassandra/10.242.5.156:9042 added 2021-05-02 22:55:42.850 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.238.220:9042 added 2021-05-02 22:55:42.883 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.CHECK_IO_DEADLOCKS is undefined, using default value true 2021-05-02 22:55:42.892 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.232.3:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:42.895 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:42.905 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] preparing to open 1 new connections, total = 2 2021-05-02 22:55:42.906 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:42.907 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:42.907 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:43.040 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.232.3:9042] Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:43.043 DEBUG 1 --- [er-nio-worker-2] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.232.3:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:43.044 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Session : Added connection pool for /10.242.232.3:9042 2021-05-02 22:55:43.673 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_dox.element_synchronization_state, scheduling delivery 2021-05-02 22:55:43.675 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:43.680 DEBUG 1 --- [er-nio-worker-2] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:43.680 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 22:55:43.970 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:43.971 DEBUG 1 --- [er-nio-worker-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.5.156:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:43.971 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.5.156:9042 2021-05-02 22:55:44.520 DEBUG 1 --- [ main] org.reflections.Reflections : going to scan these urls: file:/opt/app/aai-graphadmin/resources/ 2021-05-02 22:55:44.524 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/addmanualdata/README in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.524 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/scriptdata/addmanualdata/README in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/auth/aai_keystore in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/auth/aai_keystore in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/auth/realm.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/auth/realm.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/logging.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.525 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/logging.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/dynamic.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/dynamic.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/datatoolscrons.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/datatoolscrons.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiEventDMaaPPublisher.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiEventDMaaPPublisher.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-migration.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.526 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-migration.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-realtime.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-realtime.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-cached.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/janusgraph-cached.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/error.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/error.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiconfig.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.527 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file etc/appprops/aaiconfig.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.528 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v9/aai_oxm_v9.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.528 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v9/aai_oxm_v9.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.528 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v14/aai_oxm_v14.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.528 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v14/aai_oxm_v14.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v19/aai_oxm_v19.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v19/aai_oxm_v19.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v15/aai_oxm_v15.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v15/aai_oxm_v15.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v21/aai_oxm_v21.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.529 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v21/aai_oxm_v21.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.530 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v17/aai_oxm_v17.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.530 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v17/aai_oxm_v17.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.530 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v13/aai_oxm_v13.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.530 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v13/aai_oxm_v13.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.530 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v8/aai_oxm_v8.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.531 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v8/aai_oxm_v8.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.531 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v18/aai_oxm_v18.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.531 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v18/aai_oxm_v18.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.531 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v11/aai_oxm_v11.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.531 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v11/aai_oxm_v11.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.532 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v22/aai_oxm_v22.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.532 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v22/aai_oxm_v22.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.532 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v23/aai_oxm_v23.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.532 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v23/aai_oxm_v23.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.532 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v16/aai_oxm_v16.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v16/aai_oxm_v16.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v20/aai_oxm_v20.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v20/aai_oxm_v20.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v10/aai_oxm_v10.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v10/aai_oxm_v10.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.533 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v12/aai_oxm_v12.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.534 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/oxm/v12/aai_oxm_v12.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.534 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v9/DbEdgeRules_v9.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.534 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v9/DbEdgeRules_v9.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.534 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_esr_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_esr_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_hpa_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_hpa_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_ccvpn_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.535 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_ccvpn_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_pnp_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v14/DbEdgeRules_pnp_v14.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_esr_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_esr_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_hpa_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_hpa_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.536 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_ccvpn_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_ccvpn_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_bbs_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_bbs_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_pnp_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.537 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v19/DbEdgeRules_pnp_v19.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_hpa_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_hpa_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_pnp_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_pnp_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.538 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_ccvpn_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_ccvpn_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_esr_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v15/DbEdgeRules_esr_v15.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_esr_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.539 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_esr_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_hpa_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_hpa_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_pnp_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_pnp_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_ccvpn_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_ccvpn_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.540 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_bbs_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v21/DbEdgeRules_bbs_v21.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_hpa_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_hpa_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_ccvpn_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_ccvpn_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_bbs_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.541 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_bbs_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_pnp_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_pnp_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_esr_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_esr_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v17/DbEdgeRules_v17.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.542 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_esr_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_esr_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_hpa_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v13/DbEdgeRules_hpa_v13.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.543 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v8/DbEdgeRules_v8.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v8/DbEdgeRules_v8.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_hpa_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_hpa_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_ccvpn_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_ccvpn_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.544 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_pnp_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_pnp_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_bbs_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_bbs_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_esr_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_esr_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.545 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v18/DbEdgeRules_v18.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.546 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_esr_v11.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.546 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_esr_v11.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.546 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_v11.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.546 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v11/DbEdgeRules_v11.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.548 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_pnp_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_pnp_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_ccvpn_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_ccvpn_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_bbs_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.549 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_bbs_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_hpa_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_hpa_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_esr_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v22/DbEdgeRules_esr_v22.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_ccvpn_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.550 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_ccvpn_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_hpa_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_hpa_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_bbs_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_bbs_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_pnp_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.551 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_pnp_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_esr_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v23/DbEdgeRules_esr_v23.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_pnp_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_pnp_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_ccvpn_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.552 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_ccvpn_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_hpa_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_hpa_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_esr_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_esr_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.553 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_bbs_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v16/DbEdgeRules_bbs_v16.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_bbs_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_bbs_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_ccvpn_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_ccvpn_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_hpa_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.554 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_hpa_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_pnp_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_pnp_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_esr_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_esr_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v20/DbEdgeRules_v20.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.555 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v10/DbEdgeRules_v10.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v10/DbEdgeRules_v10.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_hpa_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_hpa_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_esr_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.556 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schema/onap/dbedgerules/v12/DbEdgeRules_esr_v12.json in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file dynamicPayloadGenerator-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file dynamicPayloadGenerator-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file forceDelete-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file forceDelete-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file updatePropertyTool-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file updatePropertyTool-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.557 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file localhost-access-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.558 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file localhost-access-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.558 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/AAIDsl.g4 in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.558 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file antlr4/org/onap/aai/AAIDsl.g4 in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.558 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.558 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file org/janusgraph/graphdb/configuration/janusgraph.internal.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file org/janusgraph/graphdb/configuration/janusgraph.internal.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schemaMod-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file schemaMod-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file retired.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.559 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file retired.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file dupeTool-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file dupeTool-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file migration-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file migration-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file uniquePropertyCheck-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file uniquePropertyCheck-logback.xml in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file application.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner TypeAnnotationsScanner 2021-05-02 22:55:44.560 DEBUG 1 --- [ main] org.reflections.Reflections : could not scan file application.properties in url file:/opt/app/aai-graphadmin/resources/ with scanner SubTypesScanner 2021-05-02 22:55:44.561 INFO 1 --- [ main] org.reflections.Reflections : Reflections took 40 ms to scan 1 urls, producing 0 keys and 0 values 2021-05-02 22:55:44.688 DEBUG 1 --- [luster-worker-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_dox.element_synchronization_state (TABLE) 2021-05-02 22:55:45.074 DEBUG 1 --- [ main] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=true] closing connection 2021-05-02 22:55:45.074 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.232.3:9042] Connection[/10.242.232.3:9042-1, inFlight=0, closed=true] closed, remaining = 0 2021-05-02 22:55:45.096 DEBUG 1 --- [ main] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=true] closing connection 2021-05-02 22:55:45.096 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=true] closed, remaining = 1 2021-05-02 22:55:45.096 DEBUG 1 --- [ main] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=true] closing connection 2021-05-02 22:55:45.096 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-1, inFlight=0, closed=true] closed, remaining = 0 2021-05-02 22:55:45.102 DEBUG 1 --- [ main] com.datastax.driver.core.Cluster : Shutting down 2021-05-02 22:55:45.103 DEBUG 1 --- [ main] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=true] closing connection 2021-05-02 22:55:45.103 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=true] closed, remaining = 0 2021-05-02 22:55:47.320 DEBUG 1 --- [er-nio-worker-0] io.netty.buffer.PoolThreadCache : Freed 42 thread-local buffer(s) from thread: JanusGraph Cluster-nio-worker-0 2021-05-02 22:55:47.320 DEBUG 1 --- [er-nio-worker-2] io.netty.buffer.PoolThreadCache : Freed 22 thread-local buffer(s) from thread: JanusGraph Cluster-nio-worker-2 2021-05-02 22:55:47.346 DEBUG 1 --- [ main] c.datastax.driver.core.SystemProperties : com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true 2021-05-02 22:55:47.346 INFO 1 --- [ main] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps. 2021-05-02 22:55:47.346 DEBUG 1 --- [ main] com.datastax.driver.core.Cluster : Starting new cluster with contact points [cassandra/10.242.5.156:9042] 2021-05-02 22:55:47.351 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:47.353 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:48.159 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:48.160 DEBUG 1 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing node list and token map 2021-05-02 22:55:48.167 DEBUG 1 --- [ main] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema 2021-05-02 22:55:48.215 DEBUG 1 --- [ main] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan. 2021-05-02 22:55:48.221 DEBUG 1 --- [ main] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 6 milliseconds 2021-05-02 22:55:48.231 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [Control connection] established to cassandra/10.242.5.156:9042 2021-05-02 22:55:48.231 INFO 1 --- [ main] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'Pod' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) 2021-05-02 22:55:48.231 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.232.3:9042 added 2021-05-02 22:55:48.232 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host cassandra/10.242.5.156:9042 added 2021-05-02 22:55:48.232 INFO 1 --- [ main] com.datastax.driver.core.Cluster : New Cassandra host /10.242.238.220:9042 added 2021-05-02 22:55:48.232 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.232.3:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:48.232 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] preparing to open 1 new connections, total = 2 2021-05-02 22:55:48.233 DEBUG 1 --- [ main] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 22:55:48.233 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:48.233 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:48.235 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 22:55:48.704 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.232.3:9042] Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:48.704 DEBUG 1 --- [er-nio-worker-2] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.232.3:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:48.704 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Session : Added connection pool for /10.242.232.3:9042 2021-05-02 22:55:48.862 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [cassandra/10.242.5.156:9042] Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:48.863 DEBUG 1 --- [er-nio-worker-0] c.d.driver.core.HostConnectionPool : Created connection pool to host cassandra/10.242.5.156:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:48.863 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Session : Added connection pool for cassandra/10.242.5.156:9042 2021-05-02 22:55:49.070 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 22:55:49.071 DEBUG 1 --- [er-nio-worker-2] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 22:55:49.071 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 22:55:50.424 INFO 1 --- [ main] org.onap.aai.GraphAdminApp : GraphAdmin MicroService Started 2021-05-02 22:55:50.424 DEBUG 1 --- [ main] org.onap.aai.GraphAdminApp : GraphAdmin MicroService Started GraphAdmin Microservice Started 2021-05-02 22:56:18.186 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:56:18.194 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:56:20.286 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:56:20.289 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:56:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 22:56:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 22:56:48.194 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:56:48.199 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:56:55.694 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:56:55.697 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:57:07.271 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE workflow, scheduling delivery 2021-05-02 22:57:07.791 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED KEYSPACE zusammen_workflow, scheduling delivery 2021-05-02 22:57:08.794 DEBUG 1 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for everything.null (null) 2021-05-02 22:57:08.849 DEBUG 1 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan. 2021-05-02 22:57:08.860 DEBUG 1 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 11 milliseconds 2021-05-02 22:57:10.628 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE workflow.unique_value, scheduling delivery 2021-05-02 22:57:11.630 DEBUG 1 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for workflow.unique_value (TABLE) 2021-05-02 22:57:12.396 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.item, scheduling delivery 2021-05-02 22:57:13.397 DEBUG 1 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.item (TABLE) 2021-05-02 22:57:13.793 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version, scheduling delivery 2021-05-02 22:57:14.794 DEBUG 1 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version (TABLE) 2021-05-02 22:57:15.460 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version_elements, scheduling delivery 2021-05-02 22:57:16.461 DEBUG 1 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version_elements (TABLE) 2021-05-02 22:57:16.797 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_namespace, scheduling delivery 2021-05-02 22:57:17.797 DEBUG 1 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element_namespace (TABLE) 2021-05-02 22:57:17.829 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element, scheduling delivery 2021-05-02 22:57:18.732 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_synchronization_state, scheduling delivery 2021-05-02 22:57:19.733 DEBUG 1 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow 2021-05-02 22:57:19.741 DEBUG 1 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Computing token to replica map for keyspace: sdctitan. 2021-05-02 22:57:19.755 DEBUG 1 --- [luster-worker-2] licationStrategy$NetworkTopologyStrategy : Token to replica map computation for keyspace sdctitan completed in 14 milliseconds 2021-05-02 22:57:19.829 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.element_stage, scheduling delivery 2021-05-02 22:57:20.830 DEBUG 1 --- [luster-worker-3] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.element_stage (TABLE) 2021-05-02 22:57:21.075 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT CREATED TABLE zusammen_workflow.version_stage, scheduling delivery 2021-05-02 22:57:22.076 DEBUG 1 --- [luster-worker-2] c.d.driver.core.ControlConnection : [Control connection] Refreshing schema for zusammen_workflow.version_stage (TABLE) 2021-05-02 22:57:25.799 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:57:25.802 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:57:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 22:57:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 22:57:50.894 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:57:50.899 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:57:52.160 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:57:52.163 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:58:11.061 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:58:11.068 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:58:22.163 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:58:22.165 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:58:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 22:58:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 22:58:51.371 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:58:51.373 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:58:52.165 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:58:52.167 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:59:21.374 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:59:21.378 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:59:22.167 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:59:22.170 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 22:59:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 22:59:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 22:59:52.171 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 22:59:52.177 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:00:06.994 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:00:06.997 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:00:22.177 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:00:22.180 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:00:27.186 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:00:27.260 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:00:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:00:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:00:52.181 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:00:52.183 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:01:22.184 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:01:22.263 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:01:27.342 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:01:27.368 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:01:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:01:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:01:52.264 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:01:52.267 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:01:57.668 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:01:57.670 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:02:17.860 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:02:17.863 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:02:22.267 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:02:22.269 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:02:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:02:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:02:52.270 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:02:52.274 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:03:22.274 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:03:22.359 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:03:23.182 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:03:23.184 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:03:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:03:37.243 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:03:43.217 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:03:43.220 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:03:52.360 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:03:52.362 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:04:18.470 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:04:18.473 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:04:22.363 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:04:22.366 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:04:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:04:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:04:48.506 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:04:48.510 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:04:52.366 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:04:52.369 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:05:22.370 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:05:22.372 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:05:28.694 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:05:28.696 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:05:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:05:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:05:52.373 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:05:52.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:05:58.697 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:05:58.700 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:06:22.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:06:22.459 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:06:34.360 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:06:34.364 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:06:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:06:37.243 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:06:50.966 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:06:50.968 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:06:52.460 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:06:52.462 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:07:04.367 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:07:04.370 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:07:22.462 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:07:22.465 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:07:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:07:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:07:50.359 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:07:50.362 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:07:52.465 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:07:52.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:08:20.576 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:08:20.579 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:08:22.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:08:22.470 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:08:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:08:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:08:52.471 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:08:52.473 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:08:56.491 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:09:00.267 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:09:16.694 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:09:16.697 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:09:22.474 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:09:22.477 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:09:30.269 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:09:31.677 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:09:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:09:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:09:52.478 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:09:52.481 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:10:22.481 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:10:22.558 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:10:32.460 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:10:32.463 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:10:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:10:37.244 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:10:52.559 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:10:52.563 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:11:14.360 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:11:14.363 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:11:22.563 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:11:22.566 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:11:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:11:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:11:51.736 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-02T23:11:35.177Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (timeout while waiting for repair of inconsistent replica) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (timeout while waiting for repair of inconsistent replica) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (timeout while waiting for repair of inconsistent replica) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-02 23:11:52.566 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:11:52.568 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:12:00.677 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:12:00.680 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:12:20.584 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:12:20.585 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:12:21.589 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:12:21.684 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:12:33.756 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-1, inFlight=1, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-02 23:12:33.760 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-02 23:12:33.760 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-1, inFlight=1, closed=false] failed, remaining = 0 2021-05-02 23:12:33.763 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=1, closed=true] closing connection 2021-05-02 23:12:33.763 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-1, inFlight=1, closed=true]: there are still pending requests 2021-05-02 23:12:33.763 DEBUG 1 --- [luster-worker-4] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:12:33.764 DEBUG 1 --- [luster-worker-4] com.datastax.driver.core.RequestHandler : onException triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false)) 2021-05-02 23:12:33.765 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2011476204-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@39c38029 at consistency LOCAL_QUORUM 2021-05-02 23:12:33.766 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2011476204-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.TransportException: [/10.242.238.220:9042] Connection has been closed 2021-05-02 23:12:33.775 DEBUG 1 --- [luster-worker-4] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:12:33.776 DEBUG 1 --- [luster-worker-4] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:12:34.776 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:12:34.781 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-2, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:12:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:12:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:12:46.859 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-2, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:12:46.860 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-2, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:12:46.861 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-2, inFlight=0, closed=true] closing connection 2021-05-02 23:12:46.861 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-2, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:12:46.861 DEBUG 1 --- [luster-worker-5] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:12:46.862 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 2000 milliseconds 2021-05-02 23:12:46.862 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 2000 ms 2021-05-02 23:12:48.862 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:12:48.868 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-3, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:12:50.589 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:12:50.591 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:13:00.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-3, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:13:00.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-3, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:13:00.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-3, inFlight=0, closed=true] closing connection 2021-05-02 23:13:00.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-3, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:13:00.956 DEBUG 1 --- [luster-worker-4] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:13:00.956 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 4000 milliseconds 2021-05-02 23:13:00.956 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 4000 ms 2021-05-02 23:13:02.372 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-1, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:04.956 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:13:04.959 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-4, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:13:17.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-4, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:13:17.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-4, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:13:17.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-4, inFlight=0, closed=true] closing connection 2021-05-02 23:13:17.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-4, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:13:17.056 DEBUG 1 --- [luster-worker-6] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:13:17.057 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 8000 milliseconds 2021-05-02 23:13:17.057 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 8000 ms 2021-05-02 23:13:17.373 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-2, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:20.592 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:13:20.595 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:13:25.057 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:13:25.060 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-5, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:13:32.374 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-3, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:33.704 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-5, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:13:33.704 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-5, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:13:33.705 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-5, inFlight=0, closed=true] closing connection 2021-05-02 23:13:33.705 DEBUG 1 --- [luster-worker-7] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:13:33.706 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-5, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:33.706 ERROR 1 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:13:33.707 ERROR 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:13:33.709 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 16000 ms 2021-05-02 23:13:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:13:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:13:47.375 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-4, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:49.710 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:13:49.715 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-6, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:13:50.595 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:13:50.597 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:13:51.683 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-6, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:13:51.683 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-6, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:13:51.684 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-6, inFlight=0, closed=true] closing connection 2021-05-02 23:13:51.684 DEBUG 1 --- [luster-worker-8] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:13:51.685 ERROR 1 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:13:51.685 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-6, inFlight=0, closed=true] has already terminated 2021-05-02 23:13:51.685 ERROR 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:13:51.687 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 32000 ms 2021-05-02 23:14:03.982 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:14:04.059 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:14:05.064 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-02 23:14:05.064 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-02 23:14:06.066 DEBUG 1 --- [luster-worker-9] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:14:06.076 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-02 23:14:06.076 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-02 23:14:06.077 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-02 23:14:06.077 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:14:06.078 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-7, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:14:24.724 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-7, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-02 23:14:24.726 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-7, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:14:24.726 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-7, inFlight=0, closed=true] closing connection 2021-05-02 23:14:24.727 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-7, inFlight=0, closed=true] has already terminated 2021-05-02 23:14:24.728 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:14:24.729 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-8, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:14:34.496 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-8, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-02 23:14:34.497 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-8, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:14:34.497 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-8, inFlight=0, closed=true] closing connection 2021-05-02 23:14:34.513 WARN 1 --- [er-nio-worker-2] com.datastax.driver.core.Session : Error creating pool to /10.242.238.220:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:14:34.515 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-8, inFlight=0, closed=true] has already terminated 2021-05-02 23:14:34.517 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Cluster : Connection pool cannot be created, not marking /10.242.238.220:9042 UP 2021-05-02 23:14:34.517 DEBUG 1 --- [uster-worker-10] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:14:34.517 DEBUG 1 --- [uster-worker-10] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:14:34.518 DEBUG 1 --- [uster-worker-10] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:14:34.518 DEBUG 1 --- [luster-worker-9] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:14:35.518 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:14:35.523 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:14:36.075 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:14:36.078 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:14:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:14:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:14:41.097 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 23:14:41.099 DEBUG 1 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:14:41.163 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to /10.242.238.220:9042, setting host UP 2021-05-02 23:14:41.163 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-02 23:14:41.163 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-02 23:14:41.164 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-02 23:14:45.201 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 1 2021-05-02 23:14:45.201 DEBUG 1 --- [-reconnection-1] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 23:14:45.202 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 23:14:45.202 DEBUG 1 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future 2021-05-02 23:15:01.896 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:15:01.896 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:15:02.898 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:15:15.635 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-02 23:15:15.636 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-02 23:15:16.637 DEBUG 1 --- [uster-worker-11] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:15:24.395 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:15:24.472 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:15:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:15:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:15:46.642 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:15:46.644 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:15:54.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:15:54.662 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:16:16.644 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:16:16.647 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:16:18.892 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:16:18.895 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:16:29.868 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:16:29.885 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:16:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:16:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:16:46.647 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:16:46.658 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:17:15.160 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:17:15.163 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:17:16.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:17:16.661 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:17:27.089 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:17:27.988 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=1, closed=false] heartbeat query succeeded 2021-05-02 23:17:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:17:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:17:46.662 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:17:46.663 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:18:16.664 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:18:16.667 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:18:23.503 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:18:23.509 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:18:28.507 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:18:28.511 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:18:32.692 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-02T23:18:15.774Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-02 23:18:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:18:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:18:46.093 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-02T23:18:15.774Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-02 23:18:46.667 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:18:46.669 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:18:53.509 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:18:53.512 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:19:16.670 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:19:16.672 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:19:23.512 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:19:23.514 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:19:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:19:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:19:46.673 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:19:46.675 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:20:16.676 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:20:16.679 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:20:33.570 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:20:33.578 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:20:35.664 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:20:35.664 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:20:36.665 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:20:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:20:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:20:40.603 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-02T23:20:29.395Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-02 23:20:54.895 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:20:54.904 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:20:57.654 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [6113577-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6ef471cc at consistency LOCAL_QUORUM 2021-05-02 23:20:57.655 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [6113577-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:20:58.835 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] Response received on stream 5312 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses. 2021-05-02 23:21:05.665 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:21:05.667 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:21:28.836 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:21:29.854 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2087090182-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1969cd7a at consistency LOCAL_QUORUM 2021-05-02 23:21:29.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2087090182-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:21:30.788 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:21:32.760 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:21:32.762 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:21:35.668 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:21:35.669 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:21:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:21:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:21:39.115 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] Response received on stream 5376 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses. 2021-05-02 23:22:05.669 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:22:05.672 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:22:09.116 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:22:21.154 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-9, inFlight=1, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-02 23:22:21.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-02 23:22:21.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-9, inFlight=1, closed=false] failed, remaining = 0 2021-05-02 23:22:21.157 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=1, closed=true] closing connection 2021-05-02 23:22:21.157 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-9, inFlight=1, closed=true]: there are still pending requests 2021-05-02 23:22:21.157 DEBUG 1 --- [uster-worker-12] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:22:21.157 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [660712124-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5517e6ac at consistency LOCAL_QUORUM 2021-05-02 23:22:21.158 DEBUG 1 --- [uster-worker-12] com.datastax.driver.core.RequestHandler : onException triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false)) 2021-05-02 23:22:21.159 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [660712124-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.TransportException: [/10.242.238.220:9042] Connection has been closed 2021-05-02 23:22:21.161 DEBUG 1 --- [uster-worker-12] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:22:21.161 DEBUG 1 --- [uster-worker-12] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:22:22.161 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:22:22.165 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-10, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:22:34.254 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-10, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:22:34.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-10, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:22:34.256 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-10, inFlight=0, closed=true] closing connection 2021-05-02 23:22:34.256 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-10, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:22:34.256 DEBUG 1 --- [uster-worker-13] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:22:34.257 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 2000 milliseconds 2021-05-02 23:22:34.257 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 2000 ms 2021-05-02 23:22:35.672 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:22:35.674 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:22:36.258 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:22:36.260 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-11, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:22:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:22:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:22:47.389 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-9, inFlight=0, closed=true] has already terminated 2021-05-02 23:22:48.354 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-11, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:22:48.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-11, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:22:48.355 DEBUG 1 --- [uster-worker-12] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:22:48.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-11, inFlight=0, closed=true] closing connection 2021-05-02 23:22:48.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-11, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:22:48.355 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 4000 milliseconds 2021-05-02 23:22:48.355 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 4000 ms 2021-05-02 23:22:52.356 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:22:52.358 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-12, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:23:01.264 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:23:01.268 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:23:02.390 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-10, inFlight=0, closed=true] has already terminated 2021-05-02 23:23:04.454 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-12, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:23:04.455 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-12, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:23:04.456 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-12, inFlight=0, closed=true] closing connection 2021-05-02 23:23:04.456 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-12, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:23:04.457 DEBUG 1 --- [uster-worker-14] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:23:04.457 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 8000 milliseconds 2021-05-02 23:23:04.457 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 8000 ms 2021-05-02 23:23:05.674 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:23:05.676 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:23:12.458 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:23:12.460 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-13, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:23:17.390 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-11, inFlight=0, closed=true] has already terminated 2021-05-02 23:23:25.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-13, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:23:25.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-13, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:23:25.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-13, inFlight=0, closed=true] closing connection 2021-05-02 23:23:25.057 DEBUG 1 --- [uster-worker-15] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:23:25.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-13, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:23:25.058 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 16000 milliseconds 2021-05-02 23:23:25.058 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 16000 ms 2021-05-02 23:23:32.391 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-12, inFlight=0, closed=true] has already terminated 2021-05-02 23:23:35.676 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:23:35.678 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:23:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:23:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:23:41.059 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:23:41.061 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-14, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:23:42.699 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-14, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:23:42.700 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-14, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:23:42.700 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-14, inFlight=0, closed=true] closing connection 2021-05-02 23:23:42.701 DEBUG 1 --- [uster-worker-16] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:23:42.701 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-14, inFlight=0, closed=true] has already terminated 2021-05-02 23:23:42.701 ERROR 1 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:23:42.702 ERROR 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:23:42.704 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 32000 ms 2021-05-02 23:24:02.392 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-13, inFlight=0, closed=true] has already terminated 2021-05-02 23:24:05.679 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:24:05.681 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:24:14.704 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:24:14.708 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-15, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:24:26.755 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-15, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:24:26.755 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-15, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:24:26.757 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-15, inFlight=0, closed=true] closing connection 2021-05-02 23:24:26.757 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-15, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:24:26.757 DEBUG 1 --- [uster-worker-17] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:24:26.757 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 64000 milliseconds 2021-05-02 23:24:26.757 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 64000 ms 2021-05-02 23:24:35.682 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:24:35.683 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:24:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:24:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:25:02.394 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-15, inFlight=0, closed=true] has already terminated 2021-05-02 23:25:05.684 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:25:05.686 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:25:30.758 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:25:30.763 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-16, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:25:34.578 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-16, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:25:34.578 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-16, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:25:34.579 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-16, inFlight=0, closed=true] closing connection 2021-05-02 23:25:34.580 DEBUG 1 --- [uster-worker-18] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:25:34.580 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-16, inFlight=0, closed=true] has already terminated 2021-05-02 23:25:34.580 ERROR 1 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:25:34.581 ERROR 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 128000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:25:34.582 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 128000 ms 2021-05-02 23:25:35.687 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:25:35.689 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:25:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:25:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:26:05.689 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:26:05.692 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:26:17.067 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:26:17.070 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:26:35.693 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:26:35.695 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:26:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:26:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:27:05.695 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:27:05.697 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:27:19.780 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-02 23:27:19.781 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-02 23:27:20.782 DEBUG 1 --- [uster-worker-19] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:27:20.787 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-02 23:27:20.787 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-02 23:27:20.787 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-02 23:27:20.787 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:27:20.789 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:27:24.175 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 23:27:24.273 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 1 2021-05-02 23:27:24.273 DEBUG 1 --- [uster-worker-19] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 23:27:24.273 DEBUG 1 --- [uster-worker-19] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 23:27:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:27:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:27:50.787 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:27:50.790 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:28:07.768 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:28:07.771 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:28:20.790 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:28:20.792 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:28:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:28:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:28:50.793 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:28:50.795 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:28:53.167 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:28:53.171 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:29:20.795 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:29:20.859 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:29:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:29:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:29:48.280 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:29:48.284 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:29:50.454 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : onTimeout triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false)) 2021-05-02 23:29:50.859 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:29:50.861 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:30:03.393 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:30:03.395 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:30:20.861 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:30:20.864 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:30:33.482 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:30:33.485 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:30:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:30:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:30:50.864 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:30:50.866 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:31:03.681 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:31:03.682 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:31:20.867 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:31:20.870 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:31:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:31:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:31:50.871 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:31:50.873 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:32:20.874 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:32:20.875 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:32:29.094 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:32:29.096 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:32:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:32:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:32:50.876 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:32:50.878 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:32:54.275 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:32:54.277 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:33:14.303 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:33:14.305 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:33:20.878 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:33:20.880 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:33:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:33:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:33:50.881 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:33:50.883 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:34:19.677 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:34:19.680 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:34:20.884 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:34:20.885 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:34:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:34:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:34:39.768 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:34:39.770 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:34:50.886 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:34:50.888 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:34:59.893 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:34:59.896 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:35:20.888 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:35:20.891 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:35:34.996 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:35:34.999 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:35:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:35:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:35:50.892 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:35:50.894 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:36:20.894 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:36:20.896 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:36:30.373 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:36:30.375 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:36:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:36:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:36:50.896 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:36:50.898 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:37:20.899 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:37:20.959 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:37:30.974 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:37:30.977 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:37:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:37:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:37:50.960 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:37:50.962 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:37:56.078 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:37:56.110 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:38:20.962 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:38:20.964 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:38:21.197 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:38:21.258 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:38:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:38:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:38:50.965 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:38:50.969 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:39:06.518 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:39:06.526 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:39:20.969 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:39:20.972 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:39:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:39:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:39:46.875 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:39:46.894 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:39:50.972 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:39:50.975 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:40:19.655 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [852715338-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5d1ffc01 at consistency LOCAL_QUORUM 2021-05-02 23:40:19.655 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [852715338-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:40:20.975 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:40:20.977 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:40:24.375 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:40:24.375 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:40:25.376 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:40:27.303 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:40:32.565 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:40:32.569 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:40:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:40:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:40:39.354 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-17, inFlight=1, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-02 23:40:39.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-02 23:40:39.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-17, inFlight=1, closed=false] failed, remaining = 0 2021-05-02 23:40:39.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=1, closed=true] closing connection 2021-05-02 23:40:39.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-17, inFlight=1, closed=true]: there are still pending requests 2021-05-02 23:40:39.356 DEBUG 1 --- [uster-worker-20] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:40:39.357 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [142204002-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@595c09c1 at consistency LOCAL_QUORUM 2021-05-02 23:40:39.357 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [142204002-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.TransportException: [/10.242.238.220:9042] Connection has been closed 2021-05-02 23:40:39.357 DEBUG 1 --- [uster-worker-20] com.datastax.driver.core.RequestHandler : onException triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false)) 2021-05-02 23:40:39.358 DEBUG 1 --- [uster-worker-20] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:40:39.359 DEBUG 1 --- [uster-worker-20] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:40:40.359 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:40:40.361 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-18, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:40:52.456 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-18, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:40:52.456 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-18, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:40:52.457 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-18, inFlight=0, closed=true] closing connection 2021-05-02 23:40:52.458 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-18, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:40:52.458 DEBUG 1 --- [uster-worker-21] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:40:52.458 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 2000 milliseconds 2021-05-02 23:40:52.459 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 2000 ms 2021-05-02 23:40:54.376 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:40:54.378 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:40:54.459 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:40:54.462 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-19, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:40:54.664 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:40:54.666 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:41:01.789 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-19, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-02 23:41:01.789 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-19, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:41:01.790 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-19, inFlight=0, closed=true] closing connection 2021-05-02 23:41:01.790 DEBUG 1 --- [uster-worker-20] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:41:01.790 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-19, inFlight=0, closed=true] has already terminated 2021-05-02 23:41:01.790 ERROR 1 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-02 23:41:01.792 ERROR 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:41:01.794 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 4000 ms 2021-05-02 23:41:05.794 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:41:05.796 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-20, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:41:17.412 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-18, inFlight=0, closed=true] has already terminated 2021-05-02 23:41:17.413 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-17, inFlight=0, closed=true] has already terminated 2021-05-02 23:41:17.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-20, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:41:17.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-20, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:41:17.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-20, inFlight=0, closed=true] closing connection 2021-05-02 23:41:17.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-20, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:41:17.856 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 8000 milliseconds 2021-05-02 23:41:17.856 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 8000 ms 2021-05-02 23:41:17.856 DEBUG 1 --- [uster-worker-21] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:41:24.378 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:41:24.458 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:41:25.856 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:41:25.859 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-21, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:41:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:41:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:41:37.954 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-21, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:41:37.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-21, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:41:37.956 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-21, inFlight=0, closed=true] closing connection 2021-05-02 23:41:37.956 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-21, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:41:37.957 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 16000 milliseconds 2021-05-02 23:41:37.957 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 16000 ms 2021-05-02 23:41:37.956 DEBUG 1 --- [uster-worker-22] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:41:47.414 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-20, inFlight=0, closed=true] has already terminated 2021-05-02 23:41:53.957 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:41:53.959 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-22, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:41:54.459 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:41:54.461 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:41:55.678 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-22, inFlight=0, closed=false] because: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:41:55.678 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-22, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:41:55.679 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-22, inFlight=0, closed=true] closing connection 2021-05-02 23:41:55.679 DEBUG 1 --- [uster-worker-23] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:41:55.679 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-22, inFlight=0, closed=true] has already terminated 2021-05-02 23:41:55.679 ERROR 1 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM 2021-05-02 23:41:55.680 ERROR 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Authentication error during reconnection to /10.242.238.220:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.238.220:9042: Error during authentication of user cassandra : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at com.datastax.driver.core.Connection$8.apply(Connection.java:503) at com.datastax.driver.core.Connection$8.apply(Connection.java:465) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1479) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2021-05-02 23:41:55.682 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 32000 ms 2021-05-02 23:42:02.414 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-21, inFlight=0, closed=true] has already terminated 2021-05-02 23:42:24.461 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:42:24.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:42:27.682 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:42:27.686 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-23, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:42:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:42:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:42:45.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-23, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:42:45.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-23, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:42:45.357 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-23, inFlight=0, closed=true] closing connection 2021-05-02 23:42:45.357 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-23, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:42:45.357 DEBUG 1 --- [uster-worker-24] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:42:45.359 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 64000 milliseconds 2021-05-02 23:42:45.359 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 64000 ms 2021-05-02 23:42:54.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:42:54.470 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:43:17.416 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-23, inFlight=0, closed=true] has already terminated 2021-05-02 23:43:24.471 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:43:24.475 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:43:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:43:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:43:37.697 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-02 23:43:37.698 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-02 23:43:38.699 DEBUG 1 --- [uster-worker-25] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:43:38.706 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-02 23:43:38.706 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-02 23:43:38.706 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-02 23:43:38.707 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:43:38.714 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:43:39.778 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 23:43:39.872 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 1 2021-05-02 23:43:39.872 DEBUG 1 --- [uster-worker-25] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 23:43:39.872 DEBUG 1 --- [uster-worker-25] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 23:43:44.971 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:43:44.993 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:44:08.705 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:44:08.759 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:44:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:44:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:44:38.759 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:44:38.762 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:45:00.003 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:00.059 DEBUG 1 --- [ scheduling-1] o.o.aai.datasnapshot.DataSnapshotTasks : Exit value of the dataSnapshot check process: 1 2021-05-02 23:45:00.060 DEBUG 1 --- [ scheduling-1] o.o.aai.datasnapshot.DataSnapshotTasks : Started cron job dataSnapshot @ 23:45:00 2021-05-02 23:45:00.061 DEBUG 1 --- [ scheduling-1] o.o.aai.datasnapshot.DataSnapshotTasks : DataSnapshot Params [-c, JUST_TAKE_SNAPSHOT] 2021-05-02 23:45:00.086 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.threads.for.create] value from aaiconfig.properties file. Property key aai.datasnapshot.threads.for.create cannot be found 2021-05-02 23:45:00.086 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.max.nodes.per.file.for.create] value from aaiconfig.properties file. Property key aai.datasnapshot.max.nodes.per.file.for.create cannot be found 2021-05-02 23:45:00.086 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.vertex.add.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.vertex.add.delay.ms cannot be found 2021-05-02 23:45:00.086 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.edge.add.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.edge.add.delay.ms cannot be found 2021-05-02 23:45:00.086 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.failure.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.failure.delay.ms cannot be found 2021-05-02 23:45:00.087 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.retry.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.retry.delay.ms cannot be found 2021-05-02 23:45:00.087 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.max.errors.per.thread] value from aaiconfig.properties file. Property key aai.datasnapshot.max.errors.per.thread cannot be found 2021-05-02 23:45:00.087 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.vertex.to.edge.proc.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.vertex.to.edge.proc.delay.ms cannot be found 2021-05-02 23:45:00.087 WARN 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : WARNING - could not get [aai.datasnapshot.stagger.thread.delay.ms] value from aaiconfig.properties file. Property key aai.datasnapshot.stagger.thread.delay.ms cannot be found 2021-05-02 23:45:00.151 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DataSnapshot command is [JUST_TAKE_SNAPSHOT] 2021-05-02 23:45:00.152 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : File name to reload snapshot [] 2021-05-02 23:45:00.152 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : snapshotType is [graphson] 2021-05-02 23:45:00.152 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Thread count is [15] 2021-05-02 23:45:00.152 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Max Nodes Per File is [120000] 2021-05-02 23:45:00.153 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Debug Flag is [] 2021-05-02 23:45:00.153 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DebugAddDelayTimer is [1] 2021-05-02 23:45:00.153 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : VertAddDelayMs is [1] 2021-05-02 23:45:00.153 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : FailureDelayMs is [50] 2021-05-02 23:45:00.153 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : RetryDelayMs is [50] 2021-05-02 23:45:00.154 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : MaxErrorsPerThread is [25] 2021-05-02 23:45:00.154 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : VertToEdgeProcDelay is [9000] 2021-05-02 23:45:00.154 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : StaggerThreadDelay is [5000] 2021-05-02 23:45:00.154 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Caller process is [] 2021-05-02 23:45:00.154 INFO 1 --- [ scheduling-1] org.onap.aai.util.AAIConfig : Initializing AAIConfig 2021-05-02 23:45:00.155 DEBUG 1 --- [ scheduling-1] org.onap.aai.util.AAIConfig : Reloading config from /opt/app/aai-graphadmin/./resources/etc/appprops/aaiconfig.properties 2021-05-02 23:45:00.155 INFO 1 --- [ scheduling-1] org.onap.aai.util.AAIConfig : Found the aaiconfig.properties in the following location: /opt/app/aai-graphadmin/./resources/etc/appprops/aaiconfig.properties 2021-05-02 23:45:00.156 INFO 1 --- [ scheduling-1] org.onap.aai.util.AAIConfig : A&AI Server Node Name = aai.config.nodename 2021-05-02 23:45:00.160 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Command = JUST_TAKE_SNAPSHOT, oldSnapshotFileName = []. 2021-05-02 23:45:00.161 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : ---- NOTE --- about to open graph (takes a little while) 2021-05-02 23:45:00.161 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Command = JUST_TAKE_SNAPSHOT 2021-05-02 23:45:00.163 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Successfully got the Graph instance. 2021-05-02 23:45:00.165 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is query.fast-propertyValue is true 2021-05-02 23:45:00.166 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is query.smart-limitValue is false 2021-05-02 23:45:00.166 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.backendValue is cql 2021-05-02 23:45:00.166 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.hostnameValue is cassandra 2021-05-02 23:45:00.167 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.cql.keyspaceValue is aaigraph 2021-05-02 23:45:00.167 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.usernameValue is cassandra 2021-05-02 23:45:00.167 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.passwordValue is cassandra 2021-05-02 23:45:00.167 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.cql.read-consistency-levelValue is LOCAL_QUORUM 2021-05-02 23:45:00.167 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.cql.write-consistency-levelValue is LOCAL_QUORUM 2021-05-02 23:45:00.168 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.cql.replication-factorValue is 3 2021-05-02 23:45:00.168 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.cql.only-use-local-consistency-for-system-operationsValue is true 2021-05-02 23:45:00.168 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is storage.lock.wait-timeValue is 300 2021-05-02 23:45:00.168 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is cache.db-cacheValue is false 2021-05-02 23:45:00.168 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is load.snapshot.fileValue is false 2021-05-02 23:45:00.169 INFO 1 --- [ scheduling-1] org.onap.aai.util.GraphAdminDBUtils : Key is graph.unique-instance-idValue is 1_aai-graphadmin_GraphAdminApp_realtime_1619996140463 2021-05-02 23:45:00.169 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Count how many nodes are in the db. 2021-05-02 23:45:00.381 WARN 1 --- [ scheduling-1] o.j.g.transaction.StandardJanusGraphTx : Query requires iterating over all vertices [()]. For better performance, use indexes 2021-05-02 23:45:02.463 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : -- To count all vertices in DB it took: 0 minutes, 2 seconds 2021-05-02 23:45:02.463 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Total Count of Nodes in DB = 426. 2021-05-02 23:45:02.464 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : We will run this many simultaneous threads: 15 2021-05-02 23:45:02.465 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Required number of passes: 1 2021-05-02 23:45:02.465 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Max Nodes per file: 120000 2021-05-02 23:45:02.465 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : We will generate this many files: 15 2021-05-02 23:45:02.465 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Each file will have (roughly): 29 nodes. 2021-05-02 23:45:02.465 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Now, divide vertexIds across this many files: 15 2021-05-02 23:45:04.087 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : -- To Loop over all vertices, and put them into sub-Arrays it took: 0 minutes, 1 seconds 2021-05-02 23:45:04.088 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 0 2021-05-02 23:45:04.091 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 1 2021-05-02 23:45:04.092 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 2 2021-05-02 23:45:04.092 INFO 1 --- [ Thread-32] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.092 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 3 2021-05-02 23:45:04.092 INFO 1 --- [ Thread-33] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.093 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 4 2021-05-02 23:45:04.093 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 5 2021-05-02 23:45:04.093 INFO 1 --- [ Thread-34] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.093 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 6 2021-05-02 23:45:04.093 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 7 2021-05-02 23:45:04.093 INFO 1 --- [ Thread-35] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.094 INFO 1 --- [ Thread-38] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.095 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 8 2021-05-02 23:45:04.095 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 9 2021-05-02 23:45:04.095 INFO 1 --- [ Thread-36] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.096 INFO 1 --- [ Thread-39] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.096 INFO 1 --- [ Thread-40] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.097 INFO 1 --- [ Thread-37] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.101 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 10 2021-05-02 23:45:04.104 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 11 2021-05-02 23:45:04.105 INFO 1 --- [ Thread-42] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.105 INFO 1 --- [ Thread-41] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.107 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 12 2021-05-02 23:45:04.107 INFO 1 --- [ Thread-43] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.109 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 13 2021-05-02 23:45:04.110 INFO 1 --- [ Thread-44] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.114 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : DEBUG >>> kick off pass # 1, thread # 14 2021-05-02 23:45:04.115 INFO 1 --- [ Thread-45] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:04.117 INFO 1 --- [ Thread-46] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:08.762 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:45:08.765 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P12 2021-05-02 23:45:13.592 DEBUG 1 --- [ Thread-44] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P12 2021-05-02 23:45:13.594 INFO 1 --- [ Thread-44] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P1 2021-05-02 23:45:13.672 DEBUG 1 --- [ Thread-33] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P1 -- Printed 21 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P14 2021-05-02 23:45:13.672 DEBUG 1 --- [ Thread-46] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 21 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P14 2021-05-02 23:45:13.673 INFO 1 --- [ Thread-33] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:13.673 INFO 1 --- [ Thread-46] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P9 2021-05-02 23:45:13.769 DEBUG 1 --- [ Thread-41] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P9 2021-05-02 23:45:13.770 INFO 1 --- [ Thread-41] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P8 2021-05-02 23:45:13.878 DEBUG 1 --- [ Thread-40] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P8 2021-05-02 23:45:13.879 INFO 1 --- [ Thread-40] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P13 -- Printed 28 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P0 2021-05-02 23:45:13.970 DEBUG 1 --- [ Thread-32] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 28 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P0 2021-05-02 23:45:13.970 DEBUG 1 --- [ Thread-45] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P13 2021-05-02 23:45:13.971 INFO 1 --- [ Thread-32] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:13.972 INFO 1 --- [ Thread-45] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P7 2021-05-02 23:45:13.985 DEBUG 1 --- [ Thread-39] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P7 2021-05-02 23:45:13.986 INFO 1 --- [ Thread-39] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P11 2021-05-02 23:45:14.072 DEBUG 1 --- [ Thread-43] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P11 -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P2 2021-05-02 23:45:14.073 DEBUG 1 --- [ Thread-34] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P2 2021-05-02 23:45:14.074 INFO 1 --- [ Thread-34] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:14.074 INFO 1 --- [ Thread-43] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P10 2021-05-02 23:45:14.280 DEBUG 1 --- [ Thread-42] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P10 2021-05-02 23:45:14.281 INFO 1 --- [ Thread-42] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P3 2021-05-02 23:45:14.316 DEBUG 1 --- [ Thread-35] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P3 2021-05-02 23:45:14.317 INFO 1 --- [ Thread-35] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P5 2021-05-02 23:45:14.399 DEBUG 1 --- [ Thread-37] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P5 2021-05-02 23:45:14.400 INFO 1 --- [ Thread-37] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P6 2021-05-02 23:45:14.472 DEBUG 1 --- [ Thread-38] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P6 2021-05-02 23:45:14.473 INFO 1 --- [ Thread-38] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P4 2021-05-02 23:45:14.584 DEBUG 1 --- [ Thread-36] o.o.aai.datasnapshot.PrintVertexDetails : PrintVertexDetails -- Printed 29 vertexes out to /opt/app/aai-graphadmin/logs/data/dataSnapshots/dataSnapshot.graphSON.202105022345.P4 2021-05-02 23:45:14.585 INFO 1 --- [ Thread-36] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:14.586 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : Pass number 1 (out of 1) took 0 minutes, 10 seconds 2021-05-02 23:45:14.586 DEBUG 1 --- [ scheduling-1] org.onap.aai.datasnapshot.DataSnapshot : -- To write all the data out to snapshot files, it took: 0 minutes, 10 seconds 2021-05-02 23:45:14.588 DEBUG 1 --- [ scheduling-1] o.o.aai.datasnapshot.DataSnapshotTasks : Ended cron job dataSnapshot @ 23:45:14 2021-05-02 23:45:14.588 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:45:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:45:38.765 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:45:38.767 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:45:44.584 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:45:44.597 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:46:08.767 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:46:08.770 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:46:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:46:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:46:38.770 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:46:38.773 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:47:08.773 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:47:08.775 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:47:21.888 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:47:21.890 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:47:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:47:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:47:38.776 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:47:38.777 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:48:08.778 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:48:08.782 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:48:17.368 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:48:17.374 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:48:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:48:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:48:38.782 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:48:38.784 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:48:45.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1162586749-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@61f26a4a at consistency LOCAL_QUORUM 2021-05-02 23:48:45.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1162586749-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:48:45.171 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:48:45.171 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:48:46.172 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:48:52.875 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:48:58.002 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:48:58.004 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:49:02.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2004767460-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@653d582d at consistency LOCAL_QUORUM 2021-05-02 23:49:02.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [2004767460-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:49:04.954 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-02 23:49:04.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-02 23:49:04.955 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-24, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:49:04.956 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=true] closing connection 2021-05-02 23:49:04.956 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-24, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:49:04.956 DEBUG 1 --- [uster-worker-26] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:49:04.958 DEBUG 1 --- [uster-worker-26] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:49:04.958 DEBUG 1 --- [uster-worker-26] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:49:05.958 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:49:05.961 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-25, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:49:15.172 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:49:15.173 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:49:18.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-25, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:49:18.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-25, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:49:18.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-25, inFlight=0, closed=true] closing connection 2021-05-02 23:49:18.056 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-25, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:49:18.056 DEBUG 1 --- [uster-worker-27] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:49:18.057 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 2000 milliseconds 2021-05-02 23:49:18.057 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 2000 ms 2021-05-02 23:49:20.057 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:49:20.060 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-26, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:49:32.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-26, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:49:32.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-26, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:49:32.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-26, inFlight=0, closed=true] closing connection 2021-05-02 23:49:32.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-26, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:49:32.155 DEBUG 1 --- [uster-worker-26] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:49:32.156 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 4000 milliseconds 2021-05-02 23:49:32.156 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 4000 ms 2021-05-02 23:49:32.423 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-24, inFlight=0, closed=true] has already terminated 2021-05-02 23:49:36.156 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:49:36.158 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-27, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:49:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:49:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:49:45.174 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:49:45.176 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:49:47.301 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:49:47.303 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:49:47.424 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-25, inFlight=0, closed=true] has already terminated 2021-05-02 23:49:48.254 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-27, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:49:48.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-27, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:49:48.256 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-27, inFlight=0, closed=true] closing connection 2021-05-02 23:49:48.256 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-27, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:49:48.256 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 8000 milliseconds 2021-05-02 23:49:48.256 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 8000 ms 2021-05-02 23:49:48.256 DEBUG 1 --- [uster-worker-28] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:49:56.257 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:49:56.261 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-28, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:50:02.425 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-26, inFlight=0, closed=true] has already terminated 2021-05-02 23:50:08.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-28, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:50:08.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-28, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:50:08.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-28, inFlight=0, closed=true] closing connection 2021-05-02 23:50:08.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-28, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:50:08.356 DEBUG 1 --- [uster-worker-29] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:50:08.357 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 16000 milliseconds 2021-05-02 23:50:08.357 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 16000 ms 2021-05-02 23:50:15.176 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:50:15.178 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:50:17.426 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-27, inFlight=0, closed=true] has already terminated 2021-05-02 23:50:24.357 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:50:24.360 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-29, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:50:32.426 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-28, inFlight=0, closed=true] has already terminated 2021-05-02 23:50:34.475 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-02 23:50:34.475 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-02 23:50:35.477 DEBUG 1 --- [uster-worker-30] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:50:35.482 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-02 23:50:35.482 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-02 23:50:35.482 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-02 23:50:35.482 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 2 2021-05-02 23:50:35.484 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:50:35.767 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-29, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 23:50:35.767 DEBUG 1 --- [-reconnection-1] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-02 23:50:35.770 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Successful reconnection to /10.242.238.220:9042, setting host UP 2021-05-02 23:50:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:50:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:50:37.975 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-02 23:50:37.983 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 2 2021-05-02 23:50:37.983 DEBUG 1 --- [uster-worker-30] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-02 23:50:37.984 DEBUG 1 --- [uster-worker-30] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-02 23:50:37.984 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-29, inFlight=0, closed=true] closing connection 2021-05-02 23:50:37.984 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-29, inFlight=0, closed=true] closed, remaining = 1 2021-05-02 23:50:37.984 DEBUG 1 --- [-reconnection-1] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future 2021-05-02 23:51:05.770 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:51:05.772 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:51:27.782 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:51:27.784 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:51:32.859 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:51:32.861 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:51:35.773 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:51:35.774 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:51:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:51:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:51:57.785 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:51:57.820 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:52:05.775 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:52:05.777 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:52:35.777 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:52:35.779 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:52:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:52:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:52:48.308 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:52:48.322 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:53:05.780 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:53:05.782 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:53:35.782 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:53:35.784 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:53:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:53:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:53:48.494 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:53:48.497 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:54:05.785 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:54:05.969 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:54:35.970 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:54:35.972 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:54:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:54:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:54:37.394 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:54:37.396 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:54:50.080 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-02T23:54:35.555Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-02 23:55:05.973 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:55:05.975 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:55:20.074 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:55:20.799 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:55:35.975 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:55:35.979 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:55:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:55:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:55:50.179 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:55:50.181 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:56:05.979 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:56:05.981 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:56:35.981 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:56:35.984 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:56:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:56:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:56:41.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1141603517-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@277cfb32 at consistency LOCAL_QUORUM 2021-05-02 23:56:41.156 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1141603517-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-02 23:56:43.967 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:56:44.467 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-02 23:56:44.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-02 23:56:45.468 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-02 23:56:54.059 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:56:54.061 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:56:56.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-30, inFlight=1, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-02 23:56:56.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-02 23:56:56.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-30, inFlight=1, closed=false] failed, remaining = 0 2021-05-02 23:56:56.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=1, closed=true] closing connection 2021-05-02 23:56:56.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-30, inFlight=1, closed=true]: there are still pending requests 2021-05-02 23:56:56.057 DEBUG 1 --- [uster-worker-31] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-02 23:56:56.058 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1257568904-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7e4f69e2 at consistency LOCAL_QUORUM 2021-05-02 23:56:56.058 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1257568904-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.TransportException: [/10.242.238.220:9042] Connection has been closed 2021-05-02 23:56:56.058 DEBUG 1 --- [uster-worker-31] com.datastax.driver.core.RequestHandler : onException triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false)) 2021-05-02 23:56:56.060 DEBUG 1 --- [uster-worker-31] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-02 23:56:56.060 DEBUG 1 --- [uster-worker-31] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-02 23:56:57.060 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:56:57.063 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-31, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:57:14.468 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:57:14.470 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:57:16.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-31, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:57:16.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-31, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:57:16.156 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-31, inFlight=0, closed=true] closing connection 2021-05-02 23:57:16.156 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-31, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:57:16.156 DEBUG 1 --- [uster-worker-32] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:57:16.157 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 2000 milliseconds 2021-05-02 23:57:16.158 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 2000 ms 2021-05-02 23:57:18.158 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:57:18.160 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-32, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:57:32.435 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-30, inFlight=0, closed=true] has already terminated 2021-05-02 23:57:32.654 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-32, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:57:32.655 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-32, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:57:32.656 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-32, inFlight=0, closed=true] closing connection 2021-05-02 23:57:32.656 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-32, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:57:32.656 DEBUG 1 --- [uster-worker-33] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:57:32.657 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 4000 milliseconds 2021-05-02 23:57:32.657 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 4000 ms 2021-05-02 23:57:36.657 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:57:36.660 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-33, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:57:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:57:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:57:44.471 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:57:44.559 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:57:47.436 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-31, inFlight=0, closed=true] has already terminated 2021-05-02 23:57:59.354 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-33, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:57:59.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-33, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:57:59.355 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-33, inFlight=0, closed=true] closing connection 2021-05-02 23:57:59.356 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-33, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:57:59.356 DEBUG 1 --- [uster-worker-34] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:57:59.356 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 8000 milliseconds 2021-05-02 23:57:59.356 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 8000 ms 2021-05-02 23:58:02.436 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-32, inFlight=0, closed=true] has already terminated 2021-05-02 23:58:07.356 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:58:07.359 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-34, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:58:14.559 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:58:14.560 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:58:23.555 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-34, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:58:23.556 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-34, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:58:23.557 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-34, inFlight=0, closed=true] closing connection 2021-05-02 23:58:23.557 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-34, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:58:23.557 DEBUG 1 --- [uster-worker-35] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:58:23.557 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 16000 milliseconds 2021-05-02 23:58:23.557 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 16000 ms 2021-05-02 23:58:32.437 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-33, inFlight=0, closed=true] has already terminated 2021-05-02 23:58:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:58:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:58:39.558 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:58:39.560 DEBUG 1 --- [er-nio-worker-1] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-35, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:58:44.561 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:58:44.562 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:58:51.654 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-35, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:58:51.655 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-35, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:58:51.656 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-35, inFlight=0, closed=true] closing connection 2021-05-02 23:58:51.656 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-35, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:58:51.656 DEBUG 1 --- [uster-worker-36] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:58:51.657 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 32000 milliseconds 2021-05-02 23:58:51.657 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 32000 ms 2021-05-02 23:59:02.438 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-34, inFlight=0, closed=true] has already terminated 2021-05-02 23:59:14.562 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:59:14.564 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-02 23:59:17.439 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-35, inFlight=0, closed=true] has already terminated 2021-05-02 23:59:23.660 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-02 23:59:23.664 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-36, inFlight=0, closed=false] Connection established, initializing transport 2021-05-02 23:59:35.755 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-36, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-02 23:59:35.756 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-36, inFlight=0, closed=false] failed, remaining = 0 2021-05-02 23:59:35.757 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-36, inFlight=0, closed=true] closing connection 2021-05-02 23:59:35.757 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-36, inFlight=0, closed=true]: there are still pending requests 2021-05-02 23:59:35.757 DEBUG 1 --- [uster-worker-37] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-02 23:59:35.757 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 64000 milliseconds 2021-05-02 23:59:35.757 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 64000 ms 2021-05-02 23:59:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-02 23:59:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-02 23:59:44.565 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-02 23:59:44.570 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:00:02.441 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-36, inFlight=0, closed=true] has already terminated 2021-05-03 00:00:14.571 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:00:14.573 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:00:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:00:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:00:39.758 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-03 00:00:39.763 DEBUG 1 --- [er-nio-worker-3] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-37, inFlight=0, closed=false] Connection established, initializing transport 2021-05-03 00:00:44.573 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:00:44.575 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:00:52.060 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:00:52.062 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:00:52.854 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-37, inFlight=0, closed=false] because: [/10.242.238.220:9042] Operation timed out 2021-05-03 00:00:52.854 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-37, inFlight=0, closed=false] failed, remaining = 0 2021-05-03 00:00:52.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-37, inFlight=0, closed=true] closing connection 2021-05-03 00:00:52.855 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-37, inFlight=0, closed=true]: there are still pending requests 2021-05-03 00:00:52.855 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Cluster : Failed reconnection to /10.242.238.220:9042 ([/10.242.238.220:9042] Operation timed out), scheduling retry in 128000 milliseconds 2021-05-03 00:00:52.855 DEBUG 1 --- [-reconnection-1] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] next reconnection attempt in 128000 ms 2021-05-03 00:00:52.855 DEBUG 1 --- [uster-worker-38] com.datastax.driver.core.Cluster : Aborting onDown because a reconnection is running on DOWN host /10.242.238.220:9042 2021-05-03 00:01:07.590 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-03 00:01:07.591 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-03 00:01:08.593 DEBUG 1 --- [uster-worker-39] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-03 00:01:08.601 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-03 00:01:08.601 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-03 00:01:08.601 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-03 00:01:08.601 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-03 00:01:08.603 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] Connection established, initializing transport 2021-05-03 00:01:10.479 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-03 00:01:10.571 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 1 2021-05-03 00:01:10.571 DEBUG 1 --- [uster-worker-39] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-03 00:01:10.571 DEBUG 1 --- [uster-worker-39] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-03 00:01:17.443 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-37, inFlight=0, closed=true] has already terminated 2021-05-03 00:01:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:01:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:01:38.600 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:01:38.602 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:01:40.571 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:01:40.572 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:01:42.297 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:01:42.299 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:02:08.602 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:02:08.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:02:27.597 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:02:27.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:02:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:02:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:02:38.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:02:38.663 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:03:08.663 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:03:08.666 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:03:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:03:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:03:38.201 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:03:38.210 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:03:38.667 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:03:38.668 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:04:08.669 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:04:08.671 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:04:28.324 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:04:28.327 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:04:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:04:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:04:38.672 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:04:38.674 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:04:54.066 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:04:54.070 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:05:08.675 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:05:08.677 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:05:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:05:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:05:38.678 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:05:38.679 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:06:00.002 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:06:00.004 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Started cron job dataGroomingCleanup @ 20210503 2021-05-03 00:06:00.005 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The logDir is /opt/app/aai-graphadmin/logs 2021-05-03 00:06:00.005 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The dataGroomingDir is /opt/app/aai-graphadmin/logs/data/dataGrooming 2021-05-03 00:06:00.006 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The archiveDir is /opt/app/aai-graphadmin/logs/data/dataGrooming/ARCHIVE 2021-05-03 00:06:00.006 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The dataGroomingArcDir is /opt/app/aai-graphadmin/logs/data/dataGrooming/ARCHIVE/dataGrooming 2021-05-03 00:06:00.006 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Directory/opt/app/aai-graphadmin/logsexists: true 2021-05-03 00:06:00.007 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The current date is Mon May 03 00:06:00 GMT 2021 2021-05-03 00:06:00.007 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The newAgeDate is Wed Apr 28 00:06:00 GMT 2021 2021-05-03 00:06:00.007 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The current date is Mon May 03 00:06:00 GMT 2021 2021-05-03 00:06:00.007 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The newAgeDate is Sat Apr 03 00:06:00 GMT 2021 2021-05-03 00:06:00.008 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Ended cron job dataGroomingCleanup @ 20210503 2021-05-03 00:06:00.008 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:06:08.680 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:06:08.682 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:06:31.397 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:06:31.588 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:06:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:06:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:06:38.682 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:06:38.683 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:06:43.762 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-03 00:06:43.762 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-03 00:06:44.763 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-03 00:07:01.995 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:07:09.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1413083045-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@119bf780 at consistency LOCAL_QUORUM 2021-05-03 00:07:09.155 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1413083045-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-03 00:07:13.763 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:07:13.765 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:07:14.054 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : Defuncting Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] because: [/10.242.238.220:9042] Heartbeat query timed out 2021-05-03 00:07:14.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preventing new connections for the next 1000 ms 2021-05-03 00:07:14.055 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-38, inFlight=0, closed=false] failed, remaining = 0 2021-05-03 00:07:14.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=true] closing connection 2021-05-03 00:07:14.057 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.Connection : Not terminating Connection[/10.242.238.220:9042-38, inFlight=0, closed=true]: there are still pending requests 2021-05-03 00:07:14.057 DEBUG 1 --- [uster-worker-40] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host DOWN 2021-05-03 00:07:14.059 DEBUG 1 --- [uster-worker-40] c.d.d.core.AbstractReconnectionHandler : First reconnection scheduled in 1000ms 2021-05-03 00:07:14.060 DEBUG 1 --- [uster-worker-40] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] starting reconnection attempt 2021-05-03 00:07:15.060 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 1 new connections, total = 1 2021-05-03 00:07:15.064 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] Connection established, initializing transport 2021-05-03 00:07:17.094 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:07:17.098 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:07:23.160 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-03 00:07:23.160 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-03 00:07:23.593 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] Transport initialized, connection ready 2021-05-03 00:07:23.594 DEBUG 1 --- [-reconnection-0] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-03 00:07:23.663 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Successful reconnection to /10.242.238.220:9042, setting host UP 2021-05-03 00:07:23.663 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] marking host UP 2021-05-03 00:07:23.663 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Cancelling reconnection attempt since node is UP 2021-05-03 00:07:23.663 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Cluster : Preparing 54 prepared queries on newly up node /10.242.238.220:9042 2021-05-03 00:07:23.686 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] preparing to open 0 new connections, total = 1 2021-05-03 00:07:23.686 DEBUG 1 --- [-reconnection-0] c.d.driver.core.HostConnectionPool : Created connection pool to host /10.242.238.220:9042 (1 connections needed, 1 successfully opened) 2021-05-03 00:07:23.686 DEBUG 1 --- [-reconnection-0] com.datastax.driver.core.Session : Added connection pool for /10.242.238.220:9042 2021-05-03 00:07:23.687 DEBUG 1 --- [-reconnection-0] c.d.d.core.AbstractReconnectionHandler : Reconnection successful, cleared the future 2021-05-03 00:07:24.162 DEBUG 1 --- [uster-worker-41] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-03 00:07:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:07:37.252 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:07:47.449 DEBUG 1 --- [ection-reaper-0] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-38, inFlight=0, closed=true] has already terminated 2021-05-03 00:07:53.686 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:07:53.767 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:07:54.165 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:07:54.167 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:08:24.167 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:08:24.169 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:08:24.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:08:24.461 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:08:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:08:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:08:54.170 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:08:54.174 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:09:14.669 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:09:14.687 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:09:24.175 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:09:24.262 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:09:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:09:37.242 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:09:44.711 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:09:44.759 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:09:54.263 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:09:54.265 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:10:24.265 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:10:24.268 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:10:35.959 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:10:35.962 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:10:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:10:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:10:51.077 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:10:51.080 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:10:54.268 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:10:54.270 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:11:11.360 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:11:11.362 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:11:24.270 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:11:24.272 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:11:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:11:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:11:46.593 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:11:46.595 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:11:54.272 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:11:54.273 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:12:11.807 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:12:12.179 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:12:24.274 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:12:24.277 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:12:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:12:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:12:46.374 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-03 00:12:46.374 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-03 00:12:47.375 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-03 00:12:52.517 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:12:53.378 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=1, closed=false] heartbeat query succeeded 2021-05-03 00:13:04.755 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1140766947-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@1932eddf at consistency LOCAL_QUORUM 2021-05-03 00:13:04.756 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1140766947-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-03 00:13:08.789 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] Response received on stream 4992 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses. 2021-05-03 00:13:16.375 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:13:16.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:13:17.697 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:13:17.699 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:13:22.289 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [979485610-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@587f2a1c at consistency null 2021-05-03 00:13:22.289 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [979485610-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:13:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:13:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:13:43.191 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1169590502-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@7e273d54 at consistency null 2021-05-03 00:13:43.192 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1169590502-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:13:46.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:13:46.379 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:13:59.811 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [992423725-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@5ee1fcd2 at consistency null 2021-05-03 00:13:59.812 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [992423725-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:14:16.383 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:14:16.385 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:14:29.813 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:14:30.372 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:14:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:14:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:14:45.296 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1099481797-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@295e0515 at consistency null 2021-05-03 00:14:45.296 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1099481797-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:14:46.386 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:14:46.387 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:14:55.159 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:14:55.163 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:15:03.190 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [447781825-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@850e5e8 at consistency null 2021-05-03 00:15:03.190 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [447781825-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:15:16.388 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:15:16.389 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:15:33.191 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:15:33.574 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:15:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:15:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:15:46.390 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:15:46.391 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:16:03.575 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:16:03.579 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:16:08.680 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [766845363-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@47230e07 at consistency null 2021-05-03 00:16:08.681 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [766845363-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:16:14.380 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1706869025-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@48d3ccae at consistency null 2021-05-03 00:16:14.381 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1706869025-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:16:16.392 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:16:16.395 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:16:25.677 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [878719147-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@52325273 at consistency null 2021-05-03 00:16:25.678 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [878719147-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:16:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:16:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:16:46.395 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:16:46.397 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:16:49.183 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-03 00:16:49.184 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-03 00:16:50.186 DEBUG 1 --- [uster-worker-42] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042 2021-05-03 00:16:55.678 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:16:55.680 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:17:00.002 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:17:00.003 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Started cron job dataSnapshotCleanup @ 20210503 2021-05-03 00:17:00.003 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The logDir is /opt/app/aai-graphadmin/logs 2021-05-03 00:17:00.004 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The dataSnapshotDir is /opt/app/aai-graphadmin/logs/data/dataSnapshots 2021-05-03 00:17:00.004 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The archiveDir is /opt/app/aai-graphadmin/logs/data/dataSnapshots/ARCHIVE 2021-05-03 00:17:00.004 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The dataSnapshotArcDir is /opt/app/aai-graphadmin/logs/data/dataSnapshots/ARCHIVE/dataSnapshots 2021-05-03 00:17:00.004 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Directory/opt/app/aai-graphadmin/logsexists: true 2021-05-03 00:17:00.005 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The current date is Mon May 03 00:17:00 GMT 2021 2021-05-03 00:17:00.005 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The newAgeDate is Wed Apr 28 00:17:00 GMT 2021 2021-05-03 00:17:00.005 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P4 2021-05-03 00:17:00.012 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.013 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P3 2021-05-03 00:17:00.013 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.013 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P2 2021-05-03 00:17:00.013 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.013 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P0 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P1 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P7 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.014 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P8 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P14 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P13 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P12 2021-05-03 00:17:00.015 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P6 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P10 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P9 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.016 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P11 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The file name in dataSnapshot: dataSnapshot.graphSON.202105022345.P5 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The fileCreateDate in dataSnapshot is Sun May 02 00:00:00 GMT 2021 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The current date is Mon May 03 00:17:00 GMT 2021 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The newAgeDate is Sat Apr 03 00:17:00 GMT 2021 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Started dmaapEventsDataCleanup @ 20210503 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The logDir is /opt/app/aai-graphadmin/logs 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : The dmaapEventsDataDir is /opt/app/aai-graphadmin/logs/data/dmaapEvents 2021-05-03 00:17:00.017 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Ended cron dmaapEventsDataCleanup @ 20210503 2021-05-03 00:17:00.031 ERROR 1 --- [ scheduling-1] org.onap.aai.logging.ErrorLogHelper : ERR.5.4.4000 ex=org.onap.aai.exceptions.AAIException: Exception running cron job for DataCleanupex=org.onap.aai.exceptions.AAIException: Property key aai.datamigration.agedelete cannot be found ClassName- org.onap.aai.util.AAIConfig :LineNumber- 157 :MethodName- get ClassName- org.onap.aai.util.AAIConfig :LineNumber- 175 :MethodName- getInt ClassName- org.onap.aai.datacleanup.DataCleanupTasks :LineNumber- 343 :MethodName- dataMigrationCleanup ClassName- org.onap.aai.datacleanup.DataCleanupTasks :LineNumber- 297 :MethodName- dataSnapshotCleanup ClassName- sun.reflect.NativeMethodAccessorImpl :LineNumber- -2 :MethodName- invoke0 ClassName- sun.reflect.NativeMethodAccessorImpl :LineNumber- 62 :MethodName- invoke ClassName- sun.reflect.DelegatingMethodAccessorImpl :LineNumber- 43 :MethodName- invoke ClassName- java.lang.reflect.Method :LineNumber- 498 :MethodName- invoke ClassName- org.springframework.scheduling.support.ScheduledMethodRunnable :LineNumber- 84 :MethodName- run ClassName- org.springframework.scheduling.support.DelegatingErrorHandlingRunnable :LineNumber- 54 :MethodName- run ClassName- org.onap.aai.logging.ErrorLogHelper :LineNumber- 616 :MethodName- logError ClassName- org.onap.aai.datacleanup.DataCleanupTasks :LineNumber- 300 :MethodName- dataSnapshotCleanup ClassName- sun.reflect.NativeMethodAccessorImpl :LineNumber- -2 :MethodName- invoke0 ClassName- sun.reflect.NativeMethodAccessorImpl :LineNumber- 62 :MethodName- invoke ClassName- sun.reflect.DelegatingMethodAccessorImpl :LineNumber- 43 :MethodName- invoke ClassName- java.lang.reflect.Method :LineNumber- 498 :MethodName- invoke ClassName- org.springframework.scheduling.support.ScheduledMethodRunnable :LineNumber- 84 :MethodName- run ClassName- org.springframework.scheduling.support.DelegatingErrorHandlingRunnable :LineNumber- 54 :MethodName- run ClassName- org.springframework.scheduling.concurrent.ReschedulingRunnable :LineNumber- 93 :MethodName- run ClassName- java.util.concurrent.Executors$RunnableAdapter :LineNumber- 511 :MethodName- call 2021-05-03 00:17:00.033 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : AAI_4000 2021-05-03 00:17:00.035 DEBUG 1 --- [ scheduling-1] o.onap.aai.datacleanup.DataCleanupTasks : Ended cron job dataSnapshotCleanup @ 20210503 2021-05-03 00:17:00.036 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:17:20.197 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:17:20.258 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:17:25.901 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:17:25.959 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:17:37.236 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:17:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:17:41.073 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:17:41.075 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:17:50.259 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:17:50.261 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:18:06.118 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:18:06.159 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:18:20.262 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:18:20.265 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:18:37.245 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:18:37.247 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:18:46.302 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:18:46.304 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:18:50.265 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:18:50.266 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:19:06.377 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:19:06.379 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:19:20.267 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:19:20.274 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:19:26.471 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:19:26.492 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:19:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:19:37.241 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:19:50.275 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:19:50.276 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:19:56.568 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:19:56.582 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:20:20.277 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:20:20.279 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:20:26.630 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:20:26.659 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:20:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:20:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:20:50.280 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:20:50.282 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:21:17.393 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:21:17.395 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:21:20.282 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:21:20.284 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:21:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:21:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:21:47.801 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:21:47.803 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:21:50.285 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:21:50.286 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:22:20.286 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:22:20.290 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:22:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:22:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:22:39.179 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:22:44.483 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=1, closed=false] heartbeat query succeeded 2021-05-03 00:22:50.291 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:22:50.292 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:23:20.293 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:23:20.295 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:23:34.562 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:23:34.564 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:23:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:23:37.240 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:23:50.295 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:23:50.298 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:24:20.298 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:24:20.306 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:24:30.025 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:24:30.027 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:24:37.238 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:24:37.246 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:24:50.107 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:24:50.110 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:24:50.307 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:24:50.309 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:25:00.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1962251576-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@575b2e8e at consistency LOCAL_QUORUM 2021-05-03 00:25:00.255 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1962251576-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-03 00:25:06.261 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT DOWN /10.242.238.220:9042, scheduling delivery 2021-05-03 00:25:06.262 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event DOWN 2021-05-03 00:25:07.263 DEBUG 1 --- [d-task-worker-0] com.datastax.driver.core.Cluster : Ignoring down event on /10.242.238.220:9042 because it still has active connections 2021-05-03 00:25:07.993 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:25:07.995 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.232.3:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:25:13.183 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=1, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:25:13.975 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=1, closed=false] heartbeat query succeeded 2021-05-03 00:25:16.844 WARN 1 --- [pool-9-thread-1] o.j.diskstorage.log.kcvs.KCVSLog : Could not read messages for timestamp [2021-05-03T00:24:47.683Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:732) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 9 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$null$2(CQLKeyColumnValueStore.java:126) at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$3(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:798) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:795) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 10 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:627) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1233) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1151) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:725) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:400) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:300) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 23 common frames omitted 2021-05-03 00:25:16.971 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] Response received on stream 8256 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses. 2021-05-03 00:25:30.264 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:25:30.266 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-2, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:25:36.262 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:25:36.264 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:25:37.237 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Entering 2021-05-03 00:25:37.239 INFO 1 --- [ scheduling-1] o.o.a.a.logs.AaiScheduledTaskAuditLog : Exiting. 2021-05-03 00:25:38.954 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1325362133-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@219ad205 at consistency LOCAL_QUORUM 2021-05-03 00:25:38.954 DEBUG 1 --- [ter-timeouter-0] com.datastax.driver.core.RequestHandler : [1325362133-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.242.238.220:9042] Timed out waiting for server response 2021-05-03 00:25:39.074 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.Connection : Connection[/10.242.238.220:9042-39, inFlight=0, closed=false] Response received on stream 8448 but no handler set anymore (either the request has timed out or it was closed due to another error). Received message is ERROR READ_TIMEOUT: Operation timed out - received only 1 responses. 2021-05-03 00:26:06.265 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat 2021-05-03 00:26:06.268 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Connection : Connection[cassandra/10.242.5.156:9042-1, inFlight=0, closed=false] heartbeat query succeeded 2021-05-03 00:26:07.076 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1916249203-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6378dae5 at consistency null 2021-05-03 00:26:07.077 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1916249203-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:26:17.688 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1015294417-0] Doing retry 1 for query com.datastax.driver.core.BoundStatement@6786054e at consistency null 2021-05-03 00:26:17.688 DEBUG 1 --- [er-nio-worker-2] com.datastax.driver.core.RequestHandler : [1015294417-0] Error querying /10.242.238.220:9042 : com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive) 2021-05-03 00:26:20.585 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Cluster : Received event EVENT UP /10.242.238.220:9042, scheduling delivery 2021-05-03 00:26:20.585 DEBUG 1 --- [er-nio-worker-0] com.datastax.driver.core.Host.STATES : [/10.242.238.220:9042] received event UP 2021-05-03 00:26:21.588 DEBUG 1 --- [uster-worker-43] c.d.driver.core.ControlConnection : [Control connection] Refreshing node info on /10.242.238.220:9042