By type
Policy clamp Kubernetes participant config file: /opt/app/policy/clamp/etc/mounted/KubernetesParticipantParameters.yaml overriding logback xml file . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v2.5.14) [2023-01-13T23:07:13.261+00:00|INFO|Application|main] Starting Application using Java 11.0.11 on onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn with PID 7 (/app/app.jar started by policy in /opt/app/policy/clamp/bin) [2023-01-13T23:07:13.266+00:00|INFO|Application|main] No active profile set, falling back to 1 default profile: "default" [2023-01-13T23:07:16.977+00:00|INFO|Http11NioProtocol|main] Initializing ProtocolHandler ["http-nio-8083"] [2023-01-13T23:07:16.978+00:00|INFO|StandardService|main] Starting service [Tomcat] [2023-01-13T23:07:16.978+00:00|INFO|StandardEngine|main] Starting Servlet engine: [Apache Tomcat/9.0.62] [2023-01-13T23:07:17.090+00:00|INFO|[/onap/policy/clamp/acm/k8sparticipant]|main] Initializing Spring embedded WebApplicationContext [2023-01-13T23:07:17.976+00:00|INFO|Version|main] HV000001: Hibernate Validator 6.2.3.Final [2023-01-13T23:07:19.743+00:00|INFO|SingleThreadedDmaapTopicSource|main] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=false, locked=false, uebThread=null, topicListeners=0, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=0]]]]: INITTED [2023-01-13T23:07:21.975+00:00|INFO|Http11NioProtocol|main] Starting ProtocolHandler ["http-nio-8083"] [2023-01-13T23:07:22.151+00:00|INFO|ServiceManager|main] service manager starting [2023-01-13T23:07:22.151+00:00|INFO|ServiceManager|main] service manager starting Topic endpoint management [2023-01-13T23:07:22.161+00:00|INFO|SingleThreadedBusTopicSource|main] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=false, locked=false, uebThread=null, topicListeners=0, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=0]]]]: starting [2023-01-13T23:07:22.163+00:00|INFO|SingleThreadedDmaapTopicSource|main] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=false, locked=false, uebThread=null, topicListeners=0, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=0]]]]: INITTED [2023-01-13T23:07:22.169+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:07:22.173+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:07:22.164+00:00|INFO|InlineBusTopicSink|main] InlineDmaapTopicSink [userName=null, password=null, getTopicCommInfrastructure()=DMAAP, toString()=InlineBusTopicSink [partitionId=b742f1d0-dbe0-422a-9f90-2b8dcf98248f, alive=false, publisher=null]]: starting [2023-01-13T23:07:22.183+00:00|INFO|InlineDmaapTopicSink|main] InlineDmaapTopicSink [userName=null, password=null, getTopicCommInfrastructure()=DMAAP, toString()=InlineBusTopicSink [partitionId=b742f1d0-dbe0-422a-9f90-2b8dcf98248f, alive=false, publisher=CambriaPublisherWrapper []]]: DMAAP SINK created [2023-01-13T23:07:22.183+00:00|INFO|ServiceManager|main] service manager starting Publisher ParticipantMessagePublisher [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener ParticipantRegisterAckListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener ParticipantStatusReqListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener AutomationCompositionStateChangeListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener ParticipantDeregisterAckListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener AutomationCompositionUpdateListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Listener ParticipantUpdateListener [2023-01-13T23:07:22.184+00:00|INFO|ServiceManager|main] service manager starting Topic Message Dispatcher [2023-01-13T23:07:22.184+00:00|INFO|TopicBase|main] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=0, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=0]]]]: registering org.onap.policy.common.endpoints.listeners.MessageTypeDispatcher@361abd01 [2023-01-13T23:07:22.185+00:00|INFO|SingleThreadedBusTopicSource|main] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: register: start not attempted [2023-01-13T23:07:22.185+00:00|INFO|ServiceManager|main] service manager started [2023-01-13T23:07:22.197+00:00|INFO|OrderedServiceImpl|main] ***** OrderedServiceImpl implementers: [] [2023-01-13T23:07:22.199+00:00|INFO|network|main] [OUT|DMAAP|POLICY-ACRUNTIME-PARTICIPANT] {"messageType":"PARTICIPANT_REGISTER","messageId":"1cab66f1-338d-42c6-9b10-58a740a2a5d4","timestamp":"2023-01-13T23:07:22.185570Z","participantType":{"name":"org.onap.policy.clamp.acm.KubernetesParticipant","version":"2.3.4"},"participantId":{"name":"K8sParticipant0","version":"1.0.0"}} [2023-01-13T23:07:22.208+00:00|INFO|Application|main] Started Application in 10.428 seconds (JVM running for 11.675) [2023-01-13T23:07:22.284+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 83 ms [2023-01-13T23:07:22.289+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:22.290+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:22.291+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:07:22.292+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:22.308+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:22.312+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:22.312+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:23.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 1132 ms [2023-01-13T23:07:23.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:23.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:23.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:23.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:23.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:24.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 2182 ms [2023-01-13T23:07:24.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:24.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:24.385+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:24.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:24.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:25.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 3232 ms [2023-01-13T23:07:25.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:25.434+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:25.441+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:25.441+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:25.442+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:26.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 4282 ms [2023-01-13T23:07:26.484+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:26.484+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:26.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:26.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:26.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:27.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 5332 ms [2023-01-13T23:07:27.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:27.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:27.536+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:27.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:27.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:28.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 6382 ms [2023-01-13T23:07:28.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:28.584+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:28.588+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:28.589+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:28.589+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:29.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 7432 ms [2023-01-13T23:07:29.634+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:29.634+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:29.637+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:29.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:29.638+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:30.684+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 8482 ms [2023-01-13T23:07:30.684+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:30.684+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:30.687+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:30.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:30.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:31.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 9532 ms [2023-01-13T23:07:31.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:31.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:31.735+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:31.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:31.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:32.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 10582 ms [2023-01-13T23:07:32.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:32.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:32.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:32.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:32.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:32.787+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 11, above threshold 10. [2023-01-13T23:07:33.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 11632 ms [2023-01-13T23:07:33.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:33.834+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:33.836+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:33.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:33.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:33.837+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 12, above threshold 10. [2023-01-13T23:07:34.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 12682 ms [2023-01-13T23:07:34.884+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:34.884+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:34.887+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:34.888+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:34.888+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:34.888+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 13, above threshold 10. [2023-01-13T23:07:35.934+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 13733 ms [2023-01-13T23:07:35.934+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:35.934+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:35.937+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:35.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:35.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:35.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 14, above threshold 10. [2023-01-13T23:07:36.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 14782 ms [2023-01-13T23:07:36.984+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:36.984+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:36.987+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:36.988+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:36.988+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:36.988+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 15, above threshold 10. [2023-01-13T23:07:37.294+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:07:37.294+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:07:37.294+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:07:37.298+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:37.298+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:37.298+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:07:38.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 15832 ms [2023-01-13T23:07:38.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:38.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:38.035+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:38.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:38.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:38.036+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 16, above threshold 10. [2023-01-13T23:07:39.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 16882 ms [2023-01-13T23:07:39.084+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:39.084+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:39.091+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:39.091+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:39.094+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:39.094+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 17, above threshold 10. [2023-01-13T23:07:40.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 17932 ms [2023-01-13T23:07:40.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:40.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:40.146+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:40.148+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:40.148+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:40.148+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 18, above threshold 10. [2023-01-13T23:07:41.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 18982 ms [2023-01-13T23:07:41.184+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:41.184+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:41.189+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:41.191+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:41.191+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:41.191+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 19, above threshold 10. [2023-01-13T23:07:42.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 20032 ms [2023-01-13T23:07:42.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:42.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:42.237+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:42.237+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:42.237+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:42.237+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 20, above threshold 10. [2023-01-13T23:07:43.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 21082 ms [2023-01-13T23:07:43.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:43.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:43.286+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:43.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:43.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:43.287+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 21, above threshold 10. [2023-01-13T23:07:44.334+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 22133 ms [2023-01-13T23:07:44.334+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:44.334+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:44.340+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:44.341+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:44.341+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:44.341+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 22, above threshold 10. [2023-01-13T23:07:45.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 23182 ms [2023-01-13T23:07:45.384+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:45.384+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:45.388+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:45.388+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:45.389+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:45.389+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 23, above threshold 10. [2023-01-13T23:07:46.434+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 24232 ms [2023-01-13T23:07:46.434+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:46.434+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:46.437+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:46.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:46.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:46.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 24, above threshold 10. [2023-01-13T23:07:47.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 25282 ms [2023-01-13T23:07:47.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:47.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:47.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:47.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:47.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:47.486+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 25, above threshold 10. [2023-01-13T23:07:48.535+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 26333 ms [2023-01-13T23:07:48.535+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:48.535+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:48.540+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:48.540+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:48.540+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:48.540+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 26, above threshold 10. [2023-01-13T23:07:49.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 27382 ms [2023-01-13T23:07:49.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:49.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:49.586+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:49.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:49.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:49.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 27, above threshold 10. [2023-01-13T23:07:50.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 28432 ms [2023-01-13T23:07:50.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:50.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:50.636+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:50.636+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:50.636+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:50.636+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 28, above threshold 10. [2023-01-13T23:07:51.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 29482 ms [2023-01-13T23:07:51.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:51.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:51.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:51.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:51.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:51.687+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 29, above threshold 10. [2023-01-13T23:07:52.299+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:07:52.300+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:07:52.300+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:07:52.303+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:52.303+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:52.303+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:07:52.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 30532 ms [2023-01-13T23:07:52.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:52.734+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:52.739+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:52.740+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:52.740+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:52.740+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 30, above threshold 10. [2023-01-13T23:07:53.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 31582 ms [2023-01-13T23:07:53.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:53.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:53.787+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:53.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:53.788+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:53.788+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 31, above threshold 10. [2023-01-13T23:07:54.835+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 32634 ms [2023-01-13T23:07:54.836+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:54.836+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:54.839+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:54.840+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:54.840+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:54.840+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 32, above threshold 10. [2023-01-13T23:07:55.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 33682 ms [2023-01-13T23:07:55.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:55.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:55.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:55.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:55.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:55.887+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 33, above threshold 10. [2023-01-13T23:07:56.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 34732 ms [2023-01-13T23:07:56.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:56.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:56.936+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:56.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:56.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:56.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 34, above threshold 10. [2023-01-13T23:07:57.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 35782 ms [2023-01-13T23:07:57.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:57.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:57.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:57.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:57.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:57.986+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 35, above threshold 10. [2023-01-13T23:07:59.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 36832 ms [2023-01-13T23:07:59.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:07:59.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:07:59.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:07:59.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:07:59.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:07:59.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 36, above threshold 10. [2023-01-13T23:08:00.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 37882 ms [2023-01-13T23:08:00.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:00.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:00.086+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:00.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:00.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:00.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 37, above threshold 10. [2023-01-13T23:08:01.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 38932 ms [2023-01-13T23:08:01.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:01.134+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:01.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:01.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:01.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:01.138+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 38, above threshold 10. [2023-01-13T23:08:02.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 39982 ms [2023-01-13T23:08:02.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:02.184+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:02.187+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:02.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:02.195+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:02.196+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 39, above threshold 10. [2023-01-13T23:08:03.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 41032 ms [2023-01-13T23:08:03.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:03.234+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:03.237+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:03.239+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:03.240+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:03.240+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 40, above threshold 10. [2023-01-13T23:08:04.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 42082 ms [2023-01-13T23:08:04.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:04.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:04.287+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:04.288+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:04.288+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:04.288+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 41, above threshold 10. [2023-01-13T23:08:05.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 43132 ms [2023-01-13T23:08:05.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:05.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:05.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:05.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:05.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:05.337+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 42, above threshold 10. [2023-01-13T23:08:06.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 44182 ms [2023-01-13T23:08:06.384+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:06.384+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:06.387+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:06.388+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:06.388+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:06.388+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 43, above threshold 10. [2023-01-13T23:08:07.304+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:08:07.304+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:08:07.304+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:08:07.308+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:07.308+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:07.308+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:08:07.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 45232 ms [2023-01-13T23:08:07.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:07.434+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:07.436+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:07.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:07.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:07.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 44, above threshold 10. [2023-01-13T23:08:08.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 46282 ms [2023-01-13T23:08:08.484+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:08.484+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:08.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:08.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:08.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:08.487+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 45, above threshold 10. [2023-01-13T23:08:09.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 47332 ms [2023-01-13T23:08:09.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:09.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:09.536+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:09.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:09.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:09.536+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 46, above threshold 10. [2023-01-13T23:08:10.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 48382 ms [2023-01-13T23:08:10.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:10.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:10.586+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:10.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:10.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:10.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 47, above threshold 10. [2023-01-13T23:08:11.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 49432 ms [2023-01-13T23:08:11.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:11.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:11.635+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:11.636+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:11.636+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:11.636+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 48, above threshold 10. [2023-01-13T23:08:12.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 50482 ms [2023-01-13T23:08:12.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:12.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:12.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:12.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:12.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:12.686+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 49, above threshold 10. [2023-01-13T23:08:13.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 51532 ms [2023-01-13T23:08:13.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:13.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:13.736+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:13.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:13.737+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:13.737+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 50, above threshold 10. [2023-01-13T23:08:14.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 52582 ms [2023-01-13T23:08:14.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:14.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:14.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:14.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:14.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:14.787+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 51, above threshold 10. [2023-01-13T23:08:15.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 53632 ms [2023-01-13T23:08:15.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:15.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:15.835+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:15.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:15.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:15.836+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 52, above threshold 10. [2023-01-13T23:08:16.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 54682 ms [2023-01-13T23:08:16.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:16.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:16.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:16.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:16.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:16.887+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 53, above threshold 10. [2023-01-13T23:08:17.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 55732 ms [2023-01-13T23:08:17.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:17.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:17.937+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:17.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:17.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:17.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 54, above threshold 10. [2023-01-13T23:08:18.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 56782 ms [2023-01-13T23:08:18.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:18.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:18.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:18.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:18.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:18.987+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 55, above threshold 10. [2023-01-13T23:08:20.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 57832 ms [2023-01-13T23:08:20.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:20.034+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:20.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:20.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:20.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:20.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 56, above threshold 10. [2023-01-13T23:08:21.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 58882 ms [2023-01-13T23:08:21.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:21.084+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:21.086+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:21.087+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:21.087+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:21.087+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 57, above threshold 10. [2023-01-13T23:08:22.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 59932 ms [2023-01-13T23:08:22.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:22.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:22.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:22.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:22.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:22.136+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 58, above threshold 10. [2023-01-13T23:08:22.308+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:08:22.309+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:08:22.309+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:08:22.312+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:22.312+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:22.312+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:08:23.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 60982 ms [2023-01-13T23:08:23.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:23.183+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:23.188+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:23.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:23.189+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:23.189+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 59, above threshold 10. [2023-01-13T23:08:24.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 62032 ms [2023-01-13T23:08:24.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:24.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:24.236+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:24.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:24.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:24.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 60, above threshold 10. [2023-01-13T23:08:25.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 63082 ms [2023-01-13T23:08:25.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:25.284+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:25.286+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:25.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:25.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:25.287+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 61, above threshold 10. [2023-01-13T23:08:26.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 64132 ms [2023-01-13T23:08:26.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:26.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:26.335+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:26.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:26.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:26.336+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 62, above threshold 10. [2023-01-13T23:08:27.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 65182 ms [2023-01-13T23:08:27.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:27.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:27.386+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:27.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:27.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:27.386+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 63, above threshold 10. [2023-01-13T23:08:28.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 66232 ms [2023-01-13T23:08:28.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:28.433+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:28.436+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:28.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:28.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:28.436+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 64, above threshold 10. [2023-01-13T23:08:29.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 67282 ms [2023-01-13T23:08:29.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:29.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:29.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:29.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:29.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:29.486+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 65, above threshold 10. [2023-01-13T23:08:30.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 68332 ms [2023-01-13T23:08:30.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:30.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:30.536+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:30.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:30.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:30.536+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 66, above threshold 10. [2023-01-13T23:08:31.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 69382 ms [2023-01-13T23:08:31.584+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:31.584+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:31.587+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:31.587+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:31.588+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:31.588+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 67, above threshold 10. [2023-01-13T23:08:32.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 70432 ms [2023-01-13T23:08:32.634+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:32.634+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:32.636+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:32.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:32.638+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:32.638+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 68, above threshold 10. [2023-01-13T23:08:33.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 71482 ms [2023-01-13T23:08:33.684+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:33.684+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:33.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:33.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:33.688+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:33.688+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 69, above threshold 10. [2023-01-13T23:08:34.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 72532 ms [2023-01-13T23:08:34.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:34.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:34.739+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:34.739+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:34.739+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:34.739+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 70, above threshold 10. [2023-01-13T23:08:35.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 73582 ms [2023-01-13T23:08:35.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:35.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:35.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:35.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:35.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:35.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 71, above threshold 10. [2023-01-13T23:08:36.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 74632 ms [2023-01-13T23:08:36.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:36.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:36.836+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:36.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:36.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:36.836+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 72, above threshold 10. [2023-01-13T23:08:37.313+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:08:37.314+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:08:37.315+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:08:37.318+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:37.318+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:37.318+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:08:37.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 75682 ms [2023-01-13T23:08:37.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:37.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:37.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:37.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:37.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:37.886+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 73, above threshold 10. [2023-01-13T23:08:38.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 76732 ms [2023-01-13T23:08:38.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:38.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:38.935+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:38.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:38.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:38.936+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 74, above threshold 10. [2023-01-13T23:08:39.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 77782 ms [2023-01-13T23:08:39.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:39.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:39.985+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:39.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:39.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:39.986+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 75, above threshold 10. [2023-01-13T23:08:41.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 78832 ms [2023-01-13T23:08:41.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:41.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:41.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:41.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:41.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:41.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 76, above threshold 10. [2023-01-13T23:08:42.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 79882 ms [2023-01-13T23:08:42.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:42.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:42.085+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:42.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:42.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:42.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 77, above threshold 10. [2023-01-13T23:08:43.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 80932 ms [2023-01-13T23:08:43.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:43.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:43.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:43.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:43.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:43.137+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 78, above threshold 10. [2023-01-13T23:08:44.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 81982 ms [2023-01-13T23:08:44.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:44.183+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:44.185+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:44.186+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:44.186+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:44.186+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 79, above threshold 10. [2023-01-13T23:08:45.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 83032 ms [2023-01-13T23:08:45.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:45.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:45.235+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:45.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:45.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:45.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 80, above threshold 10. [2023-01-13T23:08:46.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 84082 ms [2023-01-13T23:08:46.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:46.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:46.286+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:46.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:46.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:46.286+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 81, above threshold 10. [2023-01-13T23:08:47.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 85132 ms [2023-01-13T23:08:47.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:47.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:47.335+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:47.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:47.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:47.336+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 82, above threshold 10. [2023-01-13T23:08:48.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 86182 ms [2023-01-13T23:08:48.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:48.384+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:48.386+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:48.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:48.387+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:48.387+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 83, above threshold 10. [2023-01-13T23:08:49.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 87232 ms [2023-01-13T23:08:49.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:49.433+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:49.435+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:49.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:49.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:49.436+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 84, above threshold 10. [2023-01-13T23:08:50.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 88282 ms [2023-01-13T23:08:50.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:50.484+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:50.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:50.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:50.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:50.487+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 85, above threshold 10. [2023-01-13T23:08:51.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 89332 ms [2023-01-13T23:08:51.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:51.534+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:51.537+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:51.538+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:51.538+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:51.538+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 86, above threshold 10. [2023-01-13T23:08:52.319+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:08:52.319+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:08:52.319+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:08:52.322+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:52.322+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:52.322+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:08:52.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 90382 ms [2023-01-13T23:08:52.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:52.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:52.585+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:52.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:52.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:52.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 87, above threshold 10. [2023-01-13T23:08:53.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 91432 ms [2023-01-13T23:08:53.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:53.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:53.638+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:53.638+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:53.638+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:53.638+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 88, above threshold 10. [2023-01-13T23:08:54.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 92482 ms [2023-01-13T23:08:54.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:54.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:54.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:54.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:54.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:54.686+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 89, above threshold 10. [2023-01-13T23:08:55.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 93532 ms [2023-01-13T23:08:55.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:55.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:55.736+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:55.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:55.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:55.736+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 90, above threshold 10. [2023-01-13T23:08:56.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 94582 ms [2023-01-13T23:08:56.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:56.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:56.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:56.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:56.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:56.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 91, above threshold 10. [2023-01-13T23:08:57.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 95632 ms [2023-01-13T23:08:57.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:57.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:57.836+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:57.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:57.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:57.836+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 92, above threshold 10. [2023-01-13T23:08:58.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 96682 ms [2023-01-13T23:08:58.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:58.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:58.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:58.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:58.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:58.887+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 93, above threshold 10. [2023-01-13T23:08:59.934+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 97732 ms [2023-01-13T23:08:59.934+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:08:59.934+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:08:59.940+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:08:59.940+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:08:59.940+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:08:59.940+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 94, above threshold 10. [2023-01-13T23:09:00.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 98782 ms [2023-01-13T23:09:00.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:00.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:00.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:00.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:00.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:00.986+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 95, above threshold 10. [2023-01-13T23:09:02.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 99832 ms [2023-01-13T23:09:02.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:02.034+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:02.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:02.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:02.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:02.038+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 96, above threshold 10. [2023-01-13T23:09:03.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 100882 ms [2023-01-13T23:09:03.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:03.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:03.086+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:03.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:03.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:03.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 97, above threshold 10. [2023-01-13T23:09:04.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 101932 ms [2023-01-13T23:09:04.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:04.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:04.135+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:04.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:04.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:04.136+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 98, above threshold 10. [2023-01-13T23:09:05.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 102982 ms [2023-01-13T23:09:05.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:05.183+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:05.186+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:05.186+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:05.186+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:05.186+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 99, above threshold 10. [2023-01-13T23:09:06.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 104032 ms [2023-01-13T23:09:06.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:06.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:06.236+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:06.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:06.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:06.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 100, above threshold 10. [2023-01-13T23:09:07.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 105082 ms [2023-01-13T23:09:07.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:07.284+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:07.287+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:07.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:07.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:07.287+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 101, above threshold 10. [2023-01-13T23:09:07.323+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:09:07.323+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:09:07.323+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:09:07.326+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:07.327+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:07.327+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:09:08.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 106132 ms [2023-01-13T23:09:08.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:08.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:08.335+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:08.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:08.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:08.337+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 102, above threshold 10. [2023-01-13T23:09:09.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 107182 ms [2023-01-13T23:09:09.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:09.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:09.385+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:09.385+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:09.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:09.386+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 103, above threshold 10. [2023-01-13T23:09:10.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 108232 ms [2023-01-13T23:09:10.435+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:10.435+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:10.437+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:10.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:10.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:10.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 104, above threshold 10. [2023-01-13T23:09:11.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 109282 ms [2023-01-13T23:09:11.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:11.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:11.485+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:11.485+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:11.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:11.486+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 105, above threshold 10. [2023-01-13T23:09:12.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 110332 ms [2023-01-13T23:09:12.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:12.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:12.537+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:12.540+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:12.540+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:12.540+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 106, above threshold 10. [2023-01-13T23:09:13.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 111382 ms [2023-01-13T23:09:13.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:13.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:13.585+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:13.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:13.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:13.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 107, above threshold 10. [2023-01-13T23:09:14.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 112432 ms [2023-01-13T23:09:14.634+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:14.634+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:14.636+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:14.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:14.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:14.638+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 108, above threshold 10. [2023-01-13T23:09:15.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 113482 ms [2023-01-13T23:09:15.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:15.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:15.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:15.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:15.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:15.687+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 109, above threshold 10. [2023-01-13T23:09:16.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 114532 ms [2023-01-13T23:09:16.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:16.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:16.736+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:16.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:16.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:16.736+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 110, above threshold 10. [2023-01-13T23:09:17.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 115582 ms [2023-01-13T23:09:17.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:17.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:17.785+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:17.785+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:17.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:17.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 111, above threshold 10. [2023-01-13T23:09:18.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 1 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 116632 ms [2023-01-13T23:09:18.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:18.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:18.837+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:18.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:18.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 1 message to send. [2023-01-13T23:09:18.837+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 112, above threshold 10. [2023-01-13T23:09:19.170+00:00|INFO|network|pool-3-thread-1] [OUT|DMAAP|POLICY-ACRUNTIME-PARTICIPANT] {"state":"UNKNOWN","healthStatus":"UNKNOWN","participantStatistics":{"participantId":{"name":"K8sParticipant0","version":"1.0.0"},"timeStamp":"2023-01-13T23:09:19.134462Z","state":"UNKNOWN","healthStatus":"UNKNOWN","eventCount":0,"lastExecutionTime":0,"averageExecutionTime":0.0,"upTime":0,"lastEnterTime":0,"lastStart":0},"participantDefinitionUpdates":[],"automationCompositionInfoList":[],"messageType":"PARTICIPANT_STATUS","messageId":"0a7c137c-e89e-43ab-9d9a-3cb26fac99f0","timestamp":"2023-01-13T23:09:19.135012Z","participantType":{"name":"org.onap.policy.clamp.acm.KubernetesParticipant","version":"2.3.4"},"participantId":{"name":"K8sParticipant0","version":"1.0.0"}} [2023-01-13T23:09:19.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 117682 ms [2023-01-13T23:09:19.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:19.884+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:19.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:19.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:19.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:19.887+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 113, above threshold 10. [2023-01-13T23:09:20.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 118732 ms [2023-01-13T23:09:20.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:20.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:20.936+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:20.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:20.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:20.936+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 114, above threshold 10. [2023-01-13T23:09:21.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 119782 ms [2023-01-13T23:09:21.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:21.984+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:21.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:21.987+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:21.987+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:21.987+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 115, above threshold 10. [2023-01-13T23:09:22.327+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:09:22.327+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:09:22.328+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:09:22.330+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:22.331+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:22.331+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:09:23.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 120832 ms [2023-01-13T23:09:23.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:23.034+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:23.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:23.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:23.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:23.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 116, above threshold 10. [2023-01-13T23:09:24.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 121882 ms [2023-01-13T23:09:24.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:24.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:24.087+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:24.088+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:24.088+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:24.088+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 117, above threshold 10. [2023-01-13T23:09:25.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 122932 ms [2023-01-13T23:09:25.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:25.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:25.135+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:25.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:25.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:25.136+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 118, above threshold 10. [2023-01-13T23:09:26.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 123982 ms [2023-01-13T23:09:26.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:26.184+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:26.187+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:26.187+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:26.187+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:26.187+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 119, above threshold 10. [2023-01-13T23:09:27.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 125032 ms [2023-01-13T23:09:27.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:27.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:27.235+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:27.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:27.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:27.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 120, above threshold 10. [2023-01-13T23:09:28.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 126082 ms [2023-01-13T23:09:28.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:28.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:28.286+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:28.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:28.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:28.287+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 121, above threshold 10. [2023-01-13T23:09:29.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 127132 ms [2023-01-13T23:09:29.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:29.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:29.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:29.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:29.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:29.337+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 122, above threshold 10. [2023-01-13T23:09:30.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 128182 ms [2023-01-13T23:09:30.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:30.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:30.385+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:30.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:30.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:30.386+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 123, above threshold 10. [2023-01-13T23:09:31.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 129232 ms [2023-01-13T23:09:31.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:31.434+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:31.436+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:31.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:31.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:31.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 124, above threshold 10. [2023-01-13T23:09:32.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 130282 ms [2023-01-13T23:09:32.485+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:32.485+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:32.488+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:32.488+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:32.488+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:32.488+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 125, above threshold 10. [2023-01-13T23:09:33.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 131332 ms [2023-01-13T23:09:33.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:33.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:33.536+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:33.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:33.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:33.537+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 126, above threshold 10. [2023-01-13T23:09:34.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 132382 ms [2023-01-13T23:09:34.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:34.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:34.586+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:34.587+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:34.587+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:34.587+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 127, above threshold 10. [2023-01-13T23:09:35.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 133432 ms [2023-01-13T23:09:35.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:35.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:35.636+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:35.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:35.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:35.637+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 128, above threshold 10. [2023-01-13T23:09:36.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 134482 ms [2023-01-13T23:09:36.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:36.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:36.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:36.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:36.686+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:36.686+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 129, above threshold 10. [2023-01-13T23:09:37.331+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:09:37.332+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:09:37.332+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:09:37.335+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:37.335+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:37.335+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:09:37.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 135532 ms [2023-01-13T23:09:37.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:37.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:37.735+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:37.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:37.736+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:37.736+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 130, above threshold 10. [2023-01-13T23:09:38.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 136582 ms [2023-01-13T23:09:38.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:38.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:38.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:38.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:38.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:38.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 131, above threshold 10. [2023-01-13T23:09:39.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 137632 ms [2023-01-13T23:09:39.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:39.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:39.835+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:39.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:39.836+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:39.836+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 132, above threshold 10. [2023-01-13T23:09:40.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 138682 ms [2023-01-13T23:09:40.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:40.884+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:40.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:40.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:40.887+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:40.887+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 133, above threshold 10. [2023-01-13T23:09:41.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 139732 ms [2023-01-13T23:09:41.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:41.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:41.936+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:41.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:41.936+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:41.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 134, above threshold 10. [2023-01-13T23:09:42.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 140782 ms [2023-01-13T23:09:42.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:42.984+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:42.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:42.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:42.987+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:42.987+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 135, above threshold 10. [2023-01-13T23:09:44.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 141832 ms [2023-01-13T23:09:44.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:44.034+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:44.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:44.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:44.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:44.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 136, above threshold 10. [2023-01-13T23:09:45.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 142882 ms [2023-01-13T23:09:45.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:45.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:45.085+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:45.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:45.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:45.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 137, above threshold 10. [2023-01-13T23:09:46.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 143932 ms [2023-01-13T23:09:46.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:46.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:46.135+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:46.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:46.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:46.136+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 138, above threshold 10. [2023-01-13T23:09:47.187+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 144986 ms [2023-01-13T23:09:47.187+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:47.188+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:47.195+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:47.195+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:47.195+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:47.195+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 139, above threshold 10. [2023-01-13T23:09:48.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 146032 ms [2023-01-13T23:09:48.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:48.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:48.236+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:48.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:48.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:48.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 140, above threshold 10. [2023-01-13T23:09:49.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 147082 ms [2023-01-13T23:09:49.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:49.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:49.286+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:49.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:49.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:49.286+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 141, above threshold 10. [2023-01-13T23:09:50.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 148132 ms [2023-01-13T23:09:50.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:50.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:50.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:50.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:50.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:50.336+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 142, above threshold 10. [2023-01-13T23:09:51.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 149182 ms [2023-01-13T23:09:51.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:51.384+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:51.389+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:51.389+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:51.389+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:51.389+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 143, above threshold 10. [2023-01-13T23:09:52.335+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:09:52.336+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:09:52.336+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:09:52.340+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:52.341+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:52.341+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:09:52.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 150232 ms [2023-01-13T23:09:52.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:52.434+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:52.436+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:52.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:52.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:52.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 144, above threshold 10. [2023-01-13T23:09:53.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 151282 ms [2023-01-13T23:09:53.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:53.484+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:53.486+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:53.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:53.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:53.487+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 145, above threshold 10. [2023-01-13T23:09:54.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 152332 ms [2023-01-13T23:09:54.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:54.534+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:54.539+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:54.539+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:54.539+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:54.539+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 146, above threshold 10. [2023-01-13T23:09:55.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 153382 ms [2023-01-13T23:09:55.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:55.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:55.586+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:55.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:55.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:55.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 147, above threshold 10. [2023-01-13T23:09:56.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 154432 ms [2023-01-13T23:09:56.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:56.634+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:56.639+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:56.640+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:56.640+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:56.640+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 148, above threshold 10. [2023-01-13T23:09:57.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 155482 ms [2023-01-13T23:09:57.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:57.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:57.686+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:57.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:57.687+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:57.687+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 149, above threshold 10. [2023-01-13T23:09:58.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 156532 ms [2023-01-13T23:09:58.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:58.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:58.735+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:58.735+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:58.735+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:58.735+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 150, above threshold 10. [2023-01-13T23:09:59.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 157582 ms [2023-01-13T23:09:59.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:09:59.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:09:59.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:09:59.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:09:59.787+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:09:59.787+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 151, above threshold 10. [2023-01-13T23:10:00.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 158632 ms [2023-01-13T23:10:00.834+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:00.834+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:00.843+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:00.844+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:00.844+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:00.844+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 152, above threshold 10. [2023-01-13T23:10:01.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 159682 ms [2023-01-13T23:10:01.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:01.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:01.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:01.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:01.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:01.886+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 153, above threshold 10. [2023-01-13T23:10:02.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 160732 ms [2023-01-13T23:10:02.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:02.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:02.936+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:02.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:02.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:02.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 154, above threshold 10. [2023-01-13T23:10:03.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 161782 ms [2023-01-13T23:10:03.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:03.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:03.985+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:03.985+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:03.985+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:03.985+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 155, above threshold 10. [2023-01-13T23:10:05.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 162832 ms [2023-01-13T23:10:05.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:05.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:05.037+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:05.038+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:05.038+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:05.039+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 156, above threshold 10. [2023-01-13T23:10:06.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 163882 ms [2023-01-13T23:10:06.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:06.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:06.086+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:06.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:06.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:06.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 157, above threshold 10. [2023-01-13T23:10:07.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 164932 ms [2023-01-13T23:10:07.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:07.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:07.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:07.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:07.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:07.137+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 158, above threshold 10. [2023-01-13T23:10:07.341+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:10:07.342+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:10:07.342+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:10:07.346+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:07.346+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:07.346+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:10:08.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 165982 ms [2023-01-13T23:10:08.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:08.183+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:08.187+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:08.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:08.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:08.188+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 159, above threshold 10. [2023-01-13T23:10:09.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 167032 ms [2023-01-13T23:10:09.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:09.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:09.236+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:09.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:09.236+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:09.236+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 160, above threshold 10. [2023-01-13T23:10:10.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 168082 ms [2023-01-13T23:10:10.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:10.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:10.285+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:10.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:10.287+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:10.287+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 161, above threshold 10. [2023-01-13T23:10:11.334+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 169133 ms [2023-01-13T23:10:11.334+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:11.334+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:11.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:11.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:11.337+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:11.337+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 162, above threshold 10. [2023-01-13T23:10:12.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 170182 ms [2023-01-13T23:10:12.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:12.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:12.385+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:12.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:12.386+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:12.386+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 163, above threshold 10. [2023-01-13T23:10:13.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 171232 ms [2023-01-13T23:10:13.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:13.433+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:13.436+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:13.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:13.437+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:13.437+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 164, above threshold 10. [2023-01-13T23:10:14.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 172282 ms [2023-01-13T23:10:14.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:14.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:14.520+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:14.524+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:14.530+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:14.530+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 165, above threshold 10. [2023-01-13T23:10:15.535+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 173334 ms [2023-01-13T23:10:15.544+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:15.552+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:15.584+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:15.585+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:15.586+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:15.586+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 166, above threshold 10. [2023-01-13T23:10:16.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 174432 ms [2023-01-13T23:10:16.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:16.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:16.637+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:16.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:16.637+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:16.637+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 167, above threshold 10. [2023-01-13T23:10:17.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 175482 ms [2023-01-13T23:10:17.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:17.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:17.685+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:17.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:17.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:17.685+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 168, above threshold 10. [2023-01-13T23:10:18.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 176532 ms [2023-01-13T23:10:18.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:18.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:18.736+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:18.737+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:18.737+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:18.737+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 169, above threshold 10. [2023-01-13T23:10:19.786+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 177585 ms [2023-01-13T23:10:19.786+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:19.786+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:19.788+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:19.788+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:19.788+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:19.788+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 170, above threshold 10. [2023-01-13T23:10:20.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 178632 ms [2023-01-13T23:10:20.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:20.834+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:20.837+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:20.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:20.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:20.837+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 171, above threshold 10. [2023-01-13T23:10:21.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 179682 ms [2023-01-13T23:10:21.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:21.884+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:21.886+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:21.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:21.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:21.886+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 172, above threshold 10. [2023-01-13T23:10:22.347+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:10:22.349+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:10:22.349+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:10:22.352+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:22.353+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:22.353+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:10:22.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 180732 ms [2023-01-13T23:10:22.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:22.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:22.936+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:22.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:22.937+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:22.937+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 173, above threshold 10. [2023-01-13T23:10:23.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 181782 ms [2023-01-13T23:10:23.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:23.984+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:23.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:23.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:23.987+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:23.987+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 174, above threshold 10. [2023-01-13T23:10:25.035+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 182834 ms [2023-01-13T23:10:25.036+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:25.036+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:25.051+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:25.052+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:25.052+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:25.052+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 175, above threshold 10. [2023-01-13T23:10:26.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 183882 ms [2023-01-13T23:10:26.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:26.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:26.085+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:26.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:26.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:26.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 176, above threshold 10. [2023-01-13T23:10:27.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 184932 ms [2023-01-13T23:10:27.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:27.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:27.137+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:27.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:27.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:27.137+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 177, above threshold 10. [2023-01-13T23:10:28.137+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 185936 ms [2023-01-13T23:10:28.137+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:28.137+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:28.140+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:28.141+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:28.141+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:28.141+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 178, above threshold 10. [2023-01-13T23:10:29.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 186982 ms [2023-01-13T23:10:29.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:29.183+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:29.187+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:29.187+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:29.187+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:29.187+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 179, above threshold 10. [2023-01-13T23:10:30.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 188032 ms [2023-01-13T23:10:30.235+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:30.235+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:30.238+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:30.238+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:30.238+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:30.238+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 180, above threshold 10. [2023-01-13T23:10:31.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 189082 ms [2023-01-13T23:10:31.284+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:31.284+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:31.288+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:31.294+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:31.294+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:31.294+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 181, above threshold 10. [2023-01-13T23:10:32.335+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 190134 ms [2023-01-13T23:10:32.336+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:32.337+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:32.342+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:32.343+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:32.349+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:32.349+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 182, above threshold 10. [2023-01-13T23:10:33.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 191182 ms [2023-01-13T23:10:33.384+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:33.384+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:33.386+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:33.390+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:33.390+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:33.390+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 183, above threshold 10. [2023-01-13T23:10:34.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 192232 ms [2023-01-13T23:10:34.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:34.433+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:34.435+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:34.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:34.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:34.436+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 184, above threshold 10. [2023-01-13T23:10:35.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 193282 ms [2023-01-13T23:10:35.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:35.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:35.485+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:35.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:35.487+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:35.487+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 185, above threshold 10. [2023-01-13T23:10:36.535+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 194334 ms [2023-01-13T23:10:36.536+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:36.536+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:36.548+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:36.548+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:36.548+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:36.549+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 186, above threshold 10. [2023-01-13T23:10:37.355+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:10:37.356+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:10:37.356+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:10:37.364+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:37.364+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:37.364+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:10:37.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 195382 ms [2023-01-13T23:10:37.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:37.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:37.593+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:37.593+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:37.593+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:37.593+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 187, above threshold 10. [2023-01-13T23:10:38.633+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 196432 ms [2023-01-13T23:10:38.633+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:38.633+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:38.642+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:38.642+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:38.642+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:38.642+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 188, above threshold 10. [2023-01-13T23:10:39.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 197482 ms [2023-01-13T23:10:39.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:39.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:39.685+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:39.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:39.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:39.685+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 189, above threshold 10. [2023-01-13T23:10:40.735+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 198534 ms [2023-01-13T23:10:40.736+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:40.736+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:40.755+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:40.755+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:40.755+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:40.756+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 190, above threshold 10. [2023-01-13T23:10:41.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 199582 ms [2023-01-13T23:10:41.783+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:41.783+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:41.785+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:41.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:41.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:41.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 191, above threshold 10. [2023-01-13T23:10:42.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 200632 ms [2023-01-13T23:10:42.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:42.834+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:42.836+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:42.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:42.837+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:42.837+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 192, above threshold 10. [2023-01-13T23:10:43.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 201682 ms [2023-01-13T23:10:43.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:43.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:43.885+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:43.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:43.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:43.886+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 193, above threshold 10. [2023-01-13T23:10:44.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 202732 ms [2023-01-13T23:10:44.933+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:44.933+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:44.935+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:44.935+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:44.935+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:44.935+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 194, above threshold 10. [2023-01-13T23:10:45.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 203782 ms [2023-01-13T23:10:45.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:45.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:45.986+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:45.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:45.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:45.986+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 195, above threshold 10. [2023-01-13T23:10:47.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 204832 ms [2023-01-13T23:10:47.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:47.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:47.036+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:47.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:47.037+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:47.037+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 196, above threshold 10. [2023-01-13T23:10:48.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 205882 ms [2023-01-13T23:10:48.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:48.083+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:48.085+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:48.085+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:48.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:48.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 197, above threshold 10. [2023-01-13T23:10:49.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 206932 ms [2023-01-13T23:10:49.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:49.134+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:49.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:49.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:49.136+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:49.136+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 198, above threshold 10. [2023-01-13T23:10:50.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 207982 ms [2023-01-13T23:10:50.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:50.184+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:50.188+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:50.189+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:50.189+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:50.189+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 199, above threshold 10. [2023-01-13T23:10:51.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 209032 ms [2023-01-13T23:10:51.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:51.234+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:51.239+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:51.240+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:51.240+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:51.240+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 200, above threshold 10. [2023-01-13T23:10:52.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 210082 ms [2023-01-13T23:10:52.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:52.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:52.285+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:52.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:52.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:52.286+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 201, above threshold 10. [2023-01-13T23:10:52.365+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:10:52.365+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:10:52.365+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:10:52.368+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:52.368+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:52.368+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:10:53.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 211132 ms [2023-01-13T23:10:53.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:53.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:53.336+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:53.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:53.336+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:53.336+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 202, above threshold 10. [2023-01-13T23:10:54.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 212182 ms [2023-01-13T23:10:54.383+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:54.383+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:54.387+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:54.387+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:54.388+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:54.388+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 203, above threshold 10. [2023-01-13T23:10:55.388+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 213187 ms [2023-01-13T23:10:55.388+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:55.388+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:55.399+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:55.402+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:55.402+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:55.402+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 204, above threshold 10. [2023-01-13T23:10:56.433+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 214232 ms [2023-01-13T23:10:56.433+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:56.433+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:56.435+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:56.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:56.436+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:56.436+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 205, above threshold 10. [2023-01-13T23:10:57.483+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 215282 ms [2023-01-13T23:10:57.483+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:57.483+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:57.485+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:57.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:57.486+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:57.486+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 206, above threshold 10. [2023-01-13T23:10:58.533+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 216332 ms [2023-01-13T23:10:58.533+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:58.533+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:58.535+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:58.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:58.536+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:58.536+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 207, above threshold 10. [2023-01-13T23:10:59.583+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 217382 ms [2023-01-13T23:10:59.583+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:10:59.583+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:10:59.586+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:10:59.587+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:10:59.587+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:10:59.587+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 208, above threshold 10. [2023-01-13T23:11:00.636+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 218434 ms [2023-01-13T23:11:00.636+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:00.637+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:00.643+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:00.643+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:00.643+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:00.643+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 209, above threshold 10. [2023-01-13T23:11:01.683+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 219482 ms [2023-01-13T23:11:01.683+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:01.683+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:01.685+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:01.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:01.685+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:01.685+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 210, above threshold 10. [2023-01-13T23:11:02.733+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 220532 ms [2023-01-13T23:11:02.733+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:02.733+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:02.736+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:02.737+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:02.737+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:02.737+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 211, above threshold 10. [2023-01-13T23:11:03.783+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 221582 ms [2023-01-13T23:11:03.784+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:03.784+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:03.786+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:03.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:03.786+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:03.786+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 212, above threshold 10. [2023-01-13T23:11:04.833+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 222632 ms [2023-01-13T23:11:04.833+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:04.833+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:04.835+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:04.835+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:04.835+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:04.835+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 213, above threshold 10. [2023-01-13T23:11:05.883+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 223682 ms [2023-01-13T23:11:05.883+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:05.883+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:05.885+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:05.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:05.886+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:05.886+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 214, above threshold 10. [2023-01-13T23:11:06.933+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 224732 ms [2023-01-13T23:11:06.947+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:06.947+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:06.951+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:06.951+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:06.951+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:06.951+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 215, above threshold 10. [2023-01-13T23:11:07.371+00:00|ERROR|SingleThreadedBusTopicSource|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=c63309e2-6aff-4319-8c15-4a286beeba8e, consumerInstance=onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn, fetchTimeout=15000, fetchLimit=-1, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=null, apiSecret=null, useHttps=false, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=POLICY-ACRUNTIME-PARTICIPANT, effectiveTopic=POLICY-ACRUNTIME-PARTICIPANT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch java.io.IOException: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:123) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64) at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:212) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.fetchAllMessages(SingleThreadedBusTopicSource.java:244) at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:234) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384) at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368) at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87) ... 5 common frames omitted [2023-01-13T23:11:07.372+00:00|INFO|CambriaConsumerImpl|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] UEB GET /events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 [2023-01-13T23:11:07.372+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] GET http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT/c63309e2-6aff-4319-8c15-4a286beeba8e/onap-policy-clamp-ac-k8s-ppnt-5d87d6fbc6-8tmzn?timeout=15000 (anonymous) ... [2023-01-13T23:11:07.375+00:00|INFO|HttpClient|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:07.375+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:07.375+00:00|INFO|BusConsumer$FetchingBusConsumer|DMAAP-source-POLICY-ACRUNTIME-PARTICIPANT] CambriaConsumerWrapper [fetchTimeout=15000]: backoff for 15000ms [2023-01-13T23:11:07.983+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 225782 ms [2023-01-13T23:11:07.983+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:07.983+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:07.985+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:07.985+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:07.986+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:07.986+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 216, above threshold 10. [2023-01-13T23:11:08.988+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 226786 ms [2023-01-13T23:11:08.988+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:08.988+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:08.990+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:08.990+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:08.991+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:08.991+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 217, above threshold 10. [2023-01-13T23:11:10.033+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 227832 ms [2023-01-13T23:11:10.033+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:10.033+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:10.035+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:10.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:10.036+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:10.036+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 218, above threshold 10. [2023-01-13T23:11:11.083+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 228882 ms [2023-01-13T23:11:11.083+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:11.084+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:11.086+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:11.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:11.086+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:11.086+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 219, above threshold 10. [2023-01-13T23:11:12.133+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 229932 ms [2023-01-13T23:11:12.133+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:12.133+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:12.136+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:12.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:12.137+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:12.137+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 220, above threshold 10. [2023-01-13T23:11:13.183+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 230982 ms [2023-01-13T23:11:13.183+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:13.184+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:13.187+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:13.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:13.188+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:13.188+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 221, above threshold 10. [2023-01-13T23:11:14.233+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 232032 ms [2023-01-13T23:11:14.233+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:14.233+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:14.236+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:14.237+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:14.238+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:14.238+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 222, above threshold 10. [2023-01-13T23:11:15.283+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 233082 ms [2023-01-13T23:11:15.283+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:15.283+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:15.285+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:15.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:15.286+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:15.286+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 223, above threshold 10. [2023-01-13T23:11:16.333+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: 234132 ms [2023-01-13T23:11:16.333+00:00|WARN|HostSelector|pool-4-thread-1] All hosts were blacklisted; reverting to full set of hosts. [2023-01-13T23:11:16.333+00:00|INFO|HttpClient|pool-4-thread-1] POST http://message-router:3904/events/POLICY-ACRUNTIME-PARTICIPANT (anonymous) ... [2023-01-13T23:11:16.335+00:00|INFO|HttpClient|pool-4-thread-1] --> HTTP/1.1 503 Service Unavailable [2023-01-13T23:11:16.335+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] HTTP/1.1 503 Service Unavailable com.att.nsa.apiClient.http.HttpException: HTTP/1.1 503 Service Unavailable at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:816) at com.att.nsa.apiClient.http.HttpClient.post(HttpClient.java:456) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.sendBatch(CambriaSimplerBatchPublisher.java:342) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.send(CambriaSimplerBatchPublisher.java:251) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher.access$100(CambriaSimplerBatchPublisher.java:31) at com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher$1.run(CambriaSimplerBatchPublisher.java:411) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [2023-01-13T23:11:16.335+00:00|WARN|CambriaSimplerBatchPublisher|pool-4-thread-1] Send failed, 2 message to send. [2023-01-13T23:11:16.335+00:00|ERROR|CambriaSimplerBatchPublisher|pool-4-thread-1] PUB_CHRONIC_FAILURE: Send failure count is 224, above threshold 10. [2023-01-13T23:11:17.383+00:00|INFO|CambriaSimplerBatchPublisher|pool-4-thread-1] sending 2 msgs to /events/POLICY-ACRUNTIME-PARTICIPANT. Oldest: