By type
2022-07-26T00:23:47.898880Z info FLAG: --concurrency="2"
2022-07-26T00:23:47.898975Z info FLAG: --domain="onap.svc.cluster.local"
2022-07-26T00:23:47.898990Z info FLAG: --help="false"
2022-07-26T00:23:47.898999Z info FLAG: --log_as_json="false"
2022-07-26T00:23:47.899007Z info FLAG: --log_caller=""
2022-07-26T00:23:47.899015Z info FLAG: --log_output_level="default:info"
2022-07-26T00:23:47.899022Z info FLAG: --log_rotate=""
2022-07-26T00:23:47.899029Z info FLAG: --log_rotate_max_age="30"
2022-07-26T00:23:47.899037Z info FLAG: --log_rotate_max_backups="1000"
2022-07-26T00:23:47.899045Z info FLAG: --log_rotate_max_size="104857600"
2022-07-26T00:23:47.899052Z info FLAG: --log_stacktrace_level="default:none"
2022-07-26T00:23:47.899124Z info FLAG: --log_target="[stdout]"
2022-07-26T00:23:47.899140Z info FLAG: --meshConfig="./etc/istio/config/mesh"
2022-07-26T00:23:47.899148Z info FLAG: --outlierLogPath=""
2022-07-26T00:23:47.899156Z info FLAG: --proxyComponentLogLevel="misc:error"
2022-07-26T00:23:47.899164Z info FLAG: --proxyLogLevel="warning"
2022-07-26T00:23:47.899171Z info FLAG: --serviceCluster="istio-proxy"
2022-07-26T00:23:47.899179Z info FLAG: --stsPort="0"
2022-07-26T00:23:47.899186Z info FLAG: --templateFile=""
2022-07-26T00:23:47.899194Z info FLAG: --tokenManagerPlugin="GoogleTokenExchange"
2022-07-26T00:23:47.899235Z info FLAG: --vklog="0"
2022-07-26T00:23:47.899248Z info Version 1.13.1-5f3b5ed958ae75156f8656fe7b3794f78e94db84-Clean
2022-07-26T00:23:47.899853Z info Proxy role ips=[10.233.72.128] type=sidecar id=onap-so-admin-cockpit-7877596c7-d8xkl.onap domain=onap.svc.cluster.local
2022-07-26T00:23:47.900102Z info Apply proxy config from env {"tracing":{"zipkin":{"address":"istio-collector.observability:9411"}}}
2022-07-26T00:23:47.904204Z info Effective config: binaryPath: /usr/local/bin/envoy
concurrency: 2
configPath: ./etc/istio/proxy
controlPlaneAuthPolicy: MUTUAL_TLS
discoveryAddress: istiod.istio-system.svc:15012
drainDuration: 45s
parentShutdownDuration: 60s
proxyAdminPort: 15000
serviceCluster: istio-proxy
statNameLength: 189
statusPort: 15020
terminationDrainDuration: 5s
tracing:
zipkin:
address: istio-collector.observability:9411
2022-07-26T00:23:47.904230Z info JWT policy is third-party-jwt
2022-07-26T00:23:49.349016Z info platform detected is AWS
2022-07-26T00:23:50.286548Z info CA Endpoint istiod.istio-system.svc:15012, provider Citadel
2022-07-26T00:23:50.287122Z info Using CA istiod.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem
2022-07-26T00:23:50.287754Z info citadelclient Citadel client using custom root cert: var/run/secrets/istio/root-cert.pem
2022-07-26T00:23:50.298319Z info Opening status port 15020
2022-07-26T00:23:50.624556Z info ads All caches have been synced up in 2.810189954s, marking server ready
2022-07-26T00:23:50.625040Z info sds SDS server for workload certificates started, listening on "etc/istio/proxy/SDS"
2022-07-26T00:23:50.625078Z info xdsproxy Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes"
2022-07-26T00:23:50.630624Z info Pilot SAN: [istiod.istio-system.svc]
2022-07-26T00:23:50.635530Z info Starting proxy agent
2022-07-26T00:23:50.635611Z info Epoch 0 starting
2022-07-26T00:23:50.635694Z info Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --log-format %Y-%m-%dT%T.%fZ %l envoy %n %v -l warning --component-log-level misc:error --concurrency 2]
2022-07-26T00:23:50.642250Z info starting Http service at 127.0.0.1:15004
2022-07-26T00:23:50.642407Z info sds Starting SDS grpc server
2022-07-26T00:24:10.307396Z warn ca ca request failed, starting attempt 1 in 100.121552ms
2022-07-26T00:24:10.410443Z warn ca ca request failed, starting attempt 2 in 180.736987ms
2022-07-26T00:24:10.592512Z warn ca ca request failed, starting attempt 3 in 362.386908ms
2022-07-26T00:24:10.956292Z warn ca ca request failed, starting attempt 4 in 812.066794ms
2022-07-26T00:24:30.611574Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:40505->169.254.25.10:53: i/o timeout"
2022-07-26T00:24:46.347482Z warn ca ca request failed, starting attempt 1 in 100.743982ms
2022-07-26T00:24:46.357536Z warning envoy config StreamAggregatedResources gRPC config stream closed since 35s ago: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:50546->169.254.25.10:53: read: connection refused"
2022-07-26T00:24:46.449433Z warn ca ca request failed, starting attempt 2 in 208.195229ms
2022-07-26T00:24:46.659219Z warn ca ca request failed, starting attempt 3 in 372.969118ms
2022-07-26T00:24:47.033322Z warn ca ca request failed, starting attempt 4 in 867.902459ms
2022-07-26T00:24:47.903103Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:50546->169.254.25.10:53: read: connection refused"
2022-07-26T00:25:06.525918Z warning envoy config StreamAggregatedResources gRPC config stream closed since 56s ago: 14, connection error: desc = "transport: Error while dialing dial tcp: i/o timeout"
2022-07-26T00:25:06.549048Z warn ca ca request failed, starting attempt 1 in 103.74584ms
2022-07-26T00:25:06.654856Z warn ca ca request failed, starting attempt 2 in 198.886797ms
2022-07-26T00:25:06.854950Z warn ca ca request failed, starting attempt 3 in 370.629153ms
2022-07-26T00:25:07.231401Z warn ca ca request failed, starting attempt 4 in 815.433559ms
2022-07-26T00:25:08.049589Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:59962->169.254.25.10:53: read: connection refused"
2022-07-26T00:25:10.275511Z warn ca ca request failed, starting attempt 1 in 109.465907ms
2022-07-26T00:25:10.387450Z warn ca ca request failed, starting attempt 2 in 216.536511ms
2022-07-26T00:25:10.605294Z warn ca ca request failed, starting attempt 3 in 407.892903ms
2022-07-26T00:25:11.014368Z warn ca ca request failed, starting attempt 4 in 781.088283ms
2022-07-26T00:25:24.615463Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:53357->169.254.25.10:53: read: connection refused"
2022-07-26T00:25:24.617138Z warning envoy config StreamAggregatedResources gRPC config stream closed since 74s ago: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:53357->169.254.25.10:53: read: connection refused"
2022-07-26T00:25:27.063169Z warn ca ca request failed, starting attempt 1 in 101.619029ms
2022-07-26T00:25:27.168585Z warn ca ca request failed, starting attempt 2 in 213.578763ms
2022-07-26T00:25:27.383010Z warn ca ca request failed, starting attempt 3 in 418.74072ms
2022-07-26T00:25:27.805692Z warn ca ca request failed, starting attempt 4 in 777.456106ms
2022-07-26T00:25:28.584307Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:53357->169.254.25.10:53: read: connection refused"
2022-07-26T00:25:47.622558Z warning envoy config StreamAggregatedResources gRPC config stream closed since 97s ago: 14, connection error: desc = "transport: Error while dialing dial tcp: i/o timeout"
2022-07-26T00:25:47.642209Z warn ca ca request failed, starting attempt 1 in 94.479114ms
2022-07-26T00:25:47.746891Z warn ca ca request failed, starting attempt 2 in 196.045618ms
2022-07-26T00:25:47.943635Z warn ca ca request failed, starting attempt 3 in 376.182862ms
2022-07-26T00:25:48.320383Z warn ca ca request failed, starting attempt 4 in 803.817202ms
2022-07-26T00:25:49.126508Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:44620->169.254.25.10:53: i/o timeout"
2022-07-26T00:25:55.795735Z warn ca ca request failed, starting attempt 1 in 108.406207ms
2022-07-26T00:25:55.904916Z warn ca ca request failed, starting attempt 2 in 207.914835ms
2022-07-26T00:25:56.113983Z warn ca ca request failed, starting attempt 3 in 399.302127ms
2022-07-26T00:25:56.515707Z warn ca ca request failed, starting attempt 4 in 785.633999ms
2022-07-26T00:25:57.305214Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:44620->169.254.25.10:53: i/o timeout"
2022-07-26T00:26:01.024347Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T00:26:02.095853Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-1
2022-07-26T00:26:02.200626Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-2
2022-07-26T00:26:18.060611Z warn ca ca request failed, starting attempt 1 in 94.430384ms
2022-07-26T00:26:18.156378Z warn ca ca request failed, starting attempt 2 in 208.490395ms
2022-07-26T00:26:18.366249Z warn ca ca request failed, starting attempt 3 in 389.863773ms
2022-07-26T00:26:18.757156Z warn ca ca request failed, starting attempt 4 in 722.643845ms
2022-07-26T00:26:19.480306Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:19.480381Z warn cache slow generate secret lock: 17.384275605s
2022-07-26T00:26:20.663458Z warn ca ca request failed, starting attempt 1 in 92.181119ms
2022-07-26T00:26:20.757564Z warn ca ca request failed, starting attempt 2 in 203.860505ms
2022-07-26T00:26:20.962260Z warn ca ca request failed, starting attempt 3 in 411.677532ms
2022-07-26T00:26:21.374432Z warn ca ca request failed, starting attempt 4 in 831.537777ms
2022-07-26T00:26:22.207227Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-1 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:22.207344Z warn cache slow generate secret lock: 20.006467123s
2022-07-26T00:26:22.210049Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:22.426012Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-3
2022-07-26T00:26:22.693076Z warn ca ca request failed, starting attempt 1 in 91.555415ms
2022-07-26T00:26:22.785726Z warn ca ca request failed, starting attempt 2 in 198.603594ms
2022-07-26T00:26:22.985534Z warn ca ca request failed, starting attempt 3 in 409.848673ms
2022-07-26T00:26:23.396045Z warn ca ca request failed, starting attempt 4 in 772.630521ms
2022-07-26T00:26:24.169785Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:24.169363Z warn cache slow generate secret lock: 1.742715321s
2022-07-26T00:26:24.169413Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-2 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:24.184685Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-4
2022-07-26T00:26:25.186281Z warn ca ca request failed, starting attempt 1 in 108.107992ms
2022-07-26T00:26:25.295248Z warn ca ca request failed, starting attempt 2 in 201.238363ms
2022-07-26T00:26:25.496712Z warn ca ca request failed, starting attempt 3 in 414.602182ms
2022-07-26T00:26:25.912559Z warn ca ca request failed, starting attempt 4 in 736.218126ms
2022-07-26T00:26:26.649773Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-3 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:26.649892Z warn cache slow generate secret lock: 2.465021156s
2022-07-26T00:26:26.655742Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:27.529014Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-5
2022-07-26T00:26:28.095930Z warn ca ca request failed, starting attempt 1 in 95.029159ms
2022-07-26T00:26:28.191561Z warn ca ca request failed, starting attempt 2 in 192.900596ms
2022-07-26T00:26:28.385727Z warn ca ca request failed, starting attempt 3 in 437.987639ms
2022-07-26T00:26:28.826307Z warn ca ca request failed, starting attempt 4 in 843.497834ms
2022-07-26T00:26:29.670870Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-4 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:29.670985Z warn cache slow generate secret lock: 2.140799776s
2022-07-26T00:26:29.676283Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:30.147825Z warn ca ca request failed, starting attempt 1 in 96.006195ms
2022-07-26T00:26:30.246245Z warn ca ca request failed, starting attempt 2 in 199.371301ms
2022-07-26T00:26:30.333674Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-6
2022-07-26T00:26:30.446289Z warn ca ca request failed, starting attempt 3 in 432.043283ms
2022-07-26T00:26:30.879287Z warn ca ca request failed, starting attempt 4 in 735.053545ms
2022-07-26T00:26:31.615979Z warn cache slow generate secret lock: 1.282130384s
2022-07-26T00:26:31.633770Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-5 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:31.634314Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:60948->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:32.334330Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-7
2022-07-26T00:26:32.480882Z warn ca ca request failed, starting attempt 1 in 104.462511ms
2022-07-26T00:26:32.586701Z warn ca ca request failed, starting attempt 2 in 180.162561ms
2022-07-26T00:26:32.767312Z warn ca ca request failed, starting attempt 3 in 416.19363ms
2022-07-26T00:26:33.183837Z warn ca ca request failed, starting attempt 4 in 784.706646ms
2022-07-26T00:26:48.995100Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-6 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:48.995202Z warn cache slow generate secret lock: 17.0602158s
2022-07-26T00:26:49.001973Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:49.545002Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-8
2022-07-26T00:26:49.740609Z warn ca ca request failed, starting attempt 1 in 92.619534ms
2022-07-26T00:26:49.834017Z warn ca ca request failed, starting attempt 2 in 191.674032ms
2022-07-26T00:26:50.026780Z warn ca ca request failed, starting attempt 3 in 437.978843ms
2022-07-26T00:26:50.477262Z warn ca ca request failed, starting attempt 4 in 729.289619ms
2022-07-26T00:26:51.207598Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:51.207753Z warn cache slow generate secret lock: 18.873230854s
2022-07-26T00:26:52.404111Z warn ca ca request failed, starting attempt 1 in 99.00911ms
2022-07-26T00:26:52.504101Z warn ca ca request failed, starting attempt 2 in 191.13697ms
2022-07-26T00:26:52.695955Z warn ca ca request failed, starting attempt 3 in 411.172691ms
2022-07-26T00:26:53.107972Z warn ca ca request failed, starting attempt 4 in 794.255995ms
2022-07-26T00:26:53.903355Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-7 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:53.903479Z warn cache slow generate secret lock: 4.358250779s
2022-07-26T00:26:53.917993Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:54.382012Z warn ca ca request failed, starting attempt 1 in 93.81172ms
2022-07-26T00:26:54.478297Z warn ca ca request failed, starting attempt 2 in 212.891297ms
2022-07-26T00:26:54.691949Z warn ca ca request failed, starting attempt 3 in 430.808739ms
2022-07-26T00:26:55.123896Z warn ca ca request failed, starting attempt 4 in 868.92204ms
2022-07-26T00:26:55.995142Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:26:55.995055Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-8 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:56.900750Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-9
2022-07-26T00:26:56.943246Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-10
2022-07-26T00:26:57.735951Z warn ca ca request failed, starting attempt 1 in 99.758868ms
2022-07-26T00:26:57.842558Z warn ca ca request failed, starting attempt 2 in 198.08734ms
2022-07-26T00:26:58.070776Z warn ca ca request failed, starting attempt 3 in 377.298821ms
2022-07-26T00:26:58.448545Z warn ca ca request failed, starting attempt 4 in 754.284713ms
2022-07-26T00:26:59.204037Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-9 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:26:59.204131Z warn cache slow generate secret lock: 2.260674908s
2022-07-26T00:26:59.210052Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:00.010802Z warn ca ca request failed, starting attempt 1 in 104.352226ms
2022-07-26T00:27:00.116516Z warn ca ca request failed, starting attempt 2 in 214.933767ms
2022-07-26T00:27:00.333712Z warn ca ca request failed, starting attempt 3 in 398.427049ms
2022-07-26T00:27:00.737654Z warn ca ca request failed, starting attempt 4 in 763.546073ms
2022-07-26T00:27:01.505712Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-10 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:27:01.509280Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:06.416373Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-11
2022-07-26T00:27:06.722964Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-12
2022-07-26T00:27:07.260825Z warn ca ca request failed, starting attempt 1 in 90.093534ms
2022-07-26T00:27:07.351732Z warn ca ca request failed, starting attempt 2 in 218.571008ms
2022-07-26T00:27:07.571580Z warn ca ca request failed, starting attempt 3 in 391.196971ms
2022-07-26T00:27:07.970397Z warn ca ca request failed, starting attempt 4 in 821.335513ms
2022-07-26T00:27:08.797068Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-11 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:27:08.797130Z warn cache slow generate secret lock: 2.07389358s
2022-07-26T00:27:08.797781Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:09.814897Z warn ca ca request failed, starting attempt 1 in 108.098395ms
2022-07-26T00:27:09.925916Z warn ca ca request failed, starting attempt 2 in 198.307247ms
2022-07-26T00:27:10.125042Z warn ca ca request failed, starting attempt 3 in 363.704028ms
2022-07-26T00:27:10.520549Z warn ca ca request failed, starting attempt 4 in 795.940383ms
2022-07-26T00:27:11.317945Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:11.317515Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-12 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:27:11.317669Z warn cache slow generate secret lock: 3.417771866s
2022-07-26T00:27:11.785031Z warn ca ca request failed, starting attempt 1 in 99.828898ms
2022-07-26T00:27:11.889686Z warn ca ca request failed, starting attempt 2 in 189.194108ms
2022-07-26T00:27:12.083756Z warn ca ca request failed, starting attempt 3 in 426.846904ms
2022-07-26T00:27:12.514295Z warn ca ca request failed, starting attempt 4 in 868.225065ms
2022-07-26T00:27:13.383595Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:51321->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:19.480077Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-13
2022-07-26T00:27:22.306495Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-14
2022-07-26T00:27:35.078639Z warn ca ca request failed, starting attempt 1 in 105.374266ms
2022-07-26T00:27:35.184749Z warn ca ca request failed, starting attempt 2 in 217.6528ms
2022-07-26T00:27:35.404965Z warn ca ca request failed, starting attempt 3 in 418.747669ms
2022-07-26T00:27:35.824880Z warn ca ca request failed, starting attempt 4 in 780.949946ms
2022-07-26T00:27:36.609944Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-13 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:27:36.610035Z warn cache slow generate secret lock: 14.30338439s
2022-07-26T00:27:36.622077Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:37.235065Z warn ca ca request failed, starting attempt 1 in 101.376164ms
2022-07-26T00:27:37.342376Z warn ca ca request failed, starting attempt 2 in 199.566135ms
2022-07-26T00:27:37.543298Z warn ca ca request failed, starting attempt 3 in 418.012195ms
2022-07-26T00:27:37.964136Z warn ca ca request failed, starting attempt 4 in 789.047771ms
2022-07-26T00:27:38.754824Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-14 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:27:38.754814Z warn cache slow generate secret lock: 15.533698362s
2022-07-26T00:27:38.755231Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:39.692965Z warn ca ca request failed, starting attempt 1 in 95.71861ms
2022-07-26T00:27:39.789741Z warn ca ca request failed, starting attempt 2 in 192.902591ms
2022-07-26T00:27:39.983555Z warn ca ca request failed, starting attempt 3 in 431.97928ms
2022-07-26T00:27:40.416355Z warn ca ca request failed, starting attempt 4 in 854.875456ms
2022-07-26T00:27:41.271833Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:27:59.081842Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-15
2022-07-26T00:27:59.652663Z warn ca ca request failed, starting attempt 1 in 107.788973ms
2022-07-26T00:27:59.761027Z warn ca ca request failed, starting attempt 2 in 209.88399ms
2022-07-26T00:27:59.972017Z warn ca ca request failed, starting attempt 3 in 420.860045ms
2022-07-26T00:28:00.396728Z warn ca ca request failed, starting attempt 4 in 743.494565ms
2022-07-26T00:28:01.141480Z warn cache slow generate secret lock: 1.783727308s
2022-07-26T00:28:01.153870Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:01.153569Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-15 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:01.790039Z warn ca ca request failed, starting attempt 1 in 91.341261ms
2022-07-26T00:28:01.883966Z warn ca ca request failed, starting attempt 2 in 209.979823ms
2022-07-26T00:28:02.095800Z warn ca ca request failed, starting attempt 3 in 395.533873ms
2022-07-26T00:28:02.492012Z warn ca ca request failed, starting attempt 4 in 859.323957ms
2022-07-26T00:28:03.352514Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:04.119819Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-16
2022-07-26T00:28:05.244944Z warn ca ca request failed, starting attempt 1 in 96.910132ms
2022-07-26T00:28:05.343889Z warn ca ca request failed, starting attempt 2 in 216.069977ms
2022-07-26T00:28:05.563094Z warn ca ca request failed, starting attempt 3 in 424.511312ms
2022-07-26T00:28:05.988684Z warn ca ca request failed, starting attempt 4 in 765.289839ms
2022-07-26T00:28:06.755157Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-16 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:06.755677Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:07.224935Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-17
2022-07-26T00:28:08.336148Z warn ca ca request failed, starting attempt 1 in 107.931916ms
2022-07-26T00:28:08.445135Z warn ca ca request failed, starting attempt 2 in 215.52496ms
2022-07-26T00:28:08.661254Z warn ca ca request failed, starting attempt 3 in 364.849721ms
2022-07-26T00:28:09.027188Z warn ca ca request failed, starting attempt 4 in 858.209967ms
2022-07-26T00:28:09.888823Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:09.886825Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-17 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:11.010969Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-18
2022-07-26T00:28:11.870936Z warn ca ca request failed, starting attempt 1 in 106.43652ms
2022-07-26T00:28:11.978528Z warn ca ca request failed, starting attempt 2 in 205.288311ms
2022-07-26T00:28:12.187049Z warn ca ca request failed, starting attempt 3 in 410.963351ms
2022-07-26T00:28:12.599897Z warn ca ca request failed, starting attempt 4 in 864.021721ms
2022-07-26T00:28:13.464929Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-18 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:13.465811Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:34155->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:18.186976Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-19
2022-07-26T00:28:19.328332Z warn ca ca request failed, starting attempt 1 in 101.966681ms
2022-07-26T00:28:19.431202Z warn ca ca request failed, starting attempt 2 in 194.396405ms
2022-07-26T00:28:19.626717Z warn ca ca request failed, starting attempt 3 in 362.673185ms
2022-07-26T00:28:19.993239Z warn ca ca request failed, starting attempt 4 in 812.85392ms
2022-07-26T00:28:35.839936Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-19 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:35.870957Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:36.767901Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-20
2022-07-26T00:28:39.695452Z warn ca ca request failed, starting attempt 1 in 103.940795ms
2022-07-26T00:28:39.811980Z warn ca ca request failed, starting attempt 2 in 186.290764ms
2022-07-26T00:28:39.999767Z warn ca ca request failed, starting attempt 3 in 366.080815ms
2022-07-26T00:28:40.367915Z warn ca ca request failed, starting attempt 4 in 855.981674ms
2022-07-26T00:28:41.231150Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:41.229798Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-20 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:50.723505Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-21
2022-07-26T00:28:51.537660Z warn ca ca request failed, starting attempt 1 in 93.444813ms
2022-07-26T00:28:51.642366Z warn ca ca request failed, starting attempt 2 in 205.676311ms
2022-07-26T00:28:51.850106Z warn ca ca request failed, starting attempt 3 in 362.642425ms
2022-07-26T00:28:52.214093Z warn ca ca request failed, starting attempt 4 in 879.092724ms
2022-07-26T00:28:53.105150Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:28:53.101884Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-21 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:28:58.183693Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-22
2022-07-26T00:28:58.798400Z warn ca ca request failed, starting attempt 1 in 94.73726ms
2022-07-26T00:28:58.894127Z warn ca ca request failed, starting attempt 2 in 190.743498ms
2022-07-26T00:28:59.087372Z warn ca ca request failed, starting attempt 3 in 415.537156ms
2022-07-26T00:28:59.506694Z warn ca ca request failed, starting attempt 4 in 827.542009ms
2022-07-26T00:29:00.337934Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-22 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:29:00.341845Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:29:09.947859Z warn ca ca request failed, starting attempt 1 in 92.52968ms
2022-07-26T00:29:10.051272Z warn ca ca request failed, starting attempt 2 in 197.379921ms
2022-07-26T00:29:10.249780Z warn ca ca request failed, starting attempt 3 in 427.703777ms
2022-07-26T00:29:10.678104Z warn ca ca request failed, starting attempt 4 in 845.274601ms
2022-07-26T00:29:11.524664Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:29:18.465101Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-23
2022-07-26T00:29:18.505174Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-24
2022-07-26T00:29:21.048566Z warn ca ca request failed, starting attempt 1 in 92.8359ms
2022-07-26T00:29:21.143153Z warn ca ca request failed, starting attempt 2 in 192.134608ms
2022-07-26T00:29:21.336736Z warn ca ca request failed, starting attempt 3 in 433.661481ms
2022-07-26T00:29:21.773850Z warn ca ca request failed, starting attempt 4 in 745.069125ms
2022-07-26T00:29:22.521745Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-23 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:29:22.521853Z warn cache slow generate secret lock: 4.016185676s
2022-07-26T00:29:22.538163Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:29:25.843508Z warn ca ca request failed, starting attempt 1 in 91.9ms
2022-07-26T00:29:25.946929Z warn ca ca request failed, starting attempt 2 in 218.846224ms
2022-07-26T00:29:26.167588Z warn ca ca request failed, starting attempt 3 in 429.619812ms
2022-07-26T00:29:26.602250Z warn ca ca request failed, starting attempt 4 in 803.981622ms
2022-07-26T00:29:27.415782Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-24 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:29:27.420340Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:48813->169.254.25.10:53: read: connection refused"
2022-07-26T00:29:35.153356Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-25
2022-07-26T00:29:39.510647Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-26
2022-07-26T00:29:56.280130Z warn ca ca request failed, starting attempt 1 in 108.65313ms
2022-07-26T00:29:56.390071Z warn ca ca request failed, starting attempt 2 in 189.264727ms
2022-07-26T00:29:56.579845Z warn ca ca request failed, starting attempt 3 in 434.181164ms
2022-07-26T00:29:57.042525Z warn ca ca request failed, starting attempt 4 in 854.587209ms
2022-07-26T00:29:57.923133Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:29:57.905898Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-25 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:29:57.906017Z warn cache slow generate secret lock: 18.395115905s
2022-07-26T00:29:59.316314Z warn ca ca request failed, starting attempt 1 in 97.700506ms
2022-07-26T00:29:59.415435Z warn ca ca request failed, starting attempt 2 in 180.368763ms
2022-07-26T00:29:59.596428Z warn ca ca request failed, starting attempt 3 in 369.530701ms
2022-07-26T00:29:59.968921Z warn ca ca request failed, starting attempt 4 in 810.543237ms
2022-07-26T00:30:00.796802Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-26 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:00.804599Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:03.750194Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-27
2022-07-26T00:30:05.476804Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-28
2022-07-26T00:30:07.366316Z warn ca ca request failed, starting attempt 1 in 96.375764ms
2022-07-26T00:30:07.463810Z warn ca ca request failed, starting attempt 2 in 216.577591ms
2022-07-26T00:30:07.681259Z warn ca ca request failed, starting attempt 3 in 438.895656ms
2022-07-26T00:30:08.121189Z warn ca ca request failed, starting attempt 4 in 785.278678ms
2022-07-26T00:30:08.910848Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-27 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:08.911014Z warn cache slow generate secret lock: 3.432473844s
2022-07-26T00:30:08.917524Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:09.091213Z warn ca ca request failed, starting attempt 1 in 102.958814ms
2022-07-26T00:30:09.198316Z warn ca ca request failed, starting attempt 2 in 195.94859ms
2022-07-26T00:30:09.406342Z warn ca ca request failed, starting attempt 3 in 384.237596ms
2022-07-26T00:30:09.792487Z warn ca ca request failed, starting attempt 4 in 875.632447ms
2022-07-26T00:30:10.669951Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-28 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:10.671060Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:19.314728Z warn ca ca request failed, starting attempt 1 in 99.573723ms
2022-07-26T00:30:19.418022Z warn ca ca request failed, starting attempt 2 in 212.809732ms
2022-07-26T00:30:19.635079Z warn ca ca request failed, starting attempt 3 in 431.271416ms
2022-07-26T00:30:20.073590Z warn ca ca request failed, starting attempt 4 in 744.475384ms
2022-07-26T00:30:20.841985Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:22.976300Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-29
2022-07-26T00:30:25.575440Z warn ca ca request failed, starting attempt 1 in 105.287722ms
2022-07-26T00:30:25.682720Z warn ca ca request failed, starting attempt 2 in 212.132925ms
2022-07-26T00:30:25.897187Z warn ca ca request failed, starting attempt 3 in 414.583981ms
2022-07-26T00:30:26.312898Z warn ca ca request failed, starting attempt 4 in 832.987775ms
2022-07-26T00:30:27.148811Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-29 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:28.830087Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:30.328386Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused
2022-07-26T00:30:35.244050Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-30
2022-07-26T00:30:40.680583Z warn ca ca request failed, starting attempt 1 in 97.24912ms
2022-07-26T00:30:40.778987Z warn ca ca request failed, starting attempt 2 in 189.056945ms
2022-07-26T00:30:40.982466Z warn ca ca request failed, starting attempt 3 in 371.788294ms
2022-07-26T00:30:41.372162Z warn ca ca request failed, starting attempt 4 in 841.314029ms
2022-07-26T00:30:42.214565Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-30 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:42.218838Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:30:53.271049Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-31
2022-07-26T00:30:54.178718Z warn ca ca request failed, starting attempt 1 in 100.313602ms
2022-07-26T00:30:54.279643Z warn ca ca request failed, starting attempt 2 in 201.996103ms
2022-07-26T00:30:54.482917Z warn ca ca request failed, starting attempt 3 in 377.155381ms
2022-07-26T00:30:54.864760Z warn ca ca request failed, starting attempt 4 in 744.087057ms
2022-07-26T00:30:55.609973Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-31 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:30:55.620746Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:00.334273Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused
2022-07-26T00:31:01.945571Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-32
2022-07-26T00:31:02.934265Z warn ca ca request failed, starting attempt 1 in 108.239197ms
2022-07-26T00:31:03.049055Z warn ca ca request failed, starting attempt 2 in 185.499436ms
2022-07-26T00:31:03.235846Z warn ca ca request failed, starting attempt 3 in 379.805362ms
2022-07-26T00:31:03.618047Z warn ca ca request failed, starting attempt 4 in 737.950415ms
2022-07-26T00:31:04.358382Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-32 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:31:04.360164Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:11.691156Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-33
2022-07-26T00:31:15.930503Z warn ca ca request failed, starting attempt 1 in 100.55838ms
2022-07-26T00:31:16.049428Z warn ca ca request failed, starting attempt 2 in 197.443055ms
2022-07-26T00:31:16.248744Z warn ca ca request failed, starting attempt 3 in 382.701416ms
2022-07-26T00:31:16.634094Z warn ca ca request failed, starting attempt 4 in 743.207618ms
2022-07-26T00:31:17.394780Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-33 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:31:17.395248Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:30.347452Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused
2022-07-26T00:31:30.732802Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-34
2022-07-26T00:31:32.616200Z warn ca ca request failed, starting attempt 1 in 102.989112ms
2022-07-26T00:31:32.731294Z warn ca ca request failed, starting attempt 2 in 212.709416ms
2022-07-26T00:31:32.944802Z warn ca ca request failed, starting attempt 3 in 416.040966ms
2022-07-26T00:31:33.361337Z warn ca ca request failed, starting attempt 4 in 745.738012ms
2022-07-26T00:31:34.110256Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-34 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:31:34.123965Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:43.242679Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-35
2022-07-26T00:31:44.770647Z warn ca ca request failed, starting attempt 1 in 92.213584ms
2022-07-26T00:31:44.864244Z warn ca ca request failed, starting attempt 2 in 218.66406ms
2022-07-26T00:31:45.085676Z warn ca ca request failed, starting attempt 3 in 391.231063ms
2022-07-26T00:31:45.484864Z warn ca ca request failed, starting attempt 4 in 865.655266ms
2022-07-26T00:31:46.357767Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:46.358143Z warn cache slow generate secret lock: 3.114341591s
2022-07-26T00:31:47.649669Z warn ca ca request failed, starting attempt 1 in 106.018311ms
2022-07-26T00:31:47.756541Z warn ca ca request failed, starting attempt 2 in 211.783338ms
2022-07-26T00:31:47.970313Z warn ca ca request failed, starting attempt 3 in 413.058915ms
2022-07-26T00:31:48.384528Z warn ca ca request failed, starting attempt 4 in 774.34316ms
2022-07-26T00:31:49.159779Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-35 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:31:49.163865Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for ROOTCA: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:31:57.215006Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-36
2022-07-26T00:31:58.025322Z warn ca ca request failed, starting attempt 1 in 94.294757ms
2022-07-26T00:31:58.120282Z warn ca ca request failed, starting attempt 2 in 199.612136ms
2022-07-26T00:31:58.320416Z warn ca ca request failed, starting attempt 3 in 389.647525ms
2022-07-26T00:31:58.714261Z warn ca ca request failed, starting attempt 4 in 778.010081ms
2022-07-26T00:31:59.493495Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-36 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:31:59.494517Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:32:00.368710Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused
2022-07-26T00:32:02.020940Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-37
2022-07-26T00:32:03.630214Z warn ca ca request failed, starting attempt 1 in 90.437965ms
2022-07-26T00:32:03.722792Z warn ca ca request failed, starting attempt 2 in 183.932826ms
2022-07-26T00:32:03.909563Z warn ca ca request failed, starting attempt 3 in 423.659401ms
2022-07-26T00:32:04.339688Z warn ca ca request failed, starting attempt 4 in 767.377656ms
2022-07-26T00:32:05.108437Z info ads ADS: "@" onap-so-admin-cockpit-7877596c7-d8xkl.onap-37 terminated rpc error: code = Canceled desc = context canceled
2022-07-26T00:32:05.111070Z warning envoy config StreamSecrets gRPC config stream closed: 2, failed to generate secret for default: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 169.254.25.10:53: read udp 10.233.72.128:41017->169.254.25.10:53: read: connection refused"
2022-07-26T00:32:11.403719Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-38
2022-07-26T00:32:13.419046Z info cache generated new workload certificate latency=2.014957658s ttl=23h59m59.581135627s
2022-07-26T00:32:13.425419Z info ads SDS: PUSH request for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap resources:1 size:1.1kB resource:ROOTCA
2022-07-26T00:32:25.629756Z info cache returned workload certificate from cache ttl=23h59m47.370273054s
2022-07-26T00:32:25.629921Z info cache returned workload trust anchor from cache ttl=23h59m47.370088525s
2022-07-26T00:32:28.418523Z info ads ADS: new connection for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap-39
2022-07-26T00:32:28.418737Z info cache returned workload certificate from cache ttl=23h59m44.581277194s
2022-07-26T00:32:28.418995Z info ads SDS: PUSH request for node:onap-so-admin-cockpit-7877596c7-d8xkl.onap resources:1 size:4.0kB resource:default
2022-07-26T00:32:30.341926Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused
2022-07-26T00:32:33.269288Z info Readiness succeeded in 8m45.433040407s
2022-07-26T00:32:33.269969Z info Envoy proxy is ready
2022-07-26T00:58:18.342426Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T01:25:30.967593Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T01:53:13.306847Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T02:21:20.492924Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T02:50:12.715127Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2022-07-26T03:17:52.260571Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012