2022-09-07T23:04:58.742906Z info FLAG: --concurrency="2" 2022-09-07T23:04:58.745302Z info FLAG: --domain="onap.svc.cluster.local" 2022-09-07T23:04:58.745341Z info FLAG: --help="false" 2022-09-07T23:04:58.745354Z info FLAG: --log_as_json="false" 2022-09-07T23:04:58.745365Z info FLAG: --log_caller="" 2022-09-07T23:04:58.745377Z info FLAG: --log_output_level="default:info" 2022-09-07T23:04:58.745387Z info FLAG: --log_rotate="" 2022-09-07T23:04:58.745397Z info FLAG: --log_rotate_max_age="30" 2022-09-07T23:04:58.745403Z info FLAG: --log_rotate_max_backups="1000" 2022-09-07T23:04:58.745409Z info FLAG: --log_rotate_max_size="104857600" 2022-09-07T23:04:58.745415Z info FLAG: --log_stacktrace_level="default:none" 2022-09-07T23:04:58.745442Z info FLAG: --log_target="[stdout]" 2022-09-07T23:04:58.745484Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2022-09-07T23:04:58.745492Z info FLAG: --outlierLogPath="" 2022-09-07T23:04:58.745498Z info FLAG: --proxyComponentLogLevel="misc:error" 2022-09-07T23:04:58.745504Z info FLAG: --proxyLogLevel="warning" 2022-09-07T23:04:58.745510Z info FLAG: --serviceCluster="istio-proxy" 2022-09-07T23:04:58.745516Z info FLAG: --stsPort="0" 2022-09-07T23:04:58.745522Z info FLAG: --templateFile="" 2022-09-07T23:04:58.745527Z info FLAG: --tokenManagerPlugin="GoogleTokenExchange" 2022-09-07T23:04:58.745536Z info FLAG: --vklog="0" 2022-09-07T23:04:58.745543Z info Version 1.14.1-f59ce19ec6b63bbb70a65c43ac423845f1129464-Clean 2022-09-07T23:04:58.745820Z info Proxy role ips=[10.233.72.118] type=sidecar id=onap-keycloak-0.onap domain=onap.svc.cluster.local 2022-09-07T23:04:58.745971Z info Apply proxy config from env {} 2022-09-07T23:04:58.752084Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod.istio-system.svc:15012 drainDuration: 45s parentShutdownDuration: 60s proxyAdminPort: 15000 serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s tracing: zipkin: address: zipkin.istio-system:9411 2022-09-07T23:04:58.752126Z info JWT policy is third-party-jwt 2022-09-07T23:04:58.752134Z info using credential fetcher of JWT type in cluster.local trust domain 2022-09-07T23:04:58.867880Z info SDS socket not found. Starting Istio SDS Server 2022-09-07T23:04:58.867975Z info CA Endpoint istiod.istio-system.svc:15012, provider Citadel 2022-09-07T23:04:58.868037Z info Using CA istiod.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2022-09-07T23:04:58.872739Z info citadelclient Citadel client using custom root cert: var/run/secrets/istio/root-cert.pem 2022-09-07T23:04:58.894020Z info Opening status port 15020 2022-09-07T23:04:59.054107Z info ads All caches have been synced up in 341.260489ms, marking server ready 2022-09-07T23:04:59.055945Z info sds SDS server for workload certificates started, listening on "./var/run/secrets/workload-spiffe-uds/socket" 2022-09-07T23:04:59.056045Z info xdsproxy Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes" 2022-09-07T23:04:59.056273Z info sds Starting SDS grpc server 2022-09-07T23:04:59.057990Z info starting Http service at 127.0.0.1:15004 2022-09-07T23:04:59.062777Z info Pilot SAN: [istiod.istio-system.svc] 2022-09-07T23:04:59.068192Z info Starting proxy agent 2022-09-07T23:04:59.068292Z info Epoch 0 starting 2022-09-07T23:04:59.068371Z info Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --log-format %Y-%m-%dT%T.%fZ %l envoy %n %v -l warning --component-log-level misc:error --concurrency 2] 2022-09-07T23:04:59.546202Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-07T23:04:59.891192Z info ads ADS: new connection for node:onap-keycloak-0.onap-1 2022-09-07T23:04:59.929728Z info ads ADS: new connection for node:onap-keycloak-0.onap-2 2022-09-07T23:05:00.155397Z info cache generated new workload certificate latency=1.10044508s ttl=23h59m58.844638744s 2022-09-07T23:05:00.155499Z info cache Root cert has changed, start rotating root cert 2022-09-07T23:05:00.155557Z info ads XDS: Incremental Pushing:0 ConnectedEndpoints:2 Version: 2022-09-07T23:05:00.155709Z info cache returned workload trust anchor from cache ttl=23h59m58.844301093s 2022-09-07T23:05:00.155770Z info cache returned workload trust anchor from cache ttl=23h59m58.844238599s 2022-09-07T23:05:00.156065Z info cache returned workload certificate from cache ttl=23h59m58.843964338s 2022-09-07T23:05:00.156760Z info ads SDS: PUSH request for node:onap-keycloak-0.onap resources:1 size:1.1kB resource:ROOTCA 2022-09-07T23:05:00.156906Z info cache returned workload trust anchor from cache ttl=23h59m58.843104219s 2022-09-07T23:05:00.157400Z info ads SDS: PUSH request for node:onap-keycloak-0.onap resources:1 size:4.0kB resource:default 2022-09-07T23:05:00.157699Z info ads SDS: PUSH for node:onap-keycloak-0.onap resources:1 size:1.1kB resource:ROOTCA 2022-09-07T23:05:01.495002Z info Readiness succeeded in 2.778485422s 2022-09-07T23:05:01.495818Z info Envoy proxy is ready 2022-09-07T23:06:31.707166Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:06:36.717726Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:06:41.703081Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:06:46.702658Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:06:51.711096Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:06:56.704984Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:01.703402Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:06.703843Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:11.704301Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:16.703860Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:21.714979Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:26.703273Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:31.703876Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:36.702466Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:41.703382Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:46.705085Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:51.731390Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:07:56.703507Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": dial tcp 127.0.0.6:0->10.233.72.118:8080: connect: connection refused, original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:08:02.705089Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/startupz app URL path = /auth/ 2022-09-07T23:10:44.669872Z warn xdsproxy upstream [1] terminated with unexpected error rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: read tcp 10.233.72.118:53358->10.233.70.69:15012: read: connection reset by peer", received prior goaway: code: NO_ERROR 2022-09-07T23:10:45.055881Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-07T23:33:47.715578Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-07T23:33:57.809660Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-07T23:34:37.740741Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-07T23:37:47.711826Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-07T23:39:39.810115Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-07T23:41:57.713046Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-08T00:00:05.812653Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T00:09:36.836332Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T00:40:47.507903Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T01:12:01.564239Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T01:41:13.091893Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:09:16.151637Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:15:21.649003Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:24:53.098659Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:24:53.204281Z warning envoy config StreamAggregatedResources gRPC config stream closed: 8, request rate limit exceeded: rate: Wait(n=1) would exceed context deadline (previously 14, error reading from server: EOF since 0s ago) 2022-09-08T02:24:53.202879Z warn xdsproxy upstream [11] terminated with unexpected error rpc error: code = ResourceExhausted desc = request rate limit exceeded: rate: Wait(n=1) would exceed context deadline 2022-09-08T02:24:53.735810Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:24:53.752766Z warn xdsproxy upstream [12] terminated with unexpected error rpc error: code = ResourceExhausted desc = request rate limit exceeded: rate: Wait(n=1) would exceed context deadline 2022-09-08T02:24:54.259258Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T02:57:57.821478Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T03:27:27.440794Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T03:33:37.713925Z error Request to probe app failed: Get "http://10.233.72.118:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers), original URL path = /app-health/keycloak/readyz app URL path = /auth/realms/master 2022-09-08T03:40:01.959009Z warn xdsproxy upstream [15] terminated with unexpected error rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: read tcp 10.233.72.118:59964->10.233.66.89:15012: read: connection reset by peer", received prior goaway: code: NO_ERROR 2022-09-08T03:40:02.187279Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T03:40:02.199671Z warn xdsproxy upstream [16] terminated with unexpected error rpc error: code = ResourceExhausted desc = request rate limit exceeded: rate: Wait(n=1) would exceed context deadline 2022-09-08T03:40:02.203732Z warning envoy config StreamAggregatedResources gRPC config stream closed: 8, request rate limit exceeded: rate: Wait(n=1) would exceed context deadline (previously 14, closing transport due to: connection error: desc = "error reading from server: read tcp 10.233.72.118:59964->10.233.66.89:15012: read: connection reset by peer", received prior goaway: code: NO_ERROR since 0s ago) 2022-09-08T03:40:02.497786Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T03:40:02.518037Z warn xdsproxy upstream [17] terminated with unexpected error rpc error: code = ResourceExhausted desc = request rate limit exceeded: rate: Wait(n=1) would exceed context deadline 2022-09-08T03:40:03.189625Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T03:40:03.208877Z warn xdsproxy upstream [18] terminated with unexpected error rpc error: code = ResourceExhausted desc = request rate limit exceeded: rate: Wait(n=1) would exceed context deadline 2022-09-08T03:40:06.630193Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T04:12:33.684939Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2022-09-08T04:13:36.119196Z warn xdsproxy upstream [20] terminated with unexpected error rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: read tcp 10.233.72.118:42206->10.233.69.54:15012: read: connection reset by peer", received prior goaway: code: NO_ERROR 2022-09-08T04:13:36.336511Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012