*** Starting uWSGI 2.0.19.1 (64bit) on [Sun Nov 7 04:17:55 2021] *** compiled with version: 9.3.0 on 29 July 2021 10:04:23 os: Linux-4.19.0-17-cloud-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) nodename: onap-oof-has-api-84bcf67d4-hv9rf machine: x86_64 clock source: unix pcre jit disabled detected number of CPU cores: 16 current working directory: /app writing pidfile to /run/conductor/conductor-uwsgi.pid detected binary path: /usr/local/bin/uwsgi your memory page size is 4096 bytes detected max file descriptor number: 1048576 lock engine: pthread robust mutexes thunder lock: disabled (you can enable it with --thunder-lock) uwsgi socket 0 bound to UNIX address /run/conductor/uwsgi.sock fd 3 uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 4 Python version: 3.9.1 (default, Dec 11 2020, 14:29:41) [GCC 9.3.0] Python main interpreter initialized at 0x55590ae81ec0 python threads support enabled your server socket listen backlog is limited to 100 connections your mercy for graceful operations on workers is 60 seconds mapped 510440 bytes (498 KB) for 6 cores *** Operational MODE: preforking *** 2021-11-07 04:17:58,644||139999202381128|INFO|app|conductor.api.app: [-] Full WSGI config used: /usr/local/etc/conductor/api_paste.ini 2021-11-07 04:18:05,665||139999202381128|CRITICAL|log|conductor: [-] Unhandled error Traceback (most recent call last): File "/opt/has/conductor/conductor/common/etcd/api.py", line 60, in get_client return etcd3.client(host=self.host, port=self.port, File "/usr/local/lib/python3.9/site-packages/etcd3/client.py", line 1178, in client return Etcd3Client(host=host, File "/usr/local/lib/python3.9/site-packages/etcd3/client.py", line 148, in __init__ resp = self.auth_stub.Authenticate(auth_request, self.timeout) File "/usr/local/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__ return _end_unary_response_blocking(state, call, False, None) File "/usr/local/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "etcdserver: request timed out" debug_error_string = "{"created":"@1636258685.664486929","description":"Error received from peer ipv4:10.233.76.236:2379","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"etcdserver: request timed out","grpc_status":14}" > During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/etc/nginx/conductor.wsgi", line 14, in application = app.load_app(conf) File "/opt/has/conductor/conductor/api/app.py", line 126, in load_app return deploy.loadapp("config:" + cfg_file, File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp return loadobj(APP, uri, name=name, **kw) File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 278, in loadobj return context.create() File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create return self.object_type.invoke(self) File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 209, in invoke app = context.app_context.create() File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create return self.object_type.invoke(self) File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 152, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 55, in fix_call val = callable(*args, **kw) File "/opt/has/conductor/conductor/api/app.py", line 133, in app_factory return setup_app(conf=conf) File "/opt/has/conductor/conductor/api/app.py", line 74, in setup_app hooks.MessagingHook(conf), File "/opt/has/conductor/conductor/api/hooks.py", line 49, in __init__ transport = messaging.get_transport(conf=conf) File "/opt/has/conductor/conductor/messaging.py", line 56, in get_transport transport.keyspace_create(keyspace=keyspace) File "/opt/has/conductor/conductor/common/etcd/api.py", line 94, in keyspace_create self.get_client().put_if_not_exists(keyspace, "This key is a placeholder for the keyspace".encode()) File "/opt/has/conductor/conductor/common/etcd/api.py", line 69, in get_client raise EtcdClientException("Failed to establish connection with ETCD. GRPC {}".format(rpc_error.code())) conductor.common.etcd.utils.EtcdClientException: Failed to establish connection with ETCD. GRPC StatusCode.UNAVAILABLE unable to load app 0 (mountpoint='') (callable not found or import error) *** no app loaded. going in full dynamic mode *** spawned uWSGI master process (pid: 1) spawned uWSGI worker 1 (pid: 145, cores: 1) spawned uWSGI worker 2 (pid: 152, cores: 1) spawned uWSGI worker 3 (pid: 159, cores: 1) spawned uWSGI worker 4 (pid: 166, cores: 1) spawned uWSGI worker 5 (pid: 173, cores: 1) spawned uWSGI worker 6 (pid: 180, cores: 1) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 194) Respawned uWSGI worker 2 (new pid: 201) Respawned uWSGI worker 3 (new pid: 208) Respawned uWSGI worker 4 (new pid: 215) Respawned uWSGI worker 5 (new pid: 222) Respawned uWSGI worker 6 (new pid: 229) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 243) Respawned uWSGI worker 2 (new pid: 250) Respawned uWSGI worker 3 (new pid: 257) Respawned uWSGI worker 4 (new pid: 264) Respawned uWSGI worker 5 (new pid: 271) Respawned uWSGI worker 6 (new pid: 278) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 292) Respawned uWSGI worker 2 (new pid: 299) Respawned uWSGI worker 3 (new pid: 306) Respawned uWSGI worker 4 (new pid: 313) Respawned uWSGI worker 5 (new pid: 320) Respawned uWSGI worker 6 (new pid: 327) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 341) Respawned uWSGI worker 2 (new pid: 348) Respawned uWSGI worker 3 (new pid: 355) Respawned uWSGI worker 4 (new pid: 362) Respawned uWSGI worker 5 (new pid: 366) Respawned uWSGI worker 6 (new pid: 376) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 390) Respawned uWSGI worker 2 (new pid: 397) Respawned uWSGI worker 3 (new pid: 404) Respawned uWSGI worker 4 (new pid: 408) Respawned uWSGI worker 5 (new pid: 418) Respawned uWSGI worker 6 (new pid: 425) --- no python application found, check your startup logs for errors --- [pid: 408|app: -1|req: -1/1] 10.233.67.108 () {46 vars in 589 bytes} [Sun Nov 7 04:48:08 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 439) Respawned uWSGI worker 2 (new pid: 446) Respawned uWSGI worker 3 (new pid: 453) Respawned uWSGI worker 4 (new pid: 460) Respawned uWSGI worker 5 (new pid: 467) Respawned uWSGI worker 6 (new pid: 474) --- no python application found, check your startup logs for errors --- [pid: 460|app: -1|req: -1/2] 10.233.67.105 () {46 vars in 589 bytes} [Sun Nov 7 04:49:21 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 474|app: -1|req: -1/3] 10.233.67.29 () {46 vars in 586 bytes} [Sun Nov 7 04:50:16 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 453|app: -1|req: -1/4] 10.233.67.61 () {46 vars in 586 bytes} [Sun Nov 7 04:51:00 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 460|app: -1|req: -1/5] 10.233.67.238 () {46 vars in 589 bytes} [Sun Nov 7 04:51:32 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 439|app: -1|req: -1/6] 10.233.67.222 () {46 vars in 589 bytes} [Sun Nov 7 04:52:27 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 467|app: -1|req: -1/7] 10.233.67.38 () {46 vars in 586 bytes} [Sun Nov 7 04:53:07 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 488) Respawned uWSGI worker 2 (new pid: 495) Respawned uWSGI worker 3 (new pid: 502) Respawned uWSGI worker 4 (new pid: 509) Respawned uWSGI worker 5 (new pid: 516) Respawned uWSGI worker 6 (new pid: 520) --- no python application found, check your startup logs for errors --- [pid: 516|app: -1|req: -1/8] 10.233.67.236 () {46 vars in 589 bytes} [Sun Nov 7 04:54:25 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 516|app: -1|req: -1/9] 10.233.67.251 () {46 vars in 589 bytes} [Sun Nov 7 04:57:07 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 537) Respawned uWSGI worker 2 (new pid: 544) Respawned uWSGI worker 3 (new pid: 551) Respawned uWSGI worker 4 (new pid: 558) Respawned uWSGI worker 5 (new pid: 565) Respawned uWSGI worker 6 (new pid: 572) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 586) Respawned uWSGI worker 2 (new pid: 593) Respawned uWSGI worker 3 (new pid: 600) Respawned uWSGI worker 4 (new pid: 607) Respawned uWSGI worker 5 (new pid: 614) Respawned uWSGI worker 6 (new pid: 621) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 635) Respawned uWSGI worker 2 (new pid: 642) Respawned uWSGI worker 3 (new pid: 649) Respawned uWSGI worker 4 (new pid: 656) Respawned uWSGI worker 5 (new pid: 663) Respawned uWSGI worker 6 (new pid: 667) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 684) Respawned uWSGI worker 2 (new pid: 691) Respawned uWSGI worker 3 (new pid: 698) Respawned uWSGI worker 4 (new pid: 705) Respawned uWSGI worker 5 (new pid: 712) Respawned uWSGI worker 6 (new pid: 719) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 733) Respawned uWSGI worker 2 (new pid: 740) Respawned uWSGI worker 3 (new pid: 747) Respawned uWSGI worker 4 (new pid: 754) Respawned uWSGI worker 5 (new pid: 761) Respawned uWSGI worker 6 (new pid: 768) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 782) Respawned uWSGI worker 2 (new pid: 789) Respawned uWSGI worker 3 (new pid: 796) Respawned uWSGI worker 4 (new pid: 803) Respawned uWSGI worker 5 (new pid: 810) Respawned uWSGI worker 6 (new pid: 817) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 831) Respawned uWSGI worker 2 (new pid: 838) Respawned uWSGI worker 3 (new pid: 845) Respawned uWSGI worker 4 (new pid: 852) Respawned uWSGI worker 5 (new pid: 859) Respawned uWSGI worker 6 (new pid: 866) --- no python application found, check your startup logs for errors --- --- no python application found, check your startup logs for errors --- [pid: 845|app: -1|req: -1/11] 10.233.66.90 () {42 vars in 580 bytes} [Sun Nov 7 05:32:04 2021] GET /nmaplowercheck1636263124 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) [pid: 859|app: -1|req: -1/11] 10.233.66.90 () {44 vars in 568 bytes} [Sun Nov 7 05:32:04 2021] POST /sdk => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 866|app: -1|req: -1/12] 10.233.66.90 () {42 vars in 542 bytes} [Sun Nov 7 05:32:05 2021] GET /HNAP1 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 859|app: -1|req: -1/13] 10.233.66.90 () {42 vars in 552 bytes} [Sun Nov 7 05:32:05 2021] GET /evox/about => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 866|app: -1|req: -1/14] 10.233.66.90 () {36 vars in 374 bytes} [Sun Nov 7 05:32:12 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.0 500) 2 headers in 83 bytes (1 switches on core 0) --- no python application found, check your startup logs for errors --- [pid: 859|app: -1|req: -1/15] 10.233.66.90 () {38 vars in 404 bytes} [Sun Nov 7 05:32:12 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0) worker 1 lifetime reached, it was running for 301 second(s) worker 2 lifetime reached, it was running for 301 second(s) worker 3 lifetime reached, it was running for 301 second(s) worker 4 lifetime reached, it was running for 301 second(s) worker 5 lifetime reached, it was running for 301 second(s) worker 6 lifetime reached, it was running for 301 second(s) Respawned uWSGI worker 1 (new pid: 885) Respawned uWSGI worker 2 (new pid: 892) Respawned uWSGI worker 3 (new pid: 899) Respawned uWSGI worker 4 (new pid: 906) Respawned uWSGI worker 5 (new pid: 913) Respawned uWSGI worker 6 (new pid: 920) invalid request block size: 22340 (max 4096)...skip