By type
*** Starting uWSGI 2.0.19.1 (64bit) on [Sun Nov 7 04:17:55 2021] ***
compiled with version: 9.3.0 on 29 July 2021 10:04:23
os: Linux-4.19.0-17-cloud-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18)
nodename: onap-oof-has-api-84bcf67d4-hv9rf
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 16
current working directory: /app
writing pidfile to /run/conductor/conductor-uwsgi.pid
detected binary path: /usr/local/bin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/conductor/uwsgi.sock fd 3
uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 4
Python version: 3.9.1 (default, Dec 11 2020, 14:29:41) [GCC 9.3.0]
Python main interpreter initialized at 0x55590ae81ec0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 510440 bytes (498 KB) for 6 cores
*** Operational MODE: preforking ***
2021-11-07 04:17:58,644||139999202381128|INFO|app|conductor.api.app: [-] Full WSGI config used: /usr/local/etc/conductor/api_paste.ini
2021-11-07 04:18:05,665||139999202381128|CRITICAL|log|conductor: [-] Unhandled error
Traceback (most recent call last):
File "/opt/has/conductor/conductor/common/etcd/api.py", line 60, in get_client
return etcd3.client(host=self.host, port=self.port,
File "/usr/local/lib/python3.9/site-packages/etcd3/client.py", line 1178, in client
return Etcd3Client(host=host,
File "/usr/local/lib/python3.9/site-packages/etcd3/client.py", line 148, in __init__
resp = self.auth_stub.Authenticate(auth_request, self.timeout)
File "/usr/local/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "etcdserver: request timed out"
debug_error_string = "{"created":"@1636258685.664486929","description":"Error received from peer ipv4:10.233.76.236:2379","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"etcdserver: request timed out","grpc_status":14}"
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/etc/nginx/conductor.wsgi", line 14, in
application = app.load_app(conf)
File "/opt/has/conductor/conductor/api/app.py", line 126, in load_app
return deploy.loadapp("config:" + cfg_file,
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 278, in loadobj
return context.create()
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create
return self.object_type.invoke(self)
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 209, in invoke
app = context.app_context.create()
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create
return self.object_type.invoke(self)
File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 152, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/opt/has/conductor/conductor/api/app.py", line 133, in app_factory
return setup_app(conf=conf)
File "/opt/has/conductor/conductor/api/app.py", line 74, in setup_app
hooks.MessagingHook(conf),
File "/opt/has/conductor/conductor/api/hooks.py", line 49, in __init__
transport = messaging.get_transport(conf=conf)
File "/opt/has/conductor/conductor/messaging.py", line 56, in get_transport
transport.keyspace_create(keyspace=keyspace)
File "/opt/has/conductor/conductor/common/etcd/api.py", line 94, in keyspace_create
self.get_client().put_if_not_exists(keyspace, "This key is a placeholder for the keyspace".encode())
File "/opt/has/conductor/conductor/common/etcd/api.py", line 69, in get_client
raise EtcdClientException("Failed to establish connection with ETCD. GRPC {}".format(rpc_error.code()))
conductor.common.etcd.utils.EtcdClientException: Failed to establish connection with ETCD. GRPC StatusCode.UNAVAILABLE
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 145, cores: 1)
spawned uWSGI worker 2 (pid: 152, cores: 1)
spawned uWSGI worker 3 (pid: 159, cores: 1)
spawned uWSGI worker 4 (pid: 166, cores: 1)
spawned uWSGI worker 5 (pid: 173, cores: 1)
spawned uWSGI worker 6 (pid: 180, cores: 1)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 194)
Respawned uWSGI worker 2 (new pid: 201)
Respawned uWSGI worker 3 (new pid: 208)
Respawned uWSGI worker 4 (new pid: 215)
Respawned uWSGI worker 5 (new pid: 222)
Respawned uWSGI worker 6 (new pid: 229)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 243)
Respawned uWSGI worker 2 (new pid: 250)
Respawned uWSGI worker 3 (new pid: 257)
Respawned uWSGI worker 4 (new pid: 264)
Respawned uWSGI worker 5 (new pid: 271)
Respawned uWSGI worker 6 (new pid: 278)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 292)
Respawned uWSGI worker 2 (new pid: 299)
Respawned uWSGI worker 3 (new pid: 306)
Respawned uWSGI worker 4 (new pid: 313)
Respawned uWSGI worker 5 (new pid: 320)
Respawned uWSGI worker 6 (new pid: 327)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 341)
Respawned uWSGI worker 2 (new pid: 348)
Respawned uWSGI worker 3 (new pid: 355)
Respawned uWSGI worker 4 (new pid: 362)
Respawned uWSGI worker 5 (new pid: 366)
Respawned uWSGI worker 6 (new pid: 376)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 390)
Respawned uWSGI worker 2 (new pid: 397)
Respawned uWSGI worker 3 (new pid: 404)
Respawned uWSGI worker 4 (new pid: 408)
Respawned uWSGI worker 5 (new pid: 418)
Respawned uWSGI worker 6 (new pid: 425)
--- no python application found, check your startup logs for errors ---
[pid: 408|app: -1|req: -1/1] 10.233.67.108 () {46 vars in 589 bytes} [Sun Nov 7 04:48:08 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 439)
Respawned uWSGI worker 2 (new pid: 446)
Respawned uWSGI worker 3 (new pid: 453)
Respawned uWSGI worker 4 (new pid: 460)
Respawned uWSGI worker 5 (new pid: 467)
Respawned uWSGI worker 6 (new pid: 474)
--- no python application found, check your startup logs for errors ---
[pid: 460|app: -1|req: -1/2] 10.233.67.105 () {46 vars in 589 bytes} [Sun Nov 7 04:49:21 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 474|app: -1|req: -1/3] 10.233.67.29 () {46 vars in 586 bytes} [Sun Nov 7 04:50:16 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 453|app: -1|req: -1/4] 10.233.67.61 () {46 vars in 586 bytes} [Sun Nov 7 04:51:00 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 460|app: -1|req: -1/5] 10.233.67.238 () {46 vars in 589 bytes} [Sun Nov 7 04:51:32 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 439|app: -1|req: -1/6] 10.233.67.222 () {46 vars in 589 bytes} [Sun Nov 7 04:52:27 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 467|app: -1|req: -1/7] 10.233.67.38 () {46 vars in 586 bytes} [Sun Nov 7 04:53:07 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 488)
Respawned uWSGI worker 2 (new pid: 495)
Respawned uWSGI worker 3 (new pid: 502)
Respawned uWSGI worker 4 (new pid: 509)
Respawned uWSGI worker 5 (new pid: 516)
Respawned uWSGI worker 6 (new pid: 520)
--- no python application found, check your startup logs for errors ---
[pid: 516|app: -1|req: -1/8] 10.233.67.236 () {46 vars in 589 bytes} [Sun Nov 7 04:54:25 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 516|app: -1|req: -1/9] 10.233.67.251 () {46 vars in 589 bytes} [Sun Nov 7 04:57:07 2021] GET /v1/plans/healthcheck => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 537)
Respawned uWSGI worker 2 (new pid: 544)
Respawned uWSGI worker 3 (new pid: 551)
Respawned uWSGI worker 4 (new pid: 558)
Respawned uWSGI worker 5 (new pid: 565)
Respawned uWSGI worker 6 (new pid: 572)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 586)
Respawned uWSGI worker 2 (new pid: 593)
Respawned uWSGI worker 3 (new pid: 600)
Respawned uWSGI worker 4 (new pid: 607)
Respawned uWSGI worker 5 (new pid: 614)
Respawned uWSGI worker 6 (new pid: 621)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 635)
Respawned uWSGI worker 2 (new pid: 642)
Respawned uWSGI worker 3 (new pid: 649)
Respawned uWSGI worker 4 (new pid: 656)
Respawned uWSGI worker 5 (new pid: 663)
Respawned uWSGI worker 6 (new pid: 667)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 684)
Respawned uWSGI worker 2 (new pid: 691)
Respawned uWSGI worker 3 (new pid: 698)
Respawned uWSGI worker 4 (new pid: 705)
Respawned uWSGI worker 5 (new pid: 712)
Respawned uWSGI worker 6 (new pid: 719)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 733)
Respawned uWSGI worker 2 (new pid: 740)
Respawned uWSGI worker 3 (new pid: 747)
Respawned uWSGI worker 4 (new pid: 754)
Respawned uWSGI worker 5 (new pid: 761)
Respawned uWSGI worker 6 (new pid: 768)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 782)
Respawned uWSGI worker 2 (new pid: 789)
Respawned uWSGI worker 3 (new pid: 796)
Respawned uWSGI worker 4 (new pid: 803)
Respawned uWSGI worker 5 (new pid: 810)
Respawned uWSGI worker 6 (new pid: 817)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 831)
Respawned uWSGI worker 2 (new pid: 838)
Respawned uWSGI worker 3 (new pid: 845)
Respawned uWSGI worker 4 (new pid: 852)
Respawned uWSGI worker 5 (new pid: 859)
Respawned uWSGI worker 6 (new pid: 866)
--- no python application found, check your startup logs for errors ---
--- no python application found, check your startup logs for errors ---
[pid: 845|app: -1|req: -1/11] 10.233.66.90 () {42 vars in 580 bytes} [Sun Nov 7 05:32:04 2021] GET /nmaplowercheck1636263124 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
[pid: 859|app: -1|req: -1/11] 10.233.66.90 () {44 vars in 568 bytes} [Sun Nov 7 05:32:04 2021] POST /sdk => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 866|app: -1|req: -1/12] 10.233.66.90 () {42 vars in 542 bytes} [Sun Nov 7 05:32:05 2021] GET /HNAP1 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 859|app: -1|req: -1/13] 10.233.66.90 () {42 vars in 552 bytes} [Sun Nov 7 05:32:05 2021] GET /evox/about => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 866|app: -1|req: -1/14] 10.233.66.90 () {36 vars in 374 bytes} [Sun Nov 7 05:32:12 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.0 500) 2 headers in 83 bytes (1 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 859|app: -1|req: -1/15] 10.233.66.90 () {38 vars in 404 bytes} [Sun Nov 7 05:32:12 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 885)
Respawned uWSGI worker 2 (new pid: 892)
Respawned uWSGI worker 3 (new pid: 899)
Respawned uWSGI worker 4 (new pid: 906)
Respawned uWSGI worker 5 (new pid: 913)
Respawned uWSGI worker 6 (new pid: 920)
invalid request block size: 22340 (max 4096)...skip