By type
*** Starting uWSGI 2.0.19.1 (64bit) on [Wed May 12 09:25:57 2021] ***
compiled with version: 9.3.0 on 16 March 2021 05:56:01
os: Linux-4.19.0-13-cloud-amd64 #1 SMP Debian 4.19.160-2 (2020-11-28)
nodename: onap-oof-has-api-7fbfc68cfb-8hq54
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 16
current working directory: /app
writing pidfile to /run/conductor/conductor-uwsgi.pid
detected binary path: /usr/local/bin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/conductor/uwsgi.sock fd 3
uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 4
Python version: 3.9.1 (default, Dec 11 2020, 14:29:41) [GCC 9.3.0]
Python main interpreter initialized at 0x55b196de0540
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 510440 bytes (498 KB) for 6 cores
*** Operational MODE: preforking ***
2021-05-12 09:26:00,001||139650462649672|INFO|app|conductor.api.app: [-] Full WSGI config used: /usr/local/etc/conductor/api_paste.ini
2021-05-12 09:26:00,004||139650462649672|INFO|api|conductor.common.music.api: [-] Initializing Music API
WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b196de0540 pid: 1 (default app)
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 141, cores: 1)
spawned uWSGI worker 2 (pid: 142, cores: 1)
spawned uWSGI worker 3 (pid: 143, cores: 1)
spawned uWSGI worker 4 (pid: 144, cores: 1)
spawned uWSGI worker 5 (pid: 145, cores: 1)
spawned uWSGI worker 6 (pid: 146, cores: 1)
2021-05-12 09:30:33,894||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message 8420b2ec-3624-4dc4-ae5f-3b4fcade1772 on topic controller enqueued.
[pid: 144|app: 0|req: 1/1] 10.233.77.217 () {46 vars in 589 bytes} [Wed May 12 09:30:31 2021] GET /v1/plans/healthcheck => generated 159 bytes in 7412 msecs (HTTP/1.1 200) 5 headers in 134 bytes (1 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 150)
Respawned uWSGI worker 2 (new pid: 151)
Respawned uWSGI worker 3 (new pid: 152)
Respawned uWSGI worker 4 (new pid: 153)
Respawned uWSGI worker 5 (new pid: 154)
Respawned uWSGI worker 6 (new pid: 155)
2021-05-12 09:33:23,607||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message 9a0492c3-2b29-43aa-9c8f-b75ef803fe74 on topic controller enqueued.
2021-05-12 09:34:57,280||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message df7dabe5-3cae-447b-8fdd-a08effe5b7d3 on topic controller enqueued.
[pid: 154|app: 0|req: 1/2] 10.233.77.178 () {46 vars in 589 bytes} [Wed May 12 09:34:56 2021] GET /v1/plans/healthcheck => generated 159 bytes in 6342 msecs (HTTP/1.1 200) 5 headers in 134 bytes (2 switches on core 0)
2021-05-12 09:35:29,072||139650462649672|ERROR|component|conductor.common.music.messaging.component: [-] Message 9a0492c3-2b29-43aa-9c8f-b75ef803fe74 on topic controller timed out at 120 seconds
2021-05-12 09:35:29,073||139650462649672|CRITICAL|log|conductor: [-] Unhandled error
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 141, in lookup_controller
obj, remainder = find_object(obj, remainder, notfound_handlers,
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 197, in find_object
raise PecanNotFound
pecan.routing.PecanNotFound
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/middleware/recursive.py", line 56, in __call__
return self.application(environ, start_response)
File "/opt/has/conductor/conductor/api/middleware.py", line 130, in __call__
app_iter = self.app(environ, replacement_start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 852, in __call__
return super(Pecan, self).__call__(environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 692, in __call__
controller, args, kwargs = self.find_controller(state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 870, in find_controller
controller, args, kw = super(Pecan, self).find_controller(_state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 460, in find_controller
controller, remainder = self.route(req, self.root, path)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 293, in route
node, remainder = lookup_controller(node, path, req)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 158, in lookup_controller
result = handle_lookup_traversal(obj, remainder)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 175, in handle_lookup_traversal
result = obj(*args)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 311, in _lookup
return PlansItemController(uuid4), remainder
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 190, in __init__
self.plan = self.plans_get(plan_id=self.uuid)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 95, in plans_get
return self.plan_getid(plan_id)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 110, in plan_getid
result = client.call(ctx, method, args)
File "/opt/has/conductor/conductor/common/music/messaging/component.py", line 229, in call
response = rpc.response
AttributeError: 'NoneType' object has no attribute 'response'
[pid: 153|app: 0|req: 2/3] 10.233.77.17 () {46 vars in 586 bytes} [Wed May 12 09:33:08 2021] GET /v1/plans/healthcheck => generated 0 bytes in 140674 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 198)
Respawned uWSGI worker 2 (new pid: 199)
Respawned uWSGI worker 3 (new pid: 200)
Respawned uWSGI worker 4 (new pid: 201)
Respawned uWSGI worker 5 (new pid: 202)
Respawned uWSGI worker 6 (new pid: 203)
2021-05-12 09:36:51,350||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message a87fa413-77ae-4d70-bd2a-6ee0e411a409 on topic controller enqueued.
2021-05-12 09:39:00,853||139650462649672|ERROR|component|conductor.common.music.messaging.component: [-] Message a87fa413-77ae-4d70-bd2a-6ee0e411a409 on topic controller timed out at 120 seconds
2021-05-12 09:39:00,854||139650462649672|CRITICAL|log|conductor: [-] Unhandled error
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 141, in lookup_controller
obj, remainder = find_object(obj, remainder, notfound_handlers,
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 197, in find_object
raise PecanNotFound
pecan.routing.PecanNotFound
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/middleware/recursive.py", line 56, in __call__
return self.application(environ, start_response)
File "/opt/has/conductor/conductor/api/middleware.py", line 130, in __call__
app_iter = self.app(environ, replacement_start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 852, in __call__
return super(Pecan, self).__call__(environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 692, in __call__
controller, args, kwargs = self.find_controller(state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 870, in find_controller
controller, args, kw = super(Pecan, self).find_controller(_state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 460, in find_controller
controller, remainder = self.route(req, self.root, path)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 293, in route
node, remainder = lookup_controller(node, path, req)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 158, in lookup_controller
result = handle_lookup_traversal(obj, remainder)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 175, in handle_lookup_traversal
result = obj(*args)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 311, in _lookup
return PlansItemController(uuid4), remainder
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 190, in __init__
self.plan = self.plans_get(plan_id=self.uuid)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 95, in plans_get
return self.plan_getid(plan_id)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 110, in plan_getid
result = client.call(ctx, method, args)
File "/opt/has/conductor/conductor/common/music/messaging/component.py", line 229, in call
response = rpc.response
AttributeError: 'NoneType' object has no attribute 'response'
[pid: 202|app: 0|req: 2/4] 10.233.77.190 () {46 vars in 589 bytes} [Wed May 12 09:36:36 2021] GET /v1/plans/healthcheck => generated 0 bytes in 144640 msecs (HTTP/1.1 500) 0 headers in 0 bytes (1 switches on core 0)
2021-05-12 09:39:01,503||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message b0d27477-4892-4921-88b0-80c9b5f84c7f on topic controller enqueued.
2021-05-12 09:41:02,407||139650462649672|ERROR|component|conductor.common.music.messaging.component: [-] Message b0d27477-4892-4921-88b0-80c9b5f84c7f on topic controller timed out at 120 seconds
2021-05-12 09:41:02,409||139650462649672|CRITICAL|log|conductor: [-] Unhandled error
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 141, in lookup_controller
obj, remainder = find_object(obj, remainder, notfound_handlers,
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 197, in find_object
raise PecanNotFound
pecan.routing.PecanNotFound
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
resp = self.call_func(req, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/oslo_middleware/base.py", line 124, in __call__
response = req.get_response(self.application)
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
status, headers, app_iter = self.call_application(
File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
app_iter = application(self.environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/middleware/recursive.py", line 56, in __call__
return self.application(environ, start_response)
File "/opt/has/conductor/conductor/api/middleware.py", line 130, in __call__
app_iter = self.app(environ, replacement_start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 852, in __call__
return super(Pecan, self).__call__(environ, start_response)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 692, in __call__
controller, args, kwargs = self.find_controller(state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 870, in find_controller
controller, args, kw = super(Pecan, self).find_controller(_state)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 460, in find_controller
controller, remainder = self.route(req, self.root, path)
File "/usr/local/lib/python3.9/site-packages/pecan/core.py", line 293, in route
node, remainder = lookup_controller(node, path, req)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 158, in lookup_controller
result = handle_lookup_traversal(obj, remainder)
File "/usr/local/lib/python3.9/site-packages/pecan/routing.py", line 175, in handle_lookup_traversal
result = obj(*args)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 311, in _lookup
return PlansItemController(uuid4), remainder
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 190, in __init__
self.plan = self.plans_get(plan_id=self.uuid)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 95, in plans_get
return self.plan_getid(plan_id)
File "/opt/has/conductor/conductor/api/controllers/v1/plans.py", line 110, in plan_getid
result = client.call(ctx, method, args)
File "/opt/has/conductor/conductor/common/music/messaging/component.py", line 229, in call
response = rpc.response
AttributeError: 'NoneType' object has no attribute 'response'
[pid: 203|app: 0|req: 1/5] 10.233.74.11 () {46 vars in 586 bytes} [Wed May 12 09:38:46 2021] GET /v1/plans/healthcheck => generated 0 bytes in 136058 msecs (HTTP/1.1 500) 0 headers in 0 bytes (1 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 262)
Respawned uWSGI worker 2 (new pid: 263)
Respawned uWSGI worker 3 (new pid: 264)
Respawned uWSGI worker 4 (new pid: 265)
Respawned uWSGI worker 5 (new pid: 266)
Respawned uWSGI worker 6 (new pid: 267)
2021-05-12 09:41:40,227||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message f112d06d-1460-4c97-979b-fe36cb22a26e on topic controller enqueued.
[pid: 265|app: 0|req: 3/6] 10.233.77.191 () {46 vars in 589 bytes} [Wed May 12 09:41:38 2021] GET /v1/plans/healthcheck => generated 159 bytes in 7810 msecs (HTTP/1.1 200) 5 headers in 134 bytes (1 switches on core 0)
2021-05-12 09:44:44,214||139650462649672|INFO|component|conductor.common.music.messaging.component: [-] Message 8a94deb2-c701-47c4-96ad-d222108227bc on topic controller enqueued.
[pid: 263|app: 0|req: 1/7] 10.233.77.72 () {46 vars in 586 bytes} [Wed May 12 09:44:37 2021] GET /v1/plans/healthcheck => generated 159 bytes in 19032 msecs (HTTP/1.1 200) 5 headers in 134 bytes (1 switches on core 0)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 278)
Respawned uWSGI worker 2 (new pid: 279)
Respawned uWSGI worker 3 (new pid: 280)
Respawned uWSGI worker 4 (new pid: 281)
Respawned uWSGI worker 5 (new pid: 282)
Respawned uWSGI worker 6 (new pid: 283)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 284)
Respawned uWSGI worker 2 (new pid: 285)
Respawned uWSGI worker 3 (new pid: 286)
Respawned uWSGI worker 4 (new pid: 287)
Respawned uWSGI worker 5 (new pid: 288)
Respawned uWSGI worker 6 (new pid: 289)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 290)
Respawned uWSGI worker 2 (new pid: 291)
Respawned uWSGI worker 3 (new pid: 292)
Respawned uWSGI worker 4 (new pid: 293)
Respawned uWSGI worker 5 (new pid: 294)
Respawned uWSGI worker 6 (new pid: 295)
worker 1 lifetime reached, it was running for 301 second(s)
worker 2 lifetime reached, it was running for 301 second(s)
worker 3 lifetime reached, it was running for 301 second(s)
worker 4 lifetime reached, it was running for 301 second(s)
worker 5 lifetime reached, it was running for 301 second(s)
worker 6 lifetime reached, it was running for 301 second(s)
Respawned uWSGI worker 1 (new pid: 296)
Respawned uWSGI worker 2 (new pid: 297)
Respawned uWSGI worker 3 (new pid: 298)
Respawned uWSGI worker 4 (new pid: 299)
Respawned uWSGI worker 5 (new pid: 300)
Respawned uWSGI worker 6 (new pid: 301)