2022/03/22 10:27:57 Starting Kubernetes Multicloud API
2022/03/22 10:27:57 Error Getting TLS Configuration. Starting without TLS...
10.233.68.151 - - [22/Mar/2022:11:56:57 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
127.0.0.1 - - [22/Mar/2022:12:01:33 +0000] "POST /v1/rb/definition HTTP/1.1" 201 278
127.0.0.1 - - [22/Mar/2022:12:05:11 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
127.0.0.1 - - [22/Mar/2022:12:07:34 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
10.233.68.151 - - [22/Mar/2022:12:14:40 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 500 74
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/connectivity-info HTTP/1.1" 201 7665
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/rb/definition HTTP/1.1" 201 125
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "GET /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1 HTTP/1.1" 200 125
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1/content HTTP/1.1" 200 0
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "GET /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1 HTTP/1.1" 200 138
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1/profile HTTP/1.1" 201 308
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "GET /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1/profile/pnf-cnf-profile-9d3ad339-bd8a-4aa3-9574-86eda2ab8401 HTTP/1.1" 200 308
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/rb/definition/pnf-cnf-rb-c4ec8c90-1dbe-4032-932e-187c60675fc8/v1/profile/pnf-cnf-profile-9d3ad339-bd8a-4aa3-9574-86eda2ab8401/content HTTP/1.1" 200 0
2022/03/22 12:14:41 Error reading name cache from Database:  Error finding master table: mongo: no documents in result
2022/03/22 12:14:41 Error Reading from DB:  Reading cache from DB: Error finding master table: mongo: no documents in result
2022/03/22 12:14:41 Main rss info
2022/03/22 12:14:41   Path: /tmp/helm-tmpl-837970797/manifest-0
2022/03/22 12:14:41     Kind: Service
2022/03/22 12:14:41   Path: /tmp/helm-tmpl-837970797/manifest-1
2022/03/22 12:14:41     Kind: Service
2022/03/22 12:14:41   Path: /tmp/helm-tmpl-837970797/manifest-2
2022/03/22 12:14:41     Kind: Pod
2022/03/22 12:14:41   Path: /tmp/helm-tmpl-837970797/manifest-3
2022/03/22 12:14:41     Kind: Deployment
2022/03/22 12:14:41 Crd rss info
2022/03/22 12:14:41 Hook info
{"filepath":"/tmp/helm-tmpl-837970797/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:14:41Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator-mongo","time":"2022-03-22T12:14:41Z"}
{"filepath":"/tmp/helm-tmpl-837970797/manifest-1","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:14:41Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator","time":"2022-03-22T12:14:41Z"}
{"filepath":"/tmp/helm-tmpl-837970797/manifest-2","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:14:41Z"}
2022/03/22 12:14:41 No plugin for kind Pod found. Using generic Plugin
2022/03/22 12:14:41 Error converting template to map
{"gvk":{"Group":"","Version":"v1","Kind":"Pod"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator","time":"2022-03-22T12:14:41Z"}
{"filepath":"/tmp/helm-tmpl-837970797/manifest-3","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:14:41Z"}
2022/03/22 12:14:41 No plugin for kind Deployment found. Using generic Plugin
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator-mongo","time":"2022-03-22T12:14:41Z"}
10.233.68.151 - - [22/Mar/2022:12:14:41 +0000] "POST /v1/instance HTTP/1.1" 201 729
10.233.68.151 - - [22/Mar/2022:12:18:55 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
127.0.0.1 - - [22/Mar/2022:12:18:58 +0000] "POST /v1/rb/definition HTTP/1.1" 201 267
127.0.0.1 - - [22/Mar/2022:12:19:16 +0000] "POST /v1/rb/definition HTTP/1.1" 201 265
127.0.0.1 - - [22/Mar/2022:12:19:16 +0000] "POST /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/content HTTP/1.1" 200 0
127.0.0.1 - - [22/Mar/2022:12:19:32 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
127.0.0.1 - - [22/Mar/2022:12:19:51 +0000] "POST /v1/rb/definition HTTP/1.1" 201 272
10.233.68.151 - - [22/Mar/2022:12:26:49 +0000] "GET /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa HTTP/1.1" 200 270
10.233.68.151 - - [22/Mar/2022:12:26:50 +0000] "GET /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile/cnftest HTTP/1.1" 404 87
10.233.68.151 - - [22/Mar/2022:12:26:50 +0000] "POST /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile HTTP/1.1" 201 240
10.233.68.151 - - [22/Mar/2022:12:26:50 +0000] "GET /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile/cnftest HTTP/1.1" 200 240
10.233.68.151 - - [22/Mar/2022:12:26:50 +0000] "POST /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile/cnftest/content HTTP/1.1" 200 0
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"","workload_status":"GET_COMPLETE","workload_status_reason":{"stacks":[]}},"time":"2022-03-22T12:27:10Z"}
10.233.68.151 - - [22/Mar/2022:12:27:10 +0000] "GET /basicnf-owner/k8sregion/infra_workload?name=basic_cnf-82b4f6fc-e704-4aa4-944d-d7674c26ec30 HTTP/1.1" 200 114
{"Body":{},"Payload":{"generic-vnf-id":"1b2e9285-9cd4-4322-8c1f-dfd44425ff32","vf-module-id":"cb011c0d-ef6d-4663-b937-131caf6ec37a","vf-module-model-invariant-id":"c8727e24-961c-4dac-8f4b-e1ebe8458628","vf-module-model-version-id":"0da529be-c8ac-4bda-8efd-3f1e04c627cd","vf-module-model-customization-id":"5e899c0f-c0ff-46b3-bcd8-7620429011aa","oof_directives":{"attributes":null},"sdnc_directives":{"attributes":[{"attribute_name":"k8s-rb-profile-name","attribute_value":"cnftest"},{"attribute_name":"k8s-rb-profile-namespace","attribute_value":"k8s"},{"attribute_name":"availability_zone_0","attribute_value":"basicnf-availability-zone"},{"attribute_name":"dummy_name_0","attribute_value":"dummy_name"}]},"user_directives":{"attributes":null},"template_type":"","template_data":{"stack_name":"basic_cnf-82b4f6fc-e704-4aa4-944d-d7674c26ec30"}},"level":"info","msg":"Broker API Payload","time":"2022-03-22T12:27:11Z"}
{"level":"info","msg":"Instance API Payload","payload":{"rb-name":"c8727e24-961c-4dac-8f4b-e1ebe8458628","rb-version":"5e899c0f-c0ff-46b3-bcd8-7620429011aa","profile-name":"cnftest","release-name":"cb011c0d-ef6d-4663-b937-131caf6ec37a","cloud-region":"k8sregion","labels":{"stack-name":"basic_cnf-82b4f6fc-e704-4aa4-944d-d7674c26ec30"},"override-values":{"availability_zone_0":"basicnf-availability-zone","dummy_name_0":"dummy_name","k8s-rb-profile-name":"cnftest","k8s-rb-profile-namespace":"k8s"}},"time":"2022-03-22T12:27:11Z"}
2022/03/22 12:27:11 Main rss info
2022/03/22 12:27:11   Path: /tmp/helm-tmpl-148472281/manifest-0
2022/03/22 12:27:11     Kind: Service
2022/03/22 12:27:11   Path: /tmp/helm-tmpl-148472281/manifest-1
2022/03/22 12:27:11     Kind: Deployment
2022/03/22 12:27:11 Crd rss info
2022/03/22 12:27:11 Hook info
{"level":"info","msg":"Creating Namespace","namespace":"k8s","time":"2022-03-22T12:27:11Z"}
2022/03/22 12:27:11 Namespace (k8s) created
{"filepath":"/tmp/helm-tmpl-148472281/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:27:11Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"cnftest-nginx","time":"2022-03-22T12:27:11Z"}
{"filepath":"/tmp/helm-tmpl-148472281/manifest-1","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:27:11Z"}
2022/03/22 12:27:11 No plugin for kind Deployment found. Using generic Plugin
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"info","msg":"Created Kubernetes Resource","resource":"cnftest-nginx","time":"2022-03-22T12:27:11Z"}
10.233.68.151 - - [22/Mar/2022:12:27:11 +0000] "POST /basicnf-owner/k8sregion/infra_workload HTTP/1.1" 201 304
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"suspicious_shaw","template_response":[{"GVK":{"Group":"","Version":"v1","Kind":"Service"},"Name":"cnftest-nginx"},{"GVK":{"Group":"apps","Version":"v1","Kind":"Deployment"},"Name":"cnftest-nginx"}],"workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-03-22T12:27:11Z"}
10.233.68.151 - - [22/Mar/2022:12:27:11 +0000] "GET /basicnf-owner/k8sregion/infra_workload/suspicious_shaw HTTP/1.1" 200 123
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"suspicious_shaw","workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-03-22T12:27:11Z"}
10.233.68.151 - - [22/Mar/2022:12:27:12 +0000] "POST /basicnf-owner/k8sregion/infra_workload/suspicious_shaw HTTP/1.1" 405 0
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"suspicious_shaw","workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-03-22T12:27:52Z"}
10.233.68.151 - - [22/Mar/2022:12:27:52 +0000] "GET /basicnf-owner/k8sregion/infra_workload/suspicious_shaw HTTP/1.1" 200 123
2022/03/22 12:27:52 [Config Cleanup] Instance suspicious_shaw
2022/03/22 12:27:52 [scheduleResources]: START thread
2022/03/22 12:27:57 Stop config thread for suspicious_shaw
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"warning","msg":"Deleting Resource","resource":"cnftest-nginx","time":"2022-03-22T12:27:57Z"}
2022/03/22 12:27:57 No plugin for kind Deployment found. Using generic Plugin
2022/03/22 12:27:57 [scheduleResources]: STOP thread
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"warning","msg":"Deleting Resource","resource":"cnftest-nginx","time":"2022-03-22T12:27:57Z"}
2022/03/22 12:27:57 Deleting service: cnftest-nginx
2022/03/22 12:27:57 Deleted Obj with ID ObjectID("6239c09fb2de3ac935b8757b")
2022/03/22 12:27:57 Deleted Obj with ID ObjectID("6239c09f8e828e72f04a61d2")
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"suspicious_shaw","workload_status":"DELETE_COMPLETE","workload_status_reason":null},"time":"2022-03-22T12:27:57Z"}
10.233.68.151 - - [22/Mar/2022:12:27:52 +0000] "DELETE /basicnf-owner/k8sregion/infra_workload/suspicious_shaw HTTP/1.1" 202 123
10.233.68.151 - - [22/Mar/2022:12:28:15 +0000] "GET /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa HTTP/1.1" 200 270
10.233.68.151 - - [22/Mar/2022:12:28:15 +0000] "GET /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile/cnftest HTTP/1.1" 200 240
2022/03/22 12:28:15 Deleted Obj with ID ObjectID("6239c08ab2de3ac935b87579")
2022/03/22 12:28:15 Deleted Obj with ID ObjectID("6239c08ab2de3ac935b8757a")
2022/03/22 12:28:15 Deleted Obj with ID ObjectID("6239c08a8e828e72f04a61a7")
10.233.68.151 - - [22/Mar/2022:12:28:15 +0000] "DELETE /v1/rb/definition/c8727e24-961c-4dac-8f4b-e1ebe8458628/5e899c0f-c0ff-46b3-bcd8-7620429011aa/profile/cnftest HTTP/1.1" 204 0
10.233.68.151 - - [22/Mar/2022:12:28:58 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
2022/03/22 12:28:58 Deleted Obj with ID ObjectID("6239bdb1b2de3ac935b87561")
2022/03/22 12:28:58 Deleted Obj with ID ObjectID("6239bdb18e828e72f04a5f5e")
10.233.68.151 - - [22/Mar/2022:12:28:58 +0000] "DELETE /v1/connectivity-info/k8sregion HTTP/1.1" 204 0
10.233.68.151 - - [22/Mar/2022:12:29:27 +0000] "DELETE /v1/instance/dazzling_mendel HTTP/1.1" 500 177
{"error":"Getting CloudRegion Information: Get kubeconfig file: Downloading kubeconfig: Getting Connection info: Get Connection: Error finding master table: mongo: no documents in result","level":"error","msg":"Error Deleting Instance","time":"2022-03-22T12:29:27Z"}
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 500 74
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/connectivity-info HTTP/1.1" 201 7665
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/rb/definition HTTP/1.1" 201 125
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "GET /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1 HTTP/1.1" 200 125
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1/content HTTP/1.1" 200 0
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "GET /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1 HTTP/1.1" 200 138
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1/profile HTTP/1.1" 201 308
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "GET /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1/profile/pnf-cnf-profile-1b2925dc-f4b8-474b-9508-82aa21ca0ea2 HTTP/1.1" 200 308
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/rb/definition/pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5/v1/profile/pnf-cnf-profile-1b2925dc-f4b8-474b-9508-82aa21ca0ea2/content HTTP/1.1" 200 0
2022/03/22 12:32:53 Main rss info
2022/03/22 12:32:53   Path: /tmp/helm-tmpl-826726432/manifest-0
2022/03/22 12:32:53     Kind: Service
2022/03/22 12:32:53   Path: /tmp/helm-tmpl-826726432/manifest-1
2022/03/22 12:32:53     Kind: Service
2022/03/22 12:32:53   Path: /tmp/helm-tmpl-826726432/manifest-2
2022/03/22 12:32:53     Kind: Pod
2022/03/22 12:32:53   Path: /tmp/helm-tmpl-826726432/manifest-3
2022/03/22 12:32:53     Kind: Deployment
2022/03/22 12:32:53 Crd rss info
2022/03/22 12:32:53 Hook info
{"filepath":"/tmp/helm-tmpl-826726432/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-03-22T12:32:53Z"}
{"error":"Create Service error: services \"pnf-macro-test-simulator-mongo\" already exists","filepath":"/tmp/helm-tmpl-826726432/manifest-0","gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"error","msg":"Error Creating Resource","time":"2022-03-22T12:32:53Z"}
2022/03/22 12:32:53   Instance: distracted_mestorf, Main rss are failed, skip post-install and remove instance in DB
2022/03/22 12:32:53 Deleted Obj with ID ObjectID("6239c1f5b2de3ac935b87584")
2022/03/22 12:32:53 Deleted Obj with ID ObjectID("6239c1f58e828e72f04a6311")
{"error":"Create Kubernetes Resources: Error creating kind: /v1, Kind=Service: Error in plugin Service plugin: Create Service error: services \"pnf-macro-test-simulator-mongo\" already exists","level":"error","msg":"Error Creating Resource","resource":{"rb-name":"pnf-cnf-rb-81ecec39-f9dd-4c6a-ab28-4200d59018d5","rb-version":"v1","profile-name":"pnf-cnf-profile-1b2925dc-f4b8-474b-9508-82aa21ca0ea2","release-name":"","cloud-region":"k8sregion","labels":{},"override-values":{}},"time":"2022-03-22T12:32:53Z"}
10.233.68.151 - - [22/Mar/2022:12:32:53 +0000] "POST /v1/instance HTTP/1.1" 500 180