2022/04/21 02:49:15 Starting Kubernetes Multicloud API
2022/04/21 02:49:15 Error Getting TLS Configuration. Starting without TLS...
10.233.69.131 - - [21/Apr/2022:04:55:54 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:04:56:44 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:04:57:41 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:04:58:45 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
127.0.0.1 - - [21/Apr/2022:04:58:52 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
10.233.69.131 - - [21/Apr/2022:04:59:00 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
127.0.0.1 - - [21/Apr/2022:04:59:07 +0000] "POST /v1/rb/definition HTTP/1.1" 201 278
10.233.69.131 - - [21/Apr/2022:04:59:43 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:00:36 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
127.0.0.1 - - [21/Apr/2022:05:01:26 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
10.233.69.131 - - [21/Apr/2022:05:01:39 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:03:44 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:06:29 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 500 74
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/connectivity-info HTTP/1.1" 201 7665
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/rb/definition HTTP/1.1" 201 125
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "GET /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1 HTTP/1.1" 200 125
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1/content HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "GET /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1 HTTP/1.1" 200 138
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1/profile HTTP/1.1" 201 308
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "GET /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1/profile/pnf-cnf-profile-21fa503f-3482-4bd6-9a85-d6f6de384a49 HTTP/1.1" 200 308
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/rb/definition/pnf-cnf-rb-aff60cda-1b9a-4d13-b9a4-b907592a8fcd/v1/profile/pnf-cnf-profile-21fa503f-3482-4bd6-9a85-d6f6de384a49/content HTTP/1.1" 200 0
2022/04/21 05:11:36 Error reading name cache from Database:  Error finding master table: mongo: no documents in result
2022/04/21 05:11:36 Error Reading from DB:  Reading cache from DB: Error finding master table: mongo: no documents in result
2022/04/21 05:11:37 Main rss info
2022/04/21 05:11:37   Path: /tmp/helm-tmpl-397272434/manifest-0
2022/04/21 05:11:37     Kind: Service
2022/04/21 05:11:37   Path: /tmp/helm-tmpl-397272434/manifest-1
2022/04/21 05:11:37     Kind: Service
2022/04/21 05:11:37   Path: /tmp/helm-tmpl-397272434/manifest-2
2022/04/21 05:11:37     Kind: Pod
2022/04/21 05:11:37   Path: /tmp/helm-tmpl-397272434/manifest-3
2022/04/21 05:11:37     Kind: Deployment
2022/04/21 05:11:37 Crd rss info
2022/04/21 05:11:37 Hook info
{"filepath":"/tmp/helm-tmpl-397272434/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:11:37Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator-mongo","time":"2022-04-21T05:11:37Z"}
{"filepath":"/tmp/helm-tmpl-397272434/manifest-1","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:11:37Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator","time":"2022-04-21T05:11:37Z"}
{"filepath":"/tmp/helm-tmpl-397272434/manifest-2","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:11:37Z"}
2022/04/21 05:11:37 No plugin for kind Pod found. Using generic Plugin
2022/04/21 05:11:37 Error converting template to map
{"gvk":{"Group":"","Version":"v1","Kind":"Pod"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator","time":"2022-04-21T05:11:37Z"}
{"filepath":"/tmp/helm-tmpl-397272434/manifest-3","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:11:37Z"}
2022/04/21 05:11:37 No plugin for kind Deployment found. Using generic Plugin
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"info","msg":"Created Kubernetes Resource","resource":"pnf-macro-test-simulator-mongo","time":"2022-04-21T05:11:37Z"}
10.233.69.131 - - [21/Apr/2022:05:11:36 +0000] "POST /v1/instance HTTP/1.1" 201 729
10.233.69.131 - - [21/Apr/2022:05:11:49 +0000] "GET /v1/healthcheck HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:14:54 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
127.0.0.1 - - [21/Apr/2022:05:15:10 +0000] "POST /v1/rb/definition HTTP/1.1" 201 265
127.0.0.1 - - [21/Apr/2022:05:15:10 +0000] "POST /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/content HTTP/1.1" 200 0
127.0.0.1 - - [21/Apr/2022:05:15:28 +0000] "POST /v1/rb/definition HTTP/1.1" 201 270
127.0.0.1 - - [21/Apr/2022:05:15:46 +0000] "POST /v1/rb/definition HTTP/1.1" 201 267
127.0.0.1 - - [21/Apr/2022:05:16:00 +0000] "POST /v1/rb/definition HTTP/1.1" 201 272
10.233.69.131 - - [21/Apr/2022:05:22:02 +0000] "GET /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417 HTTP/1.1" 200 270
10.233.69.131 - - [21/Apr/2022:05:22:02 +0000] "GET /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile/cnftest HTTP/1.1" 404 87
10.233.69.131 - - [21/Apr/2022:05:22:02 +0000] "POST /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile HTTP/1.1" 201 240
10.233.69.131 - - [21/Apr/2022:05:22:02 +0000] "GET /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile/cnftest HTTP/1.1" 200 240
10.233.69.131 - - [21/Apr/2022:05:22:02 +0000] "POST /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile/cnftest/content HTTP/1.1" 200 0
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"","workload_status":"GET_COMPLETE","workload_status_reason":{"stacks":[]}},"time":"2022-04-21T05:22:17Z"}
10.233.69.131 - - [21/Apr/2022:05:22:17 +0000] "GET /basicnf-owner/k8sregion/infra_workload?name=basic_cnf-f77932e2-ae39-4050-b29e-a07cd4dd15de HTTP/1.1" 200 114
{"Body":{},"Payload":{"generic-vnf-id":"22d42735-56cb-466d-9499-97b9e7296763","vf-module-id":"fd197e30-4a30-4dd3-88cb-f22fff35cc29","vf-module-model-invariant-id":"6ab0ad23-b242-408c-b4b0-c9e396ac7543","vf-module-model-version-id":"dbb0d49f-23f0-4020-ab6e-236c2abaeadf","vf-module-model-customization-id":"56c2d028-2690-42bd-8a52-b4eeadbd8417","oof_directives":{"attributes":null},"sdnc_directives":{"attributes":[{"attribute_name":"k8s-rb-profile-name","attribute_value":"cnftest"},{"attribute_name":"k8s-rb-profile-namespace","attribute_value":"k8s"},{"attribute_name":"availability_zone_0","attribute_value":"basicnf-availability-zone"},{"attribute_name":"dummy_name_0","attribute_value":"dummy_name"}]},"user_directives":{"attributes":null},"template_type":"","template_data":{"stack_name":"basic_cnf-f77932e2-ae39-4050-b29e-a07cd4dd15de"}},"level":"info","msg":"Broker API Payload","time":"2022-04-21T05:22:18Z"}
{"level":"info","msg":"Instance API Payload","payload":{"rb-name":"6ab0ad23-b242-408c-b4b0-c9e396ac7543","rb-version":"56c2d028-2690-42bd-8a52-b4eeadbd8417","profile-name":"cnftest","release-name":"fd197e30-4a30-4dd3-88cb-f22fff35cc29","cloud-region":"k8sregion","labels":{"stack-name":"basic_cnf-f77932e2-ae39-4050-b29e-a07cd4dd15de"},"override-values":{"availability_zone_0":"basicnf-availability-zone","dummy_name_0":"dummy_name","k8s-rb-profile-name":"cnftest","k8s-rb-profile-namespace":"k8s"}},"time":"2022-04-21T05:22:18Z"}
2022/04/21 05:22:18 Main rss info
2022/04/21 05:22:18   Path: /tmp/helm-tmpl-864991202/manifest-0
2022/04/21 05:22:18     Kind: Service
2022/04/21 05:22:18   Path: /tmp/helm-tmpl-864991202/manifest-1
2022/04/21 05:22:18     Kind: Deployment
2022/04/21 05:22:18 Crd rss info
2022/04/21 05:22:18 Hook info
{"level":"info","msg":"Creating Namespace","namespace":"k8s","time":"2022-04-21T05:22:18Z"}
2022/04/21 05:22:18 Namespace (k8s) created
{"filepath":"/tmp/helm-tmpl-864991202/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:22:18Z"}
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"info","msg":"Created Kubernetes Resource","resource":"cnftest-nginx","time":"2022-04-21T05:22:18Z"}
{"filepath":"/tmp/helm-tmpl-864991202/manifest-1","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:22:18Z"}
2022/04/21 05:22:18 No plugin for kind Deployment found. Using generic Plugin
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"info","msg":"Created Kubernetes Resource","resource":"cnftest-nginx","time":"2022-04-21T05:22:18Z"}
10.233.69.131 - - [21/Apr/2022:05:22:18 +0000] "POST /basicnf-owner/k8sregion/infra_workload HTTP/1.1" 201 307
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"practical_goldberg","template_response":[{"GVK":{"Group":"","Version":"v1","Kind":"Service"},"Name":"cnftest-nginx"},{"GVK":{"Group":"apps","Version":"v1","Kind":"Deployment"},"Name":"cnftest-nginx"}],"workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-04-21T05:22:19Z"}
10.233.69.131 - - [21/Apr/2022:05:22:19 +0000] "GET /basicnf-owner/k8sregion/infra_workload/practical_goldberg HTTP/1.1" 200 126
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"practical_goldberg","workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-04-21T05:22:19Z"}
10.233.69.131 - - [21/Apr/2022:05:22:19 +0000] "POST /basicnf-owner/k8sregion/infra_workload/practical_goldberg HTTP/1.1" 405 0
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"practical_goldberg","workload_status":"CREATE_COMPLETE","workload_status_reason":null},"time":"2022-04-21T05:22:48Z"}
10.233.69.131 - - [21/Apr/2022:05:22:48 +0000] "GET /basicnf-owner/k8sregion/infra_workload/practical_goldberg HTTP/1.1" 200 126
2022/04/21 05:22:49 [Config Cleanup] Instance practical_goldberg
2022/04/21 05:22:49 [scheduleResources]: START thread
2022/04/21 05:22:54 Stop config thread for practical_goldberg
{"gvk":{"Group":"apps","Version":"v1","Kind":"Deployment"},"level":"warning","msg":"Deleting Resource","resource":"cnftest-nginx","time":"2022-04-21T05:22:54Z"}
2022/04/21 05:22:54 No plugin for kind Deployment found. Using generic Plugin
2022/04/21 05:22:54 [scheduleResources]: STOP thread
{"gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"warning","msg":"Deleting Resource","resource":"cnftest-nginx","time":"2022-04-21T05:22:54Z"}
2022/04/21 05:22:54 Deleting service: cnftest-nginx
2022/04/21 05:22:54 Deleted Obj with ID ObjectID("6260ea0aa56721265a6b8aaf")
2022/04/21 05:22:54 Deleted Obj with ID ObjectID("6260ea0abda24210623fafdb")
{"level":"info","msg":"Broker API Response","response":{"template_type":"heat","workload_id":"practical_goldberg","workload_status":"DELETE_COMPLETE","workload_status_reason":null},"time":"2022-04-21T05:22:54Z"}
10.233.69.131 - - [21/Apr/2022:05:22:49 +0000] "DELETE /basicnf-owner/k8sregion/infra_workload/practical_goldberg HTTP/1.1" 202 126
10.233.69.131 - - [21/Apr/2022:05:23:06 +0000] "GET /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417 HTTP/1.1" 200 270
10.233.69.131 - - [21/Apr/2022:05:23:06 +0000] "GET /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile/cnftest HTTP/1.1" 200 240
2022/04/21 05:23:06 Deleted Obj with ID ObjectID("6260e9faa56721265a6b8aad")
2022/04/21 05:23:06 Deleted Obj with ID ObjectID("6260e9faa56721265a6b8aae")
10.233.69.131 - - [21/Apr/2022:05:23:06 +0000] "DELETE /v1/rb/definition/6ab0ad23-b242-408c-b4b0-c9e396ac7543/56c2d028-2690-42bd-8a52-b4eeadbd8417/profile/cnftest HTTP/1.1" 204 0
2022/04/21 05:23:06 Deleted Obj with ID ObjectID("6260e9f9bda24210623fafb7")
10.233.69.131 - - [21/Apr/2022:05:23:38 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
2022/04/21 05:23:38 Deleted Obj with ID ObjectID("6260e788a56721265a6b8a95")
2022/04/21 05:23:38 Deleted Obj with ID ObjectID("6260e788bda24210623fadaa")
10.233.69.131 - - [21/Apr/2022:05:23:38 +0000] "DELETE /v1/connectivity-info/k8sregion HTTP/1.1" 204 0
10.233.69.131 - - [21/Apr/2022:05:26:02 +0000] "DELETE /v1/instance/optimistic_saha HTTP/1.1" 500 177
{"error":"Getting CloudRegion Information: Get kubeconfig file: Downloading kubeconfig: Getting Connection info: Get Connection: Error finding master table: mongo: no documents in result","level":"error","msg":"Error Deleting Instance","time":"2022-04-21T05:26:02Z"}
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 500 74
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "POST /v1/connectivity-info HTTP/1.1" 201 7665
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "GET /v1/connectivity-info/k8sregion HTTP/1.1" 200 7665
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "POST /v1/rb/definition HTTP/1.1" 201 125
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "GET /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1 HTTP/1.1" 200 125
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "POST /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1/content HTTP/1.1" 200 0
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "GET /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1 HTTP/1.1" 200 138
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "POST /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1/profile HTTP/1.1" 201 308
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "GET /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1/profile/pnf-cnf-profile-4e28e600-e38e-4953-8676-a3c8f64cbfaf HTTP/1.1" 200 308
10.233.69.131 - - [21/Apr/2022:05:29:00 +0000] "POST /v1/rb/definition/pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f/v1/profile/pnf-cnf-profile-4e28e600-e38e-4953-8676-a3c8f64cbfaf/content HTTP/1.1" 200 0
2022/04/21 05:29:01 Main rss info
2022/04/21 05:29:01   Path: /tmp/helm-tmpl-423059337/manifest-0
2022/04/21 05:29:01     Kind: Service
2022/04/21 05:29:01   Path: /tmp/helm-tmpl-423059337/manifest-1
2022/04/21 05:29:01     Kind: Service
2022/04/21 05:29:01   Path: /tmp/helm-tmpl-423059337/manifest-2
2022/04/21 05:29:01     Kind: Pod
2022/04/21 05:29:01   Path: /tmp/helm-tmpl-423059337/manifest-3
2022/04/21 05:29:01     Kind: Deployment
2022/04/21 05:29:01 Crd rss info
2022/04/21 05:29:01 Hook info
{"filepath":"/tmp/helm-tmpl-423059337/manifest-0","level":"info","msg":"Processing Kubernetes Resource","time":"2022-04-21T05:29:01Z"}
{"error":"Create Service error: services \"pnf-macro-test-simulator-mongo\" already exists","filepath":"/tmp/helm-tmpl-423059337/manifest-0","gvk":{"Group":"","Version":"v1","Kind":"Service"},"level":"error","msg":"Error Creating Resource","time":"2022-04-21T05:29:01Z"}
2022/04/21 05:29:01   Instance: romantic_dewdney, Main rss are failed, skip post-install and remove instance in DB
2022/04/21 05:29:01 Deleted Obj with ID ObjectID("6260eb9da56721265a6b8ab8")
2022/04/21 05:29:01 Deleted Obj with ID ObjectID("6260eb9cbda24210623fb13c")
{"error":"Create Kubernetes Resources: Error creating kind: /v1, Kind=Service: Error in plugin Service plugin: Create Service error: services \"pnf-macro-test-simulator-mongo\" already exists","level":"error","msg":"Error Creating Resource","resource":{"rb-name":"pnf-cnf-rb-bca2b8aa-e979-4d1f-aed9-213d6eee521f","rb-version":"v1","profile-name":"pnf-cnf-profile-4e28e600-e38e-4953-8676-a3c8f64cbfaf","release-name":"","cloud-region":"k8sregion","labels":{},"override-values":{}},"time":"2022-04-21T05:29:01Z"}
10.233.69.131 - - [21/Apr/2022:05:29:01 +0000] "POST /v1/instance HTTP/1.1" 500 180