Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenShift: Creating and deleting of TransportServers with F5 CIS Operator Version 2.18.1-ubi9 not working #3660

Closed
keennessch opened this issue Nov 27, 2024 · 7 comments
Closed

OpenShift: Creating and deleting of TransportServers with F5 CIS Operator Version 2.18.1-ubi9 not working #3660

keennessch opened this issue Nov 27, 2024 · 7 comments
Labels
bug JIRA

Comments

@keennessch
Copy link

Setup Details

CIS Version : 2.18.1
Build: registry.connect.redhat.com/f5networks/cntr-ingress-svcs:2.18.1-ubi9
BIGIP Version: BIG-IP 17.1.1.3 Build 0.21.5 Engineering Hotfix
AS3 Version: 3.48.0
Agent Mode: AS3
Orchestration: OSCP
Orchestration Version: 4.16.17
Pool Mode: Cluster
Additional Setup details:

Description

After updating the F5 CIS Operator to 2.18.1-ubi9, existing TransportServer resources change to Pending status. New TransportServers immediately show Pending status, stay in this state and are never created on the load balancer. Deleting TransportServer resources are deleted from the OpenShift cluster but not from the load balancer.
Switching the Operator's version back to 2.18.0-ubi9 by defining it in the .spec.version field of the F5BigIpCtlr resource, TransportServer resources change to Ok status after a couple of seconds. Creating and deleting TransportServers works as expected and have the desired effect on the load balancer.

Steps To Reproduce

  1. Change the .spec.version field to 2.18.1-ubi9 in the F5BigIpCtlr resource on the OCP cluster
  2. Wait for the new f5-bigip-ctlr pod to successfully start

Expected Result

TransportServer resources show the correct state corresponding to their virtual server counterparts on the load balancer.
New TransportServer resources can be created and show up as virtual servers on the load balancer.
Existing TransportServer resources can be deleted and are deleted on the load balancer.

Actual Result

TransportServer resources show Pending state, regardless of their actual state on the load balancer.
New TransportServer resources can be created on the OCP cluster but are not created on the load balancer.
Existing TransportServer resources can be deleted on the OCP cluster but are not deleted on the load balancer.

Diagnostic Information

The used F5BigIpCtlr definition while collecting logs for this issue:

apiVersion: cis.f5.com/v1
kind: F5BigIpCtlr
metadata:
  annotations:
    argocd.argoproj.io/tracking-id: in-cluster-f5-cis-instance:cis.f5.com/F5BigIpCtlr:f5-cis/f5-bigip-ctlr
    operator-sdk/primary-resource: /f5-bigip-ctlr
    operator-sdk/primary-resource-type: ClusterRoleBinding.rbac.authorization.k8s.io
  creationTimestamp: "2024-10-29T13:05:16Z"
  finalizers:
  - helm.sdk.operatorframework.io/uninstall-release
  name: f5-bigip-ctlr
  namespace: f5-cis
spec:
  args:
    agent: as3
    as3_validation: true
    bigip_partition: <REDACTED-PARTITION>
    bigip_url: <REDACTED-URL>
    custom_resource_mode: true
    default_route_domain: <REDACTED-ROUTE-DOMAIN>
    http_client_metrics: true
    insecure: false
    log_as3_response: true
    log_level: AS3DEBUG
    orchestration_cni: ovn-k8s
    pool_member_type: cluster
    static_routing_mode: true
    trusted-certs-cfgmap: f5-cis/user-ca-bundle
  bigip_login_secret: f5-bigip-ctlr-login
  image:
    pullPolicy: Always
    repo: f5networks/cntr-ingress-svcs
    user: registry.connect.redhat.com
  ingressClass:
    create: false
    defaultController: false
    ingressClassName: f5
  namespace: f5-cis
  podSecurityContext:
    allowPrivilegeEscalation: false
    capabilities:
      drop:
      - ALL
    runAsGroup: null
    runAsNonRoot: true
    runAsUser: null
  rbac:
    create: true
  securityContext:
    fsGroup: 1000
    runAsGroup: 1000
    runAsUser: 1000
  serviceAccount:
    create: true
    name: f5-bigip-ctrl-sa
  version: 2.18.1-ubi9
status:
  conditions:
  - lastTransitionTime: "2024-10-29T13:05:33Z"
    status: "True"
    type: Initialized
  - lastTransitionTime: "2024-10-29T13:05:35Z"
    message: |+
      Container Ingress Services controller: f5-bigip-ctlr

      Controller Documentation:
      - Kubernetes: https://clouddocs.f5.com/containers/latest/userguide/kubernetes/
      - OpenShift: https://clouddocs.f5.com/containers/latest/userguide/openshift/

    reason: UpgradeSuccessful
    status: "True"
    type: Deployed
  deployedRelease:
    manifest: ...
    name: f5-bigip-ctlr

Logs from the f5-bigip-ctlr pod. Repeating lines have been replaced by [...].

2024/11/25 10:44:42 [DEBUG] No url in credentials directory, falling back to CLI argument
2024/11/25 10:44:42 [INFO] [INIT] Starting: Container Ingress Services - Version: 2.18.1, BuildInfo: azure-6775-e3c0ec4ef25fc2fd3b994c05b63317aed2662622
2024/11/25 10:44:42 [DEBUG] [DEBUG] Adding https at the beginning of the GTM BIG IP URL as it does not start with https.
2024/11/25 10:44:43 [DEBUG] [BIGIP] Http client instrumented with metrics!
2024/11/25 10:44:43 [INFO] ConfigWriter started: 0xc0003345d0
2024/11/25 10:44:43 [WARNING] Creating GTM with default bigip credentials as GTM BIGIP Url or GTM BIGIP Username or GTM BIGIP Password is missing on CIS args.
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) writing section name global
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) successfully wrote section (global)
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) writing section name bigip
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) successfully wrote section (bigip)
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) writing section name gtm_bigip
2024/11/25 10:44:43 [DEBUG] [CCCL] ConfigWriter (0xc0003345d0) successfully wrote section (gtm_bigip)
2024/11/25 10:44:43 [INFO] Started config driver sub-process at pid: 17
2024/11/25 10:44:43 [DEBUG] [AS3] posting GET BIGIP AS3 Version request on https://<REDACTED-URL>/mgmt/shared/appsvcs/info
2024/11/25 10:44:43 [INFO] [CORE] Registered BigIP Metrics
2024/11/25 10:44:44 [DEBUG] /usr/local/bin/bigipconfigdriver.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
2024/11/25 10:44:44 [DEBUG]   __import__('pkg_resources').require('f5-ctlr-agent==0.1.0')
2024/11/25 10:44:46 [DEBUG] [AS3] BIGIP is serving with AS3 version: 3.48.0
2024/11/25 10:44:46 [DEBUG] Controller Created
2024/11/25 10:44:46 [DEBUG] Client Created
2024/11/25 10:44:46 [DEBUG] No namespaces provided. Watching all namespaces
2024/11/25 10:44:46 [DEBUG] Creating Common Resource Informers for Namespace: 
2024/11/25 10:44:46 [DEBUG] Creating Custom Resource Informers for Namespace: 
2024/11/25 10:44:46 [DEBUG] [AS3] Posting GET BIGIP Reg Key request on https://<REDACTED-URL>/mgmt/tm/shared/licensing/registration
2024/11/25 10:44:46 [INFO] Starting Controller
2024/11/25 10:44:46 [INFO] Starting  Node Informer
I1125 10:44:46.716073       1 shared_informer.go:240] Waiting for caches to sync for F5 CIS Ingress Controller
2024/11/25 10:44:46 [DEBUG]  Initialising controller monitored kubernetes nodes from cluster: local
[...]
2024/11/25 10:44:47 [DEBUG]  Initialising controller monitored kubernetes nodes from cluster: local
I1125 10:44:47.016185       1 shared_informer.go:247] Caches are synced for F5 CIS Ingress Controller 
2024/11/25 10:44:47 [DEBUG] Successfully synced node informer cache
2024/11/25 10:44:47 [INFO] Starting ExternalDNS Informer
I1125 10:44:47.016232       1 shared_informer.go:240] Waiting for caches to sync for F5 CIS Ingress Controller
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
2024/11/25 10:44:48 [DEBUG] Enqueueing Endpoints: <REDACTED>
[...]
2024/11/25 10:44:48 [DEBUG] Enqueueing Endpoints: <REDACTED>
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
* Port 8443 provides access to the OpenShift resource metrics. This port is for internal use, and no other usage is guaranteed.
* Port 9443 provides access to the internal `openshift-state-metrics` metrics. This port is for internal use, and no other usage is guaranteed. operator.openshift.io/spec-hash:700eac9efd8a924fdb3738d2a7935c5f8c8de046a1c3806df1ca3c92076a4ecc service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212 service.beta.openshift.io/serving-cert-secret-name:openshift-state-metrics-tls service.beta.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212] [] []  [{service-ca-operator Update v1 2024-04-25 12:24:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:service.alpha.openshift.io/serving-cert-signed-by":{},"f:service.beta.openshift.io/serving-cert-signed-by":{}}}}} {operator Update v1 2024-10-28 11:00:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/description":{},"f:operator.openshift.io/spec-hash":{},"f:service.beta.openshift.io/serving-cert-secret-name":{}},"f:labels":{".":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/part-of":{},"f:k8s-app":{}}},"f:spec":{"f:clusterIP":{},"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":8443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:type":{}}}}]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:https-main,Protocol:TCP,Port:8443,TargetPort:{1 0 https-main},NodePort:0,AppProtocol:nil,},ServicePort{Name:https-self,Protocol:TCP,Port:9443,TargetPort:{1 0 https-self},NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/component: exporter,app.kubernetes.io/name: openshift-state-metrics,},ClusterIP:None,Type:ClusterIP,ExternalIPs:[],SessionAffinity:None,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:nil,TopologyKeys:[],IPFamilyPolicy:*SingleStack,ClusterIPs:[None],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},} from cluster: local
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
* Port 9091 provides access to all the Prometheus endpoints. Granting access requires binding a user to the `cluster-monitoring-view` cluster role.
* Port 9092 provides access the `/metrics` and `/federate` endpoints only. This port is for internal use, and no other usage is guaranteed. operator.openshift.io/spec-hash:44db31cd73c886ad771ecc7e4731887da474737fa746e940e1abe3283798b76f service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212 service.beta.openshift.io/serving-cert-secret-name:prometheus-k8s-tls service.beta.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212] [] []  [{service-ca-operator Update v1 2024-04-25 12:24:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:service.alpha.openshift.io/serving-cert-signed-by":{},"f:service.beta.openshift.io/serving-cert-signed-by":{}}}}} {operator Update v1 2024-10-28 11:00:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/description":{},"f:operator.openshift.io/spec-hash":{},"f:service.beta.openshift.io/serving-cert-secret-name":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{}}},"f:spec":{"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":9091,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9092,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:sessionAffinityConfig":{".":{},"f:clientIP":{".":{},"f:timeoutSeconds":{}}},"f:type":{}}}}]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:web,Protocol:TCP,Port:9091,TargetPort:{1 0 web},NodePort:0,AppProtocol:nil,},ServicePort{Name:metrics,Protocol:TCP,Port:9092,TargetPort:{1 0 metrics},NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/component: prometheus,app.kubernetes.io/instance: k8s,app.kubernetes.io/name: prometheus,app.kubernetes.io/part-of: openshift-monitoring,},ClusterIP:<REDACTED-IP>,Type:ClusterIP,ExternalIPs:[],SessionAffinity:ClientIP,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:&SessionAffinityConfig{ClientIP:&ClientIPConfig{TimeoutSeconds:*10800,},},TopologyKeys:[],IPFamilyPolicy:*SingleStack,ClusterIPs:[<REDACTED-IP>],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},} from cluster: local
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
[...]
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
* Port 9091 provides access to all the Thanos Querier endpoints. Granting access requires binding a user to the `cluster-monitoring-view` cluster role.
* Port 9092 provides access to the `/api/v1/query`, `/api/v1/query_range/, `/api/v1/labels`, `/api/v1/label/*/values`, and `/api/v1/series` endpoints restricted to a given project. Granting access requires binding a user to the `view` cluster role in the project.
* Port 9093 provides access to the `/api/v1/alerts`, and `/api/v1/rules` endpoints restricted to a given project. Granting access requires binding a user to the `monitoring-rules-edit` cluster role or `monitoring-edit` cluster role or `monitoring-rules-view` cluster role in the project.
* Port 9094 provides access to the `/metrics` endpoint only. This port is for internal use, and no other usage is guaranteed. operator.openshift.io/spec-hash:0c29478f63d34bf0dd2a114c15b5518c2e967a61c11e61f2676c05a268c8b7f8 service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212 service.beta.openshift.io/serving-cert-secret-name:thanos-querier-tls service.beta.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212] [] []  [{service-ca-operator Update v1 2024-04-25 12:24:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:service.alpha.openshift.io/serving-cert-signed-by":{},"f:service.beta.openshift.io/serving-cert-signed-by":{}}}}} {operator Update v1 2024-10-28 11:00:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/description":{},"f:operator.openshift.io/spec-hash":{},"f:service.beta.openshift.io/serving-cert-secret-name":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{}}},"f:spec":{"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":9091,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9092,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9093,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9094,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:type":{}}}}]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:web,Protocol:TCP,Port:9091,TargetPort:{1 0 web},NodePort:0,AppProtocol:nil,},ServicePort{Name:tenancy,Protocol:TCP,Port:9092,TargetPort:{1 0 tenancy},NodePort:0,AppProtocol:nil,},ServicePort{Name:tenancy-rules,Protocol:TCP,Port:9093,TargetPort:{1 0 tenancy-rules},NodePort:0,AppProtocol:nil,},ServicePort{Name:metrics,Protocol:TCP,Port:9094,TargetPort:{1 0 metrics},NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/component: query-layer,app.kubernetes.io/instance: thanos-querier,app.kubernetes.io/name: thanos-query,app.kubernetes.io/part-of: openshift-monitoring,},ClusterIP:<REDACTED-IP>,Type:ClusterIP,ExternalIPs:[],SessionAffinity:None,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:nil,TopologyKeys:[],IPFamilyPolicy:*SingleStack,ClusterIPs:[<REDACTED-IP>],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},} from cluster: local
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
[...]
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
* Port 9091 provides access to the `/metrics` endpoint only. This port is for internal use, and no other usage is guaranteed.
* Port 9092 provides access to the `/federate` endpoint only. Granting access requires binding a user to the `cluster-monitoring-view` cluster role.

This also exposes the `/metrics` endpoint of the Thanos sidecar web server on port 10902. This port is for internal use, and no other usage is guaranteed. operator.openshift.io/spec-hash:5d24d54c21bc80e08e61fb22543775331c03d254b249a560e583818a38ccf404 service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212 service.beta.openshift.io/serving-cert-secret-name:prometheus-user-workload-tls service.beta.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212] [] []  [{service-ca-operator Update v1 2024-04-25 13:08:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:service.alpha.openshift.io/serving-cert-signed-by":{},"f:service.beta.openshift.io/serving-cert-signed-by":{}}}}} {operator Update v1 2024-10-28 11:00:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/description":{},"f:operator.openshift.io/spec-hash":{},"f:service.beta.openshift.io/serving-cert-secret-name":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{}}},"f:spec":{"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":9091,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9092,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":10902,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:sessionAffinityConfig":{".":{},"f:clientIP":{".":{},"f:timeoutSeconds":{}}},"f:type":{}}}}]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:metrics,Protocol:TCP,Port:9091,TargetPort:{1 0 metrics},NodePort:0,AppProtocol:nil,},ServicePort{Name:federate,Protocol:TCP,Port:9092,TargetPort:{1 0 federate},NodePort:0,AppProtocol:nil,},ServicePort{Name:thanos-proxy,Protocol:TCP,Port:10902,TargetPort:{1 0 thanos-proxy},NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/component: prometheus,app.kubernetes.io/instance: user-workload,app.kubernetes.io/name: prometheus,app.kubernetes.io/part-of: openshift-monitoring,},ClusterIP:<REDACTED-IP>,Type:ClusterIP,ExternalIPs:[],SessionAffinity:ClientIP,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:&SessionAffinityConfig{ClientIP:&ClientIPConfig{TimeoutSeconds:*10800,},},TopologyKeys:[],IPFamilyPolicy:*SingleStack,ClusterIPs:[<REDACTED-IP>],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},} from cluster: local
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
* Port 9091 provides access to all Thanos Ruler endpoints. Granting access requires binding a user to the `cluster-monitoring-view` cluster role.
* Port 9092 provides access to the `/metrics` endpoint only. This port is for internal use, and no other usage is guaranteed.

This also exposes the gRPC endpoints on port 10901. This port is for internal use, and no other usage is guaranteed. operator.openshift.io/spec-hash:958429f802736f0fb134982be126b4f7bc3ed0485008e5efc3040433d3dc36fc service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212 service.beta.openshift.io/serving-cert-secret-name:thanos-ruler-tls service.beta.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1714047212] [] []  [{service-ca-operator Update v1 2024-04-25 13:08:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:service.alpha.openshift.io/serving-cert-signed-by":{},"f:service.beta.openshift.io/serving-cert-signed-by":{}}}}} {operator Update v1 2024-10-28 11:00:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/description":{},"f:operator.openshift.io/spec-hash":{},"f:service.beta.openshift.io/serving-cert-secret-name":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{}}},"f:spec":{"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":9091,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9092,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":10901,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:sessionAffinityConfig":{".":{},"f:clientIP":{".":{},"f:timeoutSeconds":{}}},"f:type":{}}}}]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:web,Protocol:TCP,Port:9091,TargetPort:{1 0 web},NodePort:0,AppProtocol:nil,},ServicePort{Name:metrics,Protocol:TCP,Port:9092,TargetPort:{1 0 metrics},NodePort:0,AppProtocol:nil,},ServicePort{Name:grpc,Protocol:TCP,Port:10901,TargetPort:{1 0 grpc},NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/name: thanos-ruler,thanos-ruler: user-workload,},ClusterIP:<REDACTED-IP>,Type:ClusterIP,ExternalIPs:[],SessionAffinity:ClientIP,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:&SessionAffinityConfig{ClientIP:&ClientIPConfig{TimeoutSeconds:*10800,},},TopologyKeys:[],IPFamilyPolicy:*SingleStack,ClusterIPs:[<REDACTED-IP>],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},} from cluster: local
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
[...]
2024/11/25 10:44:48 [DEBUG] Enqueueing Service: <REDACTED>
2024/11/25 10:44:48 [DEBUG] Enqueueing Endpoints: <REDACTED>
[...]
2024/11/25 10:44:48 [DEBUG] Enqueueing Endpoints: <REDACTED>
2024/11/25 10:44:49 [DEBUG] [2024-11-25 10:44:49,618 icontrol.session DEBUG] get WITH uri: https://<REDACTED-URL>:443/mgmt/tm/sys/ AND suffix:  AND kwargs: {}
2024/11/25 10:44:49 [DEBUG] [2024-11-25 10:44:49,619 urllib3.connectionpool DEBUG] Starting new HTTPS connection (1): <REDACTED-URL>:443
2024/11/25 10:44:49 [DEBUG]  Initialising controller monitored kubernetes nodes from cluster: local
2024/11/25 10:44:50 [DEBUG]  Initialising controller monitored kubernetes nodes from cluster: local
2024/11/25 10:44:51 [DEBUG] [2024-11-25 10:44:51,101 urllib3.connectionpool DEBUG] https://<REDACTED-URL>:443 "POST /mgmt/shared/authn/login HTTP/1.1" 200 825
2024/11/25 10:44:51 [DEBUG] [2024-11-25 10:44:51,102 icontrol.authtoken DEBUG] Wait for 1 sec after login...
2024/11/25 10:44:52 [DEBUG] [2024-11-25 10:44:52,117 urllib3.connectionpool DEBUG] Starting new HTTPS connection (1): <REDACTED-URL>:443
2024/11/25 10:44:52 [DEBUG] [2024-11-25 10:44:52,738 urllib3.connectionpool DEBUG] https://<REDACTED-URL>:443 "GET /mgmt/tm/sys/ HTTP/1.1" 200 4136
2024/11/25 10:44:52 [DEBUG] [2024-11-25 10:44:52,739 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json;charset=utf-8 Content-Encoding: None Text: '{"kind":"tm:sys:syscollectionstate","selfLink":"https://localhost/mgmt/tm/sys?ver=17.1.1.3","items":[{"reference":{"link":"https://localhost/mgmt/tm/sys/application?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/crypto?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/daemon-log-settings?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/diags?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/disk?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/dynad?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/ecm?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/file?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/fpga?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/icall?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/ipfix?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/log-config?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/pfman?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/sflow?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/software?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/turboflex?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/url-db?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/aom?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/autoscale-group?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/cluster?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/config?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/core?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/daemon-ha?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/datastor?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/db?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/dns?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/feature-module?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/folder?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/global-settings?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/ha-group?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/httpd?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/iapp-restricted-key?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/icontrol-soap?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/internal-proxy?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/log-rotate?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/management-dhcp?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/management-ip?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/management-ovsdb?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/management-proxy-config?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/management-route?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/ntp?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/outbound-smtp?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/provision?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/scriptd?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/service?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/smtp-server?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/snmp?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/sshd?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/state-mirroring?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/syslog?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/telemd?ver=17.1.1.3"}},{"reference":{"link":"https://localhost/mgmt/tm/sys/ucs?ver=17.1.1.3"}}]}'
2024/11/25 10:44:52 [DEBUG] [2024-11-25 10:44:52,916 f5_cccl DEBUG] F5CloudServiceManager initialize
2024/11/25 10:44:52 [DEBUG] [2024-11-25 10:44:52,917 f5_cccl.bigip DEBUG] BigIPProxy.__init__()
2024/11/25 10:44:53 [DEBUG] [2024-11-25 10:44:53,018 icontrol.session DEBUG] get WITH uri: https://<REDACTED-URL>:443/mgmt/tm/sys/ AND suffix:  AND kwargs: {}
2024/11/25 10:44:53 [DEBUG] Enqueueing Secrets: <REDACTED>
[...]

cut because of maximum 65536 characters --> see file
[f5-case_only_log.txt](https://github.com/user-attachments/files/17935180/f5-case_only_log.txt)


#### Observations (if any)
@keennessch keennessch added bug untriaged no JIRA created labels Nov 27, 2024
@trinaths
Copy link
Contributor

@keennessch Please share sample transport server resources and complete CIS logs to [email protected]

@trinaths trinaths added awaiting response Awaiting response and removed untriaged no JIRA created labels Nov 27, 2024
@abvinodu2003
Copy link

i too have the same issue i'm running OCP 4.14.12 please find the arguments i'm using for CIS

args: [
              "--bigip-username=$(BIGIP_USERNAME)",
              "--bigip-password=$(BIGIP_PASSWORD)",
              "--bigip-url=192.168.110.167",
              "--bigip-partition=ocpclusterpart",
              "--pool-member-type=cluster",
              "--log-level=AS3DEBUG",
              "--insecure=true",
              "--as3-validation=true",
              "--log-as3-response=true",
              "--static-routing-mode=true",
              "--orchestration-cni=ovn-k8s",
              "--agent=as3",
              "--custom-resource-mode=true",
          ]

please find the transportserver configuration

[cislogs.txt](https://github.com/user-attachments/files/17949850/cislogs.txt)

@StevenBarre
Copy link

I'm also experiencing this issue. I've emailed the logs as requested.

@StevenBarre StevenBarre mentioned this issue Nov 29, 2024
Closed
4 tasks
@trinaths
Copy link
Contributor

trinaths commented Dec 3, 2024

@abvinodu2003 Please share sample transport server resources and complete CIS logs to [email protected]

@StevenBarre
Copy link

This works fine on 2.18.0 but not on 2.18.1

@trinaths
Copy link
Contributor

trinaths commented Dec 4, 2024

Created [CONTCNTR-5007] for internal tracking.

@trinaths trinaths added JIRA and removed awaiting response Awaiting response labels Dec 4, 2024
@trinaths
Copy link
Contributor

Fixed in 2.19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug JIRA
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@trinaths @StevenBarre @abvinodu2003 @keennessch