Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception in thread "main" java.io.IOException: Invalid keystore format #92

Open
Mihai-CMM opened this issue Dec 12, 2023 · 24 comments
Open

Comments

@Mihai-CMM
Copy link

Hello,
I am trying to install openunison on top of k0s .
looks like whatever parameters i chose i always hit this issue
Exception in thread "main" java.io.IOException: Invalid keystore format

Can you please advice what should i do to have openunison working with LDAP connector (i dont have any oidc flags on on the k0s )
I am using nginx ingress controller whee i terminate TLS with a valid and trusted certificate

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  K8S_DB_SECRET: WinSuc....
  unisonKeystorePassword: WinSu...
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

network:
  openunison_host: "openunison-in.k8s.test"
  dashboard_host: "dashboard-in.k8s.test"
  api_server_host: "k8smasters-in.k8s.test"
  k8s_url: https://k8smasters-in.k8s.test:6443
  session_inactivity_timeout_seconds: 900
  createIngressCertificate: false
  force_redirect_to_tls: false
  ingress_type: nginx
  ingress_annotations: {}


cert_template:
  ou: "Kubernetes"
  o: "MyOrg"
  l: "My Cluster"
  st: "State of Cluster"
  c: "MyCountry"

myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: datalake-in-sit
enable_impersonation: true
impersonation:
  use_jetstack: true
  explicit_certificate_trust: true

dashboard:
  namespace: "kubernetes-dashboard"
  cert_name: "kubernetes-dashboard-certs"
  label: "kube-app=kubernetes-dashboard"
  service_name: kubernetes-dashboard
  require_session: true
certs:
  use_k8s_cm: false

trusted_certs: []
#trusted_certs:
#  - name: ldaps
#    pem_b64: blabla

monitoring:
  prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s

# Uncomment one of the below options for authentication

active_directory:
  base: "DC=ad,DC=redacted,DC=redacted"
  host: "10.192.yy.xx"
 # port: "636"  for TLS
  port: "389"
 # bind_dn: "cn=Administrator,cn=users,dc=ent2k12,dc=domain,dc=com"
  bind_dn: "CN=k8s_serviceaccount,OU=Domain-Service-Accounts,DC=ad,DC=redacted,DC=redacted"
  con_type: ldap
  srv_dns: "false"

#oidc:
#  client_id: xxxxxx
#  issuer: https://xxxxxx.okta.com/
#  user_in_idtoken: false
#  domain: ""
#  scopes: openid email profile groups
#  claims:
#    sub: sub
#    email: email
#    given_name: given_name
#    family_name: family_name
#    display_name: name
#    groups: groups

#github:
#  client_id: d85d77c55a08c9bcbb15
#  teams: TremoloSecurity/

#saml:
#  idp_url: "https://portal.apps.tremolo.io/idp-test/metadata/dfbe4040-cd32-470e-a9b6-809c8f857c40"


network_policies:
  enabled: false
  ingress:
    enabled: true
    labels:
      app.kubernetes.io/name: ingress-nginx
  monitoring:
    enabled: false
    labels:
      app.kubernetes.io/name: monitoring
  apiserver:
    enabled: false
    labels:
      app.kubernetes.io/name: kube-system

services:
  enable_tokenrequest: false
  token_request_audience: api
  token_request_expiration_seconds: 600
  node_selectors: []

openunison:
  replicas: 1
  non_secret_data:
    K8S_DB_SSO: activedirectory
    PROMETHEUS_SERVICE_ACCOUNT: system:serviceaccount:monitoring:prometheus-k8s
    SHOW_PORTAL_ORGS: "false"
  secrets: []
  enable_provisioning: false
  use_standard_jit_workflow: true
  #az_groups:[]
  #- CN=k8s-users,CN=Users,DC=ent2k12,DC=domain,DC=com

#myvd_configmap: myvdconfig

# For Namespace as a Service

#database:
#  hibernate_dialect: org.hibernate.dialect.MySQL5InnoDBDialect
#  quartz_dialect: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
#  driver: com.mysql.jdbc.Driver
#  url: jdbc:mysql://mariadb.mariadb.svc.cluster.local:3306/unison
#  user: unison
#  validation: SELECT 1

#smtp:
#  host: blackhole.blackhole.svc.cluster.local
#  port: 1025
#  user: "none"
#  from: [email protected]
#  tls: false

Then this is the pod crashing log

NAME                                       READY   STATUS              RESTARTS      AGE
kube-oidc-proxy-orchestra-9649777b-mg9gq   0/1     ContainerCreating   0             7m22s
openunison-operator-6ccd5f44d7-ftjzj       1/1     Running             0             7m28s
openunison-orchestra-57546b4bcb-vp7zw      0/1     CrashLoopBackOff    6 (90s ago)   7m22s
openunison-orchestra-68d96b8695-sgds5      0/1     CrashLoopBackOff    6 (95s ago)   7m18s
`


`[2023-12-12 08:50:20,598][main] INFO  OpenUnisonOnUndertow - Loading keystore for Undertow
[2023-12-12 08:50:20,599][main] INFO  OpenUnisonOnUndertow - OpenUnison XML File : '/usr/local/openunison/work/webapp/WEB-INF/unison.xml'
[2023-12-12 08:50:20,607][main] INFO  OpenUnisonConfigLoader - No config from include files, using original
[2023-12-12 08:50:20,911][main] INFO  OpenUnisonOnUndertow - Loading keystore : '/etc/openunison/unisonKeyStore.p12'
[2023-12-12 08:50:20,911][main] INFO  OpenUnisonOnUndertow - Building Undertow
[2023-12-12 08:50:20,926][main] INFO  OpenUnisonOnUndertow - Check if enabling HTTP2 - false
[2023-12-12 08:50:20,926][main] INFO  OpenUnisonOnUndertow - Enabling HTTP2
[2023-12-12 08:50:20,927][main] INFO  OpenUnisonOnUndertow - Adding open port : '8080'
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:533)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)
```
`
@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

What are the logs from the openunison-operator-6ccd5f44d7-ftjzj say?

@Mihai-CMM
Copy link
Author

command: operator
url: https://kubernetes.default.svc
namespace: openunison
path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
versions: 2,3,4,5,6
webhooks to update: /
Testing version 6
URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Watch URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Processing {"apiVersion":"openunison.tremolo.io/v6","items":[],"kind":"OpenUnisonList","metadata":{"continue":"","resourceVersion":"4513071"}}

Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

odd, there should be some output. what method did you use for install? ouctl?

@Mihai-CMM
Copy link
Author

ouctl yes

@Mihai-CMM
Copy link
Author

Ok redeployed and i see this: (secret yaml content above)

hecking static key lastmile-oidc
the static key doesn't exist in the secret, create it
Creating a new Secret
Problem patching secret - 201 /
..............................................

QMEAgEFAAQgV+7prN0tVPLHI3p1pdrsSurqVikq6c0Si4iHGFjQWoYEFE6DS9oJfds3SFEzTtC9y/NZbrpcAgInEA=="},"kind":"Secret","type":"Opqaue"}
Starting webhook check, looking up /apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations/openunison-workflow-validation-orchestra
Webhook needs to be udpated
Webhook successfully patched
Patched /apis/apps/v1/namespaces/openunison/deployments/openunison-orchestra
{"status":{"digest":"mVI9SZ1oHMf329smTv/vu/J6FJS3b58xHTQ3+bKVpEM=","conditions":{"lastTransitionTime":"2023-12-12T13:47:17.469Z","type":"Completed","status":"True"}}}
Resource patched

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

odd, what version of Kubernetes and what distrobution (ie kubeadmin, EKS, etc)? Also, list out the Secrets in the openunison namespace?

@Mihai-CMM
Copy link
Author

k0s is the distro, https://docs.k0sproject.io/

Client Version: v1.28.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.3+k0s

k0s version
v1.28.3+k0s.0

kubectl get secret -n openunison
NAME TYPE DATA AGE
orchestra Opqaue 4 8m55s
orchestra-secrets-source Opaque 3 9m7s
orchestra-static-keys Opqaue 2 8m55s
remote-k8s-idp-sig kubernetes.io/tls 2 8m56s
sh.helm.release.v1.openunison.v1 helm.sh/release.v1 1 9m6s
sh.helm.release.v1.orchestra.v1 helm.sh/release.v1 1 9m1s
unison-saml2-rp-sig kubernetes.io/tls 2 8m56s
unison-tls kubernetes.io/tls 2 8m57s

Thanks again Marc

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

everything looks OK there. I don't think this would do it but openunison.non_secret_data.K8S_DB_SSO must be saml2 or oidc. this is internal to openunison, so the fact you're using LDAP doesn't matter. Try setting it to oidc and redeploying?

If that doesn't do it, can you run kubectl get openunison orchestra -n openunison -o yaml and set the contents here?

@Mihai-CMM
Copy link
Author

Failed the same with saml2

apiVersion: openunison.tremolo.io/v6
kind: OpenUnison
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "20"
    helm-update: Dec 12 15:38:45 2023 CET
    meta.helm.sh/release-name: orchestra
    meta.helm.sh/release-namespace: openunison
  creationTimestamp: "2023-12-12T14:38:48Z"
  generation: 1
  labels:
    app.kubernetes.io/component: openunison
    app.kubernetes.io/instance: openunison-orchestra
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: openunison
    app.kubernetes.io/part-of: openunison
  name: orchestra
  namespace: openunison
  resourceVersion: "4534933"
  uid: db300a68-c1de-45e8-9641-fb1b561c8383
spec:
  activemq_image: ghcr.io/tremolosecurity/activemq-docker:5.16.6
  deployment_data:
    liveness_probe_command:
    - /usr/local/openunison/bin/check_alive.sh
    - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration
    - issuer
    - https://127.0.0.1:8443/check_alive
    - alive
    node_selectors: []
    pull_secret: ""
    readiness_probe_command:
    - /usr/local/openunison/bin/check_alive.sh
    - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration
    - issuer
    - https://127.0.0.1:8443/check_alive
    - alive
    tokenrequest_api:
      audience: api
      enabled: false
      expirationSeconds: 600
  dest_secret: orchestra
  enable_activemq: false
  hosts:
  - annotations: []
    ingress_name: openunison
    ingress_type: nginx
    names:
    - env_var: OU_HOST
      name: openunison-in.k8s.test
    - env_var: K8S_DASHBOARD_HOST
      name: dashboard-in.k8s.test
    - env_var: K8S_API_HOST
      name: k8smasters-in.k8s.test
      service_name: kube-oidc-proxy-orchestra
    secret_name: ou-tls-certificate
  image: ghcr.io/openunison/openunison-k8s:1.0.38
  key_store:
    key_pairs:
      create_keypair_template:
      - name: ou
        value: Kubernetes
      - name: o
        value: MyOrg
      - name: l
        value: My Cluster
      - name: st
        value: State of Cluster
      - name: c
        value: MyCountry
      keys:
      - create_data:
          ca_cert: true
          key_size: 2048
          server_name: openunison-orchestra.openunison.svc
          sign_by_k8s_ca: false
          subject_alternative_names:
          - k8smasters-in.k8s.test
        import_into_ks: keypair
        name: unison-tls
      - create_data:
          ca_cert: true
          delete_pods_labels:
          - kube-app=kubernetes-dashboard
          key_size: 2048
          secret_info:
            cert_name: dashboard.crt
            key_name: dashboard.key
            type_of_secret: Opaque
          server_name: kubernetes-dashboard.kubernetes-dashboard.svc
          sign_by_k8s_ca: false
          subject_alternative_names: []
          target_namespace: kubernetes-dashboard
        import_into_ks: certificate
        name: kubernetes-dashboard
        replace_if_exists: true
        tls_secret_name: kubernetes-dashboard-certs
      - create_data:
          ca_cert: true
          key_size: 2048
          server_name: unison-saml2-rp-sig
          sign_by_k8s_ca: false
          subject_alternative_names: []
        import_into_ks: keypair
        name: unison-saml2-rp-sig
      - create_data:
          ca_cert: false
          key_size: 2048
          server_name: remote-k8s-idp-sig
          sign_by_k8s_ca: false
          subject_alternative_names: []
        import_into_ks: keypair
        name: remote-k8s-idp-sig
    static_keys:
    - name: session-unison
      version: 1
    - name: lastmile-oidc
      version: 1
    trusted_certificates: []
    update_controller:
      days_to_expire: 10
      image: ghcr.io/openunison/openunison-kubernetes-operator:1.0.4
      schedule: 0 2 * * *
  myvd_configmap: myvd
  non_secret_data:
  - name: K8S_URL
    value: https://k8smasters-in.k8s.test
  - name: SESSION_INACTIVITY_TIMEOUT_SECONDS
    value: "900"
  - name: K8S_DASHBOARD_NAMESPACE
    value: kubernetes-dashboard
  - name: K8S_DASHBOARD_SERVICE
    value: kubernetes-dashboard
  - name: K8S_CLUSTER_NAME
    value: datalake-in-sit
  - name: OPENUNISON_PROVISIONING_ENABLED
    value: "false"
  - name: K8S_IMPERSONATION
    value: "true"
  - name: PROMETHEUS_SERVICE_ACCOUNT
    value: system:serviceaccount:monitoring:prometheus-k8s
  - name: OU_SVC_NAME
    value: openunison-orchestra.openunison.svc
  - name: K8S_TOKEN_TYPE
    value: legacy
  - name: K8S_DB_SSO
    value: saml2
  - name: PROMETHEUS_SERVICE_ACCOUNT
    value: system:serviceaccount:monitoring:prometheus-k8s
  - name: SHOW_PORTAL_ORGS
    value: "false"
  - name: K8S_OPENUNISON_NS
    value: openunison
  openunison_network_configuration:
    activemq_dir: /tmp/amq
    allowed_client_names: []
    ciphers:
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
    - TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
    client_auth: none
    force_to_secure: false
    open_external_port: 80
    open_port: 8080
    path_to_deployment: /usr/local/openunison/work
    path_to_env_file: /etc/openunison/ou.env
    quartz_dir: /tmp/quartz
    secure_external_port: 443
    secure_key_alias: unison-tls
    secure_port: 8443
  replicas: 1
  secret_data:
  - AD_BIND_PASSWORD
  - K8S_DB_SECRET
  - unisonKeystorePassword
  source_secret: orchestra-secrets-source
status:
  conditions:
    lastTransitionTime: "2023-12-12T14:38:53.080Z"
    status: "True"
    type: Completed
  digest: IT1cWBkQA/9WASEW9pYkGXXmqby9K7GHNkKMBkVgJqQ=

@Mihai-CMM
Copy link
Author

I wonder what you expect here:

path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

k0s sets the kubelet under this path

# ll /var/lib/k0s/
total 20
drwxr-xr-x  2 root root 4096 Dec  1 12:08 bin
drwx--x--x 12 root root 4096 Dec  1 12:08 containerd
drwxr-xr-x  2 root root    6 Dec  1 12:08 images
drwxr-xr-x  7 root root  142 Dec  1 12:08 kubelet
-rw-r--r--  1 root root 1656 Dec  1 12:08 kubelet-config.yaml
-rw-------  1 root root 1970 Dec  1 12:08 kubelet.conf
drwxr-x--x  2 root root   20 Dec  1 12:08 pki
-rw-r--r--  1 root root 1984 Dec  1 12:08 worker-profile.yaml

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

Odd. This usually happens when the API server gets "confused" about OpenUnison object versions. But that's usually becauae the myvd config isn't loaded. What happens when you run:

kubectl get secret orchestra -n openunison -o json | jq -r '.data["ou.env"]' | base64 -d | grep myvd

Does it come back as MYVD_CONFIG_PATH=/etc/myvd/myvd.conf?

@Mihai-CMM
Copy link
Author

kubectl get secret orchestra -n openunison -o json | jq -r '.data["ou.env"]' | base64 -d | grep myvd
MYVD_CONFIG_PATH=/etc/myvd/myvd.conf

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

i'll see if i can reproduce. there's nothing really strange going on here but i've never tried on k0s before.

@Mihai-CMM
Copy link
Author

thx

@mlbiam
Copy link
Contributor

mlbiam commented Dec 12, 2023

so i took your values.yaml, deployed it onto k0s with just different host names and ldap connection info and it worked perfectly. Can you please provide the part of the logs from the beginning og the container? tere shouldn't be anything sensitive.

@Mihai-CMM
Copy link
Author

Mihai-CMM commented Dec 13, 2023

Ok : I waited on night and the cronjob was executed i think ok , but the pods are still failing

Error getting SSL certificate "openunison/ou-tls-certificate": local SSL certificate openunison/ou-tls-certificate was not found. Using default certificate

kubectl -n openunison get secret                                               NAME                               TYPE                 DATA   AGE
orchestra                          Opqaue               4      17h
orchestra-secrets-source           Opaque               3      17h
orchestra-static-keys              Opqaue               2      17h
remote-k8s-idp-sig                 kubernetes.io/tls    2      17h
sh.helm.release.v1.openunison.v1   helm.sh/release.v1   1      17h
sh.helm.release.v1.orchestra.v1    helm.sh/release.v1   1      17h
unison-saml2-rp-sig                kubernetes.io/tls    2      17h
unison-tls                         kubernetes.io/tls    2      17h
command: check-certs
url: https://kubernetes.default.svc
namespace: openunison
path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
versions: 2,3,4,5,6
webhooks to update:
Testing version 6
URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Watch URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Checking certificats in namespace
Checking openunison / orchestra
unison-tls
Secret stored in openunison / unison-tls
Checking key tls.crt
Not expiring
kubernetes-dashboard
Secret stored in kubernetes-dashboard / kubernetes-dashboard-certs
Checking key dashboard.crt
Not expiring
unison-saml2-rp-sig
Secret stored in openunison / unison-saml2-rp-sig
Checking key tls.crt
Not expiring
remote-k8s-idp-sig
Secret stored in openunison / remote-k8s-idp-sig
Checking key tls.crt
Not expiring
NAME                                       READY   STATUS              RESTARTS          AGE
check-certs-orchestra-28373820-49jlp       0/1     Completed           0                 6h44m
kube-oidc-proxy-orchestra-9649777b-tqlsv   0/1     ContainerCreating   0                 17h
openunison-operator-6ccd5f44d7-7xd6m       1/1     Running             0                 17h
openunison-orchestra-57546b4bcb-npdg9      0/1     CrashLoopBackOff    203 (3m28s ago)   17h
openunison-orchestra-6fb4c4cfc4-lq525      0/1     CrashLoopBackOff    203 (2m49s ago)   17h

here is the full log

kubectl -n openunison logs openunison-orchestra-6fb4c4cfc4-lq525
/usr/local/openunison/work/webapp/WEB-INF/lib/*:/usr/local/openunison/work/webapp/WEB-INF/classes:/tmp/quartz
[2023-12-13 07:41:14,417][main] INFO  OpenUnisonOnUndertow - Starting OpenUnison on Undertow 1.0.38-2023120501
[2023-12-13 07:41:14,423][main] INFO  OpenUnisonOnUndertow - Parsing YAML : '/etc/openunison/openunison.yaml'
[2023-12-13 07:41:14,515][main] INFO  OpenUnisonOnUndertow - Config Open Port : '8080'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Disable HTTP2 : 'false'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Allow unescaped characters : 'false'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Config Open External Port : '80'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Config Secure Port : '8443'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Config Secure External Port : '443'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Config Context Root :  '/'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Force to Secure : 'false'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - ActiveMQ Directory : '/tmp/amq'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Quartz Directory : '/tmp/quartz'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Config TLS Client Auth Mode : 'none'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Config TLS Allowed Client Subjects : '[]'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config TLS Protocols : 'null'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config TLS Ciphers : '[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384]'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config Path to Deployment : '/usr/local/openunison/work'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config Path to Environment File : '/etc/openunison/ou.env'
[2023-12-13 07:41:14,520][main] INFO  OpenUnisonOnUndertow - Redirect to contex root : 'false'
[2023-12-13 07:41:14,520][main] INFO  OpenUnisonOnUndertow - Support socket shutdown : false
[2023-12-13 07:41:14,523][main] INFO  OpenUnisonOnUndertow - true
[2023-12-13 07:41:14,523][main] INFO  OpenUnisonOnUndertow - Creating unisonServiceProps
[2023-12-13 07:41:14,541][main] INFO  OpenUnisonOnUndertow - Temporary unisonServiceProps : '/tmp/unisonService9418919455708874686props'
[2023-12-13 07:41:14,549][main] INFO  OpenUnisonOnUndertow - Loading environment file : '/etc/openunison/ou.env'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'metadata'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'data'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_SELF_LINK'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_SVC_NAME'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'type'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_TOKEN_TYPE'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_API_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'AD_BIND_PASSWORD'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_IMPERSONATION'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'SHOW_PORTAL_ORGS'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'MYVD_CONFIG_PATH'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DB_SSO'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'unisonKeystorePassword'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'kind'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DB_SECRET'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_NAMESPACE'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_QUARTZ_MASK'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_OPENUNISON_NS'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_SERVICE'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'PROMETHEUS_SERVICE_ACCOUNT'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_URL'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'namespace'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'name'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'SESSION_INACTIVITY_TIMEOUT_SECONDS'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'OPENUNISON_PROVISIONING_ENABLED'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_CLUSTER_NAME'
[2023-12-13 07:41:14,552][main] INFO  OpenUnisonOnUndertow - Loading keystore for Undertow
[2023-12-13 07:41:14,552][main] INFO  OpenUnisonOnUndertow - OpenUnison XML File : '/usr/local/openunison/work/webapp/WEB-INF/unison.xml'
[2023-12-13 07:41:14,560][main] INFO  OpenUnisonConfigLoader - No config from include files, using original
[2023-12-13 07:41:14,863][main] INFO  OpenUnisonOnUndertow - Loading keystore : '/etc/openunison/unisonKeyStore.p12'
[2023-12-13 07:41:14,863][main] INFO  OpenUnisonOnUndertow - Building Undertow
[2023-12-13 07:41:14,876][main] INFO  OpenUnisonOnUndertow - Check if enabling HTTP2 - false
[2023-12-13 07:41:14,876][main] INFO  OpenUnisonOnUndertow - Enabling HTTP2
[2023-12-13 07:41:14,877][main] INFO  OpenUnisonOnUndertow - Adding open port : '8080'
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:533)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)

@mlbiam
Copy link
Contributor

mlbiam commented Dec 13, 2023

everything looks normal. try setting image in your values.yaml to docker.io/tremolosecurity/betas:1.0.39 and redeploy. you should get an additional log line that starts with Could not create PKCS12 from.... I need that, the stack trace, and the entire preceeding log.

@Mihai-CMM
Copy link
Author

# kubectl -n openunison describe pod  openunison-orchestra-657bc46df5-v5sgw | grep -i image -C2
  openunison-orchestra:
    Container ID:   containerd://8b99c92f2b0db48a4f74f5e1f5341e1f5540c5d6c54114e0f8d02a53c0daf4d7
    Image:          docker.io/tremolosecurity/betas:1.0.39
[2023-12-14 09:11:09,047][main] WARN  OpenUnisonOnUndertow - Could not create PKCS12 from /etc/openunison/unisonKeyStore.p12, falling back to JCEKS
java.io.IOException: keystore password was incorrect
        at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2092) ~[?:?]
        at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222) ~[?:?]
        at java.base/java.security.KeyStore.load(KeyStore.java:1479) ~[?:?]
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:530) [openunison-on-undertow-1.0.39.jar:?]
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280) [openunison-on-undertow-1.0.39.jar:?]
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
        ... 5 more
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:536)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)

java.io.IOException: keystore password was incorrect - Which password?

@Mihai-CMM
Copy link
Author

Ok, i found it - thx a lot the secret file that is not a k8s should no longer have K8S_DB_SECRET and unisonKeystorePassword.

Can you please update the documentation if you feel necessary on how that secret should look like

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  K8S_DB_SECRET: WinSuc....
  unisonKeystorePassword: WinSu...
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

should only be

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

Again thanks a lot

@Mihai-CMM
Copy link
Author

Mihai-CMM commented Dec 14, 2023

Now - myabe I open a new ticket? I dont have the oicd flags enabled on k0s so i chose

enable_impersonation: true
impersonation:
  use_jetstack: true
  explicit_certificate_trust: true
NAME                                             READY   STATUS              RESTARTS   AGE
kube-oidc-proxy-orchestra-9649777b-hmv8c         0/1     ContainerCreating   0          10m
openunison-operator-6ccd5f44d7-xd6ks             1/1     Running             0          10m
openunison-orchestra-54db86f588-r58kp            1/1     Running             0          10m
ouhtml-orchestra-login-portal-749df6c7d9-c66cr   1/1     Running             0          9m19

Is it normal to have that pod stuck in creating

Later edit: I am authorized by AD but still the pod stuck in creating

@mlbiam
Copy link
Contributor

mlbiam commented Dec 17, 2023

that's odd. the orchesta-secrets-source Secret is generated by ouctl. did you change the key values after generation? Can you can delete that Secret and ouctl recreate it?

@Mihai-CMM
Copy link
Author

When I do the install I use ouctl install-auth-portal -s /home/openunison/secret.yaml /home/openunison/openunison-default.yaml. Now I did the install from scratch and I redeployed openunison. If i have other key: values in the secret except AD_BIND_PASSWORD it fails with error from ticket title. If I install without any secret provided it fails differently.
Maybe my install command is wrong?

@mysiki
Copy link

mysiki commented Dec 18, 2023

Hello, I got the problem, in my case it was password character fault. Using password with special character give me the same error. Change for password with letter and '_' only work. Don't know with character are in fault.
I don't take time to read all the thread, I hope this can help :D

@Mihai-CMM
Copy link
Author

Ok So i was able to do 3 times the deployments and it worked. I notice though another strange behavior. If dashboard was deployed before open unison there is an issue with untrusted certificate DN (Empty issuer DN not allowed in X509Certificates)
Anyway form my point of view this is ok and can be close - Whoever reads this only the secret needs to have a content like this

apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants