Troubleshooting an initial concierge-only pinniped setup. (keycloak)

[lknite]

Everything looks good yet a browser is not being launched to gather credentials. If I add --skip-browser the error message remains the same without providing a url for me to access to login to. I see this in the concierge log:

[pinniped-concierge-75989848d4-4h7bv] I0623 18:03:48.158746       1 trace.go:205] Trace[1433546369]: "create" kind:TokenCredentialRequest (23-Jun-2022 18:03:48.158) (total time: 0ms):
[pinniped-concierge-75989848d4-4h7bv] Trace[1433546369]: ---"failure" failureType:token authentication,msg:no such authenticator 0ms (18:03:48.158)
[pinniped-concierge-75989848d4-4h7bv] Trace[1433546369]: [22.003µs] [22.003µs] END

There is an authenticator though, or I believe I've done this right:

$ k get jwtauthenticators.authentication.concierge.pinniped.dev 
NAME          ISSUER                                                   AUDIENCE     AGE
oidc-config   https://keycloak.k.home.net/auth/realms/home.net         kubernetes   5d20h

I'm using this script to test with (same error when using this setup via .kube/config):

#!/bin/bash

/usr/local/bin/pinniped login oidc \
      --enable-concierge \
      --concierge-api-group-suffix=pinniped.dev \
      --concierge-authenticator-name=oidc-config \
      --concierge-authenticator-type=jwt \
      --concierge-endpoint=https://<kubernetes_kubeapi_ip>:6443 \
      --concierge-ca-bundle-data=LS0t<snip> \
      --issuer=https://keycloak.k.home.net/auth/realms/home.net \
      --client-id=kubernetes \
      --scopes=openid,email,offline_access \
      --listen-port=8000 \
      --request-audience=kubernetes \
      --ca-bundle /home/lknite/ca-bundle.crt \
      --skip-browser

In what environment did you see this bug?

• lastest install as of last week
• kubernetes version:
  Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0+vmware.wcp.2", GitCommit:"d5bb17833505d15ce5f40815bb14fede978fe8c1", GitTreeState:"clean", BuildDate:"2021-08-14T16:46:51Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.12+vmware.1", GitCommit:"768a3bbd17406f20f51df36cbd12695c8293392d", GitTreeState:"clean", BuildDate:"2021-11-01T22:45:04Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}

[lknite]

The answer was I needed to add certificateAuthorityData to the JWTAuthenticator. Perhaps a debug message could be added to the log file to make this easier to figure out?

kind: JWTAuthenticator
metadata:
  name: oidc-config
spec:
  issuer: https://keycloak.k.home.net/auth/realms/home
  audience: kubernetes
  claims:
    username: preferred_username
    groups: groups
  tls:
    certificateAuthorityData: |
      .....base64encoded_ca-bundle_goes-here

[cfryanr]

Hi @lknite,

Happy to help with that.

First, an aside... have you considered using the Pinniped Supervisor as part of your setup? It offers many benefits in a multi-cluster world. See https://pinniped.dev/docs/tutorials/concierge-and-supervisor-demo/ for some of the benefits.

Ok, going back to talking about a Concierge-only setup... Here is a basic working example. I'll use GitLab as my OIDC provider since I don't have keycloak running, but the concepts should be the same.

1. Install the Concierge. See https://pinniped.dev/docs/howto/install-concierge/ for more info.

2. Create an OIDC client in your OIDC provider. It must be a public client (meaning that it does not require authentication using a client secret). In its configuration, allow the redirect URI http://127.0.0.1:12345/callback. See https://pinniped.dev/docs/howto/configure-concierge-jwt/ for more info.

   Here is an example of what this looks like in GitLab's UI:

3. Copy the client ID of the client that you just created. For this example, let's assume that it was abc123.

4. Create a JWTAuthenticator for your Concierge. Use the same kubeconfig as when you installed the Concierge, because this step must be performed by the admin.

   Here's what mine looked like:

   apiVersion: authentication.concierge.pinniped.dev/v1alpha1
   kind: JWTAuthenticator
   metadata:
     name: my-jwt-authenticator
   spec:
     issuer: https://gitlab.com
     audience: abc123 # this is the client ID of my new OIDC client
     claims:
       username: email

5. Create a Pinniped-compatible kubeconfig by using the Pinniped CLI. Again, use the same kubeconfig as when you installed the Concierge, because this step must also be performed by the admin. Note that the listen port number must be the same one that you allowed on the OIDC client's redirect URIs.

   pinniped get kubeconfig \
     --oidc-client-id abc123 \
     --oidc-scopes openid,email \
     --oidc-listen-port 12345 \
     --concierge-authenticator-type jwt \
     --concierge-authenticator-name my-jwt-authenticator \
     > kubeconfig.yaml

   This kubeconfig can be shared with other users of this cluster, because it contains no specific identity or credentials.

6. Now, acting as an end-user, you can use that kubeconfig to run kubectl commands. A browser will open to authenticate you with your OIDC provider.

   kubectl get namespace --kubeconfig kubeconfig.yaml

7. Profit! 🚀

[cfryanr]

@lknite Oops, looks like you found your own answer while I was typing the response above. Hopefully between your answer and mine, other community users will benefit from this discussion in the future. Thanks for posting your question and answer!