Skip to content

Commit

Permalink
initial edits
Browse files Browse the repository at this point in the history
  • Loading branch information
@schavis
schavis committed Jan 10, 2025
1 parent 7efb1b1 commit 08a5025
Show file tree
Hide file tree
Showing 7 changed files with 546 additions and 408 deletions.
62 changes: 62 additions & 0 deletions website/content/docs/upgrading/deduplication/acl-policy-templates.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
layout: docs
page_title: Fix ACL policy template duplicates
description: >-
Fix duplicate identities for ACL policy templates
---

# Fix ACL policy template duplicates

#### Resolving templated ACL policies

When duplicate entities or groups are renamed by activating the
`force-identity-deduplication` flag, certain templated ACL policies might cause
access to some resources to be lost.

[Policy templating](/vault/docs/concepts/policies#templated-policies) in Vault
allows users to specify policies that dynamically resolve based on properties of
the authenticated entity. If those properties happen to include either entity or
group name then a rename has the potential to change the meaning of the policy.

In practice this is overwhelmingly unlikely to result in unintended access being
granted to an existing resource since the new name contains a UUID which should
never collide with another resource path. The rename _could_ potentially cause
the entity to _loose_ access to a resource they previously had access to
however.

For example, if a user entity named `Janine` had this policy:

```hcl
path "kv/users/{{identity.entity.name}}/*" {
capabilities = ["read", "create", "update"]
}
```

Then before the rename they would be able to read and write key-value pairs
(KVs) with the prefix `kv/users/Janine/`. After the rename, the user would
instead be granted access to KVs with prefix `kv/users/Janine-<UUID>/`, however
the KV mount doesn't know anything special about either path and so any existing
KVs the user had stored would remain at the old paths and `Janine-<UUID>` would
no longer be able to access them.

You can resolve this in two ways:
1. **Change or add policy**. You could define a new policy or a new rule in a
user-specific policy that explicitly granted access back to the old resource
path. For example you could add the following to a policy specific to the
renamed `Janine-UUID` entity:
```hcl
path "kv/users/Janine/*" {
capabilities = ["read", "create", "update"]
}
```
2. **Move the resources.** In some cases operators could move the resources to
be at the new path via the API. For example KV secrets could be renamed to match
the new path. This might be arduous if there are many to rename, and for some
other resource types it might not be possible to move them for other reasons
e.g. external resources expecting access at that path. You should decide whether
you perform any migration before or after activating the flag. If you choose to
do it afterwards, the resources will be unavailable until the migration is
complete. If you choose to do it first then the resources might be unavailable
until the flag is activated if moved, or you might need to take care to keep
updates to both old an new paths in sync until the flag is switched.

179 changes: 179 additions & 0 deletions website/content/docs/upgrading/deduplication/different-case.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
---
layout: docs
page_title: Resolve different-case entity alias duplicates
description: >-
Fix duplicate identities for entity aliases due to case differences.
---

# Fix different-case entity alias duplicates

Fix duplicate identities for entity aliases due to case differences.

**You must fix different-case entity alias duplicates before enabling forced
identity deduplication due to the potential security risk of incorrectly merging
distinct aliases**.


<Tip title="Assumptions">

- You have [identified the duplicate entity aliases](/vault/docs/upgrading/identity-deduplication).
- You have admin permission on the relevant Vault server or cluster.

</Tip>


## Why entity alias duplicates happen

Historical bugs in Vault allowed authN mounts to create duplicate entries of the
same entity alias if the aliases used difference cases. For example, if an
external LDAP system returns unnormalized usernames or changes the normalization
scheme over time. By default, Vault uses case-insensitive matching for the
usernames so entires like `username` and `uSeRnAmE` have the same alias. But,
past bugs have left some users with different-case duplicates in storage. To
avoid the security impact of incorrectly merging identities that do not
represent the same user, Vault tries to honor past behavior by switching the
identity engine to use case-sensitive matching mode.


## Example server log

```text
[WARN] identity: 2 different-case entity alias duplicates found (potential security risk)
[WARN] identity: entity-alias "alias-case" with mount accessor "auth_userpass_34aca7ec" duplicates 1 others: id=df3568a4-3b65-4104-9481-1129ecbed72f canonical_id=5f013d99-a6c7-9a00-6ad5-4ad724b14f60 force_deduplication="would merge into entity 7da76b0d-fe9b-a125-3362-2a8ff055dcf8"
[WARN] identity: entity-alias "alias-cAsE" with mount accessor "auth_userpass_34aca7ec" duplicates 1 others: id=2992253b-1e99-4e47-b6f9-afb0c7cedf7a canonical_id=7da76b0d-fe9b-a125-3362-2a8ff055dcf8 force_deduplication="would merge others into this entity"
[WARN] identity: entity-alias "alias-case" with mount accessor "auth_userpass_a555989a" duplicates 1 others: id=0e4bd46e-a868-4dd4-a34a-cb73f097e3a5 canonical_id=1ce06951-e2fd-0923-8ae0-a0e2c6c2378b force_deduplication="would merge into entity 37f41fcf-5e15-f13f-248d-9fa8405b1ecd"
[WARN] identity: entity-alias "alias-cAsE" with mount accessor "auth_userpass_a555989a" duplicates 1 others: id=6cafe546-665c-4bd0-a4fd-5e699ba413c1 canonical_id=37f41fcf-5e15-f13f-248d-9fa8405b1ecd force_deduplication="would merge others into this entity"
[WARN] identity: end of different-case entity-alias duplicates
```

The duplicate alias in the example log are distinct, despite have the same name,
because they come from different authN mounts. When resolving duplicates,
consider the set of duplicate names from each authN mount separately.

Before resolving different-case duplicates, you must determine if the case
difference is a **true duplicate** or **false positive**:

- With true duplicates, the case difference is **unintentional** and the
duplicate entries represent the same logical entity. You should deduplicate
true duplicates to reflect the expected default Vault behavior.

- With false positive, the case difference is **deliberate** and the source
identity provider **intended** to distinguish between different entities
using case. For example, `Alice` and `alice` are different entities that
correspond to different humans with distinct permission sets.


## Case 1: True duplicates

You can ignore true duplicates since Vault will force-merge them once you enable
forced identity deduplication. To confirm the merge behavior **before** enabling
the flag, review the `force_deduplication` label in the log line. During
deduplication, Vault merges any policies attached to the duplicates so the final
entry has the union of all permissions granted to the duplicates.

Users can continue to log in using any case combination of their username, but
those logins now map to a single entity gated by the same policies per the
expected, default Vault behavior.


## Case 2: False positives

<Warning>

We strongly recommend resolving duplicates and moving away from case-sensitive
usernames as soon as possible to restore the security model supported by Vault.

Vault does not officially support case-sensitive names because it carries
significant security risks. And unresolved duplicates could merge at any time
due to unrelated changes in storage such as when deleting related entities or
groups.

</Warning>

When different-case variations of an alias name actually represent different
logical users or entities in an external system, investigate each alias before
resolving duplicates. You **must** resolve the duplicates on each mount before
enabling forced identity deduplication.

**If you confirm you need to keep the duplicated aliases**, you will need to
reconfigure the authN mount or external service to stop creating case-sensitive
aliases. For example, you could append a unique ID to the alias name to
differentiate between similar usernames. Be mindful of how the modified behavior
may disrupt users as you roll out the change.

<Note>

We are not aware of specific integrations where changing mount or external
service behaviors is necessary and expect those situations to be rare. If you
determine your deployment requires changes to authN mounts or external services,
we encourage Vault Enterprise customers to work with HashiCorp support staff to
determine the best strategy and implementation.

</Note>

**If you can confirm you do not need the duplicated alias, or the entire auth
mount**, delete all but one of the entity aliases on each mount using the ID
noted in the log line with `id=`.

<Tabs>

<Tab heading="CLI" group="cli">

Use `vault delete` with the `/identity/entity-alias/id/{id}` path to delete the
duplicate identity:

```shell-session
$ vault delete /identity/entity-alias/id/<id>
```

For example, if one of the duplicate entity log entries includes
`id=df3568a4-3b65-4104-9481-1129ecbed72f`:

<CodeBlockConfig hideClipboard="true">

```shell-session
$ vault delete /identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f
Success! Data deleted (if it existed) at: identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f
```

</CodeBlockConfig>

</Tab>

<Tab heading="API" group="api">

Call the [`/identity/entity-alias/id/{id}`](/vault/api-docs/secret/identity/entity-alias#delete-entity-alias-by-id)
endpoint to delete the duplicate entry:

```shell-session
$ curl \
--request DELETE \
--header "X-Vault-Token: ${VAULT_TOKEN}" \
${VAULT_ADDR}/v1/identity/entity-alias/id/<id>
```

For example, if one of the duplicate entity log entries includes
`id=df3568a4-3b65-4104-9481-1129ecbed72f`:

<CodeBlockConfig hideClipboard="true">

```shell-session
$ curl \
--request DELETE \
--header "X-Vault-Token: ${VAULT_TOKEN}" \
${VAULT_ADDR}/v1/identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f | jq
```

</CodeBlockConfig>

</Tab>

</Tabs>


Once you resolve all duplicates by deleting or reconfiguring the relevant
services and confirm the duplicates no longer appear in the log during unseal,
you can [enable forced identity deduplication](/vault/docs/upgrading/deduplication#dedupe-flag).


83 changes: 83 additions & 0 deletions website/content/docs/upgrading/deduplication/entity-group.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
layout: docs
page_title: Fix entity and group duplicates
description: >-
Fix duplicate identities for Vault entities and groups
---

# Fix entity and group duplicates

### Resolving entity and group duplicates

```
[WARN] identity: 2 entity duplicates found
[WARN] identity: entity "entity-cAsE" with namespace ID "root" duplicates 2 others: id=2562b42d-f603-ac6b-2591-8a20dd050897 force_deduplication="would rename to entity-cAsE-2562b42d-f603-ac6b-2591-8a20dd050897"
[WARN] identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=290a643d-6043-da5e-943f-3a3d09e4ecbd force_deduplication="would rename to entity-case-290a643d-6043-da5e-943f-3a3d09e4ecbd"
[WARN] identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=b0141be5-3f03-a1c7-a57b-02f045d04426 force_deduplication="would not rename"
[WARN] identity: entity "entity-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=95f0743b-a1d5-26da-b4ef-a50490da0787 force_deduplication="would rename to entity-cAsE-95f0743b-a1d5-26da-b4ef-a50490da0787"
[WARN] identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=9d3be96f-490a-9625-3118-896bd2b3a5f3 force_deduplication="would rename to entity-case-9d3be96f-490a-9625-3118-896bd2b3a5f3"
[WARN] identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=d82231d2-3716-6b9c-e80d-7d09c0409739 force_deduplication="would not rename"
[WARN] identity: end of entity duplicates
[WARN] identity: 2 group duplicates found
[WARN] identity: 2 group duplicates found
[WARN] identity: group "group-case" with namespace ID "root" duplicates 2 others: id=8ad26e0c-8cf6-5b67-7c77-6571fa374f34 force_deduplication="would rename to group-case-8ad26e0c-8cf6-5b67-7c77-6571fa374f34"
[WARN] identity: group "group-cAsE" with namespace ID "root" duplicates 2 others: id=9fe86ea0-f80c-1131-5be1-1d6e3b70237f force_deduplication="would rename to group-cAsE-9fe86ea0-f80c-1131-5be1-1d6e3b70237f"
[WARN] identity: group "group-case" with namespace ID "root" duplicates 2 others: id=32dd070c-c1f8-c796-9a71-15887014b813 force_deduplication="would not rename"
[WARN] identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=8aaeff7e-7343-c883-1e0c-c5c9968f75a5 force_deduplication="would rename to group-case-8aaeff7e-7343-c883-1e0c-c5c9968f75a5"
[WARN] identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=f11277b3-d985-4d72-d2e9-9c8c6c0db02c force_deduplication="would rename to group-case-f11277b3-d985-4d72-d2e9-9c8c6c0db02c"
[WARN] identity: group "group-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=7c753d07-b0d9-e13b-6184-48247b8f7504 force_deduplication="would not rename"
[WARN] identity: end of group duplicates
```

Entity and group duplicates are reported as in the examples above. The
resolution steps are the same for both.

Entities or groups may exist with duplicate names. These might be exact matches,
or may differ in case such as `Admin` and `admin`. In either case, the
resolution is the same.

Note that names are only duplicates if they are in the same namespace.

Possible unexpected behavior caused by these types of duplicates includes:
* Only one of them will be returned when looking up or listing by name
* Before Vault 1.19.0, which one is returned might vary depending on which
server is hit or even after a seal and unseal on the same server
* Lookup by ID _will_ work for all duplicates
* Listing by IS will return _all_ duplicates
* Deletion by name may or may not delete all duplicates

These types of duplicates **will be resolved automatically by renaming** when
the `force-identity-deduplication` flag is activated. Automatic renaming has the
following properties:
* **Security safe**. Renaming one or more of the duplicates will not allow
existing tokens to access anything new.
* **Preserves all data**. Renaming will not delete any data in Vault and so can
be identified and reversed manually later if necessary.
* **Preserves access**. In the vast majority of cases, logins and permissions
will be unaffected and continue to work the same way. The exceptions are
noted below.

There are two know edge cases that you should be aware of before activating the
flag and triggering the renaming:
* **Templated Policies**. If access to some resource in Vault is granted via a
[templated policy](/vault/docs/concepts/policies#templated-policies) that
uses the entity or group name as part of a resource path, _and_ that policy
might be applied to the duplicated entity or group, then access to those
resources could be _lost_ when the name is changed. [Resolution details are
provided below.](#resolving-templated-acl-policies)
* **External references**. For example if entities or groups are defined in
Terraform configuration by name then one of those references will break when
the resource is renamed. The same could apply to other external references to
a named entity or group that use the name-based API methods to read or
manage. [Resolution details are
provided below.]()

In general, for each duplicate within a namespace, the possible resolutions are:
1. If the risk of either of the above edge cases is low to zero, **no action is
needed before activating the flag**.
2. If you know that the duplicates are unused or unimportant (for example
because of naming conventions such as `test-*`, or from consulting with
teams that created them) you might simply delete them by their ID using the
relevant API endpoint to clean them up.
3. If you are unsure about the risk from policy templating or external
references. Refer to the additional resolution details below.
Loading

0 comments on commit 08a5025

Please sign in to comment.