Portable SIMD subtree update #135701

calebzulawski · 2025-01-18T20:45:17Z

r? @workingjubilee

Sync up to rust-lang/portable-simd@7c7dbe0

Signed-off-by: cui fliter <[email protected]>

…table-simd-2023-11-19

stdarch no longer provide SIMD on big-endian ARM due to rust-lang/stdarch#1484

…table-simd-2024-02-18

…table-simd-2024-03-22

add without_provenance to pointer types

Add loongarch64 vendor conversions

…r=Amanieu rename ptr::from_exposed_addr -> ptr::with_exposed_provenance As discussed on [Zulip](https://rust-lang.zulipchat.com/#narrow/stream/136281-t-opsem/topic/To.20expose.20or.20not.20to.20expose/near/427757066). The old name, `from_exposed_addr`, makes little sense as it's not the address that is exposed, it's the provenance. (`ptr.expose_addr()` stays unchanged as we haven't found a better option yet. The intended interpretation is "expose the provenance and return the address".) The new name nicely matches `ptr::without_provenance`.

run Miri tests on CI

Avoid migration pitfall from packed_simd as described in rust-lang#402.

document PartialOrd difference to simd_min and simd_max

Sync 2024-04-09

Sync upstream

Add script for syncing subtree

…m-portable-simd-2025-01-18

rustbot · 2025-01-18T20:45:26Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-01-18T20:45:27Z

Portable SIMD is developed in its own repository. If possible, consider making this change to rust-lang/portable-simd instead.

cc @calebzulawski, @programmerjake

calebzulawski · 2025-01-18T20:51:44Z

r? @workingjubilee

rustbot · 2025-01-18T20:51:47Z

Could not assign reviewer from: workingjubilee.
User(s) workingjubilee are either the PR author, already assigned, or on vacation, and there are no other candidates.
Use r? to specify someone else to assign.

calebzulawski · 2025-01-18T21:05:05Z

@RalfJung (not sure who to ping for subtrees 🙃) does it make sense for some commits from rust-lang/rust to end up in this PR a second time? For example 989660c is in here a second time as b039861, I believe due to a merge conflict from the subtree-push to rust-lang/portable-simd.

programmerjake · 2025-01-18T21:10:30Z

@RalfJung (not sure who to ping for subtrees 🙃) does it make sense for some commits from rust-lang/rust to end up in this PR a second time? For example 989660c is in here a second time as b039861, I believe due to a merge conflict from the subtree-push to rust-lang/portable-simd.

that shouldn't happen from a merge conflict since all that does is change the tree you end up with in the merge commit. you'd get duplicate commits if you did any rebasing (which it sounds like is what happened before). duplicate commits may also be fine and expected (I'm not 100% sure since I last researched subtrees a while ago...)

calebzulawski · 2025-01-18T21:14:03Z

Ah, so rebasing subtree pushes in the past (not the most recent one) was a mistake because it rewrote that commit. That makes sense. I wonder if there's a fix now

programmerjake · 2025-01-18T21:17:47Z

Ah, so rebasing subtree pushes in the past (not the most recent one) was a mistake because it rewrote that commit. That makes sense. I wonder if there's a fix now

I think the fix is probably to say oh well, we messed up before, we'll just end up with some duplicates but if the final repo contents is as expected then it's mostly ok since that's what matters most rather than having perfect git history

rustbot · 2025-01-19T02:45:03Z

The Miri subtree was changed

cc @rust-lang/miri

calebzulawski · 2025-01-19T03:51:03Z

Ready to merge this as long as @rust-lang/miri is ok with removing those test cases. We now only provide bitmasks to u64 and expect users to use lane shifting + bitmask to get beyond the first 64 lanes.

RalfJung · 2025-01-19T16:29:29Z

@RalfJung (not sure who to ping for subtrees 🙃)

portable-simd doesn't use josh, does it? I know next to nothing about git subtree, so I am definitely the wrong person to ping here. :) But yes, rebase-on-merge is generally a bad idea when using any of these tools.

The git subtree you are using is hosted on https://raw.githubusercontent.com/bjorn3 so @bjorn3 seems like a good starting point.

RalfJung · 2025-01-19T16:37:33Z

But why would there have been rebasing? You didn't recently change the subtree workflow, did you? Are you sure this will not keep causing issues for future syncs? Better test what happens when you merge this PR into the main repo and then sync that back, and vice versa. This subtree sync looks rather strange so something odd might be going on. The duplicated commits go all the way back to 2023! I assume there have been other syncs since 2023. So why were they not affected?

Clippy used git subtree for a long time, maybe they have some idea what is happening here -- Cc @rust-lang/clippy . I would be wary of merging this until this is resolved.

I think the fix is probably to say oh well, we messed up before, we'll just end up with some duplicates but if the final repo contents is as expected then it's mostly ok since that's what matters most rather than having perfect git history

The question is, will the next syncs in both directions actually work, or will there be more duplication? This should be tested.

With josh, the recommended resolution would be a force-push to the "portable-simd" repo to back out the broken-by-rebase syncs. That's what the rust-analyzer folks did when they ran into the same issue. (I see a strong confirmation of my dislike for rebase-on-merge strategies. ;)

RalfJung · 2025-01-19T16:44:48Z

src/tools/miri/tests/pass/intrinsics/portable-simd.rs

-    // This used to cause an ICE. It exercises simd_select_bitmask with an array as input.
-    let bitmask = u8x4::from_array([0b00001101, 0, 0, 0]);
-    assert_eq!(
-        mask32x4::from_bitmask_vector(bitmask),
-        mask32x4::from_array([true, false, true, true]),
-    );
-    let bitmask = u8x8::from_array([0b01000101, 0, 0, 0, 0, 0, 0, 0]);
-    assert_eq!(
-        mask32x8::from_bitmask_vector(bitmask),
-        mask32x8::from_array([true, false, true, false, false, false, true, false]),
-    );
-    let bitmask =
-        u8x16::from_array([0b01000101, 0b11110000, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]);
-    assert_eq!(
-        mask32x16::from_bitmask_vector(bitmask),
-        mask32x16::from_array([
-            true, false, true, false, false, false, true, false, false, false, false, false, true,
-            true, true, true,
-        ]),
-    );


As far as I can tell, this just tests from_bitmask_vector, so yeah if that API is gone we can remove the test. We still also directly test the intrinsic.

Does this mean simd_select_bitmask on an array is no longer used? What about simd_bitmask with an array?

Neither intrinsic is used with arrays anymore. The limitation was const generics on the API so I could see it coming back at some point in the future, though

calebzulawski · 2025-01-19T16:58:06Z

No problem, I just searched zulip for subtrees and it seemed you discussed it a lot, but admittedly in the context of josh. (Once we get this all fixed, switching to josh would be nice...)

The rebasing was mostly a mistake, @workingjubilee and I have manually done subtree syncs and I know I have rebased a couple times. I only just added a sync script to ensure we get this right in the future and that's how I discovered it.

I think this won't be a problem in the future. I know if I do a subtree push now (rust -> portable-simd) the branch it creates is identical to master. Using josh to rewrite bad commit history sounds like a good idea, but I think we would need to at least merge this PR because the portable-simd changes would need to be part of that, right?

RalfJung · 2025-01-19T17:10:29Z

Using josh to rewrite bad commit history sounds like a good idea,

I didn't suggest that so I don't know what you are referring to here.

I have no idea how josh will react to these odd duplicated commits if you want to migrate to josh in the future. They will definitely also be duplicated in the portable-simd repo then, and maybe that's it, or maybe it causes more problems -- hard to say without trying.

I know I have rebased a couple times

But then why is this the first time we see duplicated commits? Or is it not the first time?

calebzulawski · 2025-01-19T17:28:39Z

Sorry I was referring to rewriting portable-simd's history:

With josh, the recommended resolution would be a force-push to the "portable-simd" repo to back out the broken-by-rebase syncs

I think we ultimately want portable-simd's history to look like this repo's, whether that includes these duplicate commits or not. I don't remember seeing this kind of duplicate commit in the past, but if there were any, they're part of this repo already. I'm guessing we just got lucky, because we try to avoid pushing changes to this repo instead of portable-simd.

My main concern is I don't feel confident that I could fix the portable-simd repo (my limited git ability got us in this mess 🙂). I would like to merge this as-is (and accept these duplicate commits), and then make sure we either use the sync script or preferably switch to josh to avoid it in the future. I'm just not ready to merge this without getting some more input.

RalfJung · 2025-01-19T17:31:15Z

Sorry I was referring to rewriting portable-simd's history:

That would be a manually constructed force-push to before the rebased PR got merged. Josh isn't used for the rewriting, the rewriting is done to make Josh work again. ;)

calebzulawski · 2025-01-19T20:19:05Z

Here's the complete list of duplicated commits in portable-simd:

git rev-list --format="%ai    %s" origin/master | sed -n '0~2p' | sort | uniq -d
2021-02-14 23:35:24 -0500    Add floating-point classification functions
2021-03-06 02:14:58 -0500    Various bug fixes
2022-04-12 11:01:22 -0400    portable-simd: use simd_arith_offset to avoid ptr-int transmutation
2022-05-20 08:54:10 -0400    Finish bumping stage0
2023-07-16 00:37:30 +0800    remove repetitive words
2023-12-02 10:49:21 -0500    Remove link to core::arch::x86_64
2023-12-12 23:26:45 +0100    Fix target_feature config in portable-simd
2024-01-30 03:40:53 +0000    Disable conversions between portable_simd and stdarch on big-endian ARM
2024-03-23 11:47:11 +0100    rename ptr::from_exposed_addr -> ptr::with_exposed_provenance
2024-03-23 23:00:53 +0100    also rename the SIMD intrinsic
2024-03-25 11:02:02 -0700    Import the 2021 prelude in the core crate
2024-04-03 15:17:00 +0200    rename `expose_addr` to `expose_provenance`

A bunch of them are in this PR, but some of them predate this PR as well and are already in this repo somewhere.
For reference, rust-lang/rust has a ton of duplicate commits:

git rev-list --format="%ai %s" origin/master | sed -n '0~2p' | sort | uniq -d | wc -l
4048

So, do I try to fix everything from around July 2023 for this PR?

RalfJung · 2025-01-19T20:33:58Z

For reference, rust-lang/rust has a ton of duplicate commits:

Spot-checking two of them, I found clippy and cranelift. So seems like other git subtree based components have the same issue. Not sure if those are caused by rebases as well -- Cc @bjorn3 @rust-lang/clippy

workingjubilee and others added 30 commits July 7, 2023 04:07

Sync portable-simd to 2023 July 07

bd7c794

Sync up to rust-lang/portable-simd@7c7dbe0

remove repetitive words

78c05f9

Signed-off-by: cui fliter <[email protected]>

use visibility to check unused imports and delete some stmts

5895729

Merge commit 'e0e9a4517f9fc021283514da387e70a56061bd3e' into sync-por…

62030c7

…table-simd-2023-11-19

Remove link to core::arch::x86_64

c6b3b35

portable-simd: add missing feature gate

08069a5

Fix target_feature config in portable-simd

98fc771

Disable conversions between portable_simd and stdarch on big-endian ARM

7e65a05

stdarch no longer provide SIMD on big-endian ARM due to rust-lang/stdarch#1484

Merge commit '649110751ef4f27440d7cc711b3e07d11bf02d4a' into sync-por…

f699661

…table-simd-2024-02-18

Merge commit 'cff979eec1ac0473fc4960ee6cde462c6aeda824' into sync-por…

8d2208e

…table-simd-2024-03-22

add without_provenance to pointer types

6c39a26

rename ptr::from_exposed_addr -> ptr::with_exposed_provenance

def711f

also rename the SIMD intrinsic

c7a9561

Merge pull request rust-lang#408 from RalfJung/without_provenance

5fa5358

add without_provenance to pointer types

Import the 2021 prelude in the core crate

61daba9

Add loongarch64 vendor conversions

bfacd14

Merge pull request rust-lang#410 from heiher/loongarch64

53fe1a3

Add loongarch64 vendor conversions

rename expose_addr to expose_provenance

b039861

run Miri tests on CI

0d6e714

Merge pull request rust-lang#411 from RalfJung/miri

48d411b

run Miri tests on CI

do not run the 4-lane tests in Miri

5533932

document PartialOrd difference to simd_min and simd_max

f79ce11

Avoid migration pitfall from packed_simd as described in rust-lang#402.

document Ord difference to simd_min and simd_max

ef5f073

Merge pull request rust-lang#413 from KonradHoeffner/patch-1

979a495

document PartialOrd difference to simd_min and simd_max

Import the 2021 prelude in the core crate

7ba49ef

rename ptr::from_exposed_addr -> ptr::with_exposed_provenance

2b03143

also rename the SIMD intrinsic

6da2337

rename expose_addr to expose_provenance

b8a18fa

Merge pull request rust-lang#414 from rust-lang/sync-2024-04-09

f0cbf40

Sync 2024-04-09

calebzulawski added 4 commits January 18, 2025 12:25

Merge pull request rust-lang#448 from rust-lang/sync-upstream

4b1a23e

Sync upstream

Add script for syncing subtree

15c1972

Merge pull request rust-lang#449 from rust-lang/subtree-sync-script

3383cfb

Add script for syncing subtree

Merge commit '3383cfbd3572465febc7a8f816a46304373de46a' into sync-fro…

1ff6c55

…m-portable-simd-2025-01-18

rustbot assigned Mark-Simulacrum Jan 18, 2025

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 18, 2025

This comment has been minimized.

Sign in to view

workingjubilee assigned workingjubilee and unassigned Mark-Simulacrum Jan 18, 2025

Update tests for std::simd subtree sync

52b42d7

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jan 19, 2025

RalfJung reviewed Jan 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Portable SIMD subtree update #135701

Portable SIMD subtree update #135701

calebzulawski commented Jan 18, 2025

rustbot commented Jan 18, 2025

rustbot commented Jan 18, 2025

calebzulawski commented Jan 18, 2025

rustbot commented Jan 18, 2025

calebzulawski commented Jan 18, 2025

This comment has been minimized.

programmerjake commented Jan 18, 2025 •

edited

Loading

calebzulawski commented Jan 18, 2025

programmerjake commented Jan 18, 2025

rustbot commented Jan 19, 2025

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading

RalfJung Jan 19, 2025

calebzulawski Jan 19, 2025

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025 •

edited

Loading

calebzulawski commented Jan 19, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading

Portable SIMD subtree update #135701

Are you sure you want to change the base?

Portable SIMD subtree update #135701

Conversation

calebzulawski commented Jan 18, 2025

rustbot commented Jan 18, 2025

rustbot commented Jan 18, 2025

calebzulawski commented Jan 18, 2025

rustbot commented Jan 18, 2025

calebzulawski commented Jan 18, 2025

This comment has been minimized.

programmerjake commented Jan 18, 2025 • edited Loading

calebzulawski commented Jan 18, 2025

programmerjake commented Jan 18, 2025

rustbot commented Jan 19, 2025

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025 • edited Loading

RalfJung commented Jan 19, 2025 • edited Loading

RalfJung Jan 19, 2025

Choose a reason for hiding this comment

calebzulawski Jan 19, 2025

Choose a reason for hiding this comment

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025

calebzulawski commented Jan 19, 2025

RalfJung commented Jan 19, 2025 • edited Loading

calebzulawski commented Jan 19, 2025 • edited Loading

RalfJung commented Jan 19, 2025 • edited Loading

programmerjake commented Jan 18, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading

calebzulawski commented Jan 19, 2025 •

edited

Loading

RalfJung commented Jan 19, 2025 •

edited

Loading