Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ESIMD][E2E] ESIMD/named_barriers/loop_extended.cpp flakily failing on PVC #16598

Open
uditagarwal97 opened this issue Jan 13, 2025 · 5 comments
Open

[ESIMD][E2E] ESIMD/named_barriers/loop_extended.cpp flakily failing on PVC #16598

uditagarwal97 opened this issue Jan 13, 2025 · 5 comments
Assignees
@sarnex
Labels
bug Something isn't working confirmed

Comments

@uditagarwal97
Copy link
Contributor

Describe the bug

Passing job: https://github.com/intel/llvm/actions/runs/12728797473/job/35500742045?pr=16595
Failing job: https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595

FAIL: SYCL :: ESIMD/named_barriers/loop_extended.cpp (889 of 2258)
******************** TEST 'SYCL :: ESIMD/named_barriers/loop_extended.cpp' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 10
/__w/llvm/llvm/toolchain/bin//clang++  -Werror  -Wno-error=deprecated-declarations -fsycl -fsycl-targets=spir64  /__w/llvm/llvm/llvm/sycl/test-e2e/ESIMD/named_barriers/loop_extended.cpp -o /__w/llvm/llvm/build-e2e/ESIMD/named_barriers/Output/loop_extended.cpp.tmp.out
# executed command: /__w/llvm/llvm/toolchain/bin//clang++ -Werror -Wno-error=deprecated-declarations -fsycl -fsycl-targets=spir64 /__w/llvm/llvm/llvm/sycl/test-e2e/ESIMD/named_barriers/loop_extended.cpp -o /__w/llvm/llvm/build-e2e/ESIMD/named_barriers/Output/loop_extended.cpp.tmp.out
# note: command had no output on stdout or stderr
# RUN: at line 11
env ONEAPI_DEVICE_SELECTOR=level_zero:gpu  /__w/llvm/llvm/build-e2e/ESIMD/named_barriers/Output/loop_extended.cpp.tmp.out
# executed command: env ONEAPI_DEVICE_SELECTOR=level_zero:gpu /__w/llvm/llvm/build-e2e/ESIMD/named_barriers/Output/loop_extended.cpp.tmp.out
# .---command stdout------------
# | Running on Intel(R) Data Center GPU Max 1100, driver=[1.6.31294.120000]
# | out[2]=0 vs dead0001
# | #1 FAILED
# | #2 Passed
# | #3 Passed
# | #4 Passed
# | #5 Passed
# `-----------------------------
# error: command failed with exit status: 1
--

To reproduce

No response

Environment

  • OS: Linux
  • Target device and vendor: PVC
  • DPC++ version: df9fba6

sycl-ls --verbose output:

[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31294.120000]
[level_zero:gpu][level_zero:1] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31294.120000]
[opencl:gpu][opencl:0] Intel(R) OpenCL Graphics, Intel(R) Data Center GPU Max 1100 OpenCL 3.0 NEO  [24.39.31294.12]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Data Center GPU Max 1100 OpenCL 3.0 NEO  [24.39.31294.12]
[opencl:cpu][opencl:2] Intel(R) OpenCL, Intel(R) Xeon(R) Platinum 8468V OpenCL 3.0 (Build 0) [2024.18.10.0.08_160000]
[opencl:fpga][opencl:3] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2024.18.10.0.08_160000]
[native_cpu:cpu][native_cpu:0] SYCL_NATIVE_CPU, SYCL Native CPU 0.1 [0.0.0]

Platforms: 5
Platform [#1]:
    Version  : 1.6
    Name     : Intel(R) oneAPI Unified Runtime over Level-Zero
    Vendor   : Intel(R) Corporation
    Devices  : 2
        Device [#0]:
        Type              : gpu
        Version           : 12.60.7
        Name              : Intel(R) Data Center GPU Max 1100
        Vendor            : Intel(R) Corporation
        Driver            : 1.6.31294.120000
        UUID              : [13](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:14)41282181147000410000000
        DeviceID          : 3034
        Num SubDevices    : 0
        Num SubSubDevices : 0
        Aspects           : gpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations ext_intel_pci_address ext_intel_gpu_eu_count ext_intel_gpu_eu_simd_width ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice atomic64 ext_intel_device_info_uuid ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_free_memory ext_intel_device_id ext_intel_memory_clock_rate ext_intel_memory_bus_width ext_intel_esimd ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_matrix ext_oneapi_limited_graph ext_oneapi_private_alloca ext_oneapi_queue_profiling_tag ext_oneapi_virtual_mem ext_oneapi_virtual_functions
        info::device::sub_group_sizes: 16 32
        Architecture: intel_gpu_pvc
        Device [#1]:
        Type              : gpu
        Version           : 12.60.7
        Name              : Intel(R) Data Center GPU Max 1100
        Vendor            : Intel(R) Corporation
        Driver            : 1.6.31294.120000
        UUID              : 1341282181[14](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:15)7000580000000
        DeviceID          : 3034
        Num SubDevices    : 0
        Num SubSubDevices : 0
        Aspects           : gpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations ext_intel_pci_address ext_intel_gpu_eu_count ext_intel_gpu_eu_simd_width ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice atomic64 ext_intel_device_info_uuid ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_free_memory ext_intel_device_id ext_intel_memory_clock_rate ext_intel_memory_bus_width ext_intel_esimd ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_matrix ext_oneapi_limited_graph ext_oneapi_private_alloca ext_oneapi_queue_profiling_tag ext_oneapi_virtual_mem ext_oneapi_virtual_functions
        info::device::sub_group_sizes: [16](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:17) 32
        Architecture: intel_gpu_pvc
Platform [#2]:
    Version  : OpenCL 3.0 
    Name     : Intel(R) OpenCL Graphics
    Vendor   : Intel(R) Corporation
    Devices  : 2
        Device [#0]:
        Type              : gpu
        Version           : OpenCL 3.0 NEO 
        Name              : Intel(R) Data Center GPU Max 1100
        Vendor            : Intel(R) Corporation
        Driver            : 24.39.31294.12
        UUID              : 1341282[18](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:19)1147000410000000
        DeviceID          : 3034
        Num SubDevices    : 0
        Num SubSubDevices : 0
        Aspects           : gpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations ext_intel_pci_address ext_intel_gpu_eu_count ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice atomic64 ext_intel_device_info_uuid ext_oneapi_srgb ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_device_id ext_intel_esimd ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_matrix ext_oneapi_private_alloca ext_oneapi_atomic16 ext_oneapi_virtual_functions
        info::device::sub_group_sizes: 16 32
        Architecture: intel_gpu_pvc
        Device [#1]:
        Type              : gpu
        Version           : OpenCL 3.0 NEO 
        Name              : Intel(R) Data Center GPU Max 1100
        Vendor            : Intel(R) Corporation
        Driver            : 24.39.31294.12
        UUID              : 1341282181147000580000000
        DeviceID          : 3034
        Num SubDevices    : 0
        Num SubSubDevices : 0
        Aspects           : gpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations ext_intel_pci_address ext_intel_gpu_eu_count ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice atomic64 ext_intel_device_info_uuid ext_oneapi_srgb ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_device_id ext_intel_esimd ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_matrix ext_oneapi_private_alloca ext_oneapi_atomic16 ext_oneapi_virtual_functions
        info::device::sub_group_sizes: 16 32
        Architecture: intel_gpu_pvc
Platform [#3]:
    Version  : OpenCL 3.0 LINUX
    Name     : Intel(R) OpenCL
    Vendor   : Intel(R) Corporation
    Devices  : 1
        Device [#2]:
        Type              : cpu
        Version           : OpenCL 3.0 (Build 0)
        Name              : Intel(R) Xeon(R) Platinum 8468V
        Vendor            : Intel(R) Corporation
        Driver            : [20](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:21)24.18.10.0.08_160000
        DeviceID          : 526072
        Num SubDevices    : 2
        Num SubSubDevices : 0
        Aspects           : cpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations usm_system_allocations ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice usm_atomic_host_allocations usm_atomic_shared_allocations atomic64 ext_oneapi_srgb ext_oneapi_native_assert ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_device_id ext_intel_legacy_image ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_matrix ext_oneapi_private_alloca ext_oneapi_virtual_functions
        info::device::sub_group_sizes: 4 8 16 32 64
        Architecture: intel_cpu_spr
Platform [#4]:
    Version  : OpenCL 1.2 Intel(R) FPGA SDK for OpenCL(TM), Version 20.3
    Name     : Intel(R) FPGA Emulation Platform for OpenCL(TM)
    Vendor   : Intel(R) Corporation
    Devices  : 1
        Device [#3]:
        Type              : fpga
        Version           : OpenCL 1.2 
        Name              : Intel(R) FPGA Emulation Device
        Vendor            : Intel(R) Corporation
        Driver            : 20[24](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:25).18.10.0.08_160000
        DeviceID          : 5[26](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:27)072
        Num SubDevices    : 2
        Num SubSubDevices : 0
        Aspects           : accelerator fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations ext_intel_gpu_slices ext_intel_gpu_subslices_per_slice ext_intel_gpu_eu_count_per_subslice usm_atomic_host_allocations usm_atomic_shared_allocations ext_oneapi_srgb ext_intel_gpu_hw_threads_per_eu ext_oneapi_cuda_async_barrier ext_intel_device_id ext_oneapi_ballot_group ext_oneapi_fixed_size_group ext_oneapi_opportunistic_group ext_oneapi_tangle_group ext_intel_fpga_task_sequence ext_oneapi_private_alloca
        info::device::sub_group_sizes: 4 8 16 32 64
        Architecture: unknown
Platform [#5]:
    Version  : 0.1
    Name     : SYCL_NATIVE_CPU
    Vendor   : tbd
    Devices  : 1
        Device [#0]:
        Type              : cpu
        Version           : 0.1
        Name              : SYCL Native CPU
        Vendor            : Intel(R) Corporation
        Driver            : 0.0.0
        DeviceID          : UNKNOWN
        Num SubDevices    : 0
        Num SubSubDevices : 0
        Aspects           : cpu fp16 fp64 online_compiler online_linker queue_profiling usm_device_allocations usm_host_allocations usm_shared_allocations usm_system_allocations usm_atomic_host_allocations usm_atomic_shared_allocations atomic64
        info::device::sub_group_sizes: 1
        Architecture: unknown
default_selector()      : gpu, Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31[29](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:30)4.120000]
accelerator_selector()  : fpga, Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2024.18.10.0.08_160000]
cpu_selector()          : cpu, Intel(R) OpenCL, Intel(R) Xeon(R) Platinum 8468V OpenCL 3.0 (Build 0) [2024.18.10.0.08_160000]
gpu_selector()          : gpu, Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.[31](https://github.com/intel/llvm/actions/runs/12728797473/job/35480027430?pr=16595#step:17:32)294.120000]
custom_selector(gpu)    : gpu, Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31294.120000]
custom_selector(cpu)    : cpu, Intel(R) OpenCL, Intel(R) Xeon(R) Platinum 8468V OpenCL 3.0 (Build 0) [2024.18.10.0.08_160000]
custom_selector(acc)    : fpga, Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2024.18.10.0.08_160000]

Additional context

No response
@uditagarwal97 uditagarwal97 added the bug Something isn't working label Jan 13, 2025
@uditagarwal97
Copy link
Contributor Author

@sarnex FYI

@sarnex
Copy link
Contributor

sarnex commented Jan 13, 2025

will take a look thx

@sarnex sarnex self-assigned this Jan 13, 2025
@sarnex
Copy link
Contributor

sarnex commented Jan 13, 2025

I wasn't able to reproduce this either on normal IGC or dev IGC (where it failed), even using a flaky test repro script that almost always works for these kind of things. Closing for now, we can reopen if it happens again.

@sarnex sarnex closed this as completed Jan 13, 2025
@sarnex
Copy link
Contributor

sarnex commented Jan 13, 2025

I can't reproduce this but I am seeing it in CI. I'll reopen it and disable the test for now.

@sarnex sarnex reopened this Jan 13, 2025
@sarnex sarnex mentioned this issue Jan 13, 2025
Merged
@aelovikov-intel
Copy link
Contributor

https://github.com/intel/llvm/actions/runs/12755033830/job/35550978609 has it as well.

sarnex added a commit that referenced this issue Jan 14, 2025
@sarnex
[SYCL][ESIMD][E2E] Disable loop_extended as it's flaky (#16612)
9bf4ee4 
#16598

Signed-off-by: Sarnie, Nick <[email protected]>
@dm-vodopyanov dm-vodopyanov mentioned this issue Jan 15, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sarnex sarnex

Labels
bug Something isn't working confirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dm-vodopyanov @sarnex @uditagarwal97 @aelovikov-intel