If sycl-ls does not list the expected devices within the system:

1. Check that there is a compatible version of the CUDA® or ROCm™ SDK installed on the system (for CUDA or HIP plugins respectively), as well as the compatible drivers.

2. Check that nvidia-smi or rocm-smi can correctly identify the devices.

3. Check that the plugins are correctly loaded. This can be done by setting the environment variable SYCL_PI_TRACE to 1 and running sycl-ls again. For example:

   SYCL_PI_TRACE=1 sycl-ls
   
   

   You should see output similar to the following:

   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.15.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.15.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.15.1 ]
   [ext_oneapi_cuda:gpu:0] NVIDIA CUDA BACKEND, NVIDIA A100-PCIE-40GB 0.0 [CUDA 11.7]
   
   

   If the plugin you’ve installed doesn’t show up in the sycl-ls output, you can run it again with SYCL_PI_TRACE this time set to -1 to see more details of the error:

   SYCL_PI_TRACE=-1 sycl-ls
   
   

   Within the output, which can be quite large, you may see errors like the following:

   SYCL_PI_TRACE[-1]: dlopen(/opt/intel/oneapi/compiler/2023.1.0/linux/lib/libpi_hip.so) failed with <libamdhip64.so.4: cannot open shared object file: No such file or directory>
   SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so
   
   

   • The CUDA plugin requires libcuda.so and libcupti.so from the CUDA SDK.

   • The HIP plugin requires libamdhip64.so from ROCm.

   Double-check your CUDA or ROCm installation and make sure that the environment is set up properly i.e. LD_LIBRARY_PATH points to the correct locations to find the above libraries.

4. Check that there isn’t any device filtering environment variable set such as SYCL_DEVICE_FILTER (note that sycl-ls will warn if this one is set), or SYCL_DEVICE_ALLOWLIST.

When running SYCL™ applications targeting CUDA or HIP, under certain circumstances the application may fail and report an error about an invalid binary. For example, for CUDA it may report CUDA_ERROR_NO_BINARY_FOR_GPU.

This means that the SYCL device selected was provided with a binary for the incorrect architecture. In that scenario, check the following points:

1. Make sure your target is in -fsycl-targets and that the correct architecture matching the available hardware is specified with the flags:

   • Flags for CUDA: -Xsycl-target-backend=nvptx64-nvidia-cuda --cuda-gpu-arch=<arch>

   • Flags for HIP: -Xsycl-target-backend=amdgcn-amd-amdhsa --offload-arch=<arch>

2. Ensure that the correct SYCL device (matching the architecture that the application was built for) is selected at run-time. The environment variable SYCL_PI_TRACE=1 can be used to display more information on which device was selected, for example:

   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic
   SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic
   SYCL_PI_TRACE[all]: Selected device: -> final score = 1500
   SYCL_PI_TRACE[all]:   platform: NVIDIA CUDA BACKEND
   SYCL_PI_TRACE[all]:   device: NVIDIA GeForce GTX 1050 Ti
   
   

3. If an incorrect device is selected, the environment variable SYCL_DEVICE_FILTER may be used to help the SYCL device selector pick the correct one - see the Environment Variables section of the Intel® oneAPI DPC++/C++ Compiler documentation.

   Note

   The SYCL_DEVICE_FILTER environment variable will be deprecated in a subsequent release.

The icpx compiler driver does not support the option -fsycl-device-code-split=per_kernel. If you wish to use device code splitting then the DPC++ clang++ compiler driver must be used.

See Install oneAPI for NVIDIA GPUs for more information.

Execution of code will hang for CUDA or AMDGPU backend when group algorithms:

• broadcast

• joint_exclusive_scan

• joint_inclusive_scan

are compiled with the icpx compiler. If you wish to use Group Algorithms then the DPC++ clang++ compiler driver must be used.

See Install oneAPI for NVIDIA GPUs for more information.

The icpx compiler driver will give incorrect results for group algorithms:

• exclusive_scan_over_group

• inclusive_scan_over_group

for CUDA or AMDGPU backends. If you wish to use Group Algorithms then the DPC++ clang++ compiler driver must be used.

See Install oneAPI for NVIDIA GPUs for more information.

This may be caused by a number of things.

1. There is currently no support for std::complex in DPC++. Please use sycl::complex instead.

1. The icpx compiler driver does not support CXX stdlib funcs such as std::cos, logf, sinf etc within device code. If you wish to use CXX stdlib funcs within kernels then the clang++ compiler driver must be used.

   See Install oneAPI for NVIDIA GPUs for more information.

If the CUDA SDK is not installed in a standard location, clang++ may fail to find it - leading to errors during compilation such as:

clang-16: error: cannot find libdevice for sm_50; provide path to different CUDA installation via '--cuda-path', or pass '-nocudalib' to build without linking with libdevice

To fix this issue, specify the path to your CUDA installation using the --cuda-path option.

Some nvptx builtins that are used by the DPC++ runtime require a minimum Compute Capability in order to compile. If you have not targeted a sufficient Compute Capability for a builtin that you’re using in your program (by using the compiler argument -Xsycl-target-backend=nvptx64-nvidia-cuda --cuda-gpu-arch=sm_xx), an error with the following pattern will be reported:

error: '__builtin_name' needs target feature (sm_70|sm_72|..),...

In order to avoid such an error, ensure that you are compiling for a device with a sufficient Compute Capability. If you are still getting such an error despite passing a supported Compute Capability to the compiler, this may be because you are passing the 32-bit triple, nvptx-nvidia-cuda to -fsycl-targets. The nvptx-nvidia-cuda triple does not allow the compilation of target feature builtins and is not officially supported by DPC++. The 64-bit triple, nvptx64-nvidia-cuda, is supported by all modern NVIDIA® devices, so it is always recommended.

Depending on the CUDA version used with the release, the compiler may output the following warning:

clang++: warning: CUDA version is newer than the latest supported version 11.5 [-Wunknown-cuda-version]

In most cases this warning can safely be ignored. It simply means that DPC++ may not use some of the latest CUDA features, but it should still work perfectly fine in most scenarios.