The current version of icpx has a known issue when compiling for Nvidia or AMD targets, and may output the following error:

LLVM ERROR: Bitcode output disabled because proprietary optimizations have been performed.

This will be fixed in the next release, but in the meantime please use the clang++ executable directly as described in this guide instead of the icpx executable, when targeting Nvidia or AMD GPUs.

If sycl-ls does not list the expected devices within the system:

1. Check that there is a compatible version of the CUDA® or ROCm™ SDK installed on the system (for CUDA or HIP plugins respectively), as well as the compatible drivers.

2. Check that nvidia-smi or rocm-smi can correctly identify the devices.

3. Check that the plugins are correctly loaded. This can be done by setting the environment variable SYCL_PI_TRACE to 1 and running sycl-ls again. For example:

   SYCL_PI_TRACE=1 sycl-ls
   
   

   You should see output similar to the following:

   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.15.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.15.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.15.1 ]
   [ext_oneapi_cuda:gpu:0] NVIDIA CUDA BACKEND, NVIDIA A100-PCIE-40GB 0.0 [CUDA 11.7]
   
   

   If the plugin you’ve installed doesn’t show up in the sycl-ls output, you can run it again with SYCL_PI_TRACE this time set to -1 to see more details of the error:

   SYCL_PI_TRACE=-1 sycl-ls
   
   

   Within the output, which can be quite large, you may see errors like the following:

   SYCL_PI_TRACE[-1]: dlopen(/opt/intel/oneapi/compiler/2023.0.0/linux/lib/libpi_hip.so) failed with <libamdhip64.so.4: cannot open shared object file: No such file or directory>
   SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so
   
   

   • The CUDA plugin requires libcuda.so and libcupti.so from the CUDA SDK.

   • The HIP plugin requires libamdhip64.so from ROCm.

   Double-check your CUDA or ROCm installation and make sure that the environment is set up properly i.e. LD_LIBRARY_PATH points to the correct locations to find the above libraries.

4. Check that there isn’t any device filtering environment variable set such as SYCL_DEVICE_FILTER (note that sycl-ls will warn if this one is set), or SYCL_DEVICE_ALLOWLIST.

When running SYCL™ applications targeting CUDA or HIP, under certain circumstances the application may fail and report an error about an invalid binary. For example, for CUDA it may report CUDA_ERROR_NO_BINARY_FOR_GPU.

This means that the SYCL device selected was provided with a binary for the incorrect architecture. In that scenario, check the following points:

1. Make sure your target is in -fsycl-targets and that the correct architecture matching the available hardware is specified with the flags:

   • Flags for CUDA: -Xsycl-target-backend=nvptx64-nvidia-cuda --cuda-gpu-arch=<arch>

   • Flags for HIP: -Xsycl-target-backend=amdgcn-amd-amdhsa --offload-arch=<arch>

2. Ensure that the correct SYCL device (matching the architecture that the application was built for) is selected at run-time. The environment variable SYCL_PI_TRACE=1 can be used to display more information on which device was selected, for example:

   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.16.1 ]
   SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic
   SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic
   SYCL_PI_TRACE[all]: Selected device: -> final score = 1500
   SYCL_PI_TRACE[all]:   platform: NVIDIA CUDA BACKEND
   SYCL_PI_TRACE[all]:   device: NVIDIA GeForce GTX 1050 Ti
   
   

3. If an incorrect device is selected, the environment variable SYCL_DEVICE_FILTER may be used to help the SYCL device selector pick the correct one - see the Environment Variables section of the Intel® oneAPI DPC++/C++ Compiler documentation.

   Note

   The SYCL_DEVICE_FILTER environment variable will be deprecated in a subsequent release.