This section covers troubleshooting tips and solutions to common issues. If the following doesn’t fix your problem, please submit a support request via Codeplay’s community support website. We cannot provide any guarantees of support, but we will try to help. Please ensure that you are using the most recent version of the software before submitting a support request.
Any issues can also be reported via the oneAPI DPC++ compiler open-source repository.
Bitcode output disabled with icpx
The current version of icpx
has a known issue when compiling for Nvidia or
AMD targets, and may output the following error:
LLVM ERROR: Bitcode output disabled because proprietary optimizations have been performed.
This will be fixed in the next release, but in the meantime please use the
clang++
executable directly as described in this guide instead of the
icpx
executable, when targeting Nvidia or AMD GPUs.
Missing Devices in sycl-ls Output
If sycl-ls
does not list the expected devices within the system:
Check that there is a compatible version of the CUDA® or ROCm™ SDK installed on the system (for CUDA or HIP plugins respectively), as well as the compatible drivers.
Check that
nvidia-smi
orrocm-smi
can correctly identify the devices.Check that the plugins are correctly loaded. This can be done by setting the environment variable
SYCL_PI_TRACE
to1
and runningsycl-ls
again. For example:SYCL_PI_TRACE=1 sycl-ls
You should see output similar to the following:
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.15.1 ] SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.15.1 ] SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.15.1 ] [ext_oneapi_cuda:gpu:0] NVIDIA CUDA BACKEND, NVIDIA A100-PCIE-40GB 0.0 [CUDA 11.7]
If the plugin you’ve installed doesn’t show up in the
sycl-ls
output, you can run it again withSYCL_PI_TRACE
this time set to-1
to see more details of the error:SYCL_PI_TRACE=-1 sycl-ls
Within the output, which can be quite large, you may see errors like the following:
SYCL_PI_TRACE[-1]: dlopen(/opt/intel/oneapi/compiler/2023.0.0/linux/lib/libpi_hip.so) failed with <libamdhip64.so.4: cannot open shared object file: No such file or directory> SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so
The CUDA plugin requires
libcuda.so
andlibcupti.so
from the CUDA SDK.The HIP plugin requires
libamdhip64.so
from ROCm.
Double-check your CUDA or ROCm installation and make sure that the environment is set up properly i.e.
LD_LIBRARY_PATH
points to the correct locations to find the above libraries.Check that there isn’t any device filtering environment variable set such as
SYCL_DEVICE_FILTER
(note thatsycl-ls
will warn if this one is set), orSYCL_DEVICE_ALLOWLIST
.
Dealing with Invalid Binary Errors
When running SYCL™ applications targeting CUDA or HIP, under certain
circumstances the application may fail and report an error about an
invalid binary. For example, for CUDA it may report
CUDA_ERROR_NO_BINARY_FOR_GPU
.
This means that the SYCL device selected was provided with a binary for the incorrect architecture. In that scenario, check the following points:
Make sure your target is in -fsycl-targets and that the correct architecture matching the available hardware is specified with the flags:
Flags for CUDA:
-Xsycl-target-backend=nvptx64-nvidia-cuda --cuda-gpu-arch=<arch>
Flags for HIP:
-Xsycl-target-backend=amdgcn-amd-amdhsa --offload-arch=<arch>
Ensure that the correct SYCL device (matching the architecture that the application was built for) is selected at run-time. The environment variable
SYCL_PI_TRACE=1
can be used to display more information on which device was selected, for example:SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 11.16.1 ] SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 11.16.1 ] SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_cuda.so [ PluginVersion: 11.16.1 ] SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic SYCL_PI_TRACE[all]: Requested device_type: info::device_type::automatic SYCL_PI_TRACE[all]: Selected device: -> final score = 1500 SYCL_PI_TRACE[all]: platform: NVIDIA CUDA BACKEND SYCL_PI_TRACE[all]: device: NVIDIA GeForce GTX 1050 Ti
If an incorrect device is selected, the environment variable
SYCL_DEVICE_FILTER
may be used to help the SYCL device selector pick the correct one - see the Environment Variables section of the Intel® oneAPI DPC++/C++ Compiler documentation.Note
The
SYCL_DEVICE_FILTER
environment variable will be deprecated in a subsequent release.
Compiler Error: “cannot find libdevice”
If the CUDA SDK is not installed in a standard location, clang++
may
fail to find it - leading to errors during compilation such as:
clang-16: error: cannot find libdevice for sm_50; provide path to different CUDA installation via '--cuda-path', or pass '-nocudalib' to build without linking with libdevice
To fix this issue, specify the path to your CUDA installation using the
--cuda-path
option.
Compiler Error: “needs target feature”
Some nvptx builtins that are used by the DPC++ runtime require a minimum
Compute Capability in order to compile. If you have not targeted a
sufficient Compute Capability for a builtin that you’re using in your
program (by using the compiler argument
-Xsycl-target-backend=nvptx64-nvidia-cuda --cuda-gpu-arch=sm_xx
), an error with the following
pattern will be reported:
error: '__builtin_name' needs target feature (sm_70|sm_72|..),...
In order to avoid such an error, ensure that you are compiling for a device
with a sufficient Compute Capability.
If you are still getting such an error despite passing a supported Compute
Capability to the compiler, this may be because you are passing the 32-bit
triple, nvptx-nvidia-cuda
to -fsycl-targets
. The
nvptx-nvidia-cuda
triple does not allow the compilation of target
feature builtins and is not officially supported by DPC++. The 64-bit
triple, nvptx64-nvidia-cuda
, is supported by all modern NVIDIA® devices,
so it is always recommended.
Compiler Warning: “CUDA version is newer than the latest supported version”
Depending on the CUDA version used with the release, the compiler may output the following warning:
clang++: warning: CUDA version is newer than the latest supported version 11.5 [-Wunknown-cuda-version]
In most cases this warning can safely be ignored. It simply means that DPC++ may not use some of the latest CUDA features, but it should still work perfectly fine in most scenarios.
Linking Error with nvvm-reflect-ftz
Currently a bug in DPC++ prevents enabling FTZ when targeting the CUDA
plugin. FTZ may also be enabled automatically by -ffast-math
or
-Ofast
, which would cause errors such as:
error: linking module flags 'nvvm-reflect-ftz': IDs have conflicting override values in '/tmp/lit-tmp-045kklh1/clang++-b050f4/libsycl-crt-a42c81.cubin' and 'llvm-link'
In this case, FTZ must be disabled to fix the issue. When using
-ffast-math
or -Ofast
, that can be done by also adding
-Xclang -fdenormal-fp-math=ieee
.
Additionally, make sure that FTZ isn’t enabled by another flag such as
-fcuda-flush-denormals-to-zero
, -fdenormal-fp-math=preserve-sign
or
-fdenormal-fp-math=positive-zero
.