XIMEA API can make use of NVIDIA's GPUDirect RDMA feature on supported configurations.
- x86_64 Linux system with Quadro or Tesla NVIDIA GPU
- PCIe cameras (xiB,xiX,xiT families)
- XIMEA API package version 4.11.13 or later, installed with
- proprietary NVIDIA video drivers
- CUDA toolkit version 6 or later (tested with version 7.5)
How to enable¶
First you need to enable GPUDirect support in XIMEA kernel driver:
Then you should set relevant xiApi parameters in your code:
xiSetParamInt(handle, XI_PRM_BUFFER_POLICY, XI_BP_UNSAFE); xiSetParamInt(handle, XI_PRM_IMAGE_DATA_FORMAT, XI_FRM_TRANSPORT_DATA); xiSetParamInt(handle, XI_PRM_TRANSPORT_DATA_TARGET, XI_TRANSPORT_DATA_TARGET_GPU_RAM); // Lower the size of acquisition buffer because it's limited by GPU's BAR size, e.g. (256 - 32) MB xiSetParamInt(handle, XI_PRM_ACQ_BUFFER_SIZE, 200000000);
Note that you can't use safe buffer policy or image format other than transport data.
How to use¶
XI_PRM_TRANSPORT_DATA_TARGET is set to
xiGetImage returns a pointer to GPU memory in
bp field of
XI_IMG structure. This saves you one
cudaMemcpy operation for copying the data from CPU to device memory on each acquired frame.
GPU memory is allocated using
xiStartAcquisition function and deallocated (using
xiStopAcquisition. You should select necessary CUDA device using
cudaSetDevice before calling
Attached is an example demonstrating xiAPI GPUDirect feature: GPUDirect-RDMA.tar.bz2. It is based on
3_Imaging/histogram from CUDA samples which computes 64-bin histogram on GPU. Results are displayed on the terminal using ASCII-art. Also this application prints time measurements, so you can compare the time needed for running the computation with and without GPUDirect enabled. Please refer to
readme.txt included in the tarball for building instructions.
More information about GPUDirect RDMA technology can be found on NVIDIA website: http://docs.nvidia.com/cuda/gpudirect-rdma/