xiAPI Image Data Flow¶
Image data delivered by the sensor is processed by multiple components of the data pipeline.
The first component is typically the Sensor Data Processor (FPGA), then data are transported through the communication link (USB, PCIe, Thunderbolt, FireWire) to the computer.
In the computer, the image data might be processed by Image Processing blocks.
Depending on camera model - some blocks are bypassed or not implemented.
The sensor data bit depth can be controlled using the SENSOR_DATA_BIT_DEPTH parameter. Decreasing this parameter can increase the maximum frame rate on some camera models.
Sensor Data Processor¶
Sensor Data Processor contains multiple blocks for shaping the data:
- FPN Correction
- Data Crop (for horizontal ROI)
- WIDTH can be used to control Data Crop unit
- Digital Gain / Offset
- GAIN_SELECTOR can be used on some cameras with value XI_GAIN_SELECTOR_ANALOG_ALL to select this data processor unit
- Defect Pixels Replacement
- Lookup Table (LUT)
- Data Packer
- The number of bits per pixel can be controlled by parameter OUTPUT_DATA_BIT_DEPTH
- Packing can be controlled by parameter OUTPUT_DATA_PACKING
- If packing is disabled and number of bits per pixel is higher than 8 -every pixel is transported as 16 bits word. E.g. Active 10 bits of value is padded by 6 bits of zeros to get 16 bits word.
- If packing is enabled - only the active bits are transported without being padded. E.g. Active 10 bits of pixel value is transported without additional padding.
Data from the camera is transported through the communication link (USB, PCIe, Thunderbolt, FireWire) to the computer.
After the transport, they are stored in computer RAM waiting for being processed or directly passed to the application.
The amount of transported data per second can be limited by parameter LIMIT_BANDWIDTH.
In computer side the transported data can be delivered to GPU (see details) or CPU memory.
This can be selected by XI_PRM_TRANSPORT_DATA_TARGET.
If data are delivered into CPU memory, they might be processed by Image Processing blocks or can be handed directly to the application bypassing the processing blocks.
By setting of XI_PRM_IMAGE_DATA_FORMAT = XI_FRM_TRANSPORT_DATA the computer data processing is completely disabled. This might save CPU and RAM access resources for other purposes.
Image Processing contains multiple blocks:
- Unpacking - transforms packed data into unpacked (e.g. 10 bits per pixel packed data converts into 16 bits per pixel for compatibility with next processing blocks).
- Auto Exposure Measurement - calculates the selected part of image characteristics to get input values for controlling exposure.
- Image Defects Correction - replaces the part of the image marked as defected by neighbouring pixels.
- White Balance Measurement - calculates the average color of the image for correction of white balance executed later in Debayering block.
- Debayering - calculates output RGB image from the input pixels values organized in Bayer matrix.
- Crop and Shift - crops the image and changing output bit depth if needed.