Qualcomm
® Snapdragon™ processors allow spectacular, true-to-life photos and vibrant videos to be captured through their advanced camera technologies, including the Spectra™ Image Signal Processor. The integrated signal processing technology allows capture of high resolution content from a distance without sacrificing visual quality and unwanted vibrations. The camera technology in Snapdragon platforms can scan for visual noise, allowing for bright and colorful photos regardless of lighting conditions by enhancing and sharpening images where it’s needed most. With high dynamic range (HDR) for a more vibrant color range and noise reduction for sharper and cleaner images, you can capture colors that appear as true to life as possible.
The Snapdragon camera sub-system enables these rich features through the Mobile Industry Processor Interface -Camera Serial Interface-2 (MIPI-CSI-2). This data path has proprietary modules whose sources are accessible only to Qualcomm Technology licensees like Penguin Edge and thus it becomes difficult, if not impossible for product makers to customize camera capabilities or implement features that aren’t part of the default deliverables made from the open source Code Aurora Forum by Qualcomm.
The application processor which hosts the OS, loads and executes the camera driver software is the most important hardware block in the camera data path. The camera driver controls all camera-related hardware blocks such as the Camera Control Interface which is a dedicated I
2C for the camera subsystem, the Image Front End engine which is the Image processing hardware, the Bayer Processing Segment which also handles Noise Reduction for snapshots and the Image Processing Engine block for handling post processing. The camera driver also sets up/controls clocks and configures GPIOs for the camera sensor. A block diagram of the camera subsystem within the newer Snapdragons like the
SD845 is shown below.
Qualcomm’s camera IPs are implemented and embedded as firmware within the Image Front End (IFE) and the Image processing (IPE) engine blocks blocks shown above. The IFE handles the processing of Bayer frames for preview and the 3A (AE/AF/AWB). Bayer processing includes lens shading, bad pixel and green imbalance corrections, black level, and channel gain among other things.
The Android HLOS code optimized for Snapdragon processors is made available by Qualcomm as open source on the code aurora server via git repositories. Using this as a base in conjunction with Qualcomm proprietary sources, Penguin Edge can enable camera sensors and provide enhanced features that aren’t part of the default releases from Qualcomm. A few important solutions are described below.
Dual Cameras
Application space use-cases like in AI enabled video surveillance or retail analytics necessitate dual cameras to operate simultaneously to ensure a wider field of view capture and preview. Snapdragon processors typically provide multiple MIPI-CSI ports but similar to mobile phones, it is possible to access either the front camera or the back camera but not both together.
Penguin Edge has created solutions to enable dual cameras to preview and also capture content simultaneously on their platforms running different versions of Android. Preview and capture of up to UltraHD content from two cameras simultaneously is thus possible on our platforms based on multiple generations of Snapdragons.
Enabling SoC Based sensors
SoC based sensors are more prevalent today and a popular example is the
OV5645 sensor from Omnivision. These sensors have the image processing pipeline implemented in hardware and thus the data need not be processed by Snapdragon’s camera sub-system and is thus bypassed. They directly output YUV frames that are consumed by the hardware encoder or display.
Penguin Edge has enabled SoC based sensors on its platforms running different versions of Android by integrating them as kernel modules.
Flash/Actuator/EEPROM drivers
Photography necessitates the use of a flash to be used in low-light settings or to remove harsh shadows. The flash output should be in sync with the camera sensor’s exposure time to ensure that the captured frame gets the required lighting when enabled. An actuator enables auto focusing and optical image stabilization
We have enabled synchronized flash for camera sensor modules along with integration of actuator and eeprom drivers that work in conjunction with our platforms running different versions of Android.
Enabling Digital and Analog video inputs
For video input, the MIPI-CSI standard is an extremely popular choice but some use-cases necessitate the presence of HDMI Input for content sharing from laptops, tablets, and mobile devices. Snapdragon processor based platforms have multiple four-lane MIPI Camera serial interfaces available but no HDMI or analog video inputs. We have enabled HDMI 2.0 input with support for UltraHD resolutions up to 60fps using MIPI-CSI to HDMI bridge ICs. This was implemented as a GPL Linux driver made available in source as part of our Android software release. The driver presents the video data to the application layer using Android’s Camera Manager APIs so that the HDMI video input appears as video coming from a camera input at the application layer. This feature is available on our
turnkey solution based on Snapdragon 820 for media applications that is being launched. With similar conversion bridge ICs, we have also enabled analog video inputs on our platforms.
Camera Solutions on Linux
Qualcomm’s camera IPs are embedded as firmware built for the Android source base. It is thus not trivial to make use of the on-board hardware accelerated encode engine to support the MIPI-CSI video frame data acquisition on a platform running an open source Linux BSP.
Penguin Edge has successfully ported the required proprietary modules and Media HAL to Linux and generated the binaries as Debian packages to create a Linux based BSP that supports hardware accelerated video encode in different (H264/H265) compression formats for various resolutions and frame rates including FullHD and UltraHD. The capture is enabled through a SoC based sensor and this solution is available for our platforms based on multiple generations of Snapdragon processors.
The solutions described above aren’t exhaustive and with its vast expertise on the camera pipeline and related IPs from Qualcomm,
Penguin Edge can provide solutions on long-life guaranteed hardware that cater to your specific use-case.
