Proper Way to Capture & Save Color Images via SDK?

Runsva

I'm attempting to capture and save color images from an ASI585MC camera on a Raspberry Pi 4 Model B, via the ASI SDK, in C++.

I'm able to capture images in monochrome by initializing the camera by calling the ASIInitCamera function, and subsequently calling the ASISetROIFormat function with the following image format settings:

iBin = 1
Img_type = ASI_IMG_RAW8

Capturing the image and then gathering the data by calling the ASIGetDataAfterExp function into an array of type unsigned char successfully produces a monochrome image.

However, I wanted to take a color image from the 585MC camera, since it's a camera capable of capturing color images.

Since I never really worked with camera SDKs before, the first question I would have is what datatype would the image data have to be saved in? The monochrome image was saved in the ASI_IMG_RAW8 type, which gives possible values of 0 - 255 for each pixel. Does that mean I have to use the ASI_IMG_RAW16 instead for a color image?

If so, how would I call the ASIGetDataAfterExp function to gather my captured image data after a camera snap? That function only accepts the image buffer array of type unsigned char, which would only work for a monochrome image. I'm assuming for a color image I would have to create a buffer array of type unsigned short or something along the lines of that. Is there a different function used for gathering color image data?

Thank you for reading my post, any guidance is appreciated.

w7ay

Runsva However, I wanted to take a color image from the 585MC camera, since it's a camera capable of capturing color images.

Capture it as a monochrome image, and Debayer that monochrome image using the "RGGB" pattern.

https://en.wikipedia.org/wiki/Bayer_filter

Except for the Foveon sensor, there is no such thing as a native CMOS color sensor. All the CMOS color sensors are made of a monochrome sensor, with an array of RGB filters over the individual photosites. I.e., some pixels collect only red wavelengths, some collect only green wavelengths, and some photosites only collect blue wavelengths.

When you read the data from the "color" sensor, each photosite gives you only one pixel value (not an RGB vector). I.e., the data that is read back is a monochrome image. Just that some of the pixels represents red, some represent green, and some represent blue.

You then reconstruct the color image through a process called DeBayering. This is why images from color sensors are lower in resolution than images from monochrome sensors.

https://en.wikipedia.org/wiki/Demosaicing

https://www.altairastro.help/info-instructions/cmos/what-is-debayering-and-why-should-i-do-it-before-stacking-my-images/

Chen

Runsva

w7ay Thank youy for your detailed reply;

Thank you for sharing the details about De-Bayering. It makes sense now to me how the monocrhome image is transformed into a color one.

I noticed that one of the properties that the color camera has, unlike the monochrome one, is the fact that it has a "Bayer Pattern" property. When I capture the monochrome image, I'm assuming I should then still save it as the ASI_IMG_RAW8 datatype. However, what ASI SDK function do I call to perform the De-Bayering that you described? Is there even a function for that in the ASI SDK, or does that need to be performed via the OpenCV library as well?

w7ay

Runsva ASI SDK function do I call to perform the De-Bayering that you described?

Just Google for Debayer algorithms and you will find a number of them. It usually involves interpolating for the "missing" pixels for a given color plane.

You can do this by apppling a lowpass filter with the proper kernel to recover all missing pixels of a color plane. It will give you a proper Nyquist limited image. This is probably the best to use if you have further processing stages that sharpens the final image, since deconvolution actually makes sense.

Notice that the green component has half the resolution of the original monochrome image, and the red and blue channels has 1/(2 . sqrt( 2) ) the resolution of the original monochrome image. So, will need some form of sharpening, otherwise the result is quite low in resolution.

Many debayering algorithms try to be "cute" and try to produce an RGB image using ad-hoc methods to try to get an image that looks sharper. Under large magnification, or deconvolution, these often produces artifacts (since they are trying to derive data that is not there (Nyquist criterion) in the first place.

Again, just google for debayering, and demosaicing (Bryce Bayer from Eastman Kodak is the inventor who placed a color mosaic over a monchrome sensor).

Good luck.

Chen

Runsva

w7ay Thank you for your reply;

Thank you, but what format do I save the original image data in prior to the conversion? ASI_IMG_RAW8, ASI_IMG_RAW16 or something else?

w7ay

Runsva ASI_IMG_RAW8, ASI_IMG_RAW16 or something else?

Either should be fine, depending on the frame rate, you may need to use 8 bits. Just read it in as if it is a monochrome sensor, an then assign the bayer components to each pixel.

In fact, a quick test to see if the color is right is to simply convert a red pixel to a color vector of (R,0,0), a Green pixel to (0,G/2,0), and a blue pixel to (0,0,B). The result is a color image that when viewed at a distance looks like a color image before any white point correction. The reason for G/2 is that there are twice as many green pixels as there are red and blue pixels.

Chen

Zzzz

w7ay Thank you very much for your professional answer.