Gain with binning

JeffMorgan

Nuts. Maybe I will trade down from the ASI174 to the ASI290.

c131frdave

Wow, I'll have to read that a few times. I'm still not 100% sure what we're talking about. In simpler terms, gain does what? It increases light sensitivity, lowers read noise, and decreases dynamic range? Physically speaking, binning obviously decreases resolution, but in doing so it decreases read noise? I've read that CMOS averages binned pixel values instead of adding them like CCD does. So "sensitivity" wouldn't increase unless you what, increased gain? But increased gain lowers noise to begin with. It seems backwards. If I increase gain on a DSLR, it greatly increases noise. Still confused.

w7ay

c131frdave In simpler terms, gain does what?

OK, I think I may have described the process in a different thread.

Photons are converted to electrons with the Photoelectric process. The electrons are called photoelectrons when they are created this way, so you may see them called that instead of electrons.

Each electron carries a negative charge. When the change moves, it creates an electrical current (i = dq/dt).

The current is then converted to a voltage (easy to do by injecting the current node into the input of a high impedance op-amp, for example).

The gain that you read about is now applied to this voltage -- i.e., it is an analog voltage gain.

The result is fed to an analog to digital converter (ADC) to convert to numbers (called analog digital units, or the ADU that you read about).

The ADU are quantized representation of the actual voltage and therefore the A/D process adds what is called quantization noise (you can check Wikipedia if any of these terms are not familiar to you).

Now, if the input signal is small, then the quantization noise would be large relatively, and be bad for SNR. Conversely, if you make the gain large, the quantization noise would become relatively small, but the ADC is saturated before the sensor is saturated -- thus losing dynamic range for no reason. You can see this effect in camera curves -- as you increase gain, the read noise (measured relative to the photoelectron e-) comes down; at the same time, the dynamic range also falls.

So, "gain" really does not directly improve the SNR of the analog signal coming from the sensor. It simply reduces the quantization noise from the ADC. Before the ADC, if you increase gain, both the analog signal and analog noise increases by the same amount and does not change the SNR.

With the presence of the ADC, increasing the gain will always improve the SNR. But you have to consider the ill effect of the dynamic range reduction too. This is why for very short exposures, where the signal is not strong enough to saturate the ADC, you should apply as much gain as possible, since it will always help, and not hurt. For long exposures, the other noises (like dark current noise and also the shot noise from the sky background) will start to eat into the dynamic range, and you should use higher gain judiciously. (This is why for guiding, I had recommend that you increase the gain much more than the "60" that you are using -- it will saturate a couple of very bright stars, but the background will still be low, and at the same time, it will improve the SNR of the majority of stars). Ditto the "lucky imaging" for planetary with millisecond-type exposures. There, you want to use as much gain as possible as long as the sensor does not saturate.

BTW, before the analog voltage is sent to the ADC, you add a constant DC voltage (this is the camera "offset" that you read about). This is because the signal and noise up to this point are always positive (signal is from discrete photons, and cannot be negative, so the shot noise has Poisson distribution) while the quantization noise is bipolar (can be modeled as uniform distribution from +/- Q where Q is one ADU). The offset makes sure the ADC does not "bottom out" which would be disastrous when you try to improve SNR by summing sub-frames.

By the way, quantization noise is quite small in practice. The variance of the noise from the ADC is only 1/12 of one ADU (i.e., RMS noise that is 0.288 of one ADU). The 1/12 is from memory; too lazy to evaluate the integral myself right now -- check wikipedia for the variance of uniform distribution noise for the true number. Basically the integral of a constant from x = -0.5 to x = +0.5.

Chen

c131frdave

I guess a better way of asking is, what advantage is having a better S/N ratio on short guide exposures? I get it for stacked imaging, but for guiding through an off axis guider at 1000mm focal length, all I want is high sensitivity because I'm trying to guide often using very dim stars because my guider's FOV is only .32 degress by .18 degrees. That's pretty tight. For guys using SCTs or larger, longer scopes, it gets super tight. What effect does binning and/or gain have on my problem? Thanks!!!

JeffMorgan

w7ay Good stuff, I look forward to putting into practice tonight. So in the context of guide cameras - exposures of 1 to 2 seconds:

The guider seems to avoid the brighter stars (if it has a choice), having any of those saturate full-well is not an issue at all.

Medium gain is in most cases should be better than Low gain. (Perhaps an exception would be a FOV where all of the guide stars were bright.)

Better yet is to try High gain. This gives the lowest noise, best SNR. No real concerns about going full well on a 2 second exposure. Even dynamic range is not a bad trade-off as the ASIAir seems to select within a narrow brightness range.

c131frdave

Ah! I get it now. Thank you for that. When you increase gain you are effectively pulling the dynamic range curve down, decreasing your well depth. So this begs the question- how do you know when you have "filled the well", so to speak? In other words, I would assume this fact would dictate how long your exposures should be. You want to keep as much dynamic range as you can, so you would want your very brightest stars to nearly fill the bucket so that you give super faint light to at least get in the game. Is there a way to identify your well "usage" to determine ideal exposure while setting up? Also, if you have a subject with a high dynamic range itself, you would use a lower gain setting, while something that is pretty much the same brightness, you might want to up the gain to reduce noise? It's an exercise in compromise, it seems. Shoot M51, use higher gain. Shoot M16, use lower gain. Yes?

w7ay

c131frdave you are effectively pulling the dynamic range curve down, decreasing your well depth

The well depth (property of the sensor) does not change, but if you use too much gain, the ADC will clip before you have reach the depth of the well. This is the destructive part of using too much gain -- you are just wasting the well depth that you pay good money for.

You want to keep as much dynamic range as you can, so you would want your very brightest stars to nearly fill the bucket so that you give super faint light to at least get in the game.

With multi-star guiding, the game has changed a little. You do want to saturate the brighter stars. That is because of the distribution of stellar magnitudes. If every star has the same absolute magnitude (e.g., every star is exactly the size and age of the Sun), then the inverse square law tells us that there are many more dim stars than there are bright stars. If you let the few bright stars saturate, and let the multi-star algorithm ignore them (the algorithm should, otherwise it is a bug to guide with saturated stars), then you have many more stars you can guide with -- and the additional gain will help with the read noise. You can't carry this to extremes though, since the fixed pattern noise on CMOS sensors start to rear its head -- remember, increasing the gain also increases the sensor noise -- the gain only reduces quantization noise from the ADC.

It's an exercise in compromise, it seems.

Now that you know what to look for, you should be able to choose a gain for your situation. Just like everything else in this hobby, you can't just use someone else's magic number (like what the clueless Facebook kids keep asking for). It will depend on your particular situation -- sky background, f-number, exposure time, etc. And then find the best compromise.

You can always pay more and compromise less. But people who buy from ZWO apparently are people who look for cheap stuff; not best stuff. Unfortunately, many of them also don't know you also need to be more careful how you use the sensors that are not designed to be used for astronomy. The same is true with ZWO's other products. I'll bet most (not just many, but most) people who buy the new ZWO mount don't know what compromises they have to make when they use a strain wave gear (Harmonic Drive [tm]) based mounts. Strain wave gears are meant for robotic arms (and in the case of the Mars Rover, used to drive the vehicle's wheels), not for low periodic errors. Some people just buy things based on internet shills.

Chen

c131frdave

Thanks Chen. Great stuff. You've helped me to understand this subject quite a bit.

JeffMorgan

Ok, results are in from first night out with guide camera gain set to High. Early results very encouraging!

Another breezy night here, average wind speed over the session (from the automated weather station at the local airport) was 7.5 knots - about 8.5 mph. Nevertheless, checking phdlogview I pulled down my RMS by about 0.05 arc seconds for RA, DEC, and Total compared to recent averages. And that was on a night I might normally have passed on due to wind! Performance did not drop off until my target approached Dobson's Hole (I'm using a TTS Panther alt-az mount).

Most importantly, all of my subs had acceptable Star Shape scores in Astro Pixel Processor. A quick glance suggests that my FWHM scores have tightened up also. I just might keep the ASI174.

Chen, one more question if I might: The Guide Camera module in ASIAir has "presets" for Gain: Low, Medium, and High. But there is also a slider control for settings within (and beyond) the preset values. I'm wondering if the presets are just a marketing consideration (as easy as 1-2-3)?

I'm curious to go beyond the High gain setting, though the performance charts suggest very marginal gains. Perhaps that is the reason for the presets?

w7ay

JeffMorgan The Guide Camera module in ASIAir has "presets" for Gain: Low, Medium, and High. But there is also a slider control for settings within (and beyond) the preset values. I'm wondering if the presets are just a marketing consideration (as easy as 1-2-3)?

The ASIAIR presets are not very meaningful for guiding. Low is usually zero analog gain (for highest dynamic range), high is usually some arbitrary value (and usually not really that high for the short exposure case), and often a medium value that could correspond to what is called "unity gain" (that is when one ADU corresponds to one electrophoton).

Gain 0 is to avoid saturation; but as we discussed earlier, you can (and in fact good to) allow brighter stars to saturate and obtain a larger population of stars to do multi-star guiding -- so you most likely want to avoid Low gain -- that setting is for when you don't want any color component of a star to saturate, else you loose star color for that bright star. We don't care about that for guiding -- we will be tossing the saturated bright stars anyway; and we won't be using color filters.

What you want to do is to experiment a bit with the gain slider, perhaps changing 6 dB (60 ZWO gain steps) at a time, and then fine tuning with 2 or 3 dB (20 to 30 ZWO gain steps) to look for a sweet spot. With multi-star guiding, there is a sweet range where you get 12 (in the case of ZWO's multi-star guiding) stars. The sweet spot will depend on the transparency of the sky that night, the seeing of the sky that night, the region of the sky you are autoguiding in, etc. You can't use the same value even on the same night -- this is one of the things where I shake my head when I read of someone asking for a preset number from someone else (be it gain, or exposure time, or calibration steps).

ZWO could have implemented an auto-gain pass to search for the sweet spot for the user, but instead waste their development effort on the "look, shiny!" stuff to please the Facebook selfie takers. As I have mentioned many times earlier, the mantra "Simple as 1,2,3" is the result of not including useful features (especially the missing parameters in autoguiding, like MinMo). If other programs start chopping functionality, they would be "Simple as 1,2,3" too!

In short, forget the L, M, H nonsense, and use the gain slider instead. If you only have the choice of L, M and H, you may not even be able to achieve 12 stars for guiding.

Just remember that there is a sweet spot (or region). If you use too low a gain, the read noise will eat you alive with the short exposures. If you use too high a gain, all the good stars become saturated, and you are left guiding with lower SNR of the really dim stars (i.e., again, SNR of the usable stars degrade). There is a goldilocks region -- once you find the range of gains that give you 12 stars, dial in the lowest gain within that range. (By the way, some of the selected stars in the ZWO user interface could be hiding behind useless ("look, shiny!") panes, or faded behind the guide graph area.)

I had forgotten that I changed my exposure times from 2 seconds to 1 second.

Mounts like the Avalons and the RainbowAstro are even happier with shorter exposure times. I have been using an RST-135 since that model was introduced a couple of years back. My earlier Takahashi EM-11 could take longer exposure times; but I seldom take it outdoors nowadays.

Predominantly, the mounts that benefit from short exposure times (== fast frame rates) have very large inherent periodic error amplitude. But the periodic error is mostly sinusoidal (with very little higher harmonics), and most importantly, they have zero or very low backlash. I had mentioned this a few times in the past -- the fast frame rate requirement has to do with the need to overcome the rate of change (d/dt for you engineers or scientists) of the gear errors so the feedback loop won't be "chasing the error" and actually cause instability. In more extreme cases, short exposures also avoid stars turning into short streaks, which is really detrimental to finding the centroid.

If you don't have a premium mount (say a 10Micron class) that is very well polar aligned, you really want to avoid long guide exposure times that are over 5 seconds.

With my RST135, a frame rate of 2 (thus exposure time of less than 0.5 seconds) easily beats out a frame rate of 1 (1 second exposure) in spite of losing SNR. The SNR deteriorates with shorter exposure times. That is why I plan on experimenting with Kalman filtering and also motion deblurring techniques to let me still use 1 second exposures -- not done with ASIAIR of course -- my experimental software platform is based on INDIGO and runs on macOS; the Mac Studio Ultra has oodles of compute power.

If you are sure that you have low backlash (for example, a Harmonic drive or belt-driven mount), you might try 0.5 second exposures to see if it gets better. Now, I wouldn't do that if you already don't have enough SNR. I use a guide scope with a 55mm objective. You can pretty much forget about using very short exposure time if you are photon starved (small guide scopes or OAG).

Have fun with experimenting -- this is what technical hobbies are all about.

Chen

JeffMorgan

A quick follow-on to my last post: Not exactly an Apples to Apples comparison. I had forgotten that I changed my exposure times from 2 seconds to 1 second.

Checking SNR values with a recent session using 1 second exposure and Medium gain, improvement looks about 2.5x. I'm not going to crunch the numbers too hard on that since the high springtime winds we get here (we're stuck in the cool breezy April weather pattern) have cut transparency noticeably.

But I'm still pretty jazzed about using the High gain setting on the guide camera.

JeffMorgan

w7ay If you are sure that you have low backlash (for example, a Harmonic drive or belt-driven mount), you might try 0.5 second exposures to see if it gets better. Now, I wouldn't do that if you already don't have enough SNR.

TTS claims spring-loaded gears and zero backlash, guess I will put that to the test.

Thanks for the explanation on gain. Calm wind forecast tonight, I'll start tuning for the lowest gain that yields 12 guide stars. On the Crescent Nebula (NGC 6888) I have a feeling that is going to be lower than I used last night. But on M 13 it will be higher.