M31 background too busy

Questions and answers about processing in StarTools and how to accomplish certain tasks.

M31 background too busy

Postby steve72614 » Wed Apr 19, 2017 11:09 pm

Hi,
I've been experimenting with Star Tools on some old images of M31 from a dark site with a less than ideal 200mm lens. I used DSS on 20 .CR2 files with no background calibration or white balancing specified. I am aware that converting the .cr2 to .tiff might prevent some inadvertent white balancing performed by DSS, so adding dcraw preconversion to .tifs is on my to do list.

In processing the .fts output from DSS, I followed the suggested procedure on the last page of the Star Tools unofficial manual. Is there a way to prevent the background stars from being so bright, like jelly beans, and still see the detail in the galaxy? The background stars don't look realistic to me. Did I push development so hard that the stars are really exaggerated noise? It just doesn't look like the stock photos I've seen of Andromeda.

Thanks for looking.

Steve
-----------------------------------------------------------
StarTools 1.3.5.289
Mon Apr 17 15:02:42 2017
-----------------------------------------------------------
File loaded [D:\users\steve\my_docs\astronomy\our astrophotographs\2017-01-23 pictures\output FITS\Autosave.fts].
---
--- Auto Develop
Parameter [Ignore Fine Detail <] set to [6.6 pixels]
Parameter [Outside ROI Influence] set to [10 %]
--- Crop
Parameter [X1] set to [10 pixels]
Parameter [Y1] set to [10 pixels]
Parameter [X2] set to [5192 pixels (-10)]
Parameter [Y2] set to [3455 pixels (-10)]
--- Bin
Parameter [Scale] set to [(scale/noise reduction 20.00%)/(2500.00%)/(+4.64 bits)]
--- Wipe
Parameter [Mode] set to [Correct Color & Brightness]
Parameter [UNKNOWN] set to [Yes]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [3 pixels]
Parameter [Drop Off Point] set to [0 %]
Parameter [Corner Aggressiveness] set to [100 %]
Parameter [Aggressiveness] set to [87 %]
--- Auto Develop
Parameter [Ignore Fine Detail <] set to [Off]
Parameter [Outside ROI Influence] set to [15 %]
--- Deconvolution
Parameter [Image Type] set to [Deep Space]
Parameter [Mask Behavior] set to [De-ring Mask Gaps, Hide Result]
Parameter [Radius] set to [1.2 pixels]
Parameter [Iterations] set to [6]
Parameter [Regularization] set to [1.00 (optimal noise and detail)]
Parameter [Mask Fuzz] set to [8.0 pixels]
--- Contrast
Parameter [Expose Dark Areas] set to [No]
Parameter [Compensate Gamma] set to [No]
Parameter [Precision] set to [128 x 128 pixels]
Parameter [Dark Anomaly Filter] set to [3 pixels]
Parameter [Aggressiveness] set to [75 %]
Parameter [Dark Anomaly Headroom] set to [15 %]
Undo.
--- Contrast
Parameter [Expose Dark Areas] set to [No]
Parameter [Compensate Gamma] set to [No]
Parameter [Precision] set to [128 x 128 pixels]
Parameter [Dark Anomaly Filter] set to [3 pixels]
Parameter [Aggressiveness] set to [75 %]
Parameter [Dark Anomaly Headroom] set to [15 %]
--- HDR
Parameter [Small Detail Precision] set to [Max]
Parameter [Channels] set to [Brightness Only]
Parameter [Algorithm] set to [Reveal DSO Core]
Parameter [Dark/Bright Response] set to [1.74]
Parameter [Detail Size Range] set to [40 pixels]
Parameter [Noise Suppression] set to [Off]
--- Wavelet Sharpen
Parameter [Intelligent Enhance] set to [Yes]
Parameter [Scale 1] set to [100 %]
Parameter [Scale 2] set to [100 %]
Parameter [Scale 3] set to [100 %]
Parameter [Scale 4] set to [100 %]
Parameter [Scale 5] set to [100 %]
Parameter [Mask Fuzz] set to [8.0 pixels]
Parameter [Amount] set to [100 %]
Parameter [Small Detail Bias] set to [75 %]
--- Color
Parameter [Cap Green] set to [To Yellow]
Parameter [Bias Slider Mode] set to [Sliders Reduce Color Bias]
Parameter [Style] set to [Scientific (Color Constancy)]
Parameter [LRGB Method Emulation] set to [Straight CIELab Luminance Retention]
Parameter [Dark Saturation] set to [2.00]
Parameter [Bright Saturation] set to [Full]
Parameter [Saturation Amount] set to [200 %]
Parameter [Blue Bias Reduce] set to [1.18]
Parameter [Green Bias Reduce] set to [1.70]
Parameter [Red Bias Reduce] set to [1.00]
Parameter [Mask Fuzz] set to [1.0 pixels]
--- Wavelet De-Noise
Parameter [Scale 1] set to [90 %]
Parameter [Scale 2] set to [90 %]
Parameter [Scale 3] set to [90 %]
Parameter [Scale 4] set to [90 %]
Parameter [Scale 5] set to [0 %]
Parameter [Mask Fuzz] set to [1.0 pixels]
Parameter [Scale Correlation] set to [3]
Parameter [Color Detail Loss] set to [20 %]
Parameter [Brightness Detail Loss] set to [5 %]
Parameter [Grain Size] set to [2.5 pixels]
Parameter [Read Noise Compensation] set to [Off]
Parameter [Smoothness] set to [75 %]
File saved [D:\users\steve\my_docs\astronomy\our astrophotographs\2017-01-23 pictures\output FITS\M31 StarTools out from FTS.tiff].

-----------------------------------------------------------
Attachments
M31 StarTools out from FTS.jpg
M31 StarTools out from FTS.jpg (319.73 KiB) Viewed 51 times
steve72614
 
Posts: 2
Joined: Mon Apr 17, 2017 10:22 pm

Re: M31 background too busy

Postby happy-kat » Thu Apr 20, 2017 6:04 pm

Why would you want to convert your cr2 files to tiff before using in DSS?
happy-kat
 
Posts: 6
Joined: Sun Feb 01, 2015 11:31 am

Re: M31 background too busy

Postby admin » Fri Apr 21, 2017 12:10 am

Hi Steve,

Looking at your image, it appears that either something went wrong during stacking, or your focus was perhaps off. A typical stellar profile would be a bright (sometimes overexposed) point, fading out into neighbouring pixels according to the diffraction pattern of your optical train (a roughly a circle for non-stopped down lenses, spikes for mirrors, possible spikes for stopped down lenses).

There is one thing that may also cause what we're seeing - if there is something wrong with the FTS image (values encoded that outside the bounds of what the FTS specifies as max/min values), then StarTools will discard those pixels as "corrupted" and use a "neutral"/interpolated value instead that will not interfere with the many algorithms that would otherwise choke on anomalous data. If you have been saving your FTS files as flotaing point, may I suggest saving them as 32-bit Integer FTS files instead.

If you'd like us to have a look at the data, do please feel free to upload it somewhere.

Hope this helps in the meantime,
Ivo Jager
StarTools creator and astronomy enthusiast
User avatar
admin
Site Admin
 
Posts: 1531
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne

Re: M31 background too busy

Postby admin » Fri Apr 21, 2017 12:15 am

happy-kat wrote:Why would you want to convert your cr2 files to tiff before using in DSS?


The answer is that DSS insists on color balancing the data. There is a small boost in signal fidelity and noise reduction by handing StarTools data that has not been color balanced yet.

Color balancing causes noise from different channels to be blended in with other channels with different weighting factors, making noise levels in the individual channels a bit of a hodgepodge as opposed to "virgin" unbalanced data where noise vs signal is a known quantity that StarTools can track from the very beginning.

Does that make sense?
Ivo Jager
StarTools creator and astronomy enthusiast
User avatar
admin
Site Admin
 
Posts: 1531
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne

Re: M31 background too busy

Postby steve72614 » Sat Apr 22, 2017 10:31 pm

Thanks for your responses. Retried everything with dcraw input and have some more questions. Hope it's not to tedious a list.

Per Ivo's advice, today I ran the CR2 files thru dcraw to produce a tiff with all the multiplier values of 1, and all the dcraw options he suggested in another post about DSS whitebalancing. The Canon Tool reports the DSS input CR2 files had a slightly red bias, and don't look to have an unnatural color cast. The images were taken at a pretty good dark site on a good night. I then ran the fts thru ST.

DSS still produced a fts with a heavy green emphasis.

I noticed that the HDR Reveal DSO Core module is what caused some of the background noise to look a little like the stars. When sampling a star field for white, and then setting the mask to the whole image, the result was still too much green. I had to raise the green bias reduction slider to about 1.7 to get this result. Why might the sampling of a star field for white not remove the green cast?

I used a 1% read noise compensation and 7 pixel grain size in the final noise reduction step. DSS had reported a sky background of roughly 0.45% for the input images. I'm guessing that the final noise reduction noise floor could always safely be set at least as high as the DSS sky background value reported in the DSS file list. Is this the correct idea?

During the ST noise reduction step I saw what looked like a bunch of hot mostly red pixels. I'm not really clear on how to make the proper adjustment on DSS hot pixel removal. Does someone know how to adjust this DSS parameter?

I used the Autosave.fts file from DSS, so I don't think its a floating point .fts file. I'm hoping the DSS default is integer data for the Autosave.fts file.

The lens I used for these pictures would not quite reach infinity focus, as it was for a different brand of camera, and I used a lens adapter. I have since got another lens that achieves good infinity focus. Wondering how much of my problem with "busy background" is related to the poor focus.

Do anyone know why Deep Sky Stacker does not white balance .tiff input files?

Since binning my DSS autosave.fts image in Star Tools is necessary anyway, how does this compare to having DSS process the .CR2 to superpixels, achieving the equivalent of a 2x2 binning before feeding the result to Star Tools? I'd imagine the white balance would still be a little off?

Anyway, I am happier with this result than the first. Dcraw, and some more aggressive final noise reduction were the main differences from the first attempt.

Thanks,
Steve
Attachments
M31_dcraw_to_DSS_to_StarTools.jpg
M31_dcraw_to_DSS_to_StarTools.jpg (302.08 KiB) Viewed 6 times
steve72614
 
Posts: 2
Joined: Mon Apr 17, 2017 10:22 pm

Re: M31 background too busy

Postby admin » Sat Apr 22, 2017 11:47 pm

steve72614 wrote:Thanks for your responses. Retried everything with dcraw input and have some more questions. Hope it's not to tedious a list.

Please feel free to ask away - it's the only way we can learn!

Per Ivo's advice, today I ran the CR2 files thru dcraw to produce a tiff with all the multiplier values of 1, and all the dcraw options he suggested in another post about DSS whitebalancing. The Canon Tool reports the DSS input CR2 files had a slightly red bias, and don't look to have an unnatural color cast. The images were taken at a pretty good dark site on a good night. I then ran the fts thru ST.

DSS still produced a fts with a heavy green emphasis.

Great! The green (or sometimes teal) is an expected "color" for the data of most DSLRs/OSCs non-color-balanced data (specifically it is the non-color balanced skyglow and light pollution signature/bias).

I noticed that the HDR Reveal DSO Core module is what caused some of the background noise to look a little like the stars.

Hmmmm... Would you be able to post a before and after screenshot? Which version of StarTools are you running?

When sampling a star field for white, and then setting the mask to the whole image, the result was still too much green. I had to raise the green bias reduction slider to about 1.7 to get this result. Why might the sampling of a star field for white not remove the green cast?

At this point' I'd love to have a look at the data to see if there are any quick pointers I could give you.
Strange... Only a light pollution filter or (heavily) modded DSLR would skew colours this severely as you describe. Normally the Color module's initial settings/analysis (with full mask set) should do well on a widefield of M31.
I used a 1% read noise compensation and 7 pixel grain size in the final noise reduction step. DSS had reported a sky background of roughly 0.45% for the input images. I'm guessing that the final noise reduction noise floor could always safely be set at least as high as the DSS sky background value reported in the DSS file list. Is this the correct idea?


The read noise compensation is really only meant to be used in cases where noise is severe and noise influences other than the usual shot/photon noise start taking over (for example read noise), or (technical jargon alert!) where there are so few samples per pixel that the Gaussian noise model is no longer a good approximate fit for the Poisson distribution of shot/photon noise (brilliant article explaining the difference for those interested in the nitty-gritty)

During the ST noise reduction step I saw what looked like a bunch of hot mostly red pixels. I'm not really clear on how to make the proper adjustment on DSS hot pixel removal. Does someone know how to adjust this DSS parameter?

Dithering between frames should take care of such noise, while bias frames may also help (also, don’t' forget to shoot flat frames if you aren't already!)

The lens I used for these pictures would not quite reach infinity focus, as it was for a different brand of camera, and I used a lens adapter. I have since got another lens that achieves good infinity focus. Wondering how much of my problem with "busy background" is related to the poor focus.


Do anyone know why Deep Sky Stacker does not white balance .tiff input files?


That's because there is no "universal" way to white balance TIFF files; the CR2 files specify what camera was used to acquire the data. DCRAW then looks up the model and the right multiplication factors/matrix. When TIFF files are used, it presumes the image (note we're not talking about "data" any more) was already white balanced.

It's the murky difference between data vs image; the less you do the the raw photon counts, the closer you are to the definition of data. The more you process the data, the closer you are to the definition of an image/picture.

Since binning my DSS autosave.fts image in Star Tools is necessary anyway, how does this compare to having DSS process the .CR2 to superpixels, achieving the equivalent of a 2x2 binning before feeding the result to Star Tools? I'd imagine the white balance would still be a little off?

It would end up being roughly the same result, though DSS may be able to better align and stack the subs with less noisy sub-frames.
Ivo Jager
StarTools creator and astronomy enthusiast
User avatar
admin
Site Admin
 
Posts: 1531
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne


Return to Image Processing Troubleshooting

Who is online

Users browsing this forum: Google [Bot] and 2 guests