Struggling with image background with new CMOS camera

Questions and answers about processing in StarTools and how to accomplish certain tasks.
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

I have been trying to get a salvageable picture from the 1st set of images that I took with my new ASI294 MC Pro camera. I think that there is a noisy background just below the surface of my stacked FITS image. This was stacked in APP but I think I may be seeing the same problem when I tried ASTAP.

There is a horizontal texture (similar to the Dark Flats pattern) that always seems to show up when I process the image in Star Tools. The pattern is a little hard to see on my MacBook screen, but when used as a background on a larger monitor it is very noticeable.

I am wondering whether there might be a way of subtracting this pattern while preserving the focal interest in the image. (M101)

I tried several sets of calibration frames. I will attach a link to my final FITS image for 3 of the different methods I tried.
M101_210610.fits: This is the original processed APP stack.
M101_210610_Try2: I stacked the lights with a later set of flats and dark flats.
M101_210610_NoFlats: This is the original lights with only darks.

https://www.dropbox.com/s/zsuagz8c64ab5 ... d.zip?dl=0

Hopefully the above comes across. In my opinion the best FITS from above to work with is the "Try2" image.

As I mentioned above I could go through a processing in Star Tools, it would look fine on my MacBook, but when made a screen desktop the true nature of the background became evident.

My original processing in Star Tools was the worst. I could actually see the textured horizontal pattern in it. I will attach this image below.
M101_210610.jpg
M101_210610.jpg (338.69 KiB) Viewed 3184 times
I am hoping that someone might be able to give me an idea of how I could salvage my 1st try with my ASI294MC Pro camera. I have never run into this type of issue when using my Nikon camera.
ionia23
Posts: 62
Joined: Fri Jul 16, 2021 7:39 pm

Re: Struggling with image background with new CMOS camera

Post by ionia23 »

I'd have a hard time giving any suggestions on this one, because I think this image is glorious. What makes it look beautiful to me is how "natural" it appears. I'm working on a series of images of the same object and I have done nowhere near as well. Is there any possibility of posting the Startools log of this image?
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Re: Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

ionia23 wrote: Fri Jul 23, 2021 9:52 pm I'd have a hard time giving any suggestions on this one, because I think this image is glorious. What makes it look beautiful to me is how "natural" it appears. I'm working on a series of images of the same object and I have done nowhere near as well. Is there any possibility of posting the Startools log of this image?
Ionia,

I thank you for the compliment on my 1st final image. To be honest with you, I would be perfectly happy with the image if only there wasn't a very distracting dark flats style pattern in the background. (You aren't the 1st to compliment this image; this also happened when I described my problem on the Cloudy Nights forum. Hopefully it's not seen that I'm crying wolf, once this same image is used as a desktop image on a fairly large monitor it really is an eye sore).

Here is a closeup of the posted image. Alternating rows of blotchy black and grey should be seen.
M101 Closeup.png
M101 Closeup.png (558.93 KiB) Viewed 3167 times
I processed this a little ways back so I don't recall the exact processing order.

If I was to make my best guess at this I would say that it was done very simply. I believe I started from the top (auto develop), skipped film and then did every other option in order all the way down to the shrink icon.

I may also recall that I forgot to region of interest select around my galaxy when I auto developed.

This was the only processing which gave me a delightful coloring of the galaxy itself. Every other processing I did had a more subdued golden color.

This processing also had the worst background of all the images that I processed in Star Tools.

If someone were to open any of the 3 FITS images (from my stacking in APP) I don't believe that they would see the same background pattern that shows up after processing in Star Tools. The best of these 3 FITS images that I linked in my 1st post I believe is Try2. I am wondering if someone might be able to process this FITS without the addition of the background pattern?
PaulE54
Posts: 40
Joined: Wed May 29, 2019 4:04 pm

Re: Struggling with image background with new CMOS camera

Post by PaulE54 »

I own the same camera and have had recent "noisy background" issues such as these. I see you stack in APP, I use DSS. One thing that worked for me is to check the sky background on the frames, and leave out of the stack those frames where the sky is lightest. I don't know if APP shows you a sky background figure, but DSS does.

Good luck!

Paul
ionia23
Posts: 62
Joined: Fri Jul 16, 2021 7:39 pm

Re: Struggling with image background with new CMOS camera

Post by ionia23 »

I know I have the terminology wrong, but I call that having a "chunky" background. This has been a frequent issue with my images. I think I've been going way overboard during the Wipe phase. A habit I'm trying very hard to break is having to fiddle with everything and just let StarTools make a judgement call based on what it can detect. I'm going to be trying that tonight. I won't be surprised if I see a huge quality improvement in some of my shots.

Still, chunky or not, your shot is simply beautiful. The detail in the dust lanes is excellent.
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Re: Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

PaulE54 wrote: Sat Jul 24, 2021 6:11 am I own the same camera and have had recent "noisy background" issues such as these. I see you stack in APP, I use DSS. One thing that worked for me is to check the sky background on the frames, and leave out of the stack those frames where the sky is lightest. I don't know if APP shows you a sky background figure, but DSS does.

Good luck!

Paul
Paul,

I do believe there is some flakiness with the ASI294MC Pro camera. I've been spending a lot of time trying to get a perfect flat with this camera, whereas with my Nikon I haven't had near the struggle in getting reasonable flats.

I am trying to get the right combination of settings with this camera.

I have learned that there is a spike in noise at a 1 second flat. I have heard that dark flats are more preferred than bias frames. I have heard that flats and dark flats at or above 3 seconds are preferred. This has been difficult to achieve. Presently I have a light box and have been trying different grey T-shirts to get the proper ADU, perhaps around 30,000. One of these grey T-shirts actually spiked the red ADU numbers. In attempting to get a flat colored grey T-shirt that is light enough I've discovered that some silkier material seems to leave some shininess to the flats. Have you arrived at a good method for getting workable flats?

I also lately have been using a gain of 121 with an offset of 30 as 120 gain triggers an optimal signal to noise with the camera. With the problem image that I've been working with I had a gain of 114 which doesn't turn on the optimal signal to noise.

As per your question, I did look through all of my lights and didn't see much variance with lighter skies, but I certainly will be looking to eliminate them from my stacking if I run into them. I don't know if APP shows a sky background figure, but I reviewed each image inside APP.

I have also tried ASTAP of which I didn't see any improvement over APP, but to be honest I am not familiar with the best practice with this software.

I also am interested in Siril, which a lot of people are sharing great stacking results.
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Re: Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

ionia23 wrote: Sat Jul 24, 2021 11:23 pm I know I have the terminology wrong, but I call that having a "chunky" background. This has been a frequent issue with my images. I think I've been going way overboard during the Wipe phase. A habit I'm trying very hard to break is having to fiddle with everything and just let StarTools make a judgement call based on what it can detect. I'm going to be trying that tonight. I won't be surprised if I see a huge quality improvement in some of my shots.

Still, chunky or not, your shot is simply beautiful. The detail in the dust lanes is excellent.
Ionia,

Thank you for your compliment; Star Tools does do a good job of pulling out detail.

I like you have the tendency to go overboard with pulling out details, which is a due to a large extent not understanding how to use this software and what the harmful effects might be.

You've suggested that this might be due to some part in how the Wipe command is used. I can look closer at this to see if I'm not applying it correctly.

I have had better results with my data than the original processing, but I still think I'm missing something.
hixx
Posts: 250
Joined: Mon Sep 02, 2019 3:36 pm

Re: Struggling with image background with new CMOS camera

Post by hixx »

Hi Paul,
this looks a bit like walking noise - I see the same in my images and it seems absolutely normal to me. I am also using APP for stacking and ST for post, but a different camera (Sony A7iii modified).
1) hence it does not only seem camera depended, it originates from stacking slight tracking errors, combined with the camera's dark pattern .
2) the amount can be dramatically reduced using Dithering when capturing.
3) in ST there is a dedicated tool to mitigate or completely remove Walking Noise. It sits in Denoise at the first screen. you just need to set the 'walking noise angle' by drawing a line in parralel to the pattern in your image and adjust. Then adjust 'walkingnoise Grain size'. If you cannot see the walking noise in that screen, reduce the Grain size temporarily to get a clear view, make walking noise adjustment, then set grain size again.
4) V 1.8 (currently alpha) will bring a further noise suppression in 'Wipe' for correlated noise (which is a different sort of noise but also an issue in many images) - of course without affecting any detail!

Further noise reduction strategies are described in the inofficial manual - (AutoDev settings, SuperStructure, Flux etc)
There is also a discussion on walking noise included
clear skies ,
jochen
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Re: Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

Yesterday afternoon while I was in APP I did another stacking with a new set of flats and dark flats, and processed it all of the way through.

I'm not exactly sure what might have made a difference but I retained a desirable luminosity, while (at least on my 32 inch monitor) reducing the infuriating background pattern. I can still see the smudgy horizontal grey to black pattern, but I really have to look to see it. I am curious how this will look on my higher resolution work monitor.
210610_M101_Try3.jpg
210610_M101_Try3.jpg (657.98 KiB) Viewed 3124 times
I can take my original (more luminous) image and alternate in and out with the new image as desktop background and see a huge difference.

I should have only made one change to be able to see where my improvement showed up from, but I was desperate so I threw together a number of changes to my flats and post processing.

As far as the Star Tools processing I did this in the following order:
Bin: (This time I tried 71%)
AutoDev: (ROI tight within the galaxy arms)
Crop: (Per a visible upper frame horizontal bright ledge) I tightened the frame a little more than usual.
Wipe: (Default)
Film Dev: I Stretched 70% and then clicked Home in 3 times
Contrast, HDR, Sharp, Decon: Defaults, but I may have chose reveal core in HDR
Color: Capped Green 100% and default
Shrink: Default
Track/NR: Applied default noise reduction
Super Structure: Chose Brilliance option

This is likely not the optimal processing but it had the most dramatic improvement (by my eyes on a larger monitor).
PaulInNorthMichigan
Posts: 35
Joined: Sat Aug 08, 2020 5:44 pm

Re: Struggling with image background with new CMOS camera

Post by PaulInNorthMichigan »

hixx wrote: Sun Jul 25, 2021 6:40 am Hi Paul,
this looks a bit like walking noise - I see the same in my images and it seems absolutely normal to me. I am also using APP for stacking and ST for post, but a different camera (Sony A7iii modified).
1) hence it does not only seem camera depended, it originates from stacking slight tracking errors, combined with the camera's dark pattern .
2) the amount can be dramatically reduced using Dithering when capturing.
3) in ST there is a dedicated tool to mitigate or completely remove Walking Noise. It sits in Denoise at the first screen. you just need to set the 'walking noise angle' by drawing a line in parralel to the pattern in your image and adjust. Then adjust 'walkingnoise Grain size'. If you cannot see the walking noise in that screen, reduce the Grain size temporarily to get a clear view, make walking noise adjustment, then set grain size again.
4) V 1.8 (currently alpha) will bring a further noise suppression in 'Wipe' for correlated noise (which is a different sort of noise but also an issue in many images) - of course without affecting any detail!

Further noise reduction strategies are described in the inofficial manual - (AutoDev settings, SuperStructure, Flux etc)
There is also a discussion on walking noise included
clear skies ,
jochen
Jochen:

So it appears there is a name to the issue I'm seeing, that is reassuring. :D

I definitely will want to look at the Denoise and walking noise angle when processing. Thank you for this information! It is great that it can be applied by grain size.

I am using Dithering, so at least that is keeping the issue less than what it could be.

Since I have been trying a lot of the tools and learning new ones, I anticipate that my images will be seeing an improvement.

Thank you everyone for the help!
Post Reply