Histogram in Super Structure Module

Requests for new features and wish-list items.
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Histogram in Super Structure Module

Post by Stefan B »

Hello Ivo,

sometimes I'm struggling to find the right strength in the Super Structure module. I really love the new preset DimSmall, it's like another denoise tool highlighting the real details instead of the noise. But it sometimes clips the background (I had a similar problem in 1.6 with Isolate) which I only notice afterwards. In order to avoid that it would be handy to have a histogram visible like in some of the other modules. And if the histogram would be a bit less flat than in the color module. IMHO, a button bringing up a new window with a histogram of what you are currently seeing (before/after) in StarTools would help me a lot in my processing decisions. It could be placed in the upper panels where the mask buttons are usually placed, for example.

Best regards
Stefan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Histogram in Super Structure Module

Post by admin »

Thank you Stefan,

I can not think of any operation that this module performs that would clip your image. :think:
Can you tell me what sort of operation you perform that causes this?

Could you perhaps attach a before-and-after example of this happening and the settings you used to accomplish this?
Thank you!
Ivo Jager
StarTools creator and astronomy enthusiast
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Re: Histogram in Super Structure Module

Post by Stefan B »

Maybe you can help me to sort out if the image is clipped at all. Astrobin gave me this histogram:
Image
Looking at the left end I thought I missed something during processing and thought it had to be the DimSmall preset since before the background clearly wasn't pitch black. But looking at the histogram in Photoshop it seems not to be clipped although the histogram doesn't appear to be a nice bell curve since it's really steep at the left end. Would you mind having a look, if it's clipped or not?

https://astrob.in/k3i9gw/0/

Image

Even if it's not clipped. Do you think the background is too dark? I think that maybe the galaxies appear too isolated from the background...and I really had problems controlling the brighter stars. Have to admit that I already tried to process this data several times and am not super happy with it. Additionally, I have weird color gradients. But there was no moon up and I live in Bortle 4. No idea where they come from... Maybe street lights, but those have never been an issue...

Your opinion on that would be greatly appreciated... I can also provide the ASTAP stack if you want to have a closer look.

Best regards
Stefan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Histogram in Super Structure Module

Post by admin »

Hi Stefan,

The image is definitely not clipped. The expected "stretched" bell curve is definitely rather tightly compressed, but is nevertheless still there (the bell curve is not a bell curve any more due to stretching, causing compression to the left and expansion to the right of the mode/peak).

On either side of the histogram's mode (peak), the bins should show values that taper off as you move further away from the peak.
Histogram_555.jpg
Histogram_555.jpg (48.53 KiB) Viewed 3861 times
There are 4 bins to the left side of the mode that contain pixels, or in other words 1.5% (4/256 * 100%) of your dynamic range is used to describe darker-than-background pixels. I'd say that's more enough, but tastes differ. Of course, on a well-calibrated screen, all brightness levels should be distinguishable from each other.

Now and then I see comments on forums from people that say an image is "clipped", followed by "space is not black!". Unless originating from a beginner, most of the time these images don't look clipped at all on my calibrated screens, and it is usually the case of the commenter using a poorly calibrated screen that is not resolving the lower dynamic range properly.

The Super Structure module should not really be used with a mask, however (not sure if that is something you are doing?).

Please do feel free to share a before-and-after (and/or the dataset) if you'd like a second pair of eyes!
Ivo Jager
StarTools creator and astronomy enthusiast
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Re: Histogram in Super Structure Module

Post by Stefan B »

Thanks for your assessment on that matter, Ivo! This made some things clearer to me. I will share the stack this evening (which will already be night for you).

I used the DimSmall on the whole image. This has been the first processing attempt during which I revisited the SuperStructure module for applying the Brighten preset. Here, I masked out the brightest stars so they wouldn't get brighter still. I thought this was okay and not different to, let's say, undoing the brightening after the fact by using the Undo filter in the layer module. Or is this rather bad practice? Can I protect brighter stars in a more appropriate way?

Thanks and regards
Stefan
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Re: Histogram in Super Structure Module

Post by Stefan B »

Hi Ivo,

you can find the stacked image here:

https://www.dropbox.com/s/x5jw13xxx51tb ... .fits?dl=0

Would be great if you could have a look.

Best regards
Stefan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Histogram in Super Structure Module

Post by admin »

Thank you for uploading that Stefan.

I had a look at the dataset, but there is not a whole lot else I can say about this, other than that everything works as expected and nothing is getting clipped.

In general though, the super structure module is meant for large to very large structures, so may not be as suitable for widefields like these. For best results, you will want to match your Airy Disk Radius closer to the radius of the stellar profiles (e.g. this is an extreme widefield, so a smaller setting, like is more appropriate). You can use the AiryOnly preset to see what small detail is being removed. You will want to pick a value that just still shows smaller stars and doesn't completely blur them away when viewing Airy Only.

As for the dataset itself, there seems to be a fair bit of correlated noise visible. E.g. larger-than-one-pixel grain, blotches, and stringy worminess;
Selection_557.jpg
Selection_557.jpg (149.66 KiB) Viewed 3824 times
StarTools will have trouble getting rid of that, as it appears like detai (some of it is as big as the stars in your image)l, rather than shot noise.

This should be your #1 priority to address. Are you are dithering? Are you are dithering sufficiently?

Addressing this will make the need to "hide" this unevenness a lot less urgent.
Ivo Jager
StarTools creator and astronomy enthusiast
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Re: Histogram in Super Structure Module

Post by Stefan B »

Hi Ivo,

thanks for your feedback. I totally agree on the problem of correlated noise. This has been MUCH better in my previous imaging sessions and I have no explanation why it should have gotten worse. I dither every second image for 10 px on the guide cam which corresponds to about 30 px on the imaging cam. Should be enough IMHO. The stack consists of about 80 frames with 5 min each. Corresponds to about 40 dithers. My understanding is that this should greatly reduce the noise. This setting hasn't changed for me since quite some time. When I blink through the images I can clearly see the huge dithers in a spiral fashion. I tried several different stacks now in DSS (since it's faster than ASTAP) and made sure to conform to the recommended settings on the StarTools website. Stacking without calibration frames revealed that noise does come from the lights and/or the stacking and NOT from flats or dark flats.

I tried median sigma clipping vs kappa median clipping, intersection vs standard mode, using only frames from before vs after meridian flip, setting black point to 0 vs not setting black point to 0. Correlated noise remains...

No idea what goes wrong here... But that's all rather pre-processing stuff and not the scope of this forum or StarTools. If you have additional ideas, I am eager to hear them. If not, thanks for your help and support so far, Ivo! :thumbsup: I will try to tinker on (and to adjust the airy disk radius appropriately) ;)

Best regards
Stefan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Histogram in Super Structure Module

Post by admin »

Please let us know if you figure out the correlated noise issue! It's rather curious it only started happening recently...
I of course definitely sounds like you're doing everything right! :think:
Ivo Jager
StarTools creator and astronomy enthusiast
Stefan B
Posts: 422
Joined: Sat Oct 31, 2020 8:59 pm

Re: Histogram in Super Structure Module

Post by Stefan B »

Here's a 800% crop of an autodev from a former imaging sessions (about 30 x 5min frames):
snip-iso800.JPG
snip-iso800.JPG (67.4 KiB) Viewed 3808 times
Looks a lot better in my eyes than a stack with the same settings from 45 x 5 min frames of the most recent session:
snip-iso1600.JPG
snip-iso1600.JPG (114.7 KiB) Viewed 3808 times
Only difference was the ISO value used. First stack was with ISO800, second with ISO1600. Already had good results with ISO1600, but with duo narrowband filter so I thought this was a better comparison since in both only the same UV/IR cut was used. ISO1600 is recommended for my EOS1100Da. Just used ISO800 for the first pic, since those frames were planned for color only and I intented to increase the dynamic range with lower ISO. Otherwise the settings in the two sessions (dithering, stacking, autodev) were the same. Really don't know...
Post Reply