Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Questions and answers about processing in StarTools and how to accomplish certain tasks.
Post Reply
xonefs
Posts: 55
Joined: Sun Jul 18, 2021 7:30 am

Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by xonefs »

Ive encountered this a number of times... sometimes I can get great images out of ST and sometimes it just doesn't seem to work.

After stacking in astro pixel processor it applies an auto stretch, and my data there looks amazing autostrethed (or if I open in pixinsight and click the screen transfer autostretch button)

but when I load into startools in compose or as a single and autodevelop, the autodevelop just looks terrible and none of the settings seem to help. i''ve just gone with it sometimes and processed anyway thinking the other modules and noise reduction at the end will save it and look good like magic, but it never does if it doesn't start looking good earlier on after auto develop and contrast.

I had issues with this data and possibly internal reflections on the left, but I was trying to test it out anyway and see what I could crop. I composed first and had issues with LRGB, but can't get a good autodevelop from just the luminance either.

autostretched straight out of APP and saved, nothing else (combined luminance):
Image

when I try to autodev

Image
what's going on? usually the presets work great and it's easy but nothing seems to save this in develop or wipe. I tried all settings with ignore fine detail, ROI, no ROI, and other sliders I've encountered quite a few projects like this in ST where the results looked so bad I just gave up thinking it was the data, but maybe not. but i've also pumped out startools projects that are up to really high standards and blown me away (usually narrowband though) so I don't get it.

I know ST autodev is supposed to be preliminary/global, but sometimes it looks really bad like this or just super noisy compared to an autostretch in other programs but seems salvageable- but then it never gets saved by later modules.

here's the data: https://drive.google.com/file/d/19bGvgv ... sp=sharing
BrendanC
Posts: 60
Joined: Sun May 17, 2020 12:23 pm

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by BrendanC »

The first Auto Dev is just to show you the image warts and all - almost to show you what's wrong with it, rather than what's right.

I agree that it's extremely unintuitive and I wish this would be changed. It could be called something like 'Auto reveal' instead.

Having said which, you can get an idea of how to improve the image even at this stage. Try moving the Ignore fine detail slider across - you'll see how it starts to bring out accents. Also, the Shadow linearity will have an effect on the overall dark/light dynamics. You can also specify a region of interest to speed things up (and make sure, if you can, that you're running the GPU-accelerated version of StarTools, it's much quicker).

So, then go to Bin, Crop (make sure you crop out any stacking artefacts around the edge by quite a margin), Wipe (always, even if you don't think you need to), then... back to Auto Dev. Yep, still very unintuitive, I think it would make more sense to have another button next to Wipe that is actually, properly called Auto Dev rather than having to go back up.

Anyway, now you really can play around with Ignore fine detail and Shadow linearity, and then move on to Contrast, HDR, Sharp, SVDecon, and Color (in that order, top to bottom, left to right). You should start to see how the image improves with each module. I hope.

Another route is to use Film Dev instead (the button to the right of Auto Dev). This isn't as sophisticated as Auto Dev, but it can be kinder to data. Having said which, your data from APP looks pristine and I think it should handle the Auto Dev treatment nicely.

So yes, it's a bit weird. You start from worse, and work your way back to better in StarTools. I've been using it a couple of years and I still regard much of it as a bit of a mystery, but when you get it right, and it works, it really does work. It can pull out huge amounts of detail while keeping stars under control.

Good luck.
Not so much boldly going as randomly stumbling where plenty of people have been before.
https://brendancooper.com
BrendanC
Posts: 60
Joined: Sun May 17, 2020 12:23 pm

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by BrendanC »

I just played around with your data quickly. It's sublime - what gear did you use?

Anyway, I tend to be a bit heavy-handed but here are three versions that show how you can come out the other end if you press on through Auto Dev: https://1drv.ms/u/s!AqovBuVZMwj3kdcqY64 ... A?e=hdbG5F

One is with no denoise at all, which looks amazing, then there's a version with StarTools denoise, and another with Topaz AI Denoise which I tend to use instead. There's no colour in the image, I didn't realise it was mono given the filenames.

I'm sure you can do better, you certainly have admirable capture skills.
Not so much boldly going as randomly stumbling where plenty of people have been before.
https://brendancooper.com
xonefs
Posts: 55
Joined: Sun Jul 18, 2021 7:30 am

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by xonefs »

BrendanC wrote: Tue Dec 07, 2021 8:26 pm I just played around with your data quickly. It's sublime - what gear did you use?

Anyway, I tend to be a bit heavy-handed but here are three versions that show how you can come out the other end if you press on through Auto Dev: https://1drv.ms/u/s!AqovBuVZMwj3kdcqY64 ... A?e=hdbG5F

I'm sure you can do better, you certainly have admirable capture skills.

thank you- yes that is just a combined luminance integration of the RGB frames and L frames together to create a super luminance (ive found this works better for RGB than adding the separate ones for synthetic L generation after in compose)- I didn't post the individual RGB. The data is actually problematic as there is some concentric reflection going on on the left from alnitak off something that will probably ruin it and idk if it is fixable, more on that here: https://www.cloudynights.com/topic/8017 ... rdrawtube/

there's also some angled banding going on i've noticed when using short exposures (I was using 30s L and 90s RGB, which shouldnt be an issue with low read noise modes of this camera). not sure if it just the cam or my stacking settings causing it I will have to investigate, it was worse when I did 10s L and 30s RGB for m42.

Gear is a zwo 2600mm, stellarvue svx102t with flattener for native 720mm fl at f7 , astrodon e series LRGB filters, ioptron cem70. This data was from a dark location in the florida keys so the seeing should have been pretty good there.

and maybe- my processing is hit or miss with how well it goes and I have more trouble with broadband.

Yours look much better from the quick glance I had. I seemed to get some better result after binning slightly and I know here the common suggestion is to aggressively bin, but I don't like that as I don't think this data requires as much aggressive binning, like 71% at most. I prefer not to lose pixel resolution as I don't think most of it is oversampled with my data. I'm always between 1-2 arcsec per pixel and my seeing should support that.

autodev seems to work a little better after some binning, but I'm confused why it doesn't seem to work at all at native resolution when just clicking autostretch in other programs looks great. and why startools makes everything look noisier in the early stages than just auto stretching in another program. sometimes the NR/tracking works fine at the end but often it seems like ST is still somehow making things noisier (on data that looked much cleaner than the final result would suggest) and the NR doesn't help that much.

I will hopefully have time to give it another chance later
BrendanC
Posts: 60
Joined: Sun May 17, 2020 12:23 pm

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by BrendanC »

I did bin your data to 25% just so I could process it quickly - binning can improve SNR, but with your data it shouldn't be a problem.

If it helps, I just added the section of the log from the StarTools session to that shared folder. It's partly human-readable, but you could also try using STReplay to replicate exactly what I did.

I also noticed those artefacts you mentioned, I'm afraid I can't help with them. But it's great to see the quality that can be achieved with your sort of kit. I tend to fight noise and poor dynamic response with my lowly EOS1000D, but I still get a kick out of it.
Not so much boldly going as randomly stumbling where plenty of people have been before.
https://brendancooper.com
User avatar
admin
Site Admin
Posts: 2996
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by admin »

Quite a few things to unpack here.

However to answer the question posed in the title of this thread;

"Because other software hides the true quality of your data from you, and ST does not."

StarTools' philosophy is that you cannot take into account what you cannot see; knowledge is power.

1. One of the major reasons why your dataset looks noisier in StarTools, is because its image rendering routines are specifically designed to retain visual representations of the noise (and even fixed pattern artifacts!) that exist at 100% zoom. All other software (that I know of) uses "standard" scaling algorithms that hide the true noise levels and artifacts in your data; they perform something similar to binning to make the image fit when rendering the image to the screen. This gives you a false sense of the noise (and any fixed pattern noise) that exists at 100% zoom. In other words, all other software - if you scale an image to fit it on the screen - roughly just shows you what your image would like like if it were binned to that size.

2. Try to learn to interpret what StarTools is telling you (through what AutoDev is showing you, or how ST makes your data look, as well as how ST reacts to your data). It can tell you so much about how to work around the idiosyncrasies of your gear and circumstances, and how you might improve your efforts going forward. For example;
  • The dataset you uploaded shows a high degree spatial pixel correlation (e.g. the value of a single pixel influences its neighbouring pixels). For example noise is not confined to a single pixel, but "bleeds" into at least one neighbouring pixel. This has several important ramifications, one of which being pertinent to your question; the Ignore Fine Detail parameter in AutoDev (and probably higher Dark Anomaly Filter settings in Wipe etc.) needs to be used if you cannot (or do not want to) address this issue.
  • Your dataset is oversampled; in this dataset, one pixel does not describe one unit of detail - detail is blurred / "smeared out" over multiple pixels. For example the "smallest" (non-overexposing) stars (which should render as a point light) take up multiple pixels to be described (this remains the case even approaching 25% bin, but leaving some oversampling for deconvolution to restore detail is usually beneficial).
3. If you haven't done so, please read the AutoDev documentation. You should have no issues achieving a superior (in terms of neutral dynamic range allocation) version of the APP stretch after a bog standard Wipe (even without a RoI, which incidentally, should not be confused with a "preview area"!);
StarTools_2810.jpg
StarTools_2810.jpg (352.43 KiB) Viewed 499 times
StarTools_2811.jpg
StarTools_2811.jpg (236.88 KiB) Viewed 499 times
(First image; Autodev after Wipe, Second image; subsequent 25% bin to negate ST's noise retention algorithm)

4. Using the Compose module (with correct exposure amounts specified) should often yield superior results and can also avoid trouble down the line (handling of highlights for the purpose of avoiding artefacts that negatively impact decon). If not improved, then at the very least, it should yield results comparable to other compositing solutions - if it does not, please submit a bug report.

If any questions remain after perusing the above information and the AutoDev documentation, do let me know!
Ivo Jager
StarTools creator and astronomy enthusiast
jhart
Posts: 13
Joined: Wed Aug 26, 2020 10:08 pm

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by jhart »

I have had similar problems with AutoDev. Many times I find using Home In in FilmDev is the best option for an "auto" stretch.
User avatar
admin
Site Admin
Posts: 2996
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by admin »

jhart wrote: Tue Dec 07, 2021 11:34 pm I have had similar problems with AutoDev. Many times I find using Home In in FilmDev is the best option for an "auto" stretch.
Then please peruse the documentation (or if anything remains unclear, please ask here on the forums). FilmDev is meant purely for emulating the look of oldschool film images. Getting a handle on AutoDev is key to getting the most out of your data (and StarTools).
Ivo Jager
StarTools creator and astronomy enthusiast
xonefs
Posts: 55
Joined: Sun Jul 18, 2021 7:30 am

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by xonefs »

admin wrote: Tue Dec 07, 2021 11:31 pm Quite a few things to unpack here.

However to answer the question posed in the title of this thread;

"Because other software hides the true quality of your data from you, and ST does not."

StarTools' philosophy is that you cannot take into account what you cannot see; knowledge is power.

1. One of the major reasons why your dataset looks noisier in StarTools, is because its image rendering routines are specifically designed to retain visual representations of the noise (and even fixed pattern artifacts!) that exist at 100% zoom. All other software (that I know of) uses "standard" scaling algorithms
Ok thank you- that answers my question about the noise/scaling from other software and makes sense.

I do have some confusion on sampling still and finding the correct level of oversampling. A non overexposed star filling a single pixel as the benchmark confuses me a little. Won’t point source light always have an airy pattern from optics, and wouldn’t that sometimes take more than a single pixel to describe all detail adequately by a pixel array? It seems that could lead to undersampling or losing detail, but this is an area I have trouble with.

I will check the autodev doc when I have time
User avatar
admin
Site Admin
Posts: 2996
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Why do some images look great autostretched in my stacker but turn into a noisy flat mess in startools?

Post by admin »

xonefs wrote: Wed Dec 08, 2021 9:57 pm n sampling still and finding the correct level of oversampling. A non overexposed star filling a single pixel as the benchmark confuses me a little. Won’t point source light always have an airy pattern from optics, and wouldn’t that sometimes take more than a single pixel to describe all detail adequately by a pixel array? It seems that could lead to undersampling or losing detail, but this is an area I have trouble with.
The way your optics diffract light is absolutely a big factor in how well-sampled your dataset needs to be to resolve detail 1:1.
A non overexposed star filling a single pixel as the benchmark confuses me a little.
That's not exactly the benchmark; Nyquist-Shannon's theorem points out it requires at least four (in 2D space, e.g a 2x2 patch) samples to describe such an ideal point light star. A point light, being - for all intents and purposes - an infinitely far away source of light that will never fit into more than one pixel..

If you require anything more than a 2x2 patch, then;

1. your point light was not a point light

or

2. your point light was diffracted or otherwise distorted

If you know you definitely imaged a star, then you can rule out 1.
2, is where it gets complicated; there are so, so many factors that can distort and diffract a point light (e.g. the Point Spread Function). From the atmosphere, to imaging through a circular opening (indeed, the Airy disc), to tracking, to field flatness/coma, to lens/mirror aberrations, to anti-aliasing filters and Bayer matrices on the sensor, to stacking and alignment errors, etc. etc. One common thing though, is that all these things spread the light of a point light across many more pixels than should be needed to describe the original point light, which is always a 2x2 patch.

Fortunately, we have deconvolution to help reconstruct point lights / detail somewhat, though it is no panacea and relies heavily on the fidelity and quality of your signal (which incidentally, you can improve by binning oversampled datasets and being careful during stacking and compositing!).

Does that help?
Ivo Jager
StarTools creator and astronomy enthusiast
Post Reply