Page 1 of 1

M31 very basic processing

Posted: Thu Feb 08, 2018 1:47 am
by jimd
Hi my first post. I'm creating this post just to get Ivo this .fts file of M31 - I couldn't send it to him via e-mail it was too big (autosave.fts 211MB). Hopefully I'll be able to attach it to this post. Since everyone can see it now take acrack at it. I can't get a presentable image from it StarTools (but was I able to with the other processing program (the "P" word)).

Ivo (or anyone else) if you can get something decent with StarTools let me know the basic WF. Thanks!

ok I cannot attach the file called autosave.fits 211MB - I get a "page cannot be displayed " on upload any ideas on how I can get this file to him?

Re: M31 very basic processing

Posted: Thu Feb 08, 2018 4:09 am
by admin
Hi Jim,

I just replied to you in an email.

Most people use Google Drive, Dropox or Microsoft OneDrive for sharing data stacks. These services should all have free plans.

Looking forward to the data and answers to some questions I had... :thumbsup:

Thanks!

Re: M31 very basic processing

Posted: Sat Feb 10, 2018 11:03 pm
by jimd
Thanks for your time on this Ivo,

Here is the stack of M31: https://drive.google.com/file/d/1I9Lw-j ... sp=sharing

Image #1 from my canon: https://drive.google.com/file/d/1LBefUu ... sp=sharing

and this is what I saved from PI after doing:

1 DynamicCrop
2 AutomaticBackgroundExtraction
3 Background Neutralization
4 ColorCalibration
5 HistogramTranformation
6 Saving the image https://drive.google.com/file/d/1xdo_w3 ... sp=sharing

I can't get any decent results from StarTools . I went over what I had tried to do in the e-mail I sent you. Nothing I have tried gives any acceptable results.
What am I doing wrong!! Thanks ..jd

Re: M31 very basic processing

Posted: Sun Feb 11, 2018 5:00 am
by admin
Hi Jim,

Many thanks for uploading this.

A couple of things;
  • Indeed, DSS is using settings that are incorrect. I don't know what the problem is here if you have followed the tutorial correctly. Are you using an older version of DSS perhaps?
    To illustrate this is what the characteristics of expected output would roughly be (scaled down a lot) after an AutoDev if correct settings were applied;
    LIGHT_60s_800iso_+23c_20171124-19h51m07s635ms_bin_autodev.jpg
    LIGHT_60s_800iso_+23c_20171124-19h51m07s635ms_bin_autodev.jpg (8.74 KiB) Viewed 6736 times
    Compared to what DSS has produced;
    Autosave(5)_bin_autodev.jpg
    Autosave(5)_bin_autodev.jpg (4.66 KiB) Viewed 6736 times
    At the very least, it appears these settings are incorrect or are being ignored;
    Image
    The point being, the stack should look the same as the single frame in appearance. The single frame just looks a lot noisier. We don't want any further meddling with the data.

    All that said, the data does not appear to be non-linearly stretched, so that's good. The "incorrect" settings only needlessly exacerbate noise at this point, but the stack is otherwise ready for processing.
  • Your stack suffers from extreme vignetting (as well as dust donuts on the sensor). In astrophotography, you *need* flats. They are not optional.
  • Your lens or scope causes rather severe chromatic aberration. It's not the end of the world, but acquiring correct star colors becomes very hard (see below for a mitigation strategy).
  • The field isn't flat, causing star elongation towards the corners (coma), (see below for a mitigation strategy).
Now for processing this;

First up, processing challenging data as a beginner using unfamiliar software is a triple-whammy. The nasty thing about this is that, if you are a beginner you are more likely to produce data that is indeed a lot more challenging to process in the first place.

Going forward, you will make your AP life infinitely easier and rewarding - and this is true regardless of the software you will be using - if you get the data acquisition/calibration down pat. In your case, flats are the #1 priority. If practical, dithering between frames should be a close #2.

Once you have taken these steps to properly calibrate your data, you will find post-processing much easier, quicker and repeatable process. You will be working with what's in your data, not around it.

Right now, as I will process this, most steps involve working around flaws. To reiterate - these flaws should be easy to fix and should be your #1 priority (flats! :bow-yellow: )

Please note I used 1.4.332 (hot off the press) for this. It includes a great many improvements over 1.3.5 (including in the Wipe module).

Loaded up stack with "Linear, was not Bayered, or was Bayered + white balanced"
--- Auto Develop
Default parameters.
To see what we got.
First off, we can see some form of normalization/channel calibration has taken place (see DSS settings screenshot above on how to avoid this). This does make the severe vignetting a lot easier to see in the process though. A big dust donut is visible to the right, with some other, smaller specks littered around the the frame as well. Coma is severe. Noise is high. Tracking is good. At 1:1 zoom we can already see blue rings around bright stars, indicating chromatic aberration in the optical path.
The data is oversampled (e.g. multiple pixels are used to describe just one unit of detail). It's a common thing with DSLRs; their resolution is just much higher than atmospheric conditions call for.
-- Bin
To trade "useless" resolution for noise reduction. I'm reducing more than needed, but it helps with noise.
Parameter [Scale] set to [(scale/noise reduction 25.00%)/(1600.00%)/(+4.00 bits)]
--- Lens
There is something we can do about the coma.
Before cropping, we'll use the Lens module to model an inverse distortion to "bend" the field back into shape. We're doing this before cropping, as the Lens module necessarily crops the image (turn Auto Crop "Off" to see why).
It really helps that you mounted the camera flush with the focuser/imaging plane really well, so we're just dealing with a simple barrel distortion. This really helps getting decent results from the Lens module until your spring for a "proper" solution (e.g. a coma corrector).
Parameter [Curvature Linked] set to [141.35 %]
--- Crop
Normally used for cropping stacking artefacts. Due to the usage of the Lens module which cropped the image slightly already, we can crop less now.
With severe vignetting like this, more cropping is better, but I am purposefully keeping things challenging here for demonstration purposes.
Parameter [X1] set to [9 pixels]
Parameter [Y1] set to [107 pixels]
Parameter [X2] set to [1019 pixels (-11)]
Parameter [Y2] set to [681 pixels (-5)]
--- Wipe
This is the part where you had trouble and I understand why - the gradients/vignetting are severe.
This is also where PI's ABE seemingly saved your bacon, while Wipe gave you bad results.

While ABE seems to do an OK job at first glance (referring to your emails), it has actually removed much of the disc. That's because with severe gradients/vignetting like this it is impossible to distinguish "real" from "fake/gradient" signal. Wipe too - unaided - has the same trouble (it errs on the side of caution, usually not removing enough in these cases). Both programs require manual intervention in challenging cases such as these. In PI, you would use DBE for this, setting manual samples. In Wipe you would aid it by creating a mask containing "off-limits" pixels. They largely do the same thing.

For Wipe, I used to Lassoo tool to draw a (rough :P ) ellipse around M31, making sure to include its full extent. Make sure it constitutes a "hole" in the mask (e.g. the pixels are not green, meaning they're "off"). Wipe will now know not to sample this area for gradients, since it is real signal and not a gradient.

My shameful attempt at an ellipse looked like this (you can download it and import it if you wish);
Autosave(5)_mas.jpg
Autosave(5)_mas.jpg (27.06 KiB) Viewed 6736 times
Again, note that all this fooling around with Masks and parameter changes is solely because of the lack of proper flats/calibration. Usually you would just sail through Wipe using default settings with an image like this.

I used the Vignetting preset with the following tweaks;
Parameter [Precision] set to [512 x 512 pixels] (only needed with very fast changing gradients).
Parameter [Dark Anomaly Filter] set to [6 pixels], increased in order to catch the small dust specks ("dark anomalies"). Without it Wipe will locally "back off" as to not clip your data, think it is real detail that needs to be preserved. Larger dust donuts will need masking out.
Parameter [Corner Aggressiveness] set to [99 %]

--- Auto Develop
Now that Wipe has cleaned up the gradients, the dynamic range that used to be occupied by the gradients can now be used for the "real" signal.
As the "Region of Interest" I just grabbed a slice of the disc around the core, since it is a good representation of the dynamic range of the object.
Parameter [Ignore Fine Detail <] set to [4.8 pixels] to ignore the finer noise (e.g. don't let AutoDev think it is real detail that needs bringing out).
Parameter [RoI X1] set to [418 pixels]
Parameter [RoI Y1] set to [198 pixels]
Parameter [RoI X2] set to [547 pixels (-463)]
Parameter [RoI Y2] set to [454 pixels (-120)]

That's all there is to stretching in StarTools. It's all based on objective analysis of detail and available signal, not on abstract/subjective histogram curve manipulation.

You can now apply decon, wavelet sharpening, contrast enhancement, HDR optimisation, etc.
...


--- HDR
I just launched this for a quick enhancement to the detail in the core.
Reveal preset.
Parameter [Algorithm] set to [Reveal DSO Core]
Parameter [Dark/Bright Response] set to [Full]
Parameter [Detail Size Range] set to [2001 pixels]
Parameter [Strength] set to [2.2]

--- Life
This is a neat trick if you have a lot of stars in the foreground and/or have copious amounts of background noise in lots of "empty" background. We could argue we have both in this image.
Clear the mask and Invert it (e.g. make sure all pixels are green/selected).
Then select the Isolate preset. It "isolates" large structures and pushes back noise and busy star fields.
Parameter [Strength] set to [50 %]

--- Color
Final color calibration. Do not underestimate the importance of color in your image! It lets you accurately convey the processes going on, chemical composition, temperatures, past, present and future (and much more) of an object.

Specifically in M31, you'd be looking for a yellowish core (older slow-burning red/yellow stars due to gas depletion) bluer outer rim (more young hot/bright blue stars due to gas abundance) and brown/red dust lanes (predominantly white light filtered by dust particles). Distinct hints of pink/purple HII areas (similar to those in our own galaxy) may become visible with enough signal.

Foreground stars should exhibit a good random distribution of temperatures from red->orange->yellow->white->blue. The chromatic aberration is causing us headaches with the latter however.

Usually the Color module does an ok job coming up with a default color calibration. The chromatic aberration (causing an over-representation of blue) is causing it to miss the mark however. I used this quick & easy method to determine a better color balance.

Finally, I set "Cap Green" to "To Yellow" to remove any spurious green dominance, as I know that nothing in that area of the sky is predominantly green (and therefore I can conclude it must be color noise).

Parameter [Dark Saturation] set to [6.70] to introduce a bit more color in the darker parts of the image.
Parameter [Green Bias Reduce] set to [1.21]
Parameter [Red Bias Reduce] set to [1.41]

--- Wavelet De-Noise
Finally, we switch Tracking off and let StarTools perform noise reduction using everything StarTools it has learned/data-mined about the image and signal thus far.
Parameter [Grain Size] set to [9.2 pixels]
--- Wavelet De-Noise
I didn't really have to touch the controls as StarTools correctly identified noise grain in the image all by itself. I bumped up "Color Detail Loss" however to remove a bit more color noise.
Parameter [Color Detail Loss] set to [17 %]

You will now have ended up with something like this;
Autosave(5)_CA.jpg
Autosave(5)_CA.jpg (156.33 KiB) Viewed 6736 times
--- Filter
It's a actually a decent image given how challenging the data is. However the thing that lets it down is the chromatic aberration.
Fortunately, there is a fringe killer feature in the "Filter" module that can help.

First, you create a star mask (Mask->Auto->Stars), then shrink a couple of times to remove the smaller stars and single-pixel noise. Then Grow a few times until the mask encompasses the offending halos around the problematic stars.

Then, back in the Filter module, select "Fringe Killer" as the "Filter Mode" .

Finally, simply click a few times on the color (e.g. the blue halos) that you find offensive and it will gradually disappear.

You will now end up with this final image;
Autosave(5).jpg
Autosave(5).jpg (151.24 KiB) Viewed 6736 times
Now this is really more like an image you would find in today's books. ;) And it was all produced by your good self.

Do let me know if you have questions/remarks/etc.