In this 2-part tutorial (second part here), we will process an LRGB dataset from start to finish to a professional standard. The purpose of this tutorial is to demonstrate the various modules, techniques and tools that make StarTools a serious contender for processing high-quality data at a professional level.
The M45 data for this tutorial was acquired by Josh Lake, at the Olmsted Observatory, Pomfret School. Josh has graciously agreed to donate this data to the public for all of us to learn from.
The data can be downloaded here.
Please note that, to be able to complete this 2-part tutorial, you will need the full version of StarTools, as saving is required. The first part of this tutorial (creating and processing the luminance frame) can be completed with the demo version.
Of course, there is no one 'correct' way of processing a data set. While StarTools uses objective measures for some aspects (detail vs noise estimation, dynamic range consumption of detail) as a baseline, beauty is ultimately in the eye of the beholder, and there are many, equally valid ways in which one can present the same data. In this case it was the data provider's express wish to bring out the Interstellar Medium (ISM) around M45, along with the object itself.
First off, some pre-processing is required; we want to create a synthetic luminance from the data that was captured. We will be processing luminance and colour data separately and combine them at the very end.
The data consists of 63 minutes of R+G+B and 108 minutes of L(uminance). By adding the R+G+B to the luminance we can gain a better and less noisy signal to work with.
StarTools 1.4 will see a new tool more user-friendly tool that was intended for this very purpose, but for now in StarTools 1.3 we will use the Layer module to obtain a similar result.
The Layer module is like a mini-Photoshop - it is extremely flexible, sporting a huge number of filters, pixel math blend modes, layering, and advanced masking modes. In the rare cases where StarTools' other modules do not provide the desired functionality, then using the Layer module (in conjunction with the other modules when desired) will act as the catch-all solution.
Without further ado, let's create our synthetic luminance data. We start off with loading R(ed). The following screen will come up;
We have to indicate that the data is stretched (although it clearly isn't!), as we will be pre-processing the data in the Layer and Repair modules which are otherwise not accessible during 'tracking' mode.
We further Ignore the 'stacking artefacts' warning, but make a mental note of it (as we will later find out StarTools was correct!).
In order to create a synthetic luminance frame, we first average R, G and B and then created a weighted average of (R+G+B) and L;
Click Layer, Open, Load G, set Blend Amount to 50%
Click 'Copy'. This copies the resulting (R + G) blend into a buffer.
Now click 'Paste->Bg'. This takes the buffer which contains our (R + G) blend, and pastes it as the new background layer.
Load B by clicking 'Open', set Blend Amount to 33%.
What you are seeing now is a perfect blend/average of R, G and B.
Again, click 'Copy' to copy the RGB blend into the buffer.
Again, click 'Paste->Bg'. As expected, this makes the RGB blend the new background layer.
Load L (luminance) by clicking 'Open', set Blend Amount to 63%.
Why 63%? Out of the 171 minutes of data, 108 minutes were shot in L (whereas 63 were shot in RGB). 108/171 * 100% = 63%.
Click Keep. Now we have a new, higher fidelity synthetic luminance frame.
From looking at the data in the Layer module, CCD blooming is visible in the brightest of stars in the form of 'cones' that protrude from very bright star cores. Fortunately StarTools comes with a de-blooming feature in the Repair module.
To avail of it, we have to create a mask that contains the offending stars. It's easier than it sounds - Click 'Mask', 'Auto', 'Stars' and set Threshold to 90%. The latter also includes anything that is between 90% white and 100% white (our star cores). Often this last step is superfluous and the 'Stars' preset does a good enough job. Even so, better safe than sorry and it's always a good idea to inspect the star mask that was generated afterwards for false positives or missing stars.
Click 'Do' to commence the Mask generation. You should end up with something like this;
The result, as far as we can see, is spot-on - the stars have all been selected correctly. Now we launch the 'Repair' module and set the algorithm to 'Debloom (Vertical Streaks)'. Commence repair by clicking 'Do'.
The improvement is quite dramatic - the blooming has been completely fixed (save for one small niggle - see below). We're happy with the result and ' Keep' it.
The Repair module has introduced small artefacts around repaired stars. This tends to happen when a star's halo has not been fully selected in the mask for repair. The following trick fixes the artefacts;
We launch, again, the Layer module, launch Mask, click Grow 2 times to 'grow' our current mask and 'Keep' the new mask. Note we will keep this mask throughout the next few processing steps - we will need it later!
Now we center the image on one of the Seven Sisters stars to see the effects of our fix.
We set 'Filter Kernel Radius' to 4.0 pixels, Mask Fuzz to 4.0, Layer Mode to 'Lighten'.
The artefacts are now repaired. We 'Keep' the result.
Now that our data is ready for post-processing (this is where StarTools' real forte is), we notify StarTools that the data is still unstretched by clicking the 'Track' button. Engaging the tracking feature in StarTools affords us a number of unique features. As we will see later, it will allow us to re-do our global stretching at any point, will allow us to perform deconvolution at any point, remove the distinction between linear and non-linear data and - the jewel in the crown of StarTools' Tracking feature - will allow use to apply extremely targeted noise reduction that seemingly 'knows' where the noise is in our image.
After engaging the Tracking feature, we now remember the stacking artefacts warning StarTools gave use earlier. It's time to address them. But first, we need to be able to see what we're doing;
We launch AutoDev and set gamma to 5.0 just so we can see what we're working with (note: touching he gamma slider in AutoDev is otherwise typically not recommended as dynamic range allocation will no longer be optimal but skewed instead. Here we just use it for non-permanent visualisation purposes).
The artefacts area readily visible to the left and bottom of the image. We 'Keep' the result. Thanks to the Tracking feature, we can redo our global stretch at any time - there is simply no need for 'screen stretching' or similar functionality in StarTools.
We proceed to crop the artefacts in the 'Crop' module. It's a simple affair - just click and drag the area that you wish to retain or use the sliders controls for fine-tuned pixel perfect cropping.
Now we re-do our global stretch (again using the 'AutoDev' module) as the cropping will have freed up dynamic range that AutoDev can put to better use.
We increase 'Ignore Detail <' to around 1.5 to make AutoDev ignore fine noise and have it refrain from allocating dynamic range to it - trying to making noise visible is just a waste of dynamic range! We 'Keep' the result.
The astronomer who provided this data has a keen interest in the InterStellar Medium (ISM) around M45, so per his request we add some steps here to push the levels a little in lieu of allocating optimal dynamic range to M45 itself.
To do so, we launch the 'Develop' module and choose to 'stretch image as-is'. The latter operates on the image as you currently see it, rather than re-doing the global stretch in its entirety. We use a value of 2.2. for 'Gamma' an keep the result.
We notice an unevenness in the levels moving from left to right (aka a gradient). We use the 'Contrast' module to even it out using an 'Aggressiveness' of 45%. We could also have used Wipe for this purpose, which is normally the go-to tool for gradients. Contrast can do the same job for black and white images, while Wipe would've asked us to reset the mask. So for those who are using the demo version of StarTools and can't save the mask, we use the Contrast module here instead, as we need the current mask later.
Next, we 'Develop' the image some more ('stretch image as-is'), pushing gamma again to 1.6. Again, note that this is all down to individual taste and totally depends on your artistic vision for your data.
Now it is time to apply deconvolution in order to bring out more detail. We launch the 'Decon' module and immediately go through to 'Mask' - we need an inverse star mask to avail of the de-ringing option.
As a matter of fact, our current mask is quite suitable with some modifications. We 'Invert' it, launch Auto, click 'Stars', set 'Feature size' to 1, and set 'Old Mask' to 'Subtract new from old'. This will 'punch' some additional holes in the inverted star mask where small stars exist.
We click Do and 'Shrink' the result twice. Notice how M45 itself is still mostly covered by the mask, but the rest of the image is not. This makes sense - we punched holes in the mask where high frequency detail (such as stars, but also noise!) exists.
Then we return to the 'Decon' module by 'Keep'ing the result.
We can ignore the warning message (Decon is just trying to be helpful by reminding us our star mask needs to be inverted, which it already is).
We define a preview area in a brighter, detailed and noise-free area to see the results.
We settle for the following parameters; 'Radius' 2.7, 'Iterations' 12 and 'Mask Fuzz' 5.5. We feel this improves our image nicely and apply it to 'All' (i.e. the whole image).
Notice how the deconvolution module enhances the detail in the masked-in areas, while refraining from creating ringing artefacts around stars. In fact, the de-ringing algorithm still applies an alternative deconvolution algorithm to the masked-out bits of the masked that can still make for an improvement over the old image (experiment with Mask Fuzz to your taste).
Also notice that, even when all pixels are masked in, the Decon module completely refrains from creating noise artefacts in noisy areas of the image and homes-in to the optimal balance of noise and detail (the latter can be controlled however by the 'Regularization' parameter). This behaviour which is unique amongst other decon implementations is courtesy of StarTools' tracking feature which, in this case, supplies the deconvolution algorithm with per-pixel signal-to-noise ratio measurements, so that the algorithm knows exactly which pixels are eligible for deconvolution and by how much.
Further notice that we are applying deconvolution to stretched data here and that the result is still mathematically correct (StarTools' engine reverses any stretching per-pixel and uses linear data as input to the deconvolution algorithm).
Happy with the result, we 'Keep' it.
We continue with another Contrast tweak, this time using it for its intended use - optimising contrast;
We use an Aggressiveness of 55% and enable 'Compensate Gamma'. The latter attempts to compensate the rest of the image for any reduction in brightness caused by running the contrast optimisation algorithm. Your mileage may vary with this parameter, based on the characteristics of your image and the amount of luminance that has been taken away. StarTools' modules invite to experiment, so you are encouraged to do exactly that!
The Aggressiveness parameter here controls how large an area of lesser contrast must be before it is picked up by the Contrast module. Smaller values will progressively make those areas bigger and the effects of the Contrast module milder and less locally defined. The Wipe module's Aggressiveness parameter works in much the same way. Both algorithms are loosely based on 'retinex theory' of human vision for detecting gradients and contrast differences.
Next up is some Wavelet/Multiscale Sharpening. We re-use the mask we made for Decon here, protecting stars and noise from being further enhanced.
We leave all scales at 100%, and just set the 'Amount' (Strength) to 200%. We create a smooth blend of sharpened and non-sharpened by setting 'Mask Fuzz' to 7.0 pixels. We then 'Keep' the result.
When compared to other Wavelet/Multiscale Sharpening implementations, StarTools' implementation is unique in two ways. Firstly, like all other modules in StarTools, it never clips your data no matter how outrageous the parameters chosen. Secondly, StarTools' implementation is inter-scale aware. This means that scales are aware of eachother when they are both competing to enhance the same pixel. Consider what happens when a larger scale decides a pixel should be brightened, while a smaller scale decides a pixel should be darkened. Normally these two conflicting results would cancel each other out and no detail would be enhanced. However, the 'Small Detail Bias' parameter lets you give priority to smaller scales when two scales are fighting to modify the same pixel, giving you another dimension of enhancing detail in your image.
The following step is optional. We haven't used the versatile HDR module at all for this image, but we wanted to show at least one of its uses at managing high dynamic range images. We launch the HDR module and click the 'Reveal' preset. To ameliorate the effect a little we select 1.00 for Dark/Brigh Response, but it's one of those things that are down to taste.
The effect is quite dramatic; where previously barely any detail was visible, the Reveal DSO Core algorithm is able to dig out any remaining latent detail, as seen in the new 'cloud fronts' detail below.
After 'Keep'ing the result from the HDR module, we are now done enhancing detail with the modules that are available during 'Tracking'. Therefore, we now click 'Track' to stop StarTools' tracking. We click 'Stop tracking, do final noise reduction'.
We just click 'Next' during the Noise reduction setup and when the Scale Extraction is complete, we answer 'Yes, set full mask'.
We change the following parameters from their default values; Scale 1 80%, Scale 2 80%, Scale 3 60%, Scale 4 42%, Scale 5 22%, Brightness Detail Loss 80%.
Notice how StarTools just seems to 'know' which pixels need noise reduction and which don't. Notice too that that it is not just a simple luminance mask - details that were dug out by heavy local dynamic range manipulation, but are just as bright as other details that were easily brought out by global stretching, are more noise reduced since they are noisier, as more local stretching had to be performed to bring out those faint details.
The final noise reduction and switching off of the Tracking feature concludes our luminance data processing. We save the resulting image for combining into a colour image (see part 2 of this tutorial).
We end up with the following result;
In the second part of this tutorial, we continue processing the colour data and combining luminance and colour into our final image.