Asterism - Image preProcessing and Conversion for Linux

General discussion about StarTools.

Asterism - Image preProcessing and Conversion for Linux

Postby Rowland » Wed May 13, 2015 11:18 pm

I have been working on a Linux preprocessing utility for use with consumer digital camera (CDC / DSLR) RAW image data, primarily for my own use, education, curiosity and fun.

Ivo has very generously permitted this thread in the StarTools forum. Asterism is an open source contribution and began as a command line image conversion script. I have taken the liberty of continuing to up-date the evolution and ongoing development of this image preprocessing utility.

Asterism (GitHub) functions and settings are clearly arranged in a tabbed user interface and described in the user manual

online user manual

Asterism direct download

0-main.png
0-main.png (84.05 KiB) Viewed 3739 times


Videos
Master frames demo
Light frames demo
Develop - digital development

The posts that follow show some of the earlier development work.
Last edited by Rowland on Tue Sep 18, 2018 3:50 am, edited 139 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Sun May 24, 2015 12:40 am

Early work. Image processing with DCRAW ImageMagick and Panotools. The highly saturated colours are just that and not the default output. I use the scientific colour module to get a good look at the luminance and saturation capability of the image. It's testing....
Attachments
M8-rawprepro - other programe calibration.jpg
M8-rawprepro - other programe calibration.jpg (193.13 KiB) Viewed 12408 times
Last edited by Rowland on Fri Jun 09, 2017 3:18 am, edited 12 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Mon May 25, 2015 11:27 pm

Image calibration / flat division with ImageMagick. This was a properly matched set of flats and lights. Sometimes not the case. Please read a few posts down for more information about flat division in Asterism.
Attachments
M8_divide_flat.jpeg
M8_divide_flat.jpeg (91.35 KiB) Viewed 12346 times
M8_no_flat.jpeg
M8_no_flat.jpeg (77.09 KiB) Viewed 12346 times
flat.jpeg
flat.jpeg (29.19 KiB) Viewed 12365 times
Last edited by Rowland on Sun Jun 17, 2018 12:22 am, edited 6 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Sat May 30, 2015 12:31 am

Further reference material - Jim Solomon's Astrophotography Cookbook
[quote] - this was the original source material and describes the traditional approach to DSLR image reduction - the bias in the dark methodology.

It is probably worth mentioning that dark frame to light frame temperature mismatch is often a source of poor image quality. Dark current (noise) is a function of temperature and exposure time and unless you are using a program that performs dark scaling (and your dark frames are several minutes longer in duration than your light frames), then matching dark and light frame exposure time and temperature is the key to proper dark noise reduction. In Asterism, the Defect map process is a work-around for mismatched dark and light frames but more often, dithering and the Asterism light frame stacking method produce a reasonable result, providing bias and flat frames are available.
Last edited by Rowland on Mon Sep 17, 2018 7:42 pm, edited 11 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Sun May 31, 2015 3:43 am

ImageMagick auto-level. This a redundant approach to the problem of differences in flat and light frame illumination. It worked for some images and not for others - confusing and inconsistent.

The problem of incorrect flat illumination is solved by simple mathematics. In Asterism, the mean exposure value of the target light frame is calculated and the master flat exposure mean value adjusted to produce near perfect, if not perfect to the eye, flat division. The process is so reliable that poor results can be traced to the image set - flats or lights.
Attachments
medianflatAUTOLEVELandNoautolevel.jpg
medianflatAUTOLEVELandNoautolevel.jpg (238.53 KiB) Viewed 12284 times
Last edited by Rowland on Sun Jun 17, 2018 12:17 am, edited 5 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Sun May 31, 2015 9:56 am

Calibrated aligned and stacked with an Asterism predecessor, using DCRAW ImageMagick and Panotools. Note that the immense star field of the Milky Way, in the vicinity of Eta Carina is revealed, showing the extent of the data captured by this set of images, nicely revealed. No attempt has been made to darken the area around EC for artistic purposes - that is, user preference.

StarTools processing - Develop, Wipe, Develop, HDR Reveal, Color, Sharp - as below - no noise reduction

Code: Select all
StarTools 1.4.302
Sun May 31 17:25:07 2015
-----------------------------------------------------------
File loaded [/home/geoland/Pictures/Eta_Carina/EtaCarina.tiff].
---
--- Bin
Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]
--- Crop
Parameter [X1] set to [61 pixels]
Parameter [Y1] set to [42 pixels]
Parameter [X2] set to [2761 pixels (-56)]
Parameter [Y2] set to [1842 pixels (-34)]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [83.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [5.0 pixels]
--- Wipe
Parameter [Mode] set to [Correct Color & Brightness]
Parameter [UNKNOWN] set to [Yes]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [2 pixels]
Parameter [Drop Off Point] set to [0 %]
Parameter [Corner Aggressiveness] set to [100 %]
Parameter [Aggressiveness] set to [75 %]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [90.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [Off]
--- HDRStarTools 1.4.302
Sun May 31 17:25:07 2015
-----------------------------------------------------------
File loaded [/home/geoland/Pictures/Eta_Carina/EtaCarina.tiff].
---
--- Bin
Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]
--- Crop
Parameter [X1] set to [61 pixels]
Parameter [Y1] set to [42 pixels]
Parameter [X2] set to [2761 pixels (-56)]
Parameter [Y2] set to [1842 pixels (-34)]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [83.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [5.0 pixels]
--- Wipe
Parameter [Mode] set to [Correct Color & Brightness]
Parameter [UNKNOWN] set to [Yes]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [2 pixels]
Parameter [Drop Off Point] set to [0 %]
Parameter [Corner Aggressiveness] set to [100 %]
Parameter [Aggressiveness] set to [75 %]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [90.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [Off]
--- HDR
Parameter [Small Detail Precision] set to [Max]
Parameter [Channels] set to [Brightness Only]
Parameter [Algorithm] set to [Reveal DSO Core]
Parameter [Dark/Bright Response] set to [Full]
Parameter [Detail Size Range] set to [64 pixels]
Parameter [Strength] set to [1.0]
Undo.
Redo.
Undo.
Redo.
--- Color
Parameter [Cap Green] set to [To Yellow]
Parameter [Bias Slider Mode] set to [Sliders Reduce Color Bias]
Parameter [Style] set to [Scientific (Color Constancy)]
Parameter [LRGB Method Emulation] set to [RGB Ratio, CIELab Luminance Retention]
Parameter [Dark Saturation] set to [3.00]
Parameter [Bright Saturation] set to [Full]
Parameter [Saturation Amount] set to [200 %]
Parameter [Blue Bias Reduce] set to [1.33]
Parameter [Green Bias Reduce] set to [1.00]
Parameter [Red Bias Reduce] set to [1.21]StarTools 1.4.302
Sun May 31 17:25:07 2015
-----------------------------------------------------------
File loaded [/home/geoland/Pictures/Eta_Carina/EtaCarina.tiff].
---
--- Bin
Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]
--- Crop
Parameter [X1] set to [61 pixels]
Parameter [Y1] set to [42 pixels]
Parameter [X2] set to [2761 pixels (-56)]
Parameter [Y2] set to [1842 pixels (-34)]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [83.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [5.0 pixels]
--- Wipe
Parameter [Mode] set to [Correct Color & Brightness]
Parameter [UNKNOWN] set to [Yes]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [2 pixels]
Parameter [Drop Off Point] set to [0 %]
Parameter [Corner Aggressiveness] set to [100 %]
Parameter [Aggressiveness] set to [75 %]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [90.00 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [Off]
--- HDR
Parameter [Small Detail Precision] set to [Max]
Parameter [Channels] set to [Brightness Only]
Parameter [Algorithm] set to [Reveal DSO Core]
Parameter [Dark/Bright Response] set to [Full]
Parameter [Detail Size Range] set to [64 pixels]
Parameter [Strength] set to [1.0]
Undo.
Redo.
Undo.
Redo.
--- Color
Parameter [Cap Green] set to [To Yellow]
Parameter [Bias Slider Mode] set to [Sliders Reduce Color Bias]
Parameter [Style] set to [Scientific (Color Constancy)]
Parameter [LRGB Method Emulation] set to [RGB Ratio, CIELab Luminance Retention]
Parameter [Dark Saturation] set to [3.00]
Parameter [Bright Saturation] set to [Full]
Parameter [Saturation Amount] set to [200 %]
Parameter [Blue Bias Reduce] set to [1.33]
Parameter [Green Bias Reduce] set to [1.00]
Parameter [Red Bias Reduce] set to [1.21]
Parameter [Mask Fuzz] set to [1.0 pixels]
--- Wavelet Sharpen
Parameter [Intelligent Enhance] set to [Yes]
Parameter [Scale 1] set to [100 %]
Parameter [Scale 2] set to [100 %]
Parameter [Scale 3] set to [100 %]
Parameter [Scale 4] set to [100 %]
Parameter [Scale 5] set to [100 %]
Parameter [Mask Fuzz] set to [8.0 pixels]
Parameter [Amount] set to [100 %]
Parameter [Small Detail Bias] set to [75 %]
Undo.
Redo.
File saved [/home/geoland/Pictures/Eta_Carina/EtaCarinaST.tiff].
Parameter [Mask Fuzz] set to [1.0 pixels]
--- Wavelet Sharpen
Parameter [Intelligent Enhance] set to [Yes]
Parameter [Scale 1] set to [100 %]
Parameter [Scale 2] set to [100 %]
Parameter [Scale 3] set to [100 %]
Parameter [Scale 4] set to [100 %]
Parameter [Scale 5] set to [100 %]
Parameter [Mask Fuzz] set to [8.0 pixels]
Parameter [Amount] set to [100 %]
Parameter [Small Detail Bias] set to [75 %]
Undo.
Redo.
File saved [/home/geoland/Pictures/Eta_Carina/EtaCarinaST.tiff].
Parameter [Small Detail Precision] set to [Max]
Parameter [Channels] set to [Brightness Only]
Parameter [Algorithm] set to [Reveal DSO Core]
Parameter [Dark/Bright Response] set to [Full]
Parameter [Detail Size Range] set to [64 pixels]
Parameter [Strength] set to [1.0]
Undo.
Redo.
Undo.
Redo.
--- Color
Parameter [Cap Green] set to [To Yellow]
Parameter [Bias Slider Mode] set to [Sliders Reduce Color Bias]
Parameter [Style] set to [Scientific (Color Constancy)]
Parameter [LRGB Method Emulation] set to [RGB Ratio, CIELab Luminance Retention]
Parameter [Dark Saturation] set to [3.00]
Parameter [Bright Saturation] set to [Full]
Parameter [Saturation Amount] set to [200 %]
Parameter [Blue Bias Reduce] set to [1.33]
Parameter [Green Bias Reduce] set to [1.00]
Parameter [Red Bias Reduce] set to [1.21]
Parameter [Mask Fuzz] set to [1.0 pixels]
--- Wavelet Sharpen
Parameter [Intelligent Enhance] set to [Yes]
Parameter [Scale 1] set to [100 %]
Parameter [Scale 2] set to [100 %]
Parameter [Scale 3] set to [100 %]
Parameter [Scale 4] set to [100 %]
Parameter [Scale 5] set to [100 %]
Parameter [Mask Fuzz] set to [8.0 pixels]
Parameter [Amount] set to [100 %]
Parameter [Small Detail Bias] set to [75 %]
Undo.
Redo.
File saved [/home/geoland/Pictures/Eta_Carina/EtaCarinaST.tiff].
Attachments
EtaCarinaSTIM.jpeg
EtaCarinaSTIM.jpeg (432.11 KiB) Viewed 12274 times
Last edited by Rowland on Sun Jun 17, 2018 12:39 am, edited 6 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: DSLR RAW 16bit CFA conversion script - master frame crea

Postby Rowland » Sun Jun 07, 2015 6:45 am

Update - new stable version

All the old and now redundant posts have been removed. In answer to the next post - Asterism replaced FAICS, which was another development version and an attempt to give the utility a meaningful name.
Last edited by Rowland on Sun Jun 17, 2018 8:56 am, edited 6 times in total.
Rowland
 
Posts: 228
Joined: Sat Mar 05, 2011 10:54 pm

Re: Image calibration, processing and RAW conversion - Linux

Postby smokey688i » Tue Dec 13, 2016 4:41 pm

Rowland,

Old thread I know...I've just discovered how much better Regim is than DSS, but I came across your FAICS program and am working on setting a Linux virtual machine on my Win10 OS to give it a trial run. I notice you've recommended Regim fairly recently though - have you dropped this project, or do you still prefer the results out of FAICS over Regim?

-Chris
smokey688i
 
Posts: 18
Joined: Mon Feb 16, 2015 2:52 am


Return to General Discussion

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron