1.7.424 alpha with new Wipe module

General discussion about StarTools.
hixx
Posts: 47
Joined: Mon Sep 02, 2019 3:36 pm

Re: 1.7.424 alpha with new Wipe module

Post by hixx »

Hi Ivo,
1) I could open 1.7.425 on Mojave just to confirm. I do not see the same issue as user @elpajare, so it might not be a serious one.

2) I found the new Wipe working a bit unreliable. I had 3 crashes but wanted to confirm the process prone to issues first:
a) run default Wipe settings to completion
b) dial in SynthFlat ( Vignetting) to completion
c) dial in Synth Dark/bias (Adaptive Multi axis Bias) and let it run on 100% SynthBias Edge Area (default) to completion
d) reduce SynthBias Edge Area to some 10% or so.
Here's what happens:
- First the wheel keeps spinning as normal for a min or so
- then the wheel gets stuck
- after another min or so, strange things start to happen to the graphics - the system does not react on any command or delayed for 30s or so
- final stage is the StarTools window may duplicate with one having black content or being translucent, the fan finally starts kicking in. A few seconds later the machine may crash, but I had one attempt that "survived" StarTools and presented a meaningful and very useful result just seconds later. So it does not seem to be a coding issue rather than handling the computing load.
The system runs normal including c) but operation d) (reducing SynthBias Edge) is starting to shake grounds. All the rest of the 1.7 processes worked smoothly and quick so far.
I am running Mojave on a late 2015 27" imac (2.7GHz quad core i7 & AMD390 graphics/16 cores)
here's the log:

StarTools 1.7.425alpha
Fri Oct 9 19:39:47 2020
-----------------------------------------------------------
File loaded [/Users/jochenscharmann/Pictures/Astro A7Mk3/20200915/NGC7822_553mm_f5_ISO1600_390m_14°-RGB-session_1.fits].
Image size is 6210 x 4436
---
Type of Data: Linear and was Bayered, but not whitebalanced
--- Auto Develop
Parameter [Ignore Fine Detail <] set to [Off]
Parameter [Outside RoI Influence] set to [15 %]
Parameter [RoI X1] set to [0 pixels]
Parameter [RoI Y1] set to [0 pixels]
Parameter [RoI X2] set to [6210 pixels (-0)]
Parameter [RoI Y2] set to [4436 pixels (-0)]
Parameter [Detector Gamma] set to [1.00]
Parameter [Shadow Linearity] set to [50 %]
--- Crop
Parameter [X1] set to [361 pixels]
Parameter [Y1] set to [173 pixels]
Parameter [X2] set to [5543 pixels (-667)]
Parameter [Y2] set to [4254 pixels (-182)]
Image size is 5182 x 4081
--- Wipe

Parameter [Synthetic Dark/Bias] set to [Adaptive Multi-Axis Bias]
Parameter [Gradient Edge Behavior] set to [Absorb 50%]
Parameter [Synthetic Flats] set to [Vignetting]
Parameter [Sampling Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [1 pixels]
Parameter [Gradient Falloff] set to [0 %]
Parameter [Synth. Bias Edge Area] set to [10 %]
Parameter [Gradient Aggressiveness] set to [75 %]



let me know if you need more information
regards,
jochen
User avatar
admin
Site Admin
Posts: 2407
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.424 alpha with new Wipe module

Post by admin »

hixx wrote: Fri Oct 09, 2020 6:15 pm 1) I could open 1.7.425 on Mojave just to confirm. I do not see the same issue as user @elpajare, so it might not be a serious one.
Thank you, that is good to hear!
2) I found the new Wipe working a bit unreliable. I had 3 crashes but wanted to confirm the process prone to issues first:
This was extremely helpful Jochen. :bow-yellow:
Also with help from @happy-kat, I tracked down this particular issue to the GPU getting overloaded and aborting. I have now capped the amount of iterations in 1.7.426 (now up for download) so this should not happen any more.
Ivo Jager
StarTools creator and astronomy enthusiast
hixx
Posts: 47
Joined: Mon Sep 02, 2019 3:36 pm

Re: 1.7.424 alpha with new Wipe module

Post by hixx »

Wow!
Issue resolving overnight! :D
I will check 426 this afternoon and let you know
And thanks happy-kat :obscene-drinkingcheers:
regards,
jochen
Attachments
image.png
image.png (739 Bytes) Viewed 189 times
happy-kat
Posts: 265
Joined: Sun Feb 01, 2015 11:31 am

Re: 1.7.424 alpha with new Wipe module

Post by happy-kat »

ooh I see there is another update, thanks, glad the data was useful.
I had an idea, when I use Deconvolution i make my parameter changes set on 'before' and only select 'after' when I am ready to see the result.
I treat the 'before' button like a 'do' switch, this helps general performance on my PC.
I am thinking I should repeat this approach when using the new Wipe module or could we add a 'Do' button to Wipe like the Life module has?
hixx
Posts: 47
Joined: Mon Sep 02, 2019 3:36 pm

Re: 1.7.424 alpha with new Wipe module

Post by hixx »

I think the Wipe DO button has just been removed in 1.7 due to GPU support - if I am not mistaken, it should be a realtime module now and any change will just immediately restart rendering. Once you set all parameters as desired the clock simply starts ticking from scratch. So there should not be an advantage to use a DO button other than the module not working in the background while you`re tweaking.
hixx
Posts: 47
Joined: Mon Sep 02, 2019 3:36 pm

Re: 1.7.424 alpha with new Wipe module

Post by hixx »

Hi Ivo,
here's the WIPE 426 retest:
WIPE is now running without crash, but it does not seem to deliver the same result it did before with a 10% Synth Bias Edge Area. :think:
Here`s what it does precisely (from a user perspective):
phase 1: 0s - 30s: After the clicking to 10%, the activity wheel updates as normal.
phase 2: 30s - 60s: the wheel updates perform laggy, every 5s or so
phase3: 60s - 70s: the wheel updates as normal again
after 70s StarTool is presenting the result
my guess is, it stops processing after phase 2 (60s), so ST is operating in safe territory now. :thumbsup:
but would it be worth to fine tune, trying to squeeze in one more iteration if favor of a potentially better result?
Would it be of advantage if the number of iterations was depending on what the user's GPU could handle, or maybe depending on processing time, e.g. 60s plus 1 iteration ??? So a user with a state of the art box GPU would be able obtain a better result than my dated imac and users with even slower boxes still would be safe.
regards,
jochen
elpajare
Posts: 92
Joined: Tue Jun 12, 2018 6:13 pm

Re: 1.7.424 alpha with new Wipe module

Post by elpajare »

The ... 426 alpha version works fine now on my Mac. :thumbsup:

The Wipe module manages to reduce the banding of my camera if I use the option: UNCALIB

What does Uncalib do and what does it mean?

Thanks fot the info...
happy-kat
Posts: 265
Joined: Sun Feb 01, 2015 11:31 am

Re: 1.7.424 alpha with new Wipe module

Post by happy-kat »

I am still plugging away, new wipe is fabulous on removing my challenging data gradients, but either wipe crashes or eventually decon crashes. I keep
happy-kat
Posts: 265
Joined: Sun Feb 01, 2015 11:31 am

Re: 1.7.424 alpha with new Wipe module

Post by happy-kat »

got there in the end, too late to start again to do a colour rendition (and preserve more of the autodev nebula reveal) as there is very strong CA on the fat stars
bin 50% then crop
autodev
Autosave v0.1.jpg
Autosave v0.1.jpg (189.22 KiB) Viewed 90 times
after autodev, wipe, HDR, decon, sharp, life and denoioise

Autosavev v1.1.jpg
Autosavev v1.1.jpg (186.95 KiB) Viewed 122 times
the bin 50% again and saved as jpgs with capped file size limit of 200
Last edited by happy-kat on Mon Oct 12, 2020 5:34 pm, edited 2 times in total.
User avatar
admin
Site Admin
Posts: 2407
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.424 alpha with new Wipe module

Post by admin »

You guys are amazing! :bow-yellow:
With a community like this, development is rapid and issues can be addressed quickly!

I just released a new version 1.7.427 with some further stability fixes, as well as an additional multi-axis bias modelling mode that is less aggressive (this mode is now used by the Uncalibrated preset).
elpajare wrote: Sun Oct 11, 2020 9:02 am The ... 426 alpha version works fine now on my Mac. :thumbsup:
Excellent! :thumbsup:
The Wipe module manages to reduce the banding of my camera if I use the option: UNCALIB
What does Uncalib do and what does it mean?
Uncalib stands for "uncalibrated", and is a preset that sets all parameters for a dataset that has not been calibrated at all with any sort of calibration frames (no flats, no bias, no darks, etc.).
For information on the specific settings, check the info buttons next to each parameter.
hixx wrote: Sun Oct 11, 2020 5:37 am WIPE is now running without crash, but it does not seem to deliver the same result it did before with a 10% Synth Bias Edge Area. :think:
Here`s what it does precisely (from a user perspective):
phase 1: 0s - 30s: After the clicking to 10%, the activity wheel updates as normal.
phase 2: 30s - 60s: the wheel updates perform laggy, every 5s or so
phase3: 60s - 70s: the wheel updates as normal again
after 70s StarTool is presenting the result
If using the same GPU for display output, then heavy compute loads on the GPU can actually cause the GPU to become intermittently unresponsive for display duties. I have two GPUs in my development rig (one of the being the P106 from this post). If I force StarTools to use my display GPU instead of the P106, I am indeed seeing the same behaviour.
but would it be worth to fine tune, trying to squeeze in one more iteration if favor of a potentially better result?
Would it be of advantage if the number of iterations was depending on what the user's GPU could handle, or maybe depending on processing time, e.g. 60s plus 1 iteration ??? So a user with a state of the art box GPU would be able obtain a better result than my dated imac and users with even slower boxes still would be safe.
I may look at something along those lines in the future (but for other reasons :).
Being smarter with the iteration limit in this case, really has to do with edge cases causing large loads. Avoiding those edge cases is more preferable as the difference in quality should be negligible and the GPU power is better used elsewhere for more legitimate purposes (e.g. seeing results faster, so you can tweak faster and more often). I did not see any significant differences in the quality of the result in my test data though...
Ivo Jager
StarTools creator and astronomy enthusiast
Post Reply