Mac 1.9.542alpha: Crash when Sharp Module is loaded

Bugs, glitches and unexpected behaviour.
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

HI Ivo,
Just tried 1.9.546 alpha and still crashing with my 25 MB file in Sharp module. Crash is when it is "Initializing ...".

I'm not sure whether I should continue reporting or wait for a message from you that I should test a a build on this large file ...

cytan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by admin »

I've audited the code again, but (un)fortunately cannot find see any problems.
As it stands, it appears the problem is an underpowered graphics solution.

As opposed to other operating systems, macOS unfortunately does not provide a means (that I know of) to change the graphics driver watchdog timer. I can change the code to allow more back and forth to "cut up the task" in smaller chunks, but that would mean performance degradation across the board for everyone else. :(

The best solution right now is to force StarTools to use your CPU instead of your GPU for bigger datasets. You can do so by opening up the 'config' file in a text editor of your choice and changing the following line;
opencl_forcecpu=false
to
opencl_forcecpu=true
Please let me know if that solves the issue.
Ivo Jager
StarTools creator and astronomy enthusiast
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

admin wrote: Sun Mar 05, 2023 2:56 am I've audited the code again, but (un)fortunately cannot find see any problems.
As it stands, it appears the problem is an underpowered graphics solution.

As opposed to other operating systems, macOS unfortunately does not provide a means (that I know of) to change the graphics driver watchdog timer. I can change the code to allow more back and forth to "cut up the task" in smaller chunks, but that would mean performance degradation across the board for everyone else. :(

The best solution right now is to force StarTools to use your CPU instead of your GPU for bigger datasets. You can do so by opening up the 'config' file in a text editor of your choice and changing the following line;
opencl_forcecpu=false
to
opencl_forcecpu=true
Please let me know if that solves the issue.
ugghh, it's worse now. With (see attached config)

"opencl_forcecpu=true"

StarTools 1.9.546alpha crashes in the Wipe Module. See StarTools.log and the crash log.

cytan
Attachments
crash.zip
(17.74 KiB) Downloaded 219 times
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by admin »

Well this is getting "interesting" :? Again a crash in the OpenCL code.

A couple of questions;
  • Does the crash happen if you bin first?
  • I just noticed that you should have an AMD Radeon Pro 5500M 8GB at your disposal. With StarTools configured to use the GPU, it does not appear your system is using that GPU. Are you able to force this GPU on? (see also here)
  • Can you confirm for me the 'resources' file has been extracted as well along with the application?
  • Is your system otherwise stable with other applications that use lots of memory?
Thank you,
Ivo Jager
StarTools creator and astronomy enthusiast
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

admin wrote: Sun Mar 05, 2023 11:30 pm Well this is getting "interesting" :? Again a crash in the OpenCL code.

A couple of questions;
  • Does the crash happen if you bin first?
  • I just noticed that you should have an AMD Radeon Pro 5500M 8GB at your disposal. With StarTools configured to use the GPU, it does not appear your system is using that GPU. Are you able to force this GPU on? (see also here)
  • Can you confirm for me the 'resources' file has been extracted as well along with the application?
  • Is your system otherwise stable with other applications that use lots of memory?
Thank you,
Binning 50% doesn't stop the crash in the wipe module.
See attached zip file.

The resources file is in the StarTools_alpha directory:
Screenshot 2023-03-05 at 6.17.35 PM.jpg
Screenshot 2023-03-05 at 6.17.35 PM.jpg (161.32 KiB) Viewed 10538 times
I did a checksum that you can check that it is the correct resources file.

Note: I don't understand what you mean that the GPU is not used because I thought by setting
opencl_forcecpu=true
ST will not use the GPU. Isn't this the expected behaviour?

I've loaded gSwitch and it says that I am using the GPU:
Screenshot 2023-03-05 at 6.32.27 PM.jpg
Screenshot 2023-03-05 at 6.32.27 PM.jpg (47.83 KiB) Viewed 10533 times
cytan
Attachments
crash1.zip
(9.25 KiB) Downloaded 218 times
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

And forcing the GPU to be used (but with "opencl_forcecpu=true") , ST still crashes in Wipe module+ 50% bin:
Screenshot 2023-03-05 at 6.40.19 PM.jpg
Screenshot 2023-03-05 at 6.40.19 PM.jpg (420.05 KiB) Viewed 10532 times
cytan
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by admin »

cytan299 wrote: Mon Mar 06, 2023 12:43 am And forcing the GPU to be used (but with "opencl_forcecpu=true") , ST still crashes in Wipe module+ 50% bin:

Screenshot 2023-03-05 at 6.40.19 PM.jpg

cytan
Hi Cytan,

You will want to have gSwitch use the Discrete GPU (not automatically switch).

opencl_forcecpu=true forces StarTools to use the CPU, not GPU

so, setting opencl_forcecpu=false will disable that and allow StarTools to use your GPU again (e.g. the default behavior).

If despite using gSwitch, StarTools keeps defaulting to the internal GPU (the Intel one, whose drivers generated the original crash) and not the discrete GPU (the AMD one, which is much more capable), then can you please run the command 'clinfo' and give me the output?

We can make StarTools use any secondary enumerated GPU if needed.
Ivo Jager
StarTools creator and astronomy enthusiast
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

admin wrote: Mon Mar 06, 2023 1:19 am
Hi Cytan,

You will want to have gSwitch use the Discrete GPU (not automatically switch).

opencl_forcecpu=true forces StarTools to use the CPU, not GPU

so, setting opencl_forcecpu=false will disable that and allow StarTools to use your GPU again (e.g. the default behavior).

If despite using gSwitch, StarTools keeps defaulting to the internal GPU (the Intel one, whose drivers generated the original crash) and not the discrete GPU (the AMD one, which is much more capable), then can you please run the command 'clinfo' and give me the output?

We can make StarTools use any secondary enumerated GPU if needed.
Good news!

I set gSwitch to use the Discrete CPU and set opencl_forcecpu=false and I went through the entire workflow and NO CRASH!!!

Where in the Startools.log can you tell whether the GPU is used or not?

cytan
cytan299
Posts: 71
Joined: Tue Nov 15, 2016 11:49 am

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by cytan299 »

And when I didn't do bin 50% which I did previously for these tests, now SVDdecon doesn't find any good stars for me to sample in *full* resolution: in fact what ST found aren't stars at all:
Screenshot 2023-03-05 at 8.19.17 PM.jpg
Screenshot 2023-03-05 at 8.19.17 PM.jpg (341.71 KiB) Viewed 10517 times
Attached is the StarTools.log file (log.zip) just in case I messed something up in SVDdecon.

cytan
Attachments
log.zip
(327.23 KiB) Downloaded 202 times
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Mac 1.9.542alpha: Crash when Sharp Module is loaded

Post by admin »

cytan299 wrote: Mon Mar 06, 2023 2:42 am Good news!

I set gSwitch to use the Discrete CPU and set opencl_forcecpu=false and I went through the entire workflow and NO CRASH!!!
Excellent! You should hopefully notice a fairly big speed up as well, as the AMD card is far more capable. :thumbsup:
Where in the Startools.log can you tell whether the GPU is used or not?
You cannot tell in the log, however you can see on the splash screen, as well as in the about dialog which resources were detected and are being used (it should show both CPU and GPU).

As for the SVDecon star detection, I cannot be 100% sure, but from the screenshot it appears your dataset is severely oversampled and may also exhibit blotchy pattern noise (both not entirely surprising for the resolution you are processing at). As a result, the apodization mask generator is latching on the blotches, rather than the oversampled stars.

If you could post a small crop of the stretched image's background, so we can try to verify if this is indeed the case.

You really should not be processing at this high resolution with this dataset - it will only work to your disadvantage. Instead, binning would be highly recommended.

If you really don't want to reduce the resolution and you cannot find the cause of the pattern noise, then you could try to clean up pattern noise in the Wipe module. using the Correlation Filtering parameter. It is more useful for zipper-artefact-like pattern noise, but it may work.
Ivo Jager
StarTools creator and astronomy enthusiast
Post Reply