1.7.440 crash

Bugs, glitches and unexpected behaviour.
clouzot
Posts: 11
Joined: Thu Jul 23, 2020 7:56 pm

1.7.440 crash

Post by clouzot »

Hi Ivo,

I don't know if there's a better place for bug/crash reports.

Using 1.7.440 alpha on my Mac, I see repeated crashes (quite similar to the one I've reported lately, "Graphics hardware encountered an error and was reset: 0x0000002b"). Startools crashes in the middle of a task, it can be the Color module (in this particular case), or the Wipe module...

I attached both the crash report and startools.log, but the latter only contains the latest module I applied, so not the one during which Startools crashed.

Unfortunately it renders Startools 1.7 unusable on my Mac. Oddly enough, I don't see any other crash reports from Mac users, so it may be my specific machine, my OS installation, nVidia's drivers...

Is there anything I could do on my side to narrow down the cause of those crashes?

Thanks
Attachments
StarTools.log.zip
(10.83 KiB) Downloaded 202 times
StarTools_2020-11-08-125056_MBP.crash.zip
(18.73 KiB) Downloaded 214 times
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.440 crash

Post by admin »

Hi,

Thank you for uploading that crash log. It is very helpful!
What are the specs of your mac? (model, processor, GPU being used - it appears to be an Nvidia Geforce model...)

Instabilities like these are likely caused by the GPU taking too long to complete its processing, and the OS resetting the driver.

You can verify this by seeing if the application stabilizes when you process a dataset at, say, quarter the resolution (bin 50%).

You can read more about GPU acceleration in StarTools here.

Sadly, I'm not aware of any way to set a custom watchdog timer delay on macOS like you can on other operating systems.

If the problem does not go away (or is not greatly reduced) when processing a smaller dataset, then you may also be looking at hardware/thermal failure.

If it is indeed caused by an underpowered GPU, then, sadly, the only remedy I have for you right now, is to use a different operating system (Linux or Windows). I will make sure, however, to provide a CPU-based version for macOS as well (as is the case for Linux and Windows).

Please do let me know the specs of your machine, and if any of the above helps/mitigates the issue.
Ivo Jager
StarTools creator and astronomy enthusiast
clouzot
Posts: 11
Joined: Thu Jul 23, 2020 7:56 pm

Re: 1.7.440 crash

Post by clouzot »

Thanks Ivo, I appreciate your taking the time to look into my issue.

My Mac is pretty old (it used to be correctly speced, but that was in 2013 :mrgreen: )

MacBook Pro (Retina, 15-inch, Late 2013)
CPU 2,3 GHz Quad-Core Intel Core i7
RAM 16 GB 1600 MHz DDR3
Internal GPU: Intel Iris Pro 1536 MB
Discrete GPU: NVIDIA GeForce GT 750M, 2 GB VRAM

Re. the custom watchdog timer, I'll ask on my side (we have pretty serious macOS low-level dev specialists at work) but as far as I understand, the kernel should be panicking then.

I'll report if I get any crashes with much smaller datasets.
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.440 crash

Post by admin »

clouzot wrote: Sun Nov 08, 2020 9:05 pm Thanks Ivo, I appreciate your taking the time to look into my issue.

My Mac is pretty old (it used to be correctly speced, but that was in 2013 :mrgreen: )

MacBook Pro (Retina, 15-inch, Late 2013)
CPU 2,3 GHz Quad-Core Intel Core i7
RAM 16 GB 1600 MHz DDR3
Internal GPU: Intel Iris Pro 1536 MB
Discrete GPU: NVIDIA GeForce GT 750M, 2 GB VRAM

Re. the custom watchdog timer, I'll ask on my side (we have pretty serious macOS low-level dev specialists at work) but as far as I understand, the kernel should be panicking then.

I'll report if I get any crashes with much smaller datasets.
Much appreciated!
I think technically, the watchdog timer should only activate for the GPU that is connected to a screen. Your specs and log show you would have two GPUs; the iGPU on your processor, and the GT 750M. I know macOS has a setting that allows you to switch between GPUs for your display. If you can get macOS to use your iGPU, rather than the GT 750M for display output, then that would leave your GT 750M for compute tasks and the watchdog should not activate.

Please let me know how you go!
Ivo Jager
StarTools creator and astronomy enthusiast
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.440 crash

Post by admin »

I forgot to mention, you can also force StarTools to use the Iris Pro iGPU instead of the GT 750M to see if that helps at all.

To do so, navigate to the folder where the StarTools 'resources', 'CHANGELOG', etc. files are located. Launch a command shell there.

Execute ;

Code: Select all

echo "0" > opencldeviceindex.cfg
A file named opencldeviceindex.cfg is now created with just one character in it ("0").
Upon (re)starting StarTools you should see your iGPU being used, rather than the GT 750M.

(if that didn't work, try a '1' instead of a '0')
Ivo Jager
StarTools creator and astronomy enthusiast
clouzot
Posts: 11
Joined: Thu Jul 23, 2020 7:56 pm

Re: 1.7.440 crash

Post by clouzot »

Hi Ivo,

just wanted to mention I'm getting the same sudden crashes on my Windows 10 x64 Pro machine, (NUC8i5BEK2, i5-8259U CPU, 16GB RAM, integrated Iris Plus Graphics 655 GPU), running Startools-x64-GPU 1.7.440

I'm not that familiar with Windows so I don't really know where to look for the Startools crash log. It doesn't appear in the Event Monitor nor in the so-called Reliability Monitor, but if you have an idea I can happily upload it if need be.

The Startools.log only shows the latest successful module I applied before it crashed in the Color module.
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.440 crash

Post by admin »

clouzot wrote: Tue Nov 10, 2020 9:39 am just wanted to mention I'm getting the same sudden crashes on my Windows 10 x64 Pro machine, (NUC8i5BEK2, i5-8259U CPU, 16GB RAM, integrated Iris Plus Graphics 655 GPU), running Startools-x64-GPU 1.7.440
For underpowered GPUs on Windows, you will want to read (and implement) this.
Ivo Jager
StarTools creator and astronomy enthusiast
clouzot
Posts: 11
Joined: Thu Jul 23, 2020 7:56 pm

Re: 1.7.440 crash

Post by clouzot »

admin wrote: Tue Nov 10, 2020 10:38 am
clouzot wrote: Tue Nov 10, 2020 9:39 am just wanted to mention I'm getting the same sudden crashes on my Windows 10 x64 Pro machine, (NUC8i5BEK2, i5-8259U CPU, 16GB RAM, integrated Iris Plus Graphics 655 GPU), running Startools-x64-GPU 1.7.440
For underpowered GPUs on Windows, you will want to read (and implement) this.
Thanks Ivo, I just disabled TDR as per your link, and ST1.7-GPU is indeed much more stable this way. :thumbsup:
clouzot
Posts: 11
Joined: Thu Jul 23, 2020 7:56 pm

Re: 1.7.440 crash

Post by clouzot »

admin wrote: Sun Nov 08, 2020 1:10 pm

If it is indeed caused by an underpowered GPU, then, sadly, the only remedy I have for you right now, is to use a different operating system (Linux or Windows). I will make sure, however, to provide a CPU-based version for macOS as well (as is the case for Linux and Windows).
Hi Ivo, just a gentle reminder about that issue: now that v1.7 is out, if you could build a non-GPU version for Mac...

Thanks!
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: 1.7.440 crash

Post by admin »

clouzot wrote: Sat Dec 19, 2020 8:57 pm If it is indeed caused by an underpowered GPU, then, sadly, the only remedy I have for you right now, is to use a different operating system (Linux or Windows). I will make sure, however, to provide a CPU-based version for macOS as well (as is the case for Linux and Windows).
In case this is still happening, and your iGPU or its drivers are simply not compatible, I just released an update (available in the download section) that should allow the system to switch to the CPU instead (though still through OpenCL).

To activate the switch, create a file named 'openclforcecpu.cfg' (case sensitive) in the 'distro' folder. Upon (re)launching StarTools, during the splash screen you should hopefully see at the bottom of the screen that it is using your CPU instead of the iGPU.

Do let me know how you go!
Ivo Jager
StarTools creator and astronomy enthusiast
Post Reply