StarTools 1.8.518 Beta 1 now available

General discussion about StarTools.
Mike in Rancho
Posts: 1141
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: StarTools 1.8.506 public alpha/preview now out

Post by Mike in Rancho »

Mike in Rancho wrote: Tue Jul 27, 2021 6:26 am
i7-8550U, 32GB, 940MX. I could go try it on my old desktop, Q9550, 4GB, N460GTX-OC. It always crashes entering the color module, but I can test this issue out before getting to that point.
I tried last night on the desktop with similar results. So I don't think it is machine dependent, but maybe data dependent. I did not see any of the artifacts in my M5 data, but did again with my Veil. Again both running Win 10, the GPU version, different machines.

This morning on the laptop again I gave a run at some more quality level data - Eyal's Corona Australis from that big practice thread on CN. I believe I cropped the edges, binned to 50, and took default on Wipe, then straight to SVD. They were hard to see, but I did again get some wild pixels. They can appear after setting the very first blue box and letting the processing finish. Some datasets they are very obvious, but others you kind of have to toggle the before/after or pre-tweak buttons and then maybe zoom in. I then set several more blue boxes, I think a total of 6 or 8 across the image, and more such pixels showed up. SVD settings were at default. I then changed resampling to intra-centroid (last option), which made some wild pixels go away, but a couple others took their place. Sometimes on the very same star, and you can see them bounce around when you do the toggling.

That's all I have to far, haven't discovered any pattern yet that would give a hint on causation.

Nobody else yet? Though I wonder if 506 is widespread for testing yet. I didn't know about it for a few days until I checked the forums.
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.8.506 public alpha/preview now out

Post by admin »

jackbak wrote: Tue Jul 27, 2021 3:46 pm Same fail with same Assertion error message.

Intel i7-3770K with in-built GPU, 16G. Tumbleweed current as of 07/27/2021.

Thanks Ivo,

Jack
Hmmm... I'm starting to run out of ideas. :(
I have prepared a debug version that outputs some information in a file named 'OpenCLError.log'. Would you be able to run this version and send me the output of the log?
Thank you!
Ivo Jager
StarTools creator and astronomy enthusiast
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.8.506 public alpha/preview now out

Post by admin »

Mike in Rancho wrote: Tue Jul 27, 2021 7:28 pm That's all I have to far, haven't discovered any pattern yet that would give a hint on causation.
Thanks MIke - I'll try to replicate this with some datasets that are outside my usual test suite.
Ivo Jager
StarTools creator and astronomy enthusiast
jackbak
Posts: 22
Joined: Sun Jun 14, 2015 3:38 pm

Re: StarTools 1.8.506 public alpha/preview now out

Post by jackbak »

Whoa, can you say "race condition". It worked (well, it came up fully, I haven't tested beyond that) when you added the write to file slow downs.

here is the OpenCLError.log - take a look,

[Num platforms
] :: [1]
[Using platform index
] :: [0]
[Num GPUS
] :: [1]
[Using device index
] :: [0]
[Found GPU
] :: [0]
[Intel(R) HD Graphics IvyBridge GT2] :: [0]
[
] :: [0]
[Device Found
] :: [834429664]
[Creating context
] :: [0]
[Device Index
] :: [0]
[Device Used
] :: [834429664]
[Creating queue
] :: [0]
[Init complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]
[Registering function
] :: [-1]
[Building program
] :: [0]
[Creating kernel
] :: [0]
[Registering function complete
] :: [0]

Thanks for your work on this,

Jack
jackbak
Posts: 22
Joined: Sun Jun 14, 2015 3:38 pm

Re: StarTools 1.8.506 public alpha/preview now out

Post by jackbak »

Ok just ran through a complete session with the 508 GPU debug code. Complete success on your part (I'm sure my technique is not perfect).

Thanks,
Jack
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.8.506 public alpha/preview now out

Post by admin »

jackbak wrote: Wed Jul 28, 2021 2:30 pm Ok just ran through a complete session with the 508 GPU debug code. Complete success on your part (I'm sure my technique is not perfect).

Thanks,
Jack
Awesome! I will roll this into the next release. Any other issues, do let me know. :thumbsup:
Ivo Jager
StarTools creator and astronomy enthusiast
alacant
Posts: 211
Joined: Mon Jan 25, 2016 7:03 am

Re: StarTools 1.8.506 public alpha/preview now out

Post by alacant »

Hi Ivo, everyone
Crash with 64bit GPU, Ubuntu 20.04. it's some Beignet stuff which is dead upstream so maybe time for a new box...

Jul 30 17:17:55 steve-desktop kernel: [ 3921.930984] traps: StarTools-Linux[7444] trap int3 ip:7faa216d6b79 sp:7ffd91953490 error:0 in libgbe.so[7faa21699000+1e35000]

Not a big deal. I'll use the non-gpu version but if it's an easy fix...
Cheers,
Steve
User avatar
admin
Site Admin
Posts: 3367
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.8.506 public alpha/preview now out

Post by admin »

alacant wrote: Fri Jul 30, 2021 3:20 pm Hi Ivo, everyone
Crash with 64bit GPU, Ubuntu 20.04. it's some Beignet stuff which is dead upstream so maybe time for a new box...

Jul 30 17:17:55 steve-desktop kernel: [ 3921.930984] traps: StarTools-Linux[7444] trap int3 ip:7faa216d6b79 sp:7ffd91953490 error:0 in libgbe.so[7faa21699000+1e35000]

Not a big deal. I'll use the non-gpu version but if it's an easy fix...
Cheers,
Steve
Does the debug version I created for @jackbak work though?
Ivo Jager
StarTools creator and astronomy enthusiast
Mike in Rancho
Posts: 1141
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: StarTools 1.8.506 public alpha/preview now out

Post by Mike in Rancho »

Ivo,

Any luck yet replicating wild pixels in SVD? If you've tried that is (I imagine you are very busy!). Or, anyone else notice it yet?

Other than backing iterations way off, I have not found a way to eliminate their generation. Altering other settings, even changing around my sample stars, will affect them, but again some will vanish but others will pop up, either on the same star or completely different stars.

The best initial suppression of them is the change to centroid tracking PSF, but that doesn't get all of them.

I did some testing and tracking last night, and the wild pixels survive all the way through denoise into the final image, so the artifacts do not seem to get caught and eliminated by the tracking.

So when I need a final image of something I've been going back to 1.7 or 505a, or in 506a I can just scan the image at high zoom at the very end and mask select the pixels and take it into heal. That works good if there aren't too many to search for. 506a has so many more improvements though that it's hard not to want to use it.

I can post some samples/screenshots if that would be helpful? But so far I am finding it to occur in nearly any dataset, whether mine or actual good data lol, and whether binned or full scale, or with other modules used first or straight to SVD after Wipe and AutoDev.
Mike in Rancho
Posts: 1141
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: StarTools 1.8.506 public alpha/preview now out

Post by Mike in Rancho »

Thought I'd post up another little snag I've run into in 506 SVD, in addition to the wild pixels.

I'm not quite sure what the process is that's going on here, but for a couple stars (and a few others strewn about that are outside of this crop) it almost seemed like some ringings were created that are offset from the star itself. I tried altering settings of course, but couldn't manage to shake it. So kind of the same as wild pixels in that way. I also changed around my sample stars in case I had picked a few clunkers, but that didn't fix things.

I then ran through a replicated workflow in 505 using the log. Exactly the same except I probably didn't pick the same blue boxes of course. I'm not sure how to replicate those and in a busy starfield, well all the stars look kind of the same. :lol: Anyway 505 does not create either these offset shadows or wild pixels either.

Any thoughts? Like the pixel artifacts I'm not sure how much here may be data driven or mistakes I make in settings, but all those things held the same it is different from 505 to 506.
ST8-506 SVD shadows.jpg
ST8-506 SVD shadows.jpg (42.63 KiB) Viewed 3035 times
Post Reply