Discussion idea - giving StarTools stdev data from stack?

[khyperia]

Okay, C++ version is all wrapped up tidy and hopefully free of debug info dumping

I've prebuilt both Windows and Linux versions. Windows is using Visual Studio 2013, Linux is using g++ 4.9.0 on my Arch Linux laptop.

I have no idea if the Linux one will actually be compatible on other linux distros, I've never actually tried to distribute a native linux prebuilt... although it needs the "cfitsio" package to run (I don't want to distribute it with the binary in case linux binaries aren't compatible or some such thing).
In any case, if the prebuilt doesn't work and you're comfortable with building from source, it's pretty darn easy to.
Make sure eigen3 and cfitsio are installed (on my system, it was as simple as "pacman -S cfitsio eigen"). I *do* use an unsupported bit of Eigen (the FFT module), so watch out for that.
Go grab this https://github.com/khyperia/Astropress, cd into Astropress/AstropressPlus, look at the build.sh and play with it until it compiles. It takes 20 seconds or so on my system. Feel free to report any issues you encounter, too, so I can fix them.

Windows should be portable, though, and I've even included a version of cfitsio.dll so that should be good to go right off the bat.

Same with the python version, it's a command-line-only app, usage is something like this (replace the filename if you're on linux):
AstropressPlus.exe --out path/to/output.fits --outstdev path/to/outputStandardDeviation.fits path/to/images/*.fits
Only .fits files are supported at the moment.

Hopefully it works! *crosses fingers*
Send me any failed datasets to let me reproduce any problems (and dump debug info, which is right now impossible to get out of the precompiled binary)

[Cheman]

khyperia
I havent had much time to devote to this yet. I downloaded the Windows version. I get the app to open with the command prompt, BUT being a windows GUI type, command prompts confuse me. I'm still not sure how to proceed. I dont know what to do with AstropressPlus.exe --out path/to/output.fits --outstdev path/to/outputStandardDeviation.fits path/to/images/*.fits
Assuming I just want everything on folders on my desktop(and that is where my fits files are and that is where the folder is that has AstropressPlus.exe)
what would I type in at the command prompt? I may have to wait for a GUI for this(hopefully), if you don't want to walk me thru. At any rate, I know this is a lot of work for you, and your efforts are not going unappreciated. If you dont have the time or patience for a "just press play" type like me, completely understandable.
Che

[khyperia]

khyperia
I havent had much time to devote to this yet. I downloaded the Windows version. I get the app to open with the command prompt, BUT being a windows GUI type, command prompts confuse me. I'm still not sure how to proceed. I dont know what to do with AstropressPlus.exe --out path/to/output.fits --outstdev path/to/outputStandardDeviation.fits path/to/images/*.fits
Assuming I just want everything on folders on my desktop(and that is where my fits files are and that is where the folder is that has AstropressPlus.exe)
what would I type in at the command prompt? I may have to wait for a GUI for this(hopefully), if you don't want to walk me thru. At any rate, I know this is a lot of work for you, and your efforts are not going unappreciated. If you dont have the time or patience for a "just press play" type like me, completely understandable.
Che

Ah... it's a bit difficult to explain, especially if something goes wrong, it'd probably be best to wait until I make a GUI version.
I was considering using Qt, although the package is utterly massive and I'm worried about how much it'd complicate distribution.

I haven't made much progress on that front, though. The past few days I've been chopping through wavelet transforms to try and get a good way of removing medium and large structures, to better improve star detection (in the precompiled version I'm using an FFT to do that, which sometimes doesn't produce the best results).

In any case, let's try explaining the command prompt:
Say you have all your .fits files in the directory Images on your desktop, like so:
Images\image1.fits
Images\image2.fits
Images\image3.fits
and so on.
Say you'd like to output the file "stack.fits" on your desktop, too.
Also, Astropress.exe is on the desktop.
Open up cmd.exe, and type in the following (or substituting whatever your username is if it's not Che):
cd C:\Users\Che\Desktop
AstropressPlus.exe --out stack.fits Images\*.fits

If something goes wrong, I'd say it'd just be easiest to wait for a GUI instead of us trying to work through it, which will be redundant work once I get the GUI going.

[Cheman]

OK getting closer!!I get the following
Loading/registering reference Images\*.fits
Exception!
FITS file library error: code 104

does that have something to do with the cfitsio file?
it is in the same folder as AstropressPlus

Che

[admin]

Apologies for not chiming in as much as I'd like to. I'm snowed under with 'the other job' at the moment (which pretty much bankrolls ST).
I tried the Linux app, but my install has a version of cfitsio (3.3) that's newer than the once you compiled against (2).
I'm going to need to upgrade my Linux box (Mint 14 is EOL) before I can install new packages and compile the code... bear with me!

[khyperia]

OK getting closer!!I get the following
Loading/registering reference Images\*.fits
Exception!
FITS file library error: code 104

does that have something to do with the cfitsio file?
it is in the same folder as AstropressPlus

Che

Okay, no, the problem is that the file pattern failed. That's the "Images\*.fits" thing - it tells the program to get all of the files in the "Images" directory, with the extension ".fits". Make sure that
a) all your images are in the Images directory on your desktop (if they're not, change the argument to read WhateverFolderNameItIs\*.fits instead of Images\*.fits)
b) all your images end with .fits (and are actually FITS files), not ending with .fit or some other thing. If they're not, change it to "Images\*.fit" or whatever extension they are (as long as they're actually FITS files)

Apologies for not chiming in as much as I'd like to. I'm snowed under with 'the other job' at the moment (which pretty much bankrolls ST).
I tried the Linux app, but my install has a version of cfitsio (3.3) that's newer than the once you compiled against (2).
I'm going to need to upgrade my Linux box (Mint 14 is EOL) before I can install new packages and compile the code... bear with me!

Okay, that's really weird. Checking the cfitsio installed package version, I have 3.360-1 installed right now. Did it fail because the .so couldn't be found / loaded properly, or because of cfitsio itself spitting volcanoes? Since I know in the fits_open_file() macro it compiles to something like ffopentest(CFITSIO_SONAME, a, b, c, d) or something like that. Possibly that compiletime constant is wrong, even though it has the right .so version? (as in I could just directly call ffopen function instead of using the "test" version the macro compiles to)

Yeah, in fact, CFITSIO_SONAME is #defined to 2 in my fitsio.h, although CFITSIO_VERSION is 3.36. That's weird.

Anyway, I'm actively hacking on the github repo, so you might nab a less-than-optimal version if you grab the latest (right now it's mostly a "commit/push whenever I change computers, so I can transfer the latest source code"). I'm considering pulling in Qt as a dependency as well, if I can't find anything else I like. Example, just earlier today I added Daubechies wavelets as opposed to Fourier transforms for removal of large structures, but it's not working too hot and I might change to some other mother wavelet... unless you can give me some tips on how to properly use a wavelet transform

[Cheman]

OK Success, sort of
Below is a link to the stacked image.
Pattern visible in image.
No color(QHY8 one shot color camera)
Hot pixels visible. I assume there is currently no provision to remove these.
Other than that, It looks like it stacked well. Why no color?
https://www.dropbox.com/s/6mf8lccp91iful7/stack.fits
this is a stack of 49 subs of M106
Che

[admin]

Okay, that's really weird. Checking the cfitsio installed package version, I have 3.360-1 installed right now. Did it fail because the .so couldn't be found / loaded properly, or because of cfitsio itself spitting volcanoes?
It's loooking for .so.2, so I tried 'fooling' it by making a copy of my .so.3 and naming it .so.2 - that made the program crash.

I'll compile from source as soon as I got some time (posting this from Mint 17 now - just need to put back my stuff and reconfigure now)

Example, just earlier today I added Daubechies wavelets as opposed to Fourier transforms for removal of large structures, but it's not working too hot and I might change to some other mother wavelet... unless you can give me some tips on how to properly use a wavelet transform

You could just use an 'à trous' like transform in the spatial domain using a fast Gaussian filter;

• make two copies of the image
• blur one using a Gaussian kernel that specifies the detail size that you want to keep
• subtract the blurred copy from the non-blurred copy; you're now left with the smaller scale structure (you've just performed a high pass filter)
• pat yourself on the back and have a beer

[khyperia]

OK Success, sort of
Below is a link to the stacked image.
Pattern visible in image.
No color(QHY8 one shot color camera)
Hot pixels visible. I assume there is currently no provision to remove these.
Other than that, It looks like it stacked well. Why no color?
https://www.dropbox.com/s/6mf8lccp91iful7/stack.fits
this is a stack of 49 subs of M106
Che

I should have been more clear - right now, only greyscale images are supported. I planned on "supporting" LRGB imaging by running the processor four times, one for each channel, giving the same L reference frame each time.
Here's my two guesses why there's a pattern:
1) The image is actually bayered, and there's metadata in the .fits file telling so that I'm not reading.
2) You have a really, really wonky dark signal that has that grid pattern on it, that I'm not subtracting since I don't support calibration yet.
I can't tell which one it is unless you send me single light frames as well as a dark frame.

Although, I'm starting to think I should just go the "use DSS's intermediate files" route instead, since DSS seems to have figured everything out and has all the features in it, like supporting bayer filters, calibration, it has a nice GUI, etc., something that's going to take a long time for me to implement (and the only purpose of my program is to create standard deviation data). Heck, it'd probably work with other stackers, too, as long as they have the option to output intermediate calibrated/registered .fits files.

You could just use an 'à trous' like transform in the spatial domain using a fast Gaussian filter;

• make two copies of the image
• blur one using a Gaussian kernel that specifies the detail size that you want to keep
• subtract the blurred copy from the non-blurred copy; you're now left with the smaller scale structure (you've just performed a high pass filter)
• pat yourself on the back and have a beer

I'd be worried about excessive negative ringing around stars if I do that, though. I guess that actually might be a good thing, as it forces a sharp threshhold around the star if I do a feature-detect threshholding at a >0 value. Perhaps skip the fourth step, though, as that'd be of questionable legality (I'm not even in college yet!)

[Cheman]

I should have been more clear - right now, only greyscale images are supported. I planned on "supporting" LRGB imaging by running the processor four times, one for each channel, giving the same L reference frame each time.
Here's my two guesses why there's a pattern:
1) The image is actually bayered, and there's metadata in the .fits file telling so that I'm not reading.
2) You have a really, really wonky dark signal that has that grid pattern on it, that I'm not subtracting since I don't support calibration yet.
I can't tell which one it is unless you send me single light frames as well as a dark frame.

Here is a light frame in case you want to look at it
https://www.dropbox.com/s/00ebxl0wutqud ... 10_001.fit
In my normal routine, I don't uses dark frames, only flats and bias. I stack and let DSS take care of hot pixels with Hot pixel removal.
Hope my input at this stage is of some help to you. If not, let me know and I'll stop bugging you
At the same time, if there is anything specifically I can do to help, say the word.
Che