DSS set black point to zero now automatic

[almcl]

...
? @almcl , do you think that there is any special relevance of this setting for processing in ST?

Well, the advice in one of Guy's tutorials was to avoid the stacking software making any changes to the data and when the check box was available it was advised to leave it unchecked.

I am not sure what difference one would see if the advice was not followed, but I suspect there was/is a reason. I also very much doubt anyone on the DSS forum has a better idea about StarTools processing requirements than Ivo does?

[Startrek]

Some practical suggestions

1/ Why not try the latest DSS version 5 with one of your data sets and see how it compares to the previous version where you had a choice of SBPTZ
Personally I don’t think it will matter , as why would David Partridge unintentionally force 1000’s of DSS/ DSLR users to have no choice in selecting SBPTZ. Doesn’t make any sense.
I don’t know whether Ivo would jump into this discussion as it’s a DSS issue , although he probably has the answer.

2/ Don’t upgrade , stay with your current version , I use version 4.2.5 ( which has the SBPTZ option ) and it’s works perfectly fine. “ If it ain’t broke don’t fix it “

3/ If your really concerned and can’t make up your mind whether to upgrade to version 5 DSS why not give ASTAP a go, I’ve used it for 3 months now and found it provided cleaner stacked images less artefacts and noise. Plus it’s much easier to use when stacking multiple nights data with multiple filters ( my Mono set up ) Im haven’t tried it on OSC data sets yet but intend to once we get some fine weather.
I’m still going to use DSS 4.2.5 and the latest ASTAP

In addition, Ivo ‘s notes on DSS settings for Startools ( link below ) relating to setting blackpoint to zero mentions that some users have found they need to check the box to get a usable image.One would assume then that this setting is a moving target and is not applicable to every DSLR.


https://www.startools.org/links--tutori ... r-settings

Cheers
Martin

[Mike in Rancho]

I don't get the sense there is any ST-unique or even "pristine unaltered data" issue here, other than your stack should be properly calibrated. Which should be true for any post-processing software.

Of course as explained earlier, I don't fully understand what exactly is happening here with SBPTZ, and what the severity of it going bad would be. But based on the discussions by the DSS experts and coders, it seems like a potential bias-level miscalibration issue. That can result in some poor datasets - something like flats over or undercorrection due to non-matching bias levels, as explained by Han in the link provided by Martin, or maybe even clipping of data, as is occasionally seen when someone unknowingly double-subtracts bias or takes darks that are too bright.

The discussions note that PI also has an option to handle this potential matter, but it goes by a different name. A lot of the discussion seemed to revolve around whether to just automate SBPTZ, since DSS can figure out which situations it should be applied and which is shouldn't. That would put it completely under the hood and hidden to the user. Perhaps ASTAP already does that? I wouldn't be surprised. There are other things that ASTAP forces, such as normalization for sigma clipping.

[decay]

In addition, Ivo ‘s notes on DSS settings for Startools ( link below ) relating to setting blackpoint to zero mentions that some users have found they need to check the box to get a usable image.One would assume then that this setting is a moving target and is not applicable to every DSLR.


https://www.startools.org/links--tutori ... r-settings

Thanks for pressing my nose onto this, Martin. This was what I was searching for and what I asked for at the beginning of this thread. I swear, I read this notes, but I must have missed this point somehow.

It sounds like one would obviously see if there's a problem with this setting (and his DSLR and calibration setup are prone to this). And it is nothing which is a special prerequisite for processing in ST.

A lot of the discussion seemed to revolve around whether to just automate SBPTZ, since DSS can figure out which situations it should be applied and which is shouldn't. That would put it completely under the hood and hidden to the user.

True, if I understand correctly they say it is automatically set depending on which kind of calibration frames are used. They don't say, it would not be set (or unset) at all.

Perhaps ASTAP already does that? I wouldn't be surprised.

True again. ASTAP seems to use DCRAW / LIBRAW as well and I would assume the same.

That can result in some poor datasets - something like flats over or undercorrection due to non-matching bias levels, as explained by Han in the link provided by Martin, or maybe even clipping of data

I would assume the latter

2/ Don’t upgrade , stay with your current version , I use version 4.2.5 ( which has the SBPTZ option ) and it’s works perfectly fine. “ If it ain’t broke don’t fix it “

Yes, of course that's a possible and totally fine solution because DSS is a very mature software. But I like the fact that development continues and that those guys spend some of their time to keep this project alive. And having multiple choices for different stackers is an advantage for all of us, isn't it? Just like we have the opportunity to use ST instead of PI, like most other people do these days. (That's the reason for me to support ST or it's community (well, don't know if I'm not having the opposite effect )) And if there is something we can do to help to find the right decision regarding this tiny checkbox in DSS that would be a benefit for DSS as well as for all of us. I think

But I'm afraid right now we cannot help in any way, unless someone can reproduce this problem. And maybe this automatic setting works just fine for all cases.

Best regards, Dietmar.

[Mike in Rancho]

I would assume the latter

Yes I think you are right, after reading those io discussions more, as well as googling Pixinsight and "no black point correction" which is their version of the exact same issue.

It's all about the extraction from RAW, by way of the options that can be fed into DCRAW or whatever. But DSS' terminology lends itself to confusion, IMO.

PI explains that you want the "no black point correction" option (and I believe it is a default setting) in order to get your pure RAW data. Or at least as pure as one can get out of a DSLR. If one does choose to use black point correction, then DCRAW will subtract off some form of the bias level, using a value from the DSLR's data (if so recorded and written to the RAW file). One does not normally want that. Instead, you want to do a full calibration yourself using more precise sets of bias, darks, etc.

In DSS, SBPTZ is the same as PI no black point correction. Zero = "no." Meaning zero (nothing) gets subtracted off of your RAW files by DCRAW.

So, if using DSLR, I imagine you would almost never want to uncheck SBPTZ. I suppose you could uncheck it and try to do a lights - flats only calibration, as DCRAW will then subtract off the camera's estimated bias level?

I can also see now why you might not want this to be fully automated. Defaulted to checked, yes. But, especially if I was trying to do some kind of unique calibration for a particular circumstance, I would be wary of DSS unchecking SBPTZ behind the scenes based on an algorithm (i.e. trying to out-think me) and having things go haywire.

[Startrek]

This is becoming a very interesting topic and have learnt so much from everyones comments especially Mikes.
In relation to ASTAP and DSLR / OSC stacking , I noticed Han has not included a checkbox for “setting black point to 0” so like Mike mentioned previously it maybe a “behind the scenes” process based on type of DSLR used and it’s RAW data or Han has intentionally recommended the use of a full Calibration suite.
From my experimentation with Mono data sets in ASTAP so far , you must use the full suite ie Darks ( or Bias ) Flats and Flat Darks ( or Bias ) otherwise your Flats don’t to the job , I ended up with an inverse Vignetting when I left the Darks tab empty. Han confirmed with me on the ASTAP forum that I should use my Bias in both Darks tab and Flat Darks tab, which I do now. Darks are a waist of time with the 2600MM as thoroughly tested by J Dupton on CN.
Getting back to ASTAP not having an option for the black point to 0 setting , the full Calibration suite might be the answer to it
Yes/ No / Maybe ??
Comments welcome
Martin

[Mike in Rancho]

Well I still haven't found a totally clear, nuts and bolts explanation of things. Maybe if one could dig into the DCRAW options it might be better explained. Or maybe not. Aren't DCRAW and LIBRAW sort of mystical in their operation?

But this is a DSLR RAW issue only, so nothing that needs to be accounted for with astrocam FITS.

Yes one has to do proper calibration when stacking, or flats will not work as intended, as explained in the link from Han that you posted earlier, Martin. How ASTAP handles SBPTZ (DSS) or No Black Point Correction (PI) would need to be directed to Han, but is not relevant to your 571s.

I do not agree that darks are a waste of time with 571s, though the difference between employing bias or dark flats (on the flats side of the equation) seems to fall into the "below detectable" category as being under the noise variance.

About a month ago there was (yet again) a CN discussion on this, in which Han joined in, then ran some experiments, and offered a conclusion that while in many cases minimal, using darks results in a cleaner stack. There is also the matter of hot pixels that in many cases you must utilize darks to handle, bias will not do it. Sure, with enough subs and dithering you might be able to get away with it. Or perhaps not. Others offered up the use of cosmetic correction techniques as the answer, which I believe Han effectively negated. Not that it will stop anyone!

The thread is here, starting at the top of page 2 where Han joined in: https://www.cloudynights.com/topic/8937 ... ry12978167

Note that while the title says OSC, the discussion revolves around calibration matters generally and I don't see that it is specific to OSC only.

[decay]

Sorry Gentlemen, I'm running a bit out of time right now, I will respond later to your posts above ...

For now I can contribute the following: The checkbox ''Set the black point to 0" of DSS sets the parameter "user_black" of DCRAW to literally 0, just like the name suggests:

2023-10-31 18_01_04-Window.jpg (41.32 KiB) Viewed 16048 times

-1 means "not set"

Some remarks contained in this part of the code:

2023-10-31 18_02_39-Window.jpg (93.6 KiB) Viewed 16048 times

Whereas ASTAP doesn't seem to use this parameter at all. So probably it always uses the camera internal black point:
(Edit: this is not true. Please see post below.)

2023-10-31 18_08_37-Window.jpg (77.46 KiB) Viewed 16048 times

I haven't found any other occurrences. But I will have a closer look later.

Best regards, Dietmar.

[Startrek]

Well I still haven't found a totally clear, nuts and bolts explanation of things. Maybe if one could dig into the DCRAW options it might be better explained. Or maybe not. Aren't DCRAW and LIBRAW sort of mystical in their operation?

But this is a DSLR RAW issue only, so nothing that needs to be accounted for with astrocam FITS.

Yes one has to do proper calibration when stacking, or flats will not work as intended, as explained in the link from Han that you posted earlier, Martin. How ASTAP handles SBPTZ (DSS) or No Black Point Correction (PI) would need to be directed to Han, but is not relevant to your 571s.

I do not agree that darks are a waste of time with 571s, though the difference between employing bias or dark flats (on the flats side of the equation) seems to fall into the "below detectable" category as being under the noise variance.

About a month ago there was (yet again) a CN discussion on this, in which Han joined in, then ran some experiments, and offered a conclusion that while in many cases minimal, using darks results in a cleaner stack. There is also the matter of hot pixels that in many cases you must utilize darks to handle, bias will not do it. Sure, with enough subs and dithering you might be able to get away with it. Or perhaps not. Others offered up the use of cosmetic correction techniques as the answer, which I believe Han effectively negated. Not that it will stop anyone!

The thread is here, starting at the top of page 2 where Han joined in: https://www.cloudynights.com/topic/8937 ... ry12978167

Note that while the title says OSC, the discussion revolves around calibration matters generally and I don't see that it is specific to OSC only.

Yes thankfully I’m getting away with using Bias in lieu of Darks and yes I do dither all my imaging.I should run a Dark Library on a cloudy night and do my own testing. Time , time …..
Although like you mentioned the title of the CN post says OSC, Han actually tested an older discontinued 1600MM camera not the later 2600MM
Dark current on the discontinued 1600MM is 0.03 e/ sec at 0 C
Dark current on the later 2600MM is less than 0.002 e/ sec at 0 C

It would have been great if Han tested the 2600MM as the results probably would have been much different. Still a full calibration set including Darks regardless of dark current is the only way forward to totally eliminate hot pixels and other anomalies
Cheers
Martin

[Startrek]

Well I still haven't found a totally clear, nuts and bolts explanation of things. Maybe if one could dig into the DCRAW options it might be better explained. Or maybe not. Aren't DCRAW and LIBRAW sort of mystical in their operation?

But this is a DSLR RAW issue only, so nothing that needs to be accounted for with astrocam FITS.

Yes one has to do proper calibration when stacking, or flats will not work as intended, as explained in the link from Han that you posted earlier, Martin. How ASTAP handles SBPTZ (DSS) or No Black Point Correction (PI) would need to be directed to Han, but is not relevant to your 571s.

I do not agree that darks are a waste of time with 571s, though the difference between employing bias or dark flats (on the flats side of the equation) seems to fall into the "below detectable" category as being under the noise variance.

About a month ago there was (yet again) a CN discussion on this, in which Han joined in, then ran some experiments, and offered a conclusion that while in many cases minimal, using darks results in a cleaner stack. There is also the matter of hot pixels that in many cases you must utilize darks to handle, bias will not do it. Sure, with enough subs and dithering you might be able to get away with it. Or perhaps not. Others offered up the use of cosmetic correction techniques as the answer, which I believe Han effectively negated. Not that it will stop anyone!

The thread is here, starting at the top of page 2 where Han joined in: https://www.cloudynights.com/topic/8937 ... ry12978167

Note that while the title says OSC, the discussion revolves around calibration matters generally and I don't see that it is specific to OSC only.

Yes thankfully I’m getting away with using Bias in lieu of Darks and yes I do dither all my imaging.I should run a Dark Library on a cloudy night and do my own testing. Time , time …..
Although like you mentioned the title of the CN post says OSC, Han actually tested an older discontinued 1600MM camera not the later 2600MM
Dark current on the discontinued 1600MM is 0.03 e/ sec at 0 C
Dark current on the later 2600MM is less than 0.002 e/ sec at 0 C
The 1600MM did suffer from some Ampglow as with many other ZWO cooled cameras
The 2600MM has almost zero Ampglow


It would have been great if Han tested the 2600MM as the results probably would have been much different. Still a full calibration set including Darks regardless of dark current as recommended is the only way forward to totally eliminate hot pixels and other anomalies
Cheers
Martin