The RAW Converters vs. the Narrow Bandpass filter

[Andy Perrin]

Following my post on using the 405nm laser to image falling snow, I decided that I needed a 405nm bandpass filter that was larger than 12mm (the size of the Omega filter that I used in that series). I bought one online earlier this week, and it arrived today, so I tested it out. The filter I bought was the MidOpt Bi405 25.4mm filter, which has this spectrum, using data given by the manufacturer and replotted by me for easier reading:

 

The filter also has an IR leak which I plugged by stacking with BG38 2mm.

 

Having acquired the filter, I took some photos with it out my window using the Sony A7S full spectrum conversion, and the EL-Nikkor 80mm/5.6 (metal) lens. I then processed the images in PhotoNinja as usual. My settings for PN were 

with the default values inside the checked boxes. At first everything seemed fine, albeit with a tiny bit of blurriness that I didn't usually get with the EL-Nikkor:

 

But a closer examination showed something was very badly awry! Here is a crop of the above image enlarged 300% with nearest neighbor interpolation:

 

It looked horrible. Mind you, it looked like this in the original TIFF and in the RAW converter, so that blockiness isn't JPEG artifacts. I did wonder at first if it might be caused by the fact that Sony uses compressed RAWs in their A7S (uncompressed is not an option) but further investigation convinced me otherwise. Because the next thing I tried was processing the image with Adobe Camera RAW, with very different results.

 

ACR, with "Adobe Monochrome" and 16 bits selected, produced the following rendition, again with all default settings unmodified:

 

Full size:

 

Crop at 300% with nearest neighbor interpolation:

 

MUCH better. So the conclusion I'm drawing is that when all the information is in one channel (blue here), PhotoNinja has serious issues processing the RAW, but Adobe does not.

 

LATE BREAKING UPDATE:

While there is about 2 stops more blue in the RAW than red and green and green2, RAW Digger (yes I own it finally) says there's plenty of the latter.

 

[Fandyus]

In this case, you'd either want to extract the color channel or you could push the color balance far enough for it to not be all just blue, then make it monochrome. The ACR way works too but you're basically getting 1/4th the sharpness because your sensor is only using 1/4 of the photosites. If you force the color balance, you should get better sharpness but more noise as well.

[Doug A]

Wow. But, Photo Ninja is supposed to be one of the best for setting UV white balance. Now what?

 

Thanks,

Doug A

[Andy Perrin]

30 minutes ago, Doug A said:

Wow. But, Photo Ninja is supposed to be one of the best for setting UV white balance. Now what?

 

Thanks,

Doug A

This doesn't have much implication for UV white balance actually - it does not do this on ordinary UV photos. (And this is technically violet anyway, not UV.) It happens, as far as I can tell, specifically with very narrowband filters centered on one of the three colors (R,G,B) when there is no signal in the other two colors.

[Andy Perrin]

41 minutes ago, Fandyus said:

In this case, you'd either want to extract the color channel or you could push the color balance far enough for it to not be all just blue, then make it monochrome. The ACR way works too but you're basically getting 1/4th the sharpness because your sensor is only using 1/4 of the photosites. If you force the color balance, you should get better sharpness but more noise as well.

Extracting the color channel would certainly work, but actually white balancing followed by monochrome conversion does not fix this issue in PN. I tried it. The PN RAW converter is doing some kind of incorrect demosaicing on the subpixels before it even gets to the rest of the software.

[colinbm]

Thanks Andy

[Andy Perrin]

*** Update*** 

 

PLOT TWIST! There is actually quite a bit of Red and Green and Green2 data in there, as shown by Raw Digger. The blue is much more intense (2 stops or so), but there's other stuff in that file.

 

 

Raw Digger shows the blue channel (with black gaps left for the other photosites) as looking like this:

 

 

[nfoto]

You ought to compare against Adobe without the "Monochrome" setting.

[JMC]

Andy, can't remember if I have posted my A7III spectral sensitivity work on here, but in that 400 to 410nm region you'd expect to see some red and green as well as the blue (curve about half way down the page) - https://jmcscientificconsulting.com/imaging/project-mirrorless-part-2-the-camera-conversion/

 

 

[Fandyus]

4 hours ago, Andy Perrin said:

Extracting the color channel would certainly work, but actually white balancing followed by monochrome conversion does not fix this issue in PN. I tried it. The PN RAW converter is doing some kind of incorrect demosaicing on the subpixels before it even gets to the rest of the software.

I would try Darktable if you want to dig in this issue more. I've done some extreme color balances followed by black and white conversion there and it didn't show any jagged edges 

[Andy Perrin]

8 hours ago, nfoto said:

You ought to compare against Adobe without the "Monochrome" setting.

Makes no difference, it still looks good, just bluer:

 

[Andy Perrin]

8 hours ago, JMC said:

Andy, can't remember if I have posted my A7III spectral sensitivity work on here, but in that 400 to 410nm region you'd expect to see some red and green as well as the blue (curve about half way down the page) - https://jmcscientificconsulting.com/imaging/project-mirrorless-part-2-the-camera-conversion/

 

 

You do see red and green, see the Raw Digger results above.

[Andy Perrin]

6 hours ago, Fandyus said:

I would try Darktable if you want to dig in this issue more. I've done some extreme color balances followed by black and white conversion there and it didn't show any jagged edges 

Darktable has many possible demosaic settings, so this is only one representative, but except for the options that pass through the original photosites, all the Darktable options render the image without artifacts I think.

 

My conclusion so far is that almost every RAW converter except PhotoNinja's can handle this image properly. That does not mean PN is awful because I'm sure there is SOME test case that every converter will fail on, but I think for people doing narrowband imaging, you need to be aware that PN is not the best choice in that scenario.

 

This is with PPG method with color smoothing disabled:

 

 

[Fandyus]

I see, it doesn't look super color balanced though. Did the color balance only push this far on auto? I know that sometimes with really extreme images, the software refuses to go further. I would mess with the green slider to see if I can get the whole image to be somewhat neutral. It might leave you with a lot of noise and probably around two discernible shades. At least that's what happened to me when I took images with only a blue LED for illumination. You can then make the image monochrome with the monochrome module to remove most of the noise.

[Andy Perrin]

18 minutes ago, Fandyus said:

I see, it doesn't look super color balanced though. Did the color balance only push this far on auto? I know that sometimes with really extreme images, the software refuses to go further. I would mess with the green slider to see if I can get the whole image to be somewhat neutral. It might leave you with a lot of noise and probably around two discernible shades. At least that's what happened to me when I took images with only a blue LED for illumination. You can then make the image monochrome with the monochrome module to remove most of the noise.

Yeah, this is the default settings with no adjustments. I'm actually not trying to get the best result possible with this image. Instead, my intent is to see if there are artifacts in the RAW conversion introduced by the various converters. I know it may be possible to improve the image in many of the converters by careful choice of settings, but that's not what this is about.

--

 

Again, let me restate the main conclusion here:

Almost every RAW converter except PhotoNinja's can handle this image properly. That does not mean PN is awful because I'm sure there is SOME test case that every converter will fail on, but I think for people doing narrowband imaging, you need to be aware that PN is not the best choice in that scenario.

[Andy Perrin]

Here is the Raw Photo Processor 64 (RPP64) result, version 27:

 

[Andy Perrin]

These are all with the converter defaults, with no effort at all made to fine-tune. So don't imagine that this or that converter is "better" than the others because these could probably all be improved with some effort. The thing to look for is the jagged artifacts present in the PhotoNinja image, which I don't see anywhere else so far.

[JMC]

3 hours ago, Andy Perrin said:

You do see red and green, see the Raw Digger results above.

Indeed. Just thought I'd mention it, as it helps explain your observation.

[dabateman]

Raw photo processor looks to be the best for your camera. 

I also have that filter, but haven't shot much with it. I will have to test it out on my various cameras to see how they hold up.

[Andrea B.]

Andy, this is a fascinating find.

I would like to contact Jim Christian (Photo Ninja) about this

because I can't help thinking that there is some sort of demosaicing bug.

 

 

To make sure I've got the facts:

• Violet-pass filter (with some UV and some blue-violet).
• Sony A7S conversion.
• EL-Nikkor 80/5.6.

 

I would like to know the Photo Ninja demosaic setting you used?

• Conservative, Moderate or Aggressive?
  If the camera has no anti-aliasing filter, then the Moderate or Aggressive setting is recommended by PN.
• Was the Balance G0/G1 setting checked?
  This default balances the even row greens with the odd row greens
  for those cameras which record the two greens differently.

 

The Sony A7S seems to record violet mostly in the blue channel with values (x,x,B) where x<B. 

Some cameras record violet as (x,z,2z) where x < z or (x,z,1.5z) or something similar.

It would be interesting (for me, anyway) to see an export of the raw composite from Raw Digger.

 

 

 

 

[Andrea B.]

Added:  The latest Photo Ninja is PhotoNinja_1.4.0d_sse423.app.

[Andy Perrin]

 

Quote

 I would like to know the Photo Ninja demosaic setting you used?

• Conservative, Moderate or Aggressive?
  If the camera has no anti-aliasing filter, then the Moderate or Aggressive setting is recommended by PN.
• Was the Balance G0/G1 setting checked?
  This default balances the even row greens with the odd row greens
  for those cameras which record the two greens differently.

Virtually the same result was observed with every combination of options in the demosaic settings. I should probably have stated this in the original post.

 

Quote

It would be interesting (for me, anyway) to see an export of the raw composite from Raw Digger.

Andrea, I did an export of the histogram from Raw Digger - it's in the original comment and below also. Did you mean you want to see the image of the raw composite? I can send the original file to play with.

 

 

[Andy Perrin]

5 hours ago, Andrea B. said:

Added:  The latest Photo Ninja is PhotoNinja_1.4.0d_sse423.app.

I just upgraded my photoninja to this version and there was no change at all in the results.

[Andrea B.]

I was wanting to see the color of the raw composite. It should look rather blue-violet?

 

Thanks for trying the latest version. 

And thank you for the info about trying all the demosaic combos.

 

If you wouldn't mind, you can upload the raw file into the next post.

After downloading it, I will remove it.

Or --- you can send a dropbox link.

Whatever is easiest. "-)

 

[Andrea B.]

I wanted to give Photo Ninja another white balance "stress test" on a monochrome image. Photo Ninja did OK. I wasn't trying to prove anything, just investigate what happens in PN when asked to WB a narrowband photo.

 

I photographed a blue (0,0,255) screen on my Macbook monitor. The photograph of the blue screen has a bit of green in it, because of the effects of ambient light in the room and because of the slight imperfections of a preset white balance in the camera (D850). I did try to minimize screen reflections as much as possible.

 

My monochrome color is something like (45,70,250) from a few sample measurements.

 

Here is the blue screen photo as seen in Photo Mechanic before conversion.

PM is displaying the internal JPG, of course.

 

 

Here is the blue screen photo in Photo Ninja.

The Color Enhancement is set to Plain 50.

The Exposure and Detail page has only the setting As Shot.

The PN histogram shows the R and G.

 

 

Here is the linear raw histogram from Raw Digger for the blue screen photo.

(This does "match" the histogram shown in Photo Ninja.)

 

 

Here is the blue screen photo after an average white balance was applied in Photo Ninja.

The enlargement in PN is at 1600% so that we can see the pixels.

I dragged the dropper in several swoops across the entire image.

A neutral white balance was achieved.  Well, as close as one can get. There are always a few pixels with some color cast. Note that the R, G and B color masses in the histogram have become well matched.