Bayer filter transmission in UV and IR

[JMC]

The regulars on here will know that I've been playing round with a home built device to measure spectral response of cameras, as I was struggling to find information on this in the UV. Here's the build thread;

 

http://www.ultraviol...ctral-response/

 

As the work progressed it became clear to me that I could use the data to calculate a transmission curve for the different channels of the Bayer filter (albeit including the effect of the microlenses as well), but in order to do that I needed a multispectral conversion - with the Bayer filter still present - of the same camera I had a monochrome conversion of, which is an EOS 5DSR. A few days ago I got one.

 

It's a bit convoluted to go through the data analysis here (to be honest I've not written it all up yet, and I think it will form the basis of a nice paper), but I want to share some of my findings.

 

Firstly, what do I get when I look in the visible region - 400nm to 700nm? There is some existing data for this, so I have something to try and compare with. Here's my data;

 

The chart shows the data for the Red (R ), Green (G and G2) and Blue (B) channels from the Raw files, as derived from RawDigger, and then processed to give the data shown. Compare this with a chart from a 2017 article ("Technology of color filter materials for image sensor", Taguchi and Enokido, 2017, available online);

 

Overall I'm happy with the shapes of the curves in my graph. My graph is made up of 2 lines for each colour of the Bayer sensor, and two pairs of lines for green as there are 2 green channels. The two sets of lines was because I collect data in two lots. 280nm to 480nm with the deuterium and halogen lamps of my light source, and then 440nm to 800nm with just the halogen source. This is done to minimise lambda/2 artefacts at the longer wavelengths. On my graph the y axis is actually expressed as a percentage of the light hitting the monochrome sensor at each wavelength. As you can see at some point this goes above 100%, and this is because my graph shows the transmission of the Bayer filter PLUS the microlenses (which get removed in the monochrome conversion).

 

With my setup I can measure between 280nm and 800nm, however the camera sensor rapidly loses sensitivity as you get towards 280nm, even with the monochrome conversion. As such I am not confident with what I'm seeing below 320nm. So, what does the graph look like if I add on the UV (320nm to 400nm) and IR (700nm to 800nm);

 

What does this tell me? The Bayer filter is very transparent in the IR, much more than in the UV. No wonder IR contamination is so damaging to UV pictures with cameras with the Bayer filter still attached. Interestingly the Green and Blue parts of the filter look to become more transparent further into the IR which I hadn't expected. The Bayer filter in the UV is highly absorbing, and rapidly drops in transmission as the wavelength gets shorter. This can be seen more obviously if I just show the graph between 320nm and 400nm;

 

If you think about it, a lot of our images are done in sunlight (which has a bias towards the longer wavelength UV), and the camera sensors are more sensitive to longer wavelength UV, and with the Bayer filter/microlenses also letting most in at the longer wavelengths, out images must be strongly biased towards long wavelength UV.

 

Dan at MaxMax told me that the monochrome conversion makes the camera about 6x more sensitive to UV, and looking at the amount absorbed by the Bayer filter and microlenses that looks to correlate quite well.

 

I must emphasize this is a first stab at the data - I've been on the computer about 4 hours to get this, so I will update if I find I've done anything wrong.

[Andy Perrin]

Quite interesting. What kind of error bars (metaphorically speaking) are we looking at in the UV region? The value is quite low there so I always wonder how big the noise is relative to the signal when reading a graph like this. In particular I am wondering why we see false yellow after white balancing in the shortwaves if the Bayer is blocking red more than green and blue?

[JMC]

Roughly 10-15% of any given value based on the assumptions I've made. Keep in mind though the filter you use on the lens. That will mean the area of interest is not going to be 320nm to 400nm. It'd be interesting to superimpose the Baader U transmission profile onto these and see what it looks like. Rough guess is it would make the red more prominent.

[Andy Perrin]

Yes, that’s right, I was forgetting the filter effect. Keep us posted!

[JMC]

I did run the Multispectral 5DSR + Rayfact 105mm UV lens + Baader U as part of my experiment yesterday, and have just gone through the data.

 

This is what it shows;

 

To explain - the y axis is now "Overall response per channel". This is not % transmission like the previous graphs, but is simply the response per channel at the different wavelengths. There is a none zero baseline, this is because of noise in the sensor I think - I run this at 6400ISO for 30s to capture the images, and I've noticed noise related to the ISO. if I lower the ISO the noise goes down, but I cannot get an image in the UV so it is a necessary evil until I get a more powerful light source. I do see some signal with the Baader U at 400nm above the baseline, but it is low. I seem to recall if I measured the transmission spectra of the Baader U, it was around 0.4% at 400nm.

 

With the filter in place the red channel becomes very prominent and the blue much less so.

 

Also I'm not sure how to deal with the green - there are 2 green channels - does the camera just add these together? If so should I be doing the same?

 

If the green channels are just added together then the red and green signals would be quite similar and much more prominent than the blue. Equal parts red and green and roughly half as much blue as either of the two others would give a yellow colour.

 

Also keep in mind this does not take into account the light source - in sunlight the curves would be more skewed to the longer wavelengths as sunlight has more of that. The effect would be to further compress the curves up around the 380nm area.

[Alex H]

How did you account for the fact that red channel is boosted during processing?

[JMC]

How did you account for the fact that red channel is boosted during processing?

I have done absolutely nothing to address that. Wouldn't even know where to begin to be honest. Does this boosting even happen in RAW files as that's what all my works done with? If this is an effect I need to consider, I'm surprised my data so closely matches the graphs from Taguchis paper.

 

Come to think of it, I'm not sure this is an issue. With my monochrome camera, each channel gives roughly the same value. The red isn't significantly different to the green and blue, so it isn't being boosted in the RAW file.

[Cadmium]

Jonathan, Very interesting!

[Andrea B.]

Also I'm not sure how to deal with the green - there are 2 green channels - does the camera just add these together? If so should I be doing the same?

 

The demosaicing algorithm handles the two green channels. You could simply average them and be on fairly safe ground for estimating color response. I'd simply leave both on the chart.

 

I can't decide if I'm bothered by the fact that they are not the same in your charts.

 

Jonathan, be sure that you are not infringing on Canon in any kind of patent way before publishing results. I do not know if these proprietary transmission details are part of a patent or not. But it might be wise to check. I doubt that Canon is going to track you down on UVP and complain. But they might see something in a published paper. (I must add, and please forgive my lack of understanding, why would a science journal accept a publication about a sensor response? I apologize for not understanding.)

 

Added: I have to ask that about patent infringement because I would be the one tasked with removing your post if heaven forbid!!! Canon ever complained. For the record, however, I do not think that is ever going to happen. And I personally think that sensor response is fair game for all of us to determine. We need to know!

[Alex H]

Come to think of it, I'm not sure this is an issue. With my monochrome camera, each channel gives roughly the same value. The red isn't significantly different to the green and blue, so it isn't being boosted in the RAW file.

 

Yes, if one opens it using "Raw composite" or renders each channel separately. But if you say that red is not different from green or blue, why in the image derived from monochrome camera red channel shows almost double the sensitivity comparing to other channels?

 

Boosting of red happens in the raw file. You can see it by opening any RAW file from unmodified camera in Raw digger and switching between "RGB render" and "Raw composite" settings. Very obvious on grey subjects.

[ulf]

I do not think this in any way could be a patent infringement.

It is just the characteristics of the Bayer dyes, open for anyone with enough skill and equipment to determine.

I cannot imagine that it is possible to get a patent approval for such information.

[JMC]

Andrea, there's nothing here that would be a problem patent wise. I have simply measured something and shared it. Nothing confidential, and it could be done with any camera. It is simply my measured sensor response, measured with a home made device.

 

As for publishing, I think this work can feed into a few areas. For instance, trying to trying to understand 'colour' outside of the visible spectrum. Also for my sunscreen work, data to support the monochrome camera sensitivity Vs one with a Bayer filter still present.

 

With regards to the green, are they averaged by the camera or are they summed though? Why have two greens? I don't know enough about sensor design.

[ulf]

Jonathan,

 

Thank you very much for this information.

It is very interesting.

 

I assume you are using some form of curve fitting or matching to your input data.

What width do your input binning have? Is that the width from your monochromator?

[JMC]

I'm measuring at 20 nm intervals Ulf. Curve fitting is simply done in Excel as part of the graph plotting.

[Andrea B.]

Jonathan - thank you for the reassurance regarding patents, etc!!

**********

 

I think we should remember that UV and IR 'colors' can be anything at all we want them to be as long as we maintain the relative contrasts or light/dark tones in the photo so as to still represent the subject in a human visible way. :)

 

*********

 

With regards to the green, are they averaged by the camera or are they summed though? Why have two greens? I don't know enough about sensor design.

 

There are references out there on sensor design.

 

We don't know what a particular camera does unless its demosaicing algorithm is known.

 

There are two greens because human eyes are most sensitive to greens. I think the greens are supposed to provide the luminance settings in the camera and the red and the blue together with one green or averaged greens in some formulaic way are used to provide to provide the color settings. Whether that approach is still valid in light of new ways of designing sensors, I do not know.

 

Fuji and Foveon tried a different approach but there doesn't seem to have been much else in DSLRs or mirrorless.There is the notion of YMC+K sensors color filters but I do not know if that has been used in DSLRs or mirrorless. I don't know how video cameras do color.

 

If you look further into this, then please let us know of any interesting facts you find.

 

I'm sure some other folks will be able to better answer this.

[DaveO]

Jonathan,

From a very decrepit chemist - I think this post - and your work - should be nominated for the Nobel Prize for UV photography

I have peered here, there and everywhere to find response curves for the Bayer dyes - thank you :D

With the normal degree of hand waving that we chemists indulge in I think it's possible to "explain" how we get the UV 'colours' when we white balance against PTFE/Spectralon through the BaaderU. The white balance on the Spectralon reflection would be an integration (if that's the correct term) of the light over the spectral window passed through the filter. If the RGB curves of the Bayer dyes could be exactly superimposed then I think we would not get the false colours. The fact that we see cross overs or mergers in the response curves mean, I think, that the colour balancing does not give white everywhere in the BaaderU window.

 

Dave

[ulf]

I'm measuring at 20 nm intervals Ulf. Curve fitting is simply done in Excel as part of the graph plotting.

 

As you are using Excel, it is easy to add gridlines for the wavelengths too.

That would improve the readability of your graphs a bit.

[JMC]

Alex, sorry I forgot to respond to your post. The graph I think you're referring to in this thread is for the colour multispectral camera, not the monochrome one. If I look at data from the monochrome camera RAW file, all the channels look the same. I'll put something up later to show what I mean, I'm currently away from the PC.

 

EDIT - so, to explain, this is a set of histograms from Rawdigger for a RAW file from the monochrome camera. In fact this was for when the monochromator was set to 660nm (far red).

 

 

All four channels are the same, or pretty much the same within a couple of percent of each other. With the Bayer filter removed I'd expect them to be. To me this says that the camera does not boost the red in the RAW file, unless specifically asked for an RGB render. I'm using the RAW composite for my work, not the RGB render.

[dabateman]

Jonathan, did you convert your regular Canon to full spectrum or did you buy a third camera that is converted?

If you still have the normal, would be interesting to compare the direct sensitivity of all three on the same curve.

Or maybe not if I remember the off the shelf droped fast at 400nm. I think I would like to see the mono vs full spectrum plotted as two seperate curves though, rather than divided out. This analysis of your gets to the old Maxmax point that full spectrum cameras don't see much below 360nm.

[JMC]

Jonathan, did you convert your regular Canon to full spectrum or did you buy a third camera that is converted?

If you still have the normal, would be interesting to compare the direct sensitivity of all three on the same curve.

Or maybe not if I remember the off the shelf droped fast at 400nm. I think I would like to see the mono vs full spectrum plotted as two seperate curves though, rather than divided out. This analysis of your gets to the old Maxmax point that full spectrum cameras don't see much below 360nm.

 

It's on the list Debateman. I have the data, I have yet to plot it all out.

 

I am fortunate enough to be in the position to have 3 of the 5DSR cameras. Unmodified, monochrome (UV-Vis-IR, Bayer filter and microlenses removed) and multispectral (UV-Vis-IR, with Bayer filter and microlenses attached).

[Andrea B.]

Everyone is aware, I'm hoping, that there is not just one set of dyes used in color array filters?

Hence the patenting of sensor packs.

[DaveO]

Andrea,

Exactly, so there is a UV elephant in the room if we make the assumption that Canon and Nikon Bayer dyes behave identically in the UV. Remember that we are operating way beyond the design constraints of the camera. The UV properties of the Bayer dyes would never have been considered. I think we are trying to use the white balanced UV colours as a proxy for the frequency of the UV reflections. Of course, if we were super rich we could buy fancy micro spectrometers to measure the UV spectra actually reflected by the plants. I have tried several times to even find out, here on the other side of the world, how many $$$ would be needed but the agents always have more important matters to pursue.

Dave

[JMC]

Absolutely Andrea. Anyone reading this should remember that it is for one camera. I've done some initial work with a Nikon and Canon which have the same filter over the sensor and there are some small differences.

[ulf]

I think a better method to get a combined result with a Baader U, or any other filter, is to calculate from measurements of a separate filter.

Then you avoid the SNR issues in the setup.

The filter measurement is relatively simple and should give a result with high confidence of the result.

[ulf]

I wonder if there is a way to increase the wavelength resolution of the measurements by taking more intermediate samples and do suitable calculations.

If the monochromator give a wavelength window with reasonably steep transitions that seams intuitively possible.

You might already do that.