Jump to content
UltravioletPhotography

Modelling UV camera response from Bayer filter meaurements


Jim Lloyd

Recommended Posts

This post presents work I have done with data kindly provided by Jonathan Crowther following his measurements as shown in this thread:

 

http://www.ultraviol...__fromsearch__1

 

I wasn't sure whether to add to that thread or start a new one - on the one hand I don't want to hijack that thread, but on the other I don't want to make it seem like this is my independent data.

 

Anyway here goes

 

Recap of Jonathan's method (apologies if I have misrepresented something):

  1. Light source: Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides reasonably flat output between about 250-1000nm
     
  2. Light passes through single diffraction grating monochromator (stray light rejection about 3 x10-5) into intergating sphere
     
  3. The integraphing sphere output is photographed using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter
     
  4. Images acquired at iso 6400 for 30 secs and imported into Rawdigger software to determine RGB values.
     
  5. Measurements made with 20 nm width from 280 - 480 nm
     
  6. Noise level(dark signal) was 6

My aims:

  1. Produce an anyltical equation to fit digital camera sensor response in the UV region and then use this to model how false colours are produced in a white balanced image in reflected sunlight
     
  2. Further compare this with the spectral sensitivity of the bee's short wave receptor

My methods

 

1. Subtract constant dark signal level of 6 from each data point and average g1 and g2

2. Set measurement at 480 to zero

3. Fit curve to data as follows. Model each channel response as sum of two lognormal curves (x-scale for fitting is wavelength difference from 500 nm - i.e. each log normal curve is skewed towrds the shorter wavelengths). The sd, mean and height of these curves are varied such that the sum of the squared errors (data minus fit) is minimised subject to the constraint that all values are non-negative and the area under the curve is that same as in the original data (using Solver in Excel)

4. Multiply modeled curves by solar spectrum - this was obtained from on-line model smoothed (see

https://www2.pvlight...calculator.aspx

5. This gives the modelled raw response in sunlight with this particular lens and filter

6. Peformed white balance step by normalising each of the response curves so that the area under each curve is the same and scale so that the maximum over all curves is 255 (i.e. the ratio between curves is preserved

7. This gives the sensor responses in sunlight after white balancing

8. sum over 10 nm bands to determine RGB values mimicing that see in "sparticle board" tests (this ignores the shape of the band pass filter, but this stage is just a sanity check to see if the colours predicted are roughly as might be expected.

9. Compare shape of response curves to the short wavelength visual receptor of bees (in sunlight) (modeled as a Gaussian curve, mean 345 sd 25 nm)

 

Results:

 

1. The model fit and the data are shown below for the raw data, after modelling sunlight and after white balance

 

post-175-0-59353600-1527450981.jpg

 

post-175-0-67014800-1527451466.jpg

 

post-175-0-49639500-1527451536.jpg

 

 

 

2. The false colours in various wavebands predicted by the model in sunlight after white balancing are shown below:

 

post-175-0-45840700-1527451307.jpg

 

3. None of the RG or B responses are close to the bee S response. The R+G signal would be closest, but this would be shifted by about 30 nm to longer wavelengths

 

post-175-0-02934200-1527451793.jpg

 

 

Discussion

 

 

Discussion:

Although the curves fit the data well, the solution is highly dependent on starting position - a simpler model was tried but did not fit the data well. Based on published data on Bayer filter response in the visible it is likely that there is structure in the response so that a simpler fit may miss important features. More resolution in the data would help to stabilize the solution

 

The false colours are believable which suggests the fitting and modelling is essentially correct (proves the concept), but it is difficult to say anything precise about this, or generalise this, due to dependence on the exact shape of the solar spectrum, variation in Bayer dyes between manufacturer and variation in lens and filter used. In very broad terms the "palette" of colours seen in UV photography is predicted - that is blues and yellows with variation in the hue of these plus grey.

 

Relating this to bee vision is complexs since the shape of the spectral response of the short wave receptor is just one input into a complex system. If one did want to produce an image illustrating the response of this channel alone then ideally one would need a sharper cut UV filter (i.e. cutting at a shorter wavelength, possibly Scott UG11 ?) or if using data provided by a system such as this it would be best to use either just the red channel as a greyscale image, or for better signal to noise add the red and green signal. This should be done before white balancing which would unduly boost the blue channel.

 

Overall this method appears useful to demonstrate in very broad terms how false colours arise in UV photography, but more detailed data is required before it could be applied in a practical application. It is suggested that the principle of providing an analytical function for the response of the Bayer filter/sensor is helpful as it allows others to incorporate this into their own system and lighting conditions.

 

Phew got that off my chest !

 

I need to get out and take more pictures now !

Link to comment

I don't think I understand that color chart in #2 at all having never seen those colours

in any white-balanced UV photo ???

And there is green on both ends of the chart, so I don't understand how you would know which one you were looking at?

Link to comment

Apologies to all if I am not very expansive on the forum for a while, but I have various competing priorities and tight deadlines st the moment. I need to produce some actual photographs (!) for an art exhibition.

 

I wrote everything I did up in detail, including aims, methods and opinions so hopefully there is sufficient there for people to make their own judgements

 

The colours produced suggest to me as proof of concept I was in the right ball park

 

If the green channel response is as predicted then no you won’t know where in the spectrum that response is coming from

 

Bear in mind I am reflecting the original data which shows a broad green response and significant transmission at 400 nm in all channels with Baader U in place. Sorry Jonathan if that sounds like a criticism ...

 

Link to comment

Here are the colors I get using bandpass filters. This shot was back-lit with a Canon 199A full spectrum modified flash with PTFE diffusion.

Of course, colors will vary a bit depending on the light you use, and exactly how it is white balanced, just like anything else.

I white balance on a very thin circle of PTFE that has the same back lighting as the bandpass filters.

My colors below differ a bit from the colors you posted above.

http://ultravioletphotography.com/content/uploads/monthly_05_2018/post-175-0-45840700-1527451307.jpg

One might refer to Weinheim's old color chart, but that was removed years ago now it seems for whatever reasons.

Regardless of that, I think the Bayer filters speak for themselves when shot through 10nm bandpass filters and the Baader U or other U filter of your choice.

So for what it's worth, here is my Sparticle version for comparison.

 

post-87-0-39691900-1527493760.jpg

Link to comment

Thanks Steve - that’s helpful

 

Where do I get one of those sparticles from - I guess it’s a matter of buying the appropriate filters and mounting them ?

 

What is the spectrum of the light source ?

 

It looks to me like the green response curve in my graph needs to shift leftwards particularly, maybe all need to shift that way a little

 

Jonathan - are you happy with the wavelength calibration of the monochromator ?

Link to comment

Just catching up with this now.

 

Jim - yes the monochromator was checked against my (calibrated) Ocean Optics spectrometer, and was out by a max of 2nm across the entire range of 280nm to 800nm. I would assume here that the choice of light source - solar spectrum vs flash will have a big impact on the white balance. After all it does in visible light imaging.

 

Steve - this comes back to something we chatted about a while ago, and getting hold of a Sparticle RAW image, so I can run it through the same type of analysis I use for my images as a RAW composite before any white balancing has taken. It would be great to know whether the data from the Sparticle matches that from my monochromator. Even if the light source if different (you used flash, I used the OO light source, monochromator and integrating sphere) the RGB data should be similar for the specific wavelength, assuming of course the Bayer filter transmission is the same - which is a big assumption, I accept.

 

Interestingly this comes back a little bit to my recent post about the colours from the Daisy and Buttercup. The Daisy reflect strongly around the 380nm to 400nm point, and looks blue/purple in my final white balanced picture. The Buttercup has more reflection in the 350-360nm region and looks yellow after white balancing. So post white balancing, I get similar colours to you Steve.

Link to comment

Jim, I think you are probably the nicest person on this whole board.

You don't get a Sparticle, you make one.

The filters are the hard part, you have to find them from Omega, and they don't list as many as they use to it seems.

So best idea is to ask Omega for the list you want, see if they have any surplus... try to get the 12.5mm diameter size, all the same size.

The spectrum of the light source I used for that pic I posted above is a Canon 199A flash, which works pretty close to sunlight, but my sense is that real sunlight has an even lower reach if using a really good lens.

Link to comment
Thanks Steve ! - I went to a convent junior school and if you weren’t nice the nuns hit you over the head with a bible !
Link to comment

Ideally, I would like to remove the 360BP15. I would like to have 360BP10 and 370BP10, but I have not seen those available as surplus.

Those might be available at a premium price however.

Link to comment

Jim: Apologies to all if I am not very expansive on the forum for a while, but I have various competing priorities and tight deadlines st the moment. I need to produce some actual photographs (!) for an art exhibition.

 

That is exciting! Wishing you well on this effort.

Link to comment

Found a little bit of time for this ... Still work in progress ...

 

I took Cadmium's image above and read off the RGB values (in View NX) and then fitted bi-normal curves to get this model of RGB camera response (this includes the light source and white balancing effects)

 

post-175-0-37762400-1527679152.jpg

 

Fit seems good except that around 370 there is uncertainty.

 

Comparison of colours read off from image, fitted values and original image look good (IMHO)

 

post-175-0-98342700-1527680097.jpg

 

This shows value at 360 scaled to 10 nm bandwith as well as original

 

 

 

Then looked at the FRED database and selected three common flowers:

 

post-175-0-58318700-1527679247.jpg

 

Then multiplied these reflectance spectra by the model fit to get these model responses. I also looked at estimating what false colour this would produce. It looked about right for the daisy, but when I tried with dandelion and buttercup the results were yellowy green - i.e rather too green. However There are lots of factors coming into play - particularly the light source so I didn't think it was worth taking that much further. I was interested particularly in the bird's foot trefoil as I thought that was a good example fo a UV dark flower that can be tricky in UV photography - to my naive eyes it looks like it might reflect in a similar way to the dandelion (same pigment?), but at a much lower level. So maybe a dark dullish yellow (that I found) is valid? ( see http://www.ultraviol...__fromsearch__1) I was also having a similar finding with the centre of ox-eye daisys which are coming out dark yellow rather than black.

 

post-175-0-60954700-1527679637.jpg

 

post-175-0-41137000-1527679661.jpg

 

post-175-0-41984000-1527679681.jpg

 

This looks promising to me.

 

Way forward:

 

Make own monochromator measurements (I have access to one in the hospital dermatology Dept used for skin testing) - normalise to output level to give source independent response

 

Fit this data

 

Make own Sparticle (Joanathan is kindly obtaining two sets of filters so we can have one each which will be great for comparisons)

 

Check model and sparticle agree

 

Use model for applications re flowers, bee and birds ... !!

Link to comment

Please forgive me, but I think I see a few risks or flaws in the assumptions below that might affect the results.

If I am wrong, please let me know how and help me understand better.

 

Recap of Jonathan's method (apologies if I have misrepresented something):

  1. Light source: Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides reasonably flat output between about 250-1000nm
     
  2. Light passes through single diffraction grating monochromator (stray light rejection about 3 x10-5) into intergating sphere
     
  3. The integraphing sphere output is photographed using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter
     
  4. Images acquired at iso 6400 for 30 secs and imported into Rawdigger software to determine RGB values.
     
  5. Measurements made with 20 nm width from 280 - 480 nm
     
  6. Noise level(dark signal) was 6

 

 

1. The Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides a reasonably flat output between about 250-1000nm.

 

Reasonably, is it good enough? It has, as all deuterium light sources a lot of variation over the wavelength range. It is much better than most other, but not very flat.

 

The graph OO provide do not have a linear, but logarithmic intensity scale: http://oceanoptics.c..._spectra-01.jpg

The intensity level of the visual light part can be adjusted with a trimmer at the rear panel, but the intensity curve never get close to constant.

In a linear graph there will be a lot of variation, peaks and dips, regardless of settings.

These variations can possibly affect the total result a bit.

However Jonathan might already have compensated for this, but then I missed that detail reading the posts.

 

3. The output data using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter show quite a lot of intensity above 400nm, especially in the blue channel.

http://ultravioletph...-1526752775.jpg

I think this is caused by the smearing effect of the monochromator wavelength-window width and possibly also by background noise.

As far as I know the Baader U has a lot better attenuation above 400nm.

It might be better to use the pure sensor response data instead, without the Baader U attached and then multiply that with the Baader U transmission.

Link to comment

Hi Ulf - comments always welcome as that is how we learn and advance

 

Please bear in mind that my interest in this thread is how you model the camera RGB sensor response from data measurements and then apply that model. I have used Jonathan's data that he kindly provided, but questions of his method should really be addressed to him in his thread. Having said that I understand that Joanthan's measurements are normalised to the source output - i.e. are relative response per waveband. Therefore in theory the source spectrum does not matter. I say in theory because there can be a problem with out of band rejection if the out of band irradiance is particularly high compared to the band of interest.

 

Regarding your second point I agree that something doesn't seem right with the amount of response recorded above 400 nm (again a question for Jonathan) and again in principle what you suggest is reasonable, but in practice I think using some band pass filters in addition to the monochromator should help to cut out any out of band signal. It also as in this case gives some idea of potential issues and noise level.

Link to comment

Ulf - the intensity issue with the light source was accounted for by. I measured the transmission through the monochromator at every 20nm interval, and then corrected using the absolute irradiance correction on the spectrometer. Essentially I ended up with a calibration files which I can use to correct for each 20nm band.

 

Ulf, Jim - the 400nm and above response confuses me too. Well, to be honest the response at 400nm doesn't confuse me, but the one at 420nm does. I have theories as to why there may be a signal but no answers, and no way to go further at the moment. Keep in mind the blue Bayer filter has a rapid increase in transmission above 40nm as well which doesn't help.

Link to comment

Thanks Steve

 

I didn’t expect to see the blue sensitivity rising again in the very short wavelengths - mostly academic considering most lenses and light sources.

 

What came of attempts at standardisation that klaus talks of ?

 

It would be helpful to have something a bit like this in a tutorial / Introduction section sticky somewhere

Link to comment

Jim, I don't know how that chart was derived or generated.

The Bayer filters produce typical colors per wavelength of the UV-A spectrum, just as you illustrated above, similar colors to results with the Sparticle,

both of those and the chart tend to agree which wavelength has which color. That is just how the Bayer filters react to the UV range they are sensitive to.

Link to comment

What came of attempts at standardisation that klaus talks of ?

 

Well not much because it is a difficult way to do spectrometric measurements of UV-reflective subjects. As has been discussed ad infinitum. :D

 

A Little Bit of History (which I seem to be writing about a lot these days......)

 

Everyone was temporarily enthusiastic about Klaus' Lumix GH1 UV-wavelength to false-color chart made in 2011 until we all realized how many factors stand in the way of applying it accurately. Then there is that pesky non-invertibility problem. Then Klaus got mad at us for saying all that. And Klaus also got mad at us for asking him to provide details about his methods. We got mad at Klaus for keeping secrets. Crankiness ensued on all sides about what is and is not Proper Science and who is and is not a Proper Scientist.

 

All that revisiting of the past reminds me to HEARTILY and SINCERELY THANK YOU JIM and JONATHAN for your openess in sharing your methods and results. You don't know how refreshing it is to see that.

 

There is a 4-page discussion on UVP where all this is discussed and re-discussed.

http://www.ultraviol...age__hl__sascha

 

That linked topic contains another Panasonic Lumix G3 color map posted by Sascha. He and Klaus got mad at each other over the Lumix maps they each had made. Sascha thought the maps weren't predictive. Especially because Klaus would not say how he made his map. Then Sascha got mad at us for acting like we were a "scientific peer reviewed panel" rather than the UV-enthusiast website we are. Of course everyone here having a Ph.D. (a lot!!) then got mad at Sascha. [[[Added: OK, I have exaggerated a little bit here. :D But not by much.]]]

 

So this entire topic involving UV-wavelength to False-colour mapping has historically generated large amounts of anger, rancor and dramatic exits from websites. Does anyone really wonder why I am not eager to put all this history in a Sticky somewhere?

 

Let me say formally: at this point neither I nor Bjørn Birna have been convinced of either the feasibility or the utility of UV-wavelength to false-colour mappings so we will not support a Sticky on the subject at this time. That can certainly change in the future. And this does not mean we are trying to discourage any efforts anywhere to work on this. Problems like this should be tackled to see what comes of it.

Link to comment

Heavy indeed! :-)

I had never seen that topic before, quite interesting, I am worn out now. ;-)

Andrea, Thanks for posting that.

Link to comment

Thanks for that 4-page memory bump Andrea, wow I certainly was enthusiastic. Let me say formally: at that point I was not mad at anyone and surely am not now. However, with repetition enthusiasm wanes and so I also happily thank the currently enthusiastic.

 

As to the feasibility of color to wavelength mapping I wish to add the following, which may seem painfully obvious to some but perhaps not others. Apologies in advance.

 

We understand in a general sense that metameric and spectral color are not mutually exclusive. One cannot distinguish if a visualized color is one or the other because perception has processed it down to a single channel - the color we name. This is the case for a real seen object, a photograph in a book or an image on the screen.

 

We know spectral UV false-color exists because the Sparticle shows us so. What has not been shown to my knowledge is a clear example of a UV false-metamerism. That would be a fine experiment for one of us to try.

 

The Sparticle is in effect a crude multiband spectrometer. This may be easily understood by illuminating it with a 365nm LED. Likewise the deferentially filtered pixels of Bayer filtered pixel array are a form of multiband spectrometer if the raw signals under each filter are considered independently. This is why the current line of inquiry has potential.

 

That said, it will predict that the relative signals in each channel can at best only be expected to roughly correlate to wavelength under reproducible controlled lighting conditions. This is why we have to white balance our cameras in the Vis under different lighting. In UV the Bayer CFA is not divided into distinct spectral passbands which must also limit spectral resolution in a manner similar to how the CFA looses chromatic distinction near the NIR limit of sensitivity.

 

Always enjoy this perennial topic!

Link to comment

Let me say formally: at that point I was not mad at anyone and surely am not now.

 

It is always best to keep a cool head. :D

I was telling a story up there and might have exaggerated. I'll go back and see if I need to tone it down. :rolleyes:

 

*******

 

About the UV-wavelength to false-colour mapping:

  • I fail to see why white balance should play a role at all. I think the mappings should be done using the demosaiced colour before white balance is applied. Note that I'm not saying that the illuminant should be disregarded. But why add another algorithm to the process. WB algorithms can vary according to which camera or conversion app you are using. They won't vary by much, but still.....

  • And speaking of demosaicing........It might be useful to consider the various demosaicing algorithms to see whether any method has an edge when demosaicing an image made in the narrower UV waveband??
    Many of us have had strange experiences with UV conversions. Ten years ago Bjørn and I and some others could not get a very clean UV conversion from Adobe Raw and took to using Bibble. I never figured out why ACR could not handle UV images well back then. It seemed partly to be a demosaicing problem (maze artifacts, color noise) and partly a temperature problem when attempting white balance. I've heard that ACR can now be used OK for UV conversions. But I gave up on Adobe when I started UV photography and never went back.
    The point here in this second bullet point is this: a good conversion of a UV image will lead to a cleaner, less noisy image and thus possibly to more accurate mappings.

  • We have talked (a lot) about the problem of, say, yellow and R+G both being recorded as the same color by the digital camera. But there is that other kind of lumping that occurs when the camera assigns the same RGB value to two different spectral wavelengths. IIRC, this is "camera metamerism". (Not sure, corrections welcomed on the terminology.)

Well, enough. It is late.

 

 


EDITOR'S NOTE: This topic was split. Some posts were split. Post paragraphs remaining in this topic are added where appropriate to other posts.


 

CADMIUM's RESPONSE:

White balance is essential. Everything is white balanced. We all know what a UV shot looks like when we shoot it with auto white balance or daylight, etc..

So, I am pretty sure I am not understanding your point.

 

Lumping doesn't happen with a Sparticle because all the narrow bands are using the same full spectrum light. Of course it does happen with reflected light.

 

Of course, the concept of wavelength mapping is different than the idea of a Sparticle, which was only intended to observe lens transmission depth (and handy for a few others tests measurements),

however, the Sparticle does illustrate the colors that the sensor sees through the Bayer filters when limited to the range of your UV filter.

Link to comment
Andy Perrin
But there is that other kind of lumping that happens where the camera assigns the same RGB value to two different spectral wavelengths. IIRC, this is "camera metamerism". (Not sure, corrections welcomed on the terminology.)

Wait, does that really happen? I did think that kind of one-way mapping was not an issue. Assuming you meant the hue, not the literal RGB value.

--

Thought: if Klaus's chart was correct, I suppose "blue" might repeat based on the earlier discussion, which would be an example. But most of our cameras don't go out to such short wavelengths, so it might not be a practical issue even so.

Link to comment

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
×
×
  • Create New...