• Ultraviolet Photography
  •  

Modelling UV camera response from Bayer filter meaurements

49 replies to this topic

#1 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 27 May 2018 - 20:15

This post presents work I have done with data kindly provided by Jonathan Crowther following his measurements as shown in this thread:

http://www.ultraviol...__fromsearch__1

I wasn't sure whether to add to that thread or start a new one - on the one hand I don't want to hijack that thread, but on the other I don't want to make it seem like this is my independent data.

Anyway here goes

Recap of Jonathan's method (apologies if I have misrepresented something):
  • Light source: Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides reasonably flat output between about 250-1000nm

  • Light passes through single diffraction grating monochromator (stray light rejection about 3 x10-5) into intergating sphere

  • The integraphing sphere output is photographed using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter

  • Images acquired at iso 6400 for 30 secs and imported into Rawdigger software to determine RGB values.

  • Measurements made with 20 nm width from 280 - 480 nm

  • Noise level(dark signal) was 6
My aims:
  • Produce an anyltical equation to fit digital camera sensor response in the UV region and then use this to model how false colours are produced in a white balanced image in reflected sunlight

  • Further compare this with the spectral sensitivity of the bee's short wave receptor
My methods

1. Subtract constant dark signal level of 6 from each data point and average g1 and g2
2. Set measurement at 480 to zero
3. Fit curve to data as follows. Model each channel response as sum of two lognormal curves (x-scale for fitting is wavelength difference from 500 nm - i.e. each log normal curve is skewed towrds the shorter wavelengths). The sd, mean and height of these curves are varied such that the sum of the squared errors (data minus fit) is minimised subject to the constraint that all values are non-negative and the area under the curve is that same as in the original data (using Solver in Excel)
4. Multiply modeled curves by solar spectrum - this was obtained from on-line model smoothed (see
https://www2.pvlight...calculator.aspx
5. This gives the modelled raw response in sunlight with this particular lens and filter
6. Peformed white balance step by normalising each of the response curves so that the area under each curve is the same and scale so that the maximum over all curves is 255 (i.e. the ratio between curves is preserved
7. This gives the sensor responses in sunlight after white balancing
8. sum over 10 nm bands to determine RGB values mimicing that see in "sparticle board" tests (this ignores the shape of the band pass filter, but this stage is just a sanity check to see if the colours predicted are roughly as might be expected.
9. Compare shape of response curves to the short wavelength visual receptor of bees (in sunlight) (modeled as a Gaussian curve, mean 345 sd 25 nm)

Results:

1. The model fit and the data are shown below for the raw data, after modelling sunlight and after white balance

Attached Image: Capture Bayer response raw.JPG

Attached Image: Capture raw response in sun.JPG

Attached Image: Capture response after WB.JPG



2. The false colours in various wavebands predicted by the model in sunlight after white balancing are shown below:

Attached Image: Capture modelled false colours.JPG

3. None of the RG or B responses are close to the bee S response. The R+G signal would be closest, but this would be shifted by about 30 nm to longer wavelengths

Attached Image: Capture bee response.JPG


Discussion


Discussion:
Although the curves fit the data well, the solution is highly dependent on starting position - a simpler model was tried but did not fit the data well. Based on published data on Bayer filter response in the visible it is likely that there is structure in the response so that a simpler fit may miss important features. More resolution in the data would help to stabilize the solution

The false colours are believable which suggests the fitting and modelling is essentially correct (proves the concept), but it is difficult to say anything precise about this, or generalise this, due to dependence on the exact shape of the solar spectrum, variation in Bayer dyes between manufacturer and variation in lens and filter used. In very broad terms the "palette" of colours seen in UV photography is predicted - that is blues and yellows with variation in the hue of these plus grey.

Relating this to bee vision is complexs since the shape of the spectral response of the short wave receptor is just one input into a complex system. If one did want to produce an image illustrating the response of this channel alone then ideally one would need a sharper cut UV filter (i.e. cutting at a shorter wavelength, possibly Scott UG11 ?) or if using data provided by a system such as this it would be best to use either just the red channel as a greyscale image, or for better signal to noise add the red and green signal. This should be done before white balancing which would unduly boost the blue channel.

Overall this method appears useful to demonstrate in very broad terms how false colours arise in UV photography, but more detailed data is required before it could be applied in a practical application. It is suggested that the principle of providing an analytical function for the response of the Bayer filter/sensor is helpful as it allows others to incorporate this into their own system and lighting conditions.

Phew got that off my chest !

I need to get out and take more pictures now !

Edited by Jim Lloyd, 27 May 2018 - 20:18.


#2 BruceG

    Member

  • Members
  • 66 posts

Posted 28 May 2018 - 02:24

Hi Jim, how is the bee response curve measured in this case?

#3 Andrea B.

    Desert Dancer

  • Owner-Administrator
  • 7,566 posts
  • Location: USA

Posted 28 May 2018 - 05:07

I don't think I understand that color chart in #2 at all having never seen those colours
in any white-balanced UV photo ???
And there is green on both ends of the chart, so I don't understand how you would know which one you were looking at?
Andrea G. Blum
Often found hanging out with flowers & bees.

#4 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 28 May 2018 - 05:32

Bruce - the bee response is estimated from published data

https://www.research...in_the_honeybee

#5 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 28 May 2018 - 06:58

Apologies to all if I am not very expansive on the forum for a while, but I have various competing priorities and tight deadlines st the moment. I need to produce some actual photographs (!) for an art exhibition.

I wrote everything I did up in detail, including aims, methods and opinions so hopefully there is sufficient there for people to make their own judgements

The colours produced suggest to me as proof of concept I was in the right ball park

If the green channel response is as predicted then no you won’t know where in the spectrum that response is coming from

Bear in mind I am reflecting the original data which shows a broad green response and significant transmission at 400 nm in all channels with Baader U in place. Sorry Jonathan if that sounds like a criticism ...


#6 Cadmium

    Member

  • Members(+)
  • 3,069 posts

Posted 28 May 2018 - 07:52

Here are the colors I get using bandpass filters. This shot was back-lit with a Canon 199A full spectrum modified flash with PTFE diffusion.
Of course, colors will vary a bit depending on the light you use, and exactly how it is white balanced, just like anything else.
I white balance on a very thin circle of PTFE that has the same back lighting as the bandpass filters.
My colors below differ a bit from the colors you posted above.
http://ultravioletph...-1527451307.jpg
One might refer to Weinheim's old color chart, but that was removed years ago now it seems for whatever reasons.
Regardless of that, I think the Bayer filters speak for themselves when shot through 10nm bandpass filters and the Baader U or other U filter of your choice.
So for what it's worth, here is my Sparticle version for comparison.

Attached Image: Sparticle_199A_Diffuser_Kuri_35_1020.jpg

Edited by Cadmium, 28 May 2018 - 07:55.


#7 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 28 May 2018 - 08:41

Thanks Steve - that’s helpful

Where do I get one of those sparticles from - I guess it’s a matter of buying the appropriate filters and mounting them ?

What is the spectrum of the light source ?

It looks to me like the green response curve in my graph needs to shift leftwards particularly, maybe all need to shift that way a little

Jonathan - are you happy with the wavelength calibration of the monochromator ?

#8 JMC

    Member

  • Members(+)
  • 836 posts
  • Location: London, UK

Posted 28 May 2018 - 08:58

Just catching up with this now.

Jim - yes the monochromator was checked against my (calibrated) Ocean Optics spectrometer, and was out by a max of 2nm across the entire range of 280nm to 800nm. I would assume here that the choice of light source - solar spectrum vs flash will have a big impact on the white balance. After all it does in visible light imaging.

Steve - this comes back to something we chatted about a while ago, and getting hold of a Sparticle RAW image, so I can run it through the same type of analysis I use for my images as a RAW composite before any white balancing has taken. It would be great to know whether the data from the Sparticle matches that from my monochromator. Even if the light source if different (you used flash, I used the OO light source, monochromator and integrating sphere) the RGB data should be similar for the specific wavelength, assuming of course the Bayer filter transmission is the same - which is a big assumption, I accept.

Interestingly this comes back a little bit to my recent post about the colours from the Daisy and Buttercup. The Daisy reflect strongly around the 380nm to 400nm point, and looks blue/purple in my final white balanced picture. The Buttercup has more reflection in the 350-360nm region and looks yellow after white balancing. So post white balancing, I get similar colours to you Steve.

#9 Cadmium

    Member

  • Members(+)
  • 3,069 posts

Posted 28 May 2018 - 09:06

Jim, I think you are probably the nicest person on this whole board.
You don't get a Sparticle, you make one.
The filters are the hard part, you have to find them from Omega, and they don't list as many as they use to it seems.
So best idea is to ask Omega for the list you want, see if they have any surplus... try to get the 12.5mm diameter size, all the same size.
The spectrum of the light source I used for that pic I posted above is a Canon 199A flash, which works pretty close to sunlight, but my sense is that real sunlight has an even lower reach if using a really good lens.

Edited by Cadmium, 28 May 2018 - 14:13.


#10 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 28 May 2018 - 10:15

Thanks Steve ! - I went to a convent junior school and if you weren’t nice the nuns hit you over the head with a bible !

Edited by Jim Lloyd, 28 May 2018 - 19:41.


#11 Cadmium

    Member

  • Members(+)
  • 3,069 posts

Posted 28 May 2018 - 14:16

Ideally, I would like to remove the 360BP15. I would like to have 360BP10 and 370BP10, but I have not seen those available as surplus.
Those might be available at a premium price however.

Edited by Cadmium, 29 May 2018 - 02:19.


#12 Andrea B.

    Desert Dancer

  • Owner-Administrator
  • 7,566 posts
  • Location: USA

Posted 28 May 2018 - 16:21

Jim: Apologies to all if I am not very expansive on the forum for a while, but I have various competing priorities and tight deadlines st the moment. I need to produce some actual photographs (!) for an art exhibition.

That is exciting! Wishing you well on this effort.
Andrea G. Blum
Often found hanging out with flowers & bees.

#13 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 30 May 2018 - 11:32

Found a little bit of time for this ... Still work in progress ...

I took Cadmium's image above and read off the RGB values (in View NX) and then fitted bi-normal curves to get this model of RGB camera response (this includes the light source and white balancing effects)

Attached Image: Capture fitted curves to sparticle WB sun Baader U Cadmium.JPG

Fit seems good except that around 370 there is uncertainty.

Comparison of colours read off from image, fitted values and original image look good (IMHO)

Attached Image: Capture sparticle sun baader U colours.JPG

This shows value at 360 scaled to 10 nm bandwith as well as original



Then looked at the FRED database and selected three common flowers:

Attached Image: Capture FRED database.JPG

Then multiplied these reflectance spectra by the model fit to get these model responses. I also looked at estimating what false colour this would produce. It looked about right for the daisy, but when I tried with dandelion and buttercup the results were yellowy green - i.e rather too green. However There are lots of factors coming into play - particularly the light source so I didn't think it was worth taking that much further. I was interested particularly in the bird's foot trefoil as I thought that was a good example fo a UV dark flower that can be tricky in UV photography - to my naive eyes it looks like it might reflect in a similar way to the dandelion (same pigment?), but at a much lower level. So maybe a dark dullish yellow (that I found) is valid? ( see http://www.ultraviol...__fromsearch__1) I was also having a similar finding with the centre of ox-eye daisys which are coming out dark yellow rather than black.

Attached Image: Capture daisy.JPG

Attached Image: Capture dandelion.JPG

Attached Image: Capture bft.JPG

This looks promising to me.

Way forward:

Make own monochromator measurements (I have access to one in the hospital dermatology Dept used for skin testing) - normalise to output level to give source independent response

Fit this data

Make own Sparticle (Joanathan is kindly obtaining two sets of filters so we can have one each which will be great for comparisons)

Check model and sparticle agree

Use model for applications re flowers, bee and birds ... !!

Edited by Jim Lloyd, 30 May 2018 - 11:43.


#14 UlfW

    Ulf W

  • Members(+)
  • 859 posts
  • Location: Sweden, Malmö

Posted 30 May 2018 - 14:24

Please forgive me, but I think I see a few risks or flaws in the assumptions below that might affect the results.
If I am wrong, please let me know how and help me understand better.

View PostJim Lloyd, on 27 May 2018 - 20:15, said:

Recap of Jonathan's method (apologies if I have misrepresented something):
  • Light source: Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides reasonably flat output between about 250-1000nm

  • Light passes through single diffraction grating monochromator (stray light rejection about 3 x10-5) into intergating sphere

  • The integraphing sphere output is photographed using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter

  • Images acquired at iso 6400 for 30 secs and imported into Rawdigger software to determine RGB values.

  • Measurements made with 20 nm width from 280 - 480 nm

  • Noise level(dark signal) was 6


1. The Ocean Optics DH-2000-BAL - dual deuterium and hallogen provides a reasonably flat output between about 250-1000nm.

Reasonably, is it good enough? It has, as all deuterium light sources a lot of variation over the wavelength range. It is much better than most other, but not very flat.

The graph OO provide do not have a linear, but logarithmic intensity scale: http://oceanoptics.c..._spectra-01.jpg
The intensity level of the visual light part can be adjusted with a trimmer at the rear panel, but the intensity curve never get close to constant.
In a linear graph there will be a lot of variation, peaks and dips, regardless of settings.
These variations can possibly affect the total result a bit.
However Jonathan might already have compensated for this, but then I missed that detail reading the posts.

3. The output data using Canon EOS 5DSR with Rayfacts 105 mm lens and Baader U filter show quite a lot of intensity above 400nm, especially in the blue channel.
http://ultravioletph...-1526752775.jpg
I think this is caused by the smearing effect of the monochromator wavelength-window width and possibly also by background noise.
As far as I know the Baader U has a lot better attenuation above 400nm.
It might be better to use the pure sensor response data instead, without the Baader U attached and then multiply that with the Baader U transmission.
Ulf Wilhelmson
Curious and trying to see the invisible.

#15 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 30 May 2018 - 15:30

Hi Ulf - comments always welcome as that is how we learn and advance

Please bear in mind that my interest in this thread is how you model the camera RGB sensor response from data measurements and then apply that model. I have used Jonathan's data that he kindly provided, but questions of his method should really be addressed to him in his thread. Having said that I understand that Joanthan's measurements are normalised to the source output - i.e. are relative response per waveband. Therefore in theory the source spectrum does not matter. I say in theory because there can be a problem with out of band rejection if the out of band irradiance is particularly high compared to the band of interest.

Regarding your second point I agree that something doesn't seem right with the amount of response recorded above 400 nm (again a question for Jonathan) and again in principle what you suggest is reasonable, but in practice I think using some band pass filters in addition to the monochromator should help to cut out any out of band signal. It also as in this case gives some idea of potential issues and noise level.

#16 JMC

    Member

  • Members(+)
  • 836 posts
  • Location: London, UK

Posted 30 May 2018 - 16:27

Ulf - the intensity issue with the light source was accounted for by. I measured the transmission through the monochromator at every 20nm interval, and then corrected using the absolute irradiance correction on the spectrometer. Essentially I ended up with a calibration files which I can use to correct for each 20nm band.

Ulf, Jim - the 400nm and above response confuses me too. Well, to be honest the response at 400nm doesn't confuse me, but the one at 420nm does. I have theories as to why there may be a signal but no answers, and no way to go further at the moment. Keep in mind the blue Bayer filter has a rapid increase in transmission above 40nm as well which doesn't help.

#17 Cadmium

    Member

  • Members(+)
  • 3,069 posts

Posted 30 May 2018 - 23:36

I found the old color chart that I thought didn't exist anymore:
https://www.fotozone...%2Blens%2B%2Buv

#18 Jim Lloyd

    Member

  • Members(+)
  • 312 posts

Posted 31 May 2018 - 13:39

Thanks Steve

I didn’t expect to see the blue sensitivity rising again in the very short wavelengths - mostly academic considering most lenses and light sources.

What came of attempts at standardisation that klaus talks of ?

It would be helpful to have something a bit like this in a tutorial / Introduction section sticky somewhere

#19 Cadmium

    Member

  • Members(+)
  • 3,069 posts

Posted 31 May 2018 - 16:13

Jim, I don't know how that chart was derived or generated.
The Bayer filters produce typical colors per wavelength of the UV-A spectrum, just as you illustrated above, similar colors to results with the Sparticle,
both of those and the chart tend to agree which wavelength has which color. That is just how the Bayer filters react to the UV range they are sensitive to.

Edited by Cadmium, 31 May 2018 - 20:38.


#20 Andrea B.

    Desert Dancer

  • Owner-Administrator
  • 7,566 posts
  • Location: USA

Posted 31 May 2018 - 19:10

What came of attempts at standardisation that klaus talks of ?

Well not much because it is a difficult way to do spectrometric measurements of UV-reflective subjects. As has been discussed ad infinitum. :D

A Little Bit of History (which I seem to be writing about a lot these days......)

Everyone was temporarily enthusiastic about Klaus' Lumix GH1 UV-wavelength to false-color chart made in 2011 until we all realized how many factors stand in the way of applying it accurately. Then there is that pesky non-invertibility problem. Then Klaus got mad at us for saying all that. And Klaus also got mad at us for asking him to provide details about his methods. We got mad at Klaus for keeping secrets. Crankiness ensued on all sides about what is and is not Proper Science and who is and is not a Proper Scientist. <andrea here heaves a huge sigh>

All that revisiting of the past reminds me to HEARTILY and SINCERELY THANK YOU JIM and JONATHAN for your openess in sharing your methods and results. You don't know how refreshing it is to see that.

There is a 4-page discussion on UVP where all this is discussed and re-discussed.
http://www.ultraviol...age__hl__sascha

That linked topic contains another Panasonic Lumix G3 color map posted by Sascha. He and Klaus got mad at each other over the Lumix maps they each had made. Sascha thought the maps weren't predictive. Especially because Klaus would not say how he made his map. Then Sascha got mad at us for acting like we were a "scientific peer reviewed panel" rather than the UV-enthusiast website we are. Of course everyone here having a Ph.D. (a lot!!) then got mad at Sascha. [[[Added: OK, I have exaggerated a little bit here. :D But not by much.]]]

So this entire topic involving UV-wavelength to False-colour mapping has historically generated large amounts of anger, rancor and dramatic exits from websites. Does anyone really wonder why I am not eager to put all this history in a Sticky somewhere?

Let me say formally: at this point neither I nor Bjørn Birna have been convinced of either the feasibility or the utility of UV-wavelength to false-colour mappings so we will not support a Sticky on the subject at this time. That can certainly change in the future. And this does not mean we are trying to discourage any efforts anywhere to work on this. Problems like this should be tackled to see what comes of it.
Andrea G. Blum
Often found hanging out with flowers & bees.