What is the real response of Panasonic Lumix G3? False colors vs. wavelengths. Spectroscopy.

[Andrea B.]

So a Spectrometer can be turned into a Spectro-Reflectometer by using a UV grating and a good UV-Pass filter on the input fiber to block all visible/infrared light.

 

"Would that be a UV single-grating or a UV double-grating?", she asked.

[JCDowdy]

Sure Col,

 

I am not the greatest Ocean Optics fan but they have a wealth of information you which is also relevant to your spectrometer. Here are the basics:

1) Example Setups (click on UV-VIS REFLECTION)

2) Reflectance & Transmittance Measurement Techniques is a more detailed page with lots of info.

3) They have a video and also on

demonstrating the apparatus.

 

The minimum you will need is a bifurcated fiber optic reflectance probe. This is a multicore bundle which carries light to the target on some fibers and then carries the reflected light to the spectrometer on another fiber and you may also want a probe holder of some type. The NA of the fiber should be compatible with your spectrometer since on many inexpensive spectrometers the diameter and NA of the fiber serves as the entrance slit. Be sure to get a UV grade fiber or you wont get very deep into the UV. If you have deep pockets you might graduate up to some type of reflectance sphere to attach to the fiber, like this or this,

 

You will obviously need some sort of light source. OO has a couple of options depending on the the application. It needs to be very stable and the ability to add a filter may be helpful. The coupling must of course accept the fiber. OO rates their Tungsten halogen light source for 360-2400 nm and their Deuterium-tungsten halogen light source for ~215-2500 nm. I think there is also a Xenon source. OO makes a variety of them and of course are not the only supplier.

 

You will also need a diffuse reflectance standard, see also Labsphere and Avian. You likely already have a Spectralon std for UV white balance which will suffice. Unless you are going to do some serious professional work save some money and just get the un-calibrated one.

[JCDowdy]

So a Spectrometer can be turned into a Spectro-Reflectometer by using a UV grating and a good UV-Pass filter on the input fiber to block all visible/infrared light.

 

"Would that be a UV single-grating or a UV double-grating?", she asked.

 

What makes a spectrometer into a spectroreflectometer is the input optics to collect the reflected light. Measuring reflected UV is just like measuring emitted or transmitted UV, you must have UV gratings and optics and detectors. The quality, precision and dynamic range is dependent on the type of monochromator, with fixed grating array type being the least, single grating scanning type intermediate and double grating scanning type the best. Price, weight and portability of course run in the opposite direction.

[Shane]

Do not underestimate the performance of the USB2000.

 

In the lab I used several UV-Vis & UV-Vis-NIR (Hitachi, Perkin Elmer, Pye Unicam etc) spectrometers ranging from $50K to $100K to measure certain gem materials. However, I needed to build a portable system for air travel to measure the spectrum of gems at the source location. Gems are difficult to characterize correctly, especially diamonds, as they are cut to maximize light return. The system I eventually pieced together cost $7K, completed the task in 15 secs vs 5 minutes and in many cases provided better spectral quality with regards to polarization artifacts, detector changes etc.

 

In a spin off project related measuring gem materials at cryogenic temperatures, the "lab" equipment was $50K plus a $10K cryogenic unit and took 30 minutes to complete the characterization. I undertook a similar approach for a small, cheaper, portable system including designing a $5 !! cryogenic unit prototype (patented) configured with OO USB detector and reduced the data collection time to 10secs at a cost of ~$8K. In many cases the system provided cleaner data than the lab system could and more importantly with reduced speed and cheaper cost, significantly improved the testing throughput.

 

Obviously the CCD detector has some issues and limitations but so do most monochromator/photometer systems. Its a matter of whether those limitations can be utilized, reduced, eliminated, compensated for, or are just plain not significant to the the analytical test.

 

One of the biggest limitations I found using these detectors was actually the light sources, especially the mini deuterium sources. I tried several different makes and models and found that most of these mini deuterium sources just don't really have what it takes in most cases. Pulsed xenon sources were OK but I didn't like them as much as halogen/D2 sources as they seemed much more unstable.

 

Obviously, I'm not saying the USB CCD detector can confidently replace all expensive lab monochromator/photometer systems but there is a place for it and it is not just in the field or as a survey system.

 

It is possible to pick these USB detectors up as used items, e.g. ebay, however most ebay postings just provide the general parameter of wavelength range which is insufficient. You need to know how that wavelength range is configured with regards to grating and slit width (resolution, peak wavelength performance etc). For UV you need to know if it is optimized for UV by utilizing detector enhancements that are typically not standard configuration but add-on options at the time of purchase. So it is buyer beware.

 

I used an ocean optics integration sphere between the monochromator ............. This appears to be in accordance with the list provided by Shane Elen.

 

I did specify an integrating sphere however I did not specify the OO integrating sphere. In fact I found the OO IS quite poorly designed due to lack of internal baffling. A similar sized but better designed sphere is made by Avantes. However, both these spheres are really too small for the characterization of sensors.

[Shane]

The spectrum on this page was collected using a USB CCD spectrometer in conjunction with a reflectance probe.

http://www.beyondvisible.com/BV3-IRcontamination.html

I published it but was never really happy with actual spectrum itself because using a reflectance probe on this kind of subject can be tricky due to alignment, stray light etc. It can be done but much better spectra can be obtained using an integrating sphere which may require destroying the flower in some cases. Then you have the issue of contaminating the sphere with pollen etc.

 

The spectra on this page

http://www.beyondvisible.com/BV1-UVIRLum2006GRC.html

was collected using a $50K UV-Vis-NIR spectrophotometer system. These benchtop or free standing systems do have the advantage of being self-contained thereby eliminating stray light and sample positioning issues (but it is difficult to put in your carry-on luggage!).

[Andrea B.]

Shane, thanks for adding some more good info to this topic. It is cool also to hear about your gem work a little bit.

 

This is all very fascinating to me. Hopefully to others as well.

 

I am attempting to make a list of variables to control (see Post #2) and equipment (see Post #21) to use for "solving" the UV Wavelength to False Colour Correlation problem. Also a list of questions which must be answered - chief amongst those questions being the matter of how to set camera white balance when measuring sensor response.

 

BTW, like Dave, I would also love to be able to get some kind of UV reflected wavelength measurement on a flower while 'in the field'.

I understand that it would be indicative only and not rigourous when not done in the lab. But it would still be interesting.

[Shane]

If you are measuring sensor response you need to measure "sensor response" i.e. raw data before WB has been implemented.

 

I think you are really interested in "measuring" image system response ie. the combined effect of filter, lens, sensor response, custom WB, demosaicing, interpolation and colour space implementation. In this case your results are specific to that combination only. If under those conditions, you want to relate the ratio of R,G,B to wavelengths then you will need to come up with a spectral response chart and if you want to relate them to "false colours" then you would have to produce a very complex algorithm to provide results for variable ratios of UV wavelengths.

[Andrea B.]

Well, duh, Andrea, duh !!! :P

 

It just occurred to me how to handle the problem of in-camera white balance when attempting to produce false colour charts.

I'll get back to you after a couple of days experiment.

I also have to buy some software.

It's so obvious I'm sure it will occur to someone else.

 

Or I am so wrong that I will return here and have to strike out this post. :D B) :)

I am not afraid whatsover of being wrong. It is how we learn and advance factual knowledge.

 

Stay tuned.

 

Added 2015.01.13 5:03 PM GMT-5: See Shane's next post. I'm also going to make direct use of Dcraw.

[Shane]

In the past I used DCRaw and Rawnalyze combined with statistical analysis of a small central region from each channel but now RawDigger should provide you the ratios and statistics you require.

[enricosavazzi]

I missed the start of this thread and tomorrow will be off traveling until the weekend, so I won't be able to follow up or reply for a few more days. I have been aware of this specific work by Sascha for a couple of months and have discussed aspects of it privately with him well before this thread started. Several aspects are difficult and may still be open to personal interpretation, as long as we do not have access to "perfect" lab equipment (which may not exist).

 

What we do have for sure is that the G3 records amounts of UV useful for imaging down to about 300 nm. I agree with Sascha that, by using narrow-band UV-pass filters, we do record the response of the firmware/sensor/lens/filter system. It does not really matter to me whether we are recording the direct conversion of UV photons to electrons in the photosite junctions, or indirectly through an intermediate conversion to longer wavelengths by fluorescence of parts of the sensor. What passes through the filter is UV with a relatively narrow, approximately gaussian distribution around the transmission peak of the filter, and the image is a recording of this transmitted radiation (biased by a wavelength-dependent sensitivity function of the camera).

 

I have played in the past with false color produced in UV images by this and other cameras. My impression is that there is a definite (albeit certainly complex) relationship between false color and the predominant UV wavelength. A given camera with a given UV source and white balance always produces false-color green with a 340 nm narrow-band filter and a subject that reflects this wavelength, for instance. I don't expect to wake up tomorrow and discover that the same combination of factors will suddenly produce a false-color red. Well, it has actually happened a couple of times with certain subjects, but there were always verifiable explanations, like the subject being so dark at 340 nm that a slight red and NIR leak through the filter became the predominant wavelength interval. Adding a red- and NIR-cut filter removed the red in the image, and that confirms the explanation in those particular cases.

 

On the other hand, we do know that WB and other camera and post-processing settings make it very difficult to standardize the false color and use it across individual cameras and camera models. My three full-spectrum cameras produce broadly different false-color UV palettes with custom WB chosen to give quite similar results when shooting through the same UV-cut, VIS-pass and IR-cut filter, same radiation source and same subject. Another difficulty, which has been mentioned I believe in other recent threads, is that none of our filters completely cut everything outside their transmission peak. Whenever we increase the exposure time, sensitivity and illumination strength to compensate for the fading image at shorter wavelengths, we increase the chance that some radiation leak, current leak or other bias will become strong enough to produce spurious results. All our precious filters probably transmit across the whole spectrum if we look at optical densities above 4 or 5.

 

It is of course possible to publish BW versions of images recorded with narrow-band UV pass filters and avoid all discussion about different interpretations of false color. This will not remove any actual image bias, but will hide the problem well enough that we will not have to keep discussing it, which I believe would lead us nowhere really fast.

[Andrea B.]

Enrico, thank you for these excellent observations.

I will be modifiying the list of variables, equipment and questions with everyone's input.

 

*************

 

Enrico wrote: I have played in the past with false color produced in UV images by this and other cameras. My impression is that there is a definite (albeit certainly complex) relationship between false color and the predominant UV wavelength.

 

I think we all agree - tentatively - on that there is a one-way connection. But there is not a 1-to-1 connection.

To finish the experiment, we must determine how to handle those mixed signals and figure out how to handle the various variables in a standardized fashion so that the experiment is always repeatable on a per-camera basis.

Even then using such a false colour chart will never be rigorous, only indicative.

 

*******

 

I am using "we" in a generic sense. I as an individual or as part of any team am not going to be the one to figure all this out because I do not have any equipment with which to do so. I simply have been interested in the colour chart problem from the get-go and hope to see it worked out someday.

[JCDowdy]

Someone other than me may recall more clearly, but I think KDS's UV color palette(s) that were linked earlier were made using his apochromatic CERCO f4.1 / 94mm quartz fluorite lens and a Baader-U filter on, if memory serves, a converted Panasonic G3. I also am pretty sure he white balanced that with his custom made PTFE reflection disk. I am not sure what his light source was but also seem to recall it was some type of UV enabled Xe flash. Obviously all these factors will influence the resultant color palette. I do not recall that the color palette(s) KDS posted were for the bare unbalanced sensor.

[Andrea B.]

Yes, we have mentioned that lens, filter, sensor, wb standard and illumination can affect production of false colour (or any colour). :)

 

(Klaus has removed his original bog post until he can refine it.)

 

***************

 

Spectral Color: http://en.wikipedia..../Spectral_color

I always list Wikipedia links with trepidation. But they at least serve as an introduction to a subject which can be followed up in a proper reference.

 

The problem of Non-Spectral Colours must also be dealt with for a false colour chart to be useful.

 

When we make all these UV photographs and "standardize" them,

how nearly do they match actual wavelengths given that there are massive amounts of non-spectral colour.

Actually, there is almost totally non-spectral colour in most "standardized" UV images.

Even the blues we get (blue being spectral) are never the blue of any (0,0,255) representation

which might be mapped to the blue wavelength.

 

How can a Spectral Colour best be represented in a Colour Space?

 

Choices of Color Spaces (via ICC profiles) plays a huge role.

For example, sRGB green is non-spectral.

But wide-gamut RGB green is almost spectral.

Apparently NO commercial colour space is considered spectral.

Who knew ??!!?? Well, not me, but some of you all might have known this stuff.

 

There is a formula out there somewhere for converting wavelength to RGB.

Could we simply use that in some way to produce a colour chart??

I have it linked somewhere and will add it when I find it.

 

When comparing a UV photograph to a false colour chart, you also have to account for the native white point of the monitor illuminant.

 

It just gets more and more complicated - all the things that can affect the colors we see in photographs and charts.

[Andrea B.]

This is so pretty. It is just a little side-trip in this long adventurous thread.

CREDIT: Photo Research, Inc.

LINK: http://hyperphysics....cie1976.html#c1

 

Shouldn't they call that "violet" instead of "purple"?

 

1976 CIE Chromacity Diagram

http://hyperphysics.phy-astr.gsu.edu/hbase/vision/vispic/cie1976b.jpg

[Andrea B.]

One last item from the Hyperphysics site: http://hyperphysics....ion/spd.html#c1

 

There they re-state the central objection - in a slightly different context - which some of us have to using a False Colour to determine a UV Wavelength. The bolding is mine.

 

[excerpt from above link]

 

From the Spectral Power Distribution (wavelength, wavelength power)

both the luminance and the chromaticity of a color may be derived to precisely describe the color in the CIE system.

Other systems of color measurement can also be related to the SPD.

 

These systems have been successful in predicting color perception from the SPD,

but it is not possible to proceed in the opposite direction.

That is, the Spectral Power Distribution (wavelength, wavelength power)

cannot be predicted from the characteristics of the color as perceived by the human eye.

 

[/excerpt]

 

 

Here I would add: ....nor as perceived by the digital camera.

 

And I would also add: We already knew this. We've already all mentioned this several times. The question that could be answered is whether there a useful work-around that will work well enough to indicate what UV wavelengths a flower might be reflecting subject to confirmation with actual measurements.

 

Guys, ya can't argue with the physicists. So do proceed with caution for those of you who still wish to work more on the False Colour to UV Wavelength correlation.

 

I'll keep tabs on the variables and equipment and questions to be answered along the way.

Some of the questions raised are of interest to some other aspects of our UV photography.

[JCDowdy]

Do not underestimate the performance of the USB2000.

 

There are applications where portability and speed of acquisition are enabling factors. Your experience proves that some applications have less stringent requirements for stray light rejection and limited single measurement dynamic range. For a measuring spectra where there is plenty of energy and not a lot of spectral intensity range these instruments perform adequately. If all one needs is a line spectrum, like an astronomer, even better. Indeed, fixed grating array spectrometers have wide acceptance in LED measurement where there is not much out of band energy to worry about.

 

What I can tell you is that in my experience a USB2000 cannot reliably measure a sharp spectral cut off. Consequently it is unable, for example, to measure the out of band blocking of a bandpass filter lower than an OD less than ~2.5-3 before it hits baseline noise irregardless of input energy. The stray radiation will also cause the cut-off of such a filter to appear less sharp, rendering an incorrect slope for the cutoff.

 

We are still using an older software version for our USB2000 because the upgrade removed the ability to display a spectrum with a log y-axis scale. I suppose that is a good analogy for where I draw the line on usability. If an application can be satisfied with spectra plotted on a linear scale the instrument may acceptable. If I need to be able to define a sharp cutoff to a high OD then a fixed grating array spectrometer will simply not suffice. If in need to apply a spectral weighting function that drops 3+ orders of magnitude in the opposite direction from a high OD cutoff in the UV then my USB2000 is not even considered.

 

Here is a somewhat outdated white paper on the subject that explains in greater detail how these limitations impact what I do. The compact spectrometers have improved since then but so have the scanning systems.

[Shane]

I agree with what you have said but wanted to make the point not to just discount the potential of the USB spectrometer in some lab applications.

Obviously the CCD detector has some issues and limitations but so do most monochromator/photometer systems. Its a matter of whether those limitations can be utilized, reduced, eliminated, compensated for, or are just plain not significant to the analytical test.

[JCDowdy]

I always list Wikipedia links with trepidation.

 

Well, since you took the path of trepidation see also: Metamerism (color)

 

It is counter intuitive that one can calculate color,CIE L*a*b*, from a reflectance spectrum but one cannot do the reverse. That said, intuition keeps nagging me something else is happening.

 

Aren't most of our UV cameras only really responding in two channels? When the occasional odd color pops out are we seeing a wobble out of the false-bichromic response into the edge of a false-trichromic?

 

No, I have not been drinking.........

[DaveO]

The people who wrote the words for the colours on the CIE diagram were obviously NOT photographers who had ever been in a darkroom in the good old days making colour prints from negatives. The whole photographic world knows that blue-green = cyan and red-blue = magenta (at least they couldn't call yellow anything else).

 

All schoolkids in my day knew that Richard Of York Gained Battles In Vain when looking at the rainbow (red, orange, yellow, green, blue, indigo, violet).

 

I never did know what indigo looked like!

 

Dave

[colinbm]

Thanks very much for the contributions on using a USB Spectrometer for reflected light spectral output & particularly the useful links.

Col

[colinbm]

Dave, Indigo is like a Navy blue.

As for purple & magenta, I can live without them.

Violet, our cameras & colour spaces just don't go there unfortunately ;)

Col

[Andrea B.]

Aren't most of our UV cameras only really responding in two channels?

 

No. My Raw Digger histograms show response in all three channels. Usually the R channel has more.

 

****

 

Speaking of drinking.....I still have some egg nog base left. Right now it sounds real good !!

[Andrea B.]

Richard Of York Gained Battles In Vain

 

For some reason we learned that as a guy's name: Roy G. Biv

I like yours better, Dave.

[DaveO]

This has to win the prize for the greatest ever diversion from the topic ;)

 

My "rainbow" referred to ancient English history and Richard III http://en.wikipedia.org/wiki/Richard_III_%28play%29

 

Dave

[Andrea B.]

Well we needed the diversion, methinks. ;)

 

***********

 

John writes: It is counter intuitive that one can calculate color, CIE L*a*b*, from a reflectance spectrum

but one cannot do the reverse.

 

We are not mapping between a chromacity chart and a wavelength.

We are trying to map between a camera colour record and a wavelength.

 

For example, when our camera records yellow, how do we know whether that yellow came from a mix of red & green wavelengths or from a wavelength-yellow?

(What Bayer dye would wavelength-yellow pass through anyway?)

 

I hope I have phrased that example correctly. As I try to always stress, I want to hear about any factual errors so I can correct them.

 

Added Later: I forgot that the transmission bands of the Bayer dyes "overlap". A yellow wavelength would pass mostly thru the red & green areas.

 

Added Later: I need to check whether our cameras are currently using RGB Bayer filters or yellow-cyan-magenta filters.

***********

 

As a reminder, Klaus has said that his actual spectrometric measurements on flowers supported the false colour chart. I know we do not have a formally published scientific paper about this so we could understand the instrumentation and methodology.

(I wish I knew more about flower pigmentation.)