What color profile to use?

[Fandyus]

I mostly process my images in Darktable. Darktable has many input color profiles to pick from.

I am mostly asking, since I noticed a peculiar problem with the standard color matrix and I don't really get why it has to happen. Basically, standard color matrix seems to make areas of certain channels completely black sometimes to achieve saturation.

Normal photo with standard color matrix:

Blue channel isolated with the color calibration tab:

 

 

This is very annoying to me, since it is data loss/distortion. I have lately been splitting images into channels, subtracting channels from one another etc. I figure that there is no way the blue photosites on the sensor actually captured an image this dark and contrasty.

If I switch to Lab or linear Rec709/Rec2020, I don't encounter this problem anymore, but the problem is that for some reason, the colors shift. Yellow becomes orange, blue becomes cyan, etc.

 

Again, isolated blue channel:

 

My theory here is that Lab actually shows you how the sensor itself sees the world, and image processing programs are just designed to automatically shift the colors back to where they should be and clip certain channels (such as with the dandelion) to achieve saturation akin to that you see in real life. But I am not sure. What I know is that I would like to see my data represented as objectively as possible.

Here's another example. I got myself a a diffraction grating from AliExpress yesterday, it has 600 lines/mm, actually works very well, even though I've been told here it's not enough.

I attached my Rowi GO-2 orange filter on the lens, color balanced on PTFE and then tilted the camera away from the lightsource so that the pattern would show. I used a halogen bulb.

Standard color matrix full color and blue channel isolated:

If you look carefully, you notice that towards the shortwave end, there is a completely black spot. It makes zero sense for it to be there optically.

Lab:

Lab has no such thing, and also the blue channel seems to be working as some sort of bizarre triple bandpass (any explenation anyone?), which might be why my single shot aerochrome attempts have been failing.

 

Looking at this, I am very annoyed that I don't have Rawdigger and can't really afford it. I am even more annoyed at the fact that something as simple as OBJECTIVELY splitting a raw file into the four channels (red, blue, green #1 and #2) without any color profiles or demosaicing is not available in any free software. It would be priceless to be able to do that so that I could analyse what's going on with my images. This way I am not even sure if the triple bandpass effect is not just some artifact of the software combining the channels together to achieve some visual standard. Makes me wonder how well do cameras really see.

Here's the RAW file if anyone wanted to investigate it themselves.

DSC03329_halogen_rowiGO2.ARW

[photoni]

@Fandyus the sRGB profile is the most used because the majority of monitors "see" only that limited part of the colours
Better to use a wider color palette like AdobeRGB (if you have a good Eizo) or DCI-P3 (if you have a good recent Macbook Pro)
Use what your PC sees ... LINK

 

P.S. I ONLY use AdobeRGB
All PCs reason in LAB but do not see all colors
I believe Lightroom displays RAWs in ProPhoto (which no monitor sees), but images, Jpeg, Tiff, ecc are saved with "visible" profiles

[Fandyus]

@photoni

sRGB and AdobeRGB both make the image very underexposed.

 

sRGB

 

AdobeRGB

[photoni]

2 hours ago, Fandyus said:

DSC03329_halogen_rowiGO2.ARW 23.47 MB · 1 download

I downloaded the RAW, it's a useless file, it doesn't have a reference for the white balance.
I think you need to study a lot of sensitometry, you're making a big mess

.
What you see on the monitor is a translation of LAB information.
If you have a bad monitor you only see part of the information.
How can you process and evaluate what you don't see?

[Andrea B.]

hold on... can't you get channels from Darktable? I'll go check.

[Fedia]

Very interesting post @Fandyus !

 

Color profile are really obscure. I'm surprised that when using RAW files there even is an input color profile. I thought Raw files weren't really images at this point... I would have thought in RAW you just had to set an output color profile.

 

2 hours ago, Fandyus said:

I am mostly asking, since I noticed a peculiar problem with the standard color matrix and I don't really get why it has to happen. Basically, standard color matrix seems to make areas of certain channels completely black sometimes to achieve saturation

I remember @Andrea say somewhere that in order to have a fully saturated color you had to have at least one of the RGB values to zero. This of course doesn't ever happen when reading what the sensor outputs, only in the display.

 

The standard color matrix obvioulsy doesn't represent what the sensor's channels output. But nothing proves that the lab profile does it either. The Lab profile displays color in a less saturated way, it's close to a log profile in video, and this absence of strong saturation is enough to explain that the monochrome image of the blue channel in lab profile looks something like the original Blue channel of the Raw file. But it doesn't mean it is !

[photoni]

OMG
I leave you
Raw means... from raw material, i.e. without color profile, without color temperature.
Different programs extract different information (numbers).
How many "numbers" does your monitor see?

Raws are a soft processing of the camera which does not indicate the pure parameters of each pixel but mixes them with neighboring ones so as not to have a moiré effect

[Fedia]

13 minutes ago, photoni said:

it doesn't have a reference for the white balance

Fandyus said he white balanced the image before taking the photo (that's why the infrared tail is blue). He couldn't have the grating on his lens and a white target in the frame at the same time. The white balance coordinates are in the metadata. Why would a would a reference be so important ?

[Fedia]

I really like this picture becuse it shows how digital photography relies on color substraction beetween the channels to produce saturated colors and while maintaining soft contrast.

 

 

Here we can see clearly the green information that was substracted from the blue channel.

 

So in digital photography, when a strong color is present in one channel, it's very probable that you will find a sort of "negative ghost" in the other channels

 

This process can't happen in film, can it ? I suppose to saturate an image with film you have to raise the contrast, or use very strong dyes.

 

Also the more I look to this line the more I start to see a comet in space. It's maybe time that I stop looking at it.

 

I understand your frustration not to be able to just display R, G and B, each one in its respective channel without any interaction between them.

[Fedia]

Maybe these to brighter areas in the line from the Lab image make sense

 

 

They could come from here :

 

[Fandyus]

2 hours ago, photoni said:

OMG
I leave you
Raw means... from raw material, i.e. without color profile, without color temperature.
Different programs extract different information (numbers).
How many "numbers" does your monitor see?

Raws are a soft processing of the camera which does not indicate the pure parameters of each pixel but mixes them with neighboring ones so as not to have a moiré effect

I'm sorry but I have no idea what you're trying to say. It doesn't matter what my monitor sees, I could have the worst monitor in the world. What I care about is the way the images come out, and like Fedia just said, the apparent inability to have a program process my image without letting the R, G and B channels interact with each other.

 

Edit: @photoniI didn't see your comment earlier where you said I'm making a mess. I'm not sure if that's something lost in translation but I don't think that's a very polite way to express yourself. And as Fedia said, the file is pre color balanced. Even if it wasn't, you could just turn the colour balance off and process the data knowing there was an orange filter on the lens at the time of the image being taken. Please read my post properly next time before you accuse me of making a mess. Thanks!

[Fandyus]

@Fediayes, colour subtraction seems to be it. Thanks for the insight. I don't think this process can happen in film, and that might be one of the reasons why colour film has a sought after look. Because apparently no normal program lets you disable this weird colour maths that goes on for some reason. It really is frustrating, I like to be in control of every layer of processing applied on my data, but apparently I can't have this one.

Thanks for posting that graph also, explains a lot. I wonder why the small blue peak at 650. One would think that since that's pure red, it would be benefitial to let the camera record it as so.

 

Does anyone have any idea if there are perhaps any colour profiles I could download that when applied would let me minimize the amount of processing the software in question does? Or if I could make one myself. Or maybe if any of the profiles I have available is like that. I haven't noticed any "nagative ghosting" with the linear Rec profiles and the Lab profile yet.

[Andrea B.]

I am even more annoyed at the fact that something as simple as OBJECTIVELY splitting a raw file into the four channels (red, blue, green #1 and #2) without any color profiles or demosaicing is not available in any free software.

 

Fandy, you already had the correct procedure for obtaining channels in Darktable. I'm going to write it up here for future reference for other viewers.

 

Channels in Darktable

EDITED Fri Aug 25 2023

 

Initialize

Input Profile:  Set it to "standard color matrix" so that Darktable applies a camera model specific profile to translate the camera's raw colors into the working space colors used by Darktable's tools.

Working Profile: Set it to Linear Rec2020 RGB as recommended by the Darktable developers. You can use Adobe RGB here if you like.

Output Profile:  Set it to either aRGB if printing or sRGB if photo will be displayed on website.

Exposure Tool:  OFF. You don't want Darktable applying any of its auto corrections to exposure when splitting channels.

 

For RAW RGB channels, White Balance Tool:  OFF.

For As Shot RGB channels, White Balance Tool: ON. Set it to Camera.

 

Go to the Color Calibration Module.

Right click on the Color Calibration bar and select basic channel mixer.

Click gray (on the right).

 

For the red channel, move the input red slider all the way to the right until value 1.0 is attained.

Right click on the Color Calibration bar and select store as preset. 

Give the preset a name like Red Channel.

 

Reset the Color Calibration Module by clicking the Reset Parameters button.

 

Repeat the procedure to create Green Channel and Blue Channel presets.

 

Now when you go to the Color Calibration Module, you can right click on the bar

to select those channel presets.

 

*****

 

I'm going to run your file through Raw Digger to verify that the Darktable channels as described above, are valid.  There might be some minor differences in contrast, etc.  That is because photo apps may demosaic slightly differently from one another in the areas of curve and gamma corrections. 

 

 

 

[Andrea B.]

Yes, the channels from Darktable are OK.

Here are some screen shots. The top is the channel from Raw Digger, the bottom is from Darktable.

(Please do not make any contrast/lightness/darkness conclusions from screen shots!)

 

Red Channel

 

 

Green Channel

 

 

Blue Channel

[Andrea B.]

I have something else to tell you about that Input Profile in Darktable, but I have to look some stuff up first.

[Fandyus]

Thank you very much for your assistance @Andrea B., It's very kind of you and much appreciated. I have been using the method of splitting channels like this for a long time, it's how I made many of my multispectral images, but just recently I have realized the weird channel subtracting that seems to be going on with certain input colour profiles and it's been driving me crazy, I'd like to actually figure out how this works.

[Andrea B.]

I'm writing it up now. But it will take a while.

[Andrea B.]

Sorry I couldn't get this posted sooner. Dinner intervened. Then a couple of phone calls.

 

I worked through this with a yellow flower of my own to see if I could determine what was happening in Fandyus' photo above.

 

First of all, you definitely want the Darktable Input Profile to be standard color matrix so that a camera specific translation is made between your camera's raw color space and Darktable's working color space. When you switched to Lab or Linear Rec as an input profile, then of course the colors changed. Those profiles don't know anything at all about your Sony a6000.

 

For a working profile, I agree (with Darktable) that the Darktable default Linear Rec2020 RGB is a wise choice. It has a slightly smaller gamut than the favorite of recent years, ProPhoto RGB. But the colors in Linear Rec2020 RGB are based on human vision. There aren't any imaginary colors and fewer un-displayable colors. That is better, I think, when you later assign an output aRGB profile (for printing) or sRGB profile (for website display). There are fewer colors in that Linear Rec working space which must be "re-rendered" to fit in either of those smaller gamuts.

 

Finally, I set gamut clipping to Off. (I don't use Darktable often enough to decide how that setting affects output.)

 

Reference Photo:  Here's my yellow flower (visible light). This is a crop.

 

Raw Digger:  raw blue channel. No edits.

 

Darktable:  as shot blue channel. No edits.

 

 

EDIT:  26 Aug 2023. I should not have tried to make a comparison between the two blue channels because  the blue channel from Raw Digger was for raw data and the blue channel from Darktable was from the "As Shot" mode, i.e. using white balance.

There is a difference in contrast. The 2nd Darktable blue channel is more contrasty. I don't think this should bother anyone? There is no universal definition of how much the raw data should be "stretched" and "curved" during conversion to turn the raw data into something resembling what would be seen by human vision. As one example, I've noticed that Raw Digger applies contrast (and saturation, where applicable) with a very light hand as compared to similar output from Photo Ninja. 

HOWEVER, neither of my blue channel examples shows the extreme darkness in the blue channel that Fandyus has shown above. So what the heck is going on?

 

I did discover one interesting thing.

When I first loaded the raw file of the yellow flower into Darktable, I noticed that Exposure (mode automatic), and White Balance (as shot) were turned on. Are those setting automatically turned on when you load a file? I must investigate that. In other words, does Darktable do some auto-adjustments when you load a raw file? I've seen this kind of minor auto-adjustment in other apps when you load a photo. It seems to be an attempt to present a "nice" view of the photo before the editing begins.

 

So, I wondered, what would the blue channel look like if I turned off the Exposure & WB tools?

 

Darktable:  blue channel with Exposure OFF and White Balance OFF

EDIT:  If White Balance is OFF, then this should be a raw blue channel. But it is too dark for that.

This looks much more like what Fandyus was getting above for the blue channel of a yellow flower. Way too dark, yes!!

 

 

Sooooo, next step.....

Investigate the initialization of a photo in Darktable.

Did I set something somewhere which caused Exposure (Automatic) and White Balance (as shot) to be initially ON?

In Photo Ninja (which I use regularly), the Exposure tool makes an initial highlight adjustment and applies a contrast curve setting chosen from As Shot, Unadjusted or Absolute. Does Darktable also do this? (Looking forward, the answer is YES.)

 

Time to dig further into either the Darktable manual or some Darktable discussions.

 

Fandyus, try extracting the blue channel of your yellow flower with Exposure turned ON to mode Automatic and White Balance turned ON to As Shot. Does this produce a lighter version of the blue channel?

 

We've almost got this figured out!!

EDIT:  Ha-ha, was I ever wrong about that.

[Andrea B.]

Here is the Raw Digger raw composite of the diffraction grating. (FWIW, . I'm always curious about this stuff.) This represents the colors after demosaicing but before white balance has been applied.

 

In Raw Digger I selected along the color band. (Translucent white rectangle represents selection area.) And then created a raw histogram for that selection.

 

 

[Andrea B.]

Fedia writes:  Color profile are really obscure. I'm surprised that when using RAW files there even is an input color profile. I thought Raw files weren't really images at this point... I would have thought in RAW you just had to set an output color profile.

 

Please ignore the following if you already know it, OK? I add these things sometimes so that other readers who may not know as much can learn.

 

A color profile translates between two different color spaces. 

 

The camera has a raw color space. [[....which is not sRGB or aRGB....]]

The photo converter app (like Darktable or Lightroom) has a working color space.

The Input Profile translates the camera's raw colors to the colors of the working color space.

The Input Profile can also be referred to as a Camera Profile.

Camera Profiles are specific to brand and model.

 

Most photo converter apps do not force a choice of Input Profile as does Darktable. In most converters, a specific camera color profile is automatically applied when a raw file is loaded. In Darktable if you select standard color matrix as the Input Profile, then such a specific camera color profile is automatically applied.

 

I should digress for a moment here to say that not every converter app uses the same profile for a particular camera model. You would think that this might be a good idea. But the photo software business has their own ideas about how to make your photos look "good". 

 

The photo converter app (like Darktable or Lightroom) has a working color space.

The computer/tablet/phone monitor has a monitor color space.

The Output Profile translates the converter app's working colors to the colors of the monitor.

Typically we select sRGB output profiling for photos which will be displayed on a website.

 

Similarly, the printer has a printer color space.

The Output Profile translates the converter app's working colors to the colors of the printer.

Usually we select Adobe RGB output profiling for printing photos.

 

You can use Adobe RGB for website photos, but it is not yet universally recognized. sRGB is still considered the safest for website display. (I am not taking a stance on that, I'm just reporting. The world is a big place and not everyone has aRGB capable monitors.)

 

SUMMARY:  

camera's raw color space ->

camera (input) profile ->

converter's working color space ->

output profile ->

output devices's color space

 

[photoni]

.

.

.

 

[photoni]

.

.

.

 

[Fandyus]

8 hours ago, Andrea B. said:

Fandyus, try extracting the blue channel of your yellow flower with Exposure turned ON to mode Automatic and White Balance turned ON to As Shot. Does this produce a lighter version of the blue channel?

No issues with the dinner intervention, please do take as much time as you need.

I might try this evening but that is unlikely to do anything. The image I used as an example was well exposed and well color balanced to begin with (I used "as shot" for color balance as far as I know anyway). Plus the dark image you got looks simply underexposed (since the blue channel got tuned all the way down when white balance was switched off). It doesn't have the black "dead zone" that I got. You could confirm this by going back to your image and turning up the exposure. If you bring up the image enough so that you can see the flower without any weird artifacts, then it's just underexposure. If you get a black spot that stays there prettymuch no matter what, you got the same as I did.

[Fandyus]

I thought it might be a good idea to upload the raw file for the dandelion as well, in case anyone wanted to try replicating my results.

IMG_5092.CR2

[Fandyus]

5 hours ago, photoni said:

I do not know what you mean

In the comment I posted, I posted two other images.

One image uses sRGB, the other AdobeRGB. I simply switched to these profiles without making any other adjustments. The images are what I got, they are very dark, as you can surely tell.