Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintenance and read more Dng exif colordata #17695

Merged
merged 2 commits into from
Oct 21, 2024

Conversation

jenshannoschwalm
Copy link
Collaborator

DNG Exif: Use AnalogBalance, read more tags and maintenance

Now also read Exif.Image.AnalogBalance and Exif.Image.ForwardMatrix.
To allow easier checking of read exif data we initialize the matrixes with usable defaults instead of marking them as invalid and keep track about found data via gbooleans.
Avoid reading all dng tags if no version has been provided.

Name all matrixes as used in the DNG specs to avoid confusion.

For easy of maintenance all Illuminant specific data are kept in struct dt_dng_illuminant_data_t illuminant_data having struct dt_dng_illuminant_data_t per Illuminant.

  • temperature
  • human readable Illuminant name for better logs
  • the xy coordinate
  • the CA matrix to transform illuminant data to D65

Implemented use of Exif.Image.AnalogBalance if provided.

Added pretty extensive information about found tags and used matrixes via -d imageio.

This all requires re-read exif data - done via user request button or fresh import - so no break of old edits.

Note about currently unsupported dng features:

  1. According to DNG specs we would have to interpolate between illuminants if there is no given D65 illuminant and at least 2 provided illuminant temperatures are one below and one above D65.
  2. To improve quality the specs describe and favor the usage of ForwardMatrix if provided. There seems to be more maths involved (matrix diagonalisation ...).
  3. The specs describe how to handle DT_LS_Other illuminant.
  4. No support for 4 channel images

Camera DNG files with (1) or (3) are still pretty rare, for (2) there are at least files from sigma.

If we want to have dt progress in this field it would be very nice to get hold on such samples. Thus we check for these cases, report in log and ask the user for feedback via control log to have better chances to get hold on such images to investigate and test implementations.

Tag problematic DNG image files

As commented in commit cf9d9f6 there a dng files still problematic for darktable.
While reading exif data they now automatically are tagged with "darktable|github|dng_issue"
so they can be found easily by the user.

@jenshannoschwalm jenshannoschwalm added feature: enhancement current features to improve scope: image processing correcting pixels scope: codebase making darktable source code easier to manage labels Oct 20, 2024
@jenshannoschwalm jenshannoschwalm added this to the 5.0 milestone Oct 20, 2024
@jenshannoschwalm
Copy link
Collaborator Author

  1. Release note would be "DNG files make use of AnalogBalance information"
  2. Not sure i will be able to implement at least the ForwardMatrix correction for 5.0 so let's go with this for now. At least the code is more easy to read/use now so chances might be better other people stepping in ...

@kmilos
Copy link
Contributor

kmilos commented Oct 20, 2024

Nice! Too bad RPU is down so we can recheck existing samples...

One question, as I'm not too familiar w/ the WB nor CC internals:

I noticed the iPhone 16 DNGs (needs JPEG XL support in rawspeed though) have 1 1 1 for AsShotNeutral, and AnalogBalance that is not identity. (Maybe there are other devices doing this as well, but can't check RPU). Rawspeed will populate wb_coeffs as 1 then. Are there any side effects on WB and/or CC then?

@jenshannoschwalm
Copy link
Collaborator Author

I guess this pr would help for those images. Could you share one?

@kmilos
Copy link
Contributor

kmilos commented Oct 20, 2024

Can't remember where I found a sample, couldn've been here...

@TurboGit
Copy link
Member

Needs a rebase and conflict resolution after the big code cleanup.

@jenshannoschwalm jenshannoschwalm force-pushed the dng_exif_colordata branch 2 times, most recently from 47742c6 to bfeeb91 Compare October 21, 2024 06:14
@kmilos
Copy link
Contributor

kmilos commented Oct 21, 2024

@jenshannoschwalm That link doesn't seem to contain raws... I now remembered I got the one sample I have from here.

Copy link
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

src/common/exif.cc Outdated Show resolved Hide resolved
Now also read Exif.Image.AnalogBalance and Exif.Image.ForwardMatrix.
To allow easier checking of read exif data we initialize the matrixes with usable defaults
instead of marking them as invalid and keep track about found data via gbooleans.

Name all matrixes as used in the DNG specs to avoid confusion.

For easy of maintenance all Illuminant specific data are kept in
`struct dt_dng_illuminant_data_t illuminant_data`
having `struct dt_dng_illuminant_data_t` per Illuminant.
  - temperature
  - human readable Illuminant name for better logs
  - the xy coordinate
  - the CA matrix to transform illuminant data to D65

Implemented usage of Exif.Image.AnalogBalance if provided.

Added pretty extensive information about found tags and used matrixes via -d imageio.

This all requires re-read exif data - done via user request button or fresh import - so no break of old edits.

Note about currently unsupported dng features:
1. According to DNG specs we would have to interpolate between illuminants if there is no given
   D65 illuminant and at least 2 provided illuminant temperatures are one below and one above D65.
2. To improve quality the specs describe and favor the usage of ForwardMatrix if provided.
   There seems to be more maths involved (matrix diagonalisation ...).
3. The specs describe how to handle DT_LS_Other illuminant.
4. No support for 4 channel images

Camera DNG files with (1) or (3) are still pretty rare, for (2) there are at least files from sigma.

If we want to have dt progress in this field it would be very nice to get hold on such samples.
Thus we check for these cases, report in log and ask the user for feedback via control log to have
better chances to get hold on such images to investigate and test implementations.

S S

adfb
As commented in commit cf9d9f6 there a dng files still problematic for darktable.
While reading exif data they now automatically are tagged with "darktable|issue|no-samples"
so they can be found easily by the user.
Copy link
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@TurboGit TurboGit merged commit a10aab3 into darktable-org:master Oct 21, 2024
6 checks passed
@jenshannoschwalm jenshannoschwalm deleted the dng_exif_colordata branch October 21, 2024 17:10
@TurboGit
Copy link
Member

@jenshannoschwalm : Can you draft a release note entry please? TIA.

@jenshannoschwalm
Copy link
Collaborator Author

Release note:

  1. Added support for exif tags 'AnalogBalance' used for color calibration and LinearResponseLimit used in highlights reconstruction.
  2. If we find currently unsupported color calibration data in DNG specific tags we tag the image by darktable|issue|no-samples for better support,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature: enhancement current features to improve scope: codebase making darktable source code easier to manage scope: image processing correcting pixels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants