KDE and Colour Management

Colour Management has a long way to come to the Linux desktop. Like on other computing environments first came single applications like Scribus, CinePaint or Krita and proved colour management be useful and mature. Now the open source Desktop stacks are following. Most advanced and wide spread inside colour managed applications is colour correction for monitors.

For KDE exists the KolorManager systemsettings panel for configuring a ICC profile per monitor devices. It´s Oyranos CMS cares as well about setup of standard Xorg _ICC_PROFILE(_xxx) root window atoms and video card graphics table (VCGT). The later is a one dimensional per channel colour correction feature. So expect no magic all included solution by this means.

The heart of today most wide spread colour management systems is the ICC file specification. And to get ICC style Colour Management working inside KDE, the framework has to handle several needs.

One is to provide a overall desktop colour correction, which applies to the whole monitor screen area. A Desktop Colour Server will improve the overall Desktop usability and appearance in heterogeneous  environments like Linux. Obviously colour measurement devices and early colour correcting applications need native access to the monitor colour space. The currently only full featured and backward compatible implementation of that scheme for Linux is the CompICC plugin to Compiz. The CompICC colour server assumes sRGB for all input. Output target is the actual monitor profile set e.g. by KolorManager. More details can be read int a older blog post. Given that colour servers are still pretty rare, only osX and CompICC provide that, it might not be a good goal for the foreseeable future to let all applications rely solely on that feature for their colour corrections. But it has the potential to the most slick and flexible desktop colour experience.

On the Input side ICC profiles and according Exif meta data need to be detected from the input streams and get appropriately assigned as ICC meta data to graphics objects. That can be application specific, but is as a very common task, which will be better implemented on the toolkit or framework level. The display component should then make use of this ICC meta data and do on the fly colour correction. As well file output needs support of embedding the ICC profile data back into the data stream.

Alongside the ICC data handling come options and preference management, which is covered by the KolorManager GUI to the Oyranos API.

Colour Management in openSUSE-12.1

The Oyranos Colour Management System will be in the upcoming openSUSE-12.1 release. With the new library users can configure their ICC profiles and settings in one central place. It brings as well a set of command line tools like oyranos-policy for handling policy configuration files and oyranos-profiles for installation of ICC profiles. KDE users can install the KolorManager package. This Oyranos front end adds a system settings control panel for individual settings adaption. But most systems will run fine with Oyranos defaults.

KolorManager is a front end to Oyranos´ settings and device configuration. It can be found in KDE´s system settings panel.

The first tab in the Settings frame contains the policies. Different settings can be grouped into a policy for easier switching workflow configurations. By selecting a policy the Default Profiles and the Behaviour tabs are updated accordingly. If you prefer a very simple workflow with all images converted to sRGB by default, then the Office policy might be right for you. Other users should select one of the quality preserving policies or the inbuilt defaults.

The Devices frame contains all detected colour devices. The currently correctly working device class is monitor. For these devices the setup of the _ICC_PROFILE in Xorg works. Applications can use this information to actively colour correct from their blending or editing colour space to the actual monitor colour space. Be prepared, that at the moment some applications might not use the system profile from Xorg. In some cases that can be changed in the colour management preferences of affected applications.

Below is a list of colour management packages in openSUSE and what they are for.

  • Oyranos – the Colour Management System library and tools
  • KolorManager – front end to Oyranos
  • ICC Examin – for looking at the content of ICC profiles and their 3D gamut
  • libXcm – parse monitor information and handle X colour management protocols
  • xcm – command line tools for libXcm
  • qcmsevents – libXcm based system applet observes X colour management events
  • icc-profile-openicc and icc-profile-basiccolor-printing2009 – high quality ICC profiles
  • oyranos-monitor – monitor command line tool
  • oyranos-monitor-nvidia – fetches monitor information from nvidia proprietary driver
  • lcms2 – high quality ICC Colour Matching Module (CMM) and command line tools
  • Xcalib – command line to upload linear curves from ICC profiles into graphics cards

More packages like ArgyllCMS, it´s front end dispcalGUI, SampleICC and CompICC can be installed from opensuse buildservice.

OCIO 1.0.1 released

Jeremy Selan released a new version of the open source color management library OpenColorIO.

Changes:

  • simplification of ocio2icc / ociobakelut
  • bug fixes

About:
Based on development started in 2003, OCIO enables color transforms and image display to be handled in a consistent manner across multiple graphics applications. Unlike other color management solutions, OCIO is geared towards motion-picture post production, with an emphasis on visual effects and animation color pipelines.

Download:
Github

Google Summer of Code Mentors Summit 2011

Last week end from 21th to 23th October mentors meet in San Francisco / US to share our experiences in organising and mentoring students during Google Summer of Code 2011, talk about open source projects and of course get in contact with other teams and meet in person. The Mentors Summit was well organised and it was fun to be there. It was my first time in the south of North America. So chances where good to meet new people.

The VLC and FFMPEG people where interested in colour management. I tried to give them some idea, what colour managed applications need to know about media streams. It seems, that awareness rises about colour management in the open source video community, which is wonderful.

Two members of the Scribus team where present, Peter Linell and Malex. We could discuss quite some topics around distributions. It was great to meet them both.MountainView Google Summer of Code Mentors Summit 2011

The Open Source in Visual Effects was missed be me due to a swapping of rooms. Luckily Peter knew the OpenColorIO author and we could get in contact in a small four people meeting. We discussed Linux colour management, distribution and the relation of movie picture to ICC style colour management. Jeremy Selan pointed out the very difference of HDR scene referred imagery compared to the style of HDR which is typical in ICC workflows. I could convince Jeremy to look at the ICC floating point extension and it’s two open source implementations. He was interested and might work on implementing that in OCIO too. If that happens, it would be a great step toward joining movie with still graphics workflows. In the past the issues around round tripping and HDR handling had lead to the decision of movie studios to develop own colour management systems. OCIO supports already the export of colour correction tables to ICC profiles for use in photo and paint applications, which traditionally feature ICC workflows. Once the input side through ICC profiles is implemented in OCIO the same library could be used as a CMM inside ICC and movie workflows.

Thanks to Google and its OPSO team for sponsoring and organising a great event.SF_near_port_after_GSoC_summit

Colour Correction Concepts for Monitors

Colour management for the desktop is a long standing issue not only for Linux. The following text will concentrate on colour correction of monitors. This means the experience, when you switch your computer or handheld on and look on the screen.

Why colour correct the desktop?
People tend to compensate for quite a lot of different types of monitors. They are most often able to adapt to one full screen and see colours as they are intended to look like. That’s fine as long as they are concentrated on the visual event on that one display. But in this article we want to discuss how to compensate the monitor colours to our human visual needs in environments with various displays side by side, as is the typical situation for more and more people today. They take pictures with mobile phones or other digital cameras and look at them on laptops, tablets, picture frames, TV sets and printouts often side by side. Or we look at them over the internet and want to share the same visual impression with other people. Many people with uncorrected systems see quite a difference between various colour devices and try to figure out how to conveniently synchronise them. Colour management should help accomplish that task.

What is used today on Linux for colour improvements?
Different display devices feature different colour gamuts and colour appearance. We are especially sensible to the gray balance. This is a important property for our visual experience. Gray balance is since long time maintained by placing three linear correction curves into the graphics card. They are known as VCGT tag. Properly done, this helps in maintaining a equally stepped gradient from black to white with neutral shades in between. This calibration technic is fast and deployed since a long time by tools like xcalib. Calibration means in ICC terms, to setup a device driver in order to deliver a good and stable colour response. Strictly it is not part of the ICC specification. But for practical reasons it is usually embedded in ICC profiles. This basic calibration with linear curves is still in use on Windows, osX and X11 systems.

Acer Extensa 5630EZ laptop monitor

Different primary colours become more and more an issue through evolving LCD display technology. The above and below CIE shoe diagrams show quite different gamuts between the three primary colours red, green and blue of a typical laptop above and a wide gamut monitor below.

HP LP2481zx wide gamut monitor

It is generally desirable to use a screen, which is able to show deeply saturated colours. But uncorrected images show much shifted colour primaries to better cover human visible colours. Compared to traditional monitor primaries green typical becomes more cyan and red is moved toward the purple line. Of course the same RGB number triple looks quite different on booth devices. A calibration technic like VCGT linear per channel curves can not correct in any way for these saturation and hue shifts. Colours look strange not only on side by side comparisons of such devices. We need a colorimetric description of the monitor to properly colour correct the given device response.

Historically we have seen three approaches to cover the colour correction of the full screen.

  • X11 systems are colour management wise much like Windows. They predate ICC colour management and therefore have no historical APIs to enforce throughout colour correction. Maintainance of gray balance through VCGT is possible. The X11 XCMS extension was to our knowledge never in real use. Never APIs can bring a requirement for colour management. Few do today. Differences between corrected and non corrected content on one screen can be very irritating.
  • A very rigorous answer to the situation of missed colour management APIs is a fixed colour correction in hardware. But this can limit the capabilities of monitors to sRGB and thus the experience of wide gamut monitor technology not available. On the other side it provides a way to synchronise tightly controlled display devices.
  • On osX the whole rendering pipe line is colour managed. Each colour must have a colour space assigned to or it is not accepted by the systems graphic APIs. As a result all content from applications is converted on the fly by Apples ColorSync to the according monitor and looks correct. ColorSync takes over the work to bring colours from a source colour space like LStar-RGB to a native destination colour space like from a monitor. This helps programmers to concentrate on their actual task, which is most often not colour management related at all.

What is done so far to get the colour correction ball rolling?
In 2008 we started a project for OpenICC to do colour correction on the GPU on top of X11. The project allows to colour correct all non colour characterised desktop areas. Non colour characterised means by definition all window regions, except actively marked regions for opt out of system colour management. If no colour managed application is running, this means a full screen colour correction. These non colour characterised regions or areas are handled similar to other non colour characterised content. Such colours are considered to be in sRGB In the printing world for instance. sRGB is the internets default colour space. This approach is a combination of the above outlined ones and an answer to the historical X11 specifics. The advantage is, that legacy APIs and applications can benefit from a colour corrected desktop, while updated applications will be able to opt-out of colour correction. Thus a situation like on osX, with all content clearly characterised, can be reached gradually. During the project was a protocol designed for server client communication. The belonging spec is the X Color Management spec and is maintained by me over OpenICC. The spec was previously called net-color spec, but was renamed due to name space conflicts. For a smooth transition of legacy applications, the implementation of the spec in libXcm contains the xcm tool. This tool allows to set colour regions manually as a workaround. LibXcm covers just the communication protocol for applications and toolkits to talk to a colour server. The colour server, for instance CompICC, is responsible for doing the colour correction on GPU. If you want to add a colour server to your favorite compositing window manager or have questions around the spec, get in contact with me.

Can the X Color Management spec do compositing?
During a discussion on a Qt email list came out the question about compositing and colour correction. Most obviously the spec misses a freely selectable compositing colour space to do full blown up compositing. this would mean blurry region borders and other technics where transparency is involved. That is simply out of scope for the X Color Management spec. It is logical in the responsibility of the compositor to decide about the compositing colour space. Compositing window managers on Linux use today just the plain monitor colour space for their compositing. That is not perfect but seems to be good enough for simple transparency effects.  One correct way would be to meld the compositing of the window manager into the compositing of the client, which is in many cases the toolkit. But that is a bold adventure.taking the freedom of toolkits away and making traditionally modular stuff very monolithic. A architectural more sensible approach would be to use an intermediate blending space on top of the X Color Management spec. Thus the blending space is correctly selectable and modularity, interoperability and so on is much better. The toolkit can then rely on the compositing window manager to do all complicated transforms with the provided window and do colour correction as a final step. That would then essentially become a per window colour correction. From a today’s point of view the intermediate per window blending space needs a lot to do inside toolkits. It might take years to get there as toolkits just started about first steps. So the per region colour corrected desktop is still a valuable concept for today’s needs.

What is needed in the future?
Future developments have to take into account that the X11 architecture might change and other approaches become possible. The design of new architectures should cover colour management right from the beginning. With Wayland on the horizon and OpenGL-ES this is technically not a problem and would solve legacy issues like we see today in X11, on Windows and all major mobile platforms.

Colour management results of Google Summer of Code 2011

We are pleased to announce the final results of this years OpenICC participation in the Google Summer of Code program [1]. All students could reach successfully the project goals and obtained the second part of the Google stipends.

OpenICC mentored two students directly and one student through the collaboration with the openSUSE organisation. All three worked on colour management projects, which covered in parts new ground breaking technology and maintain existing one.

Yiannis Belias worked on the “API stabilization for Oyranos Colour Management System II” project [2]. The new classes, code generator improvements and tools will walk into the Oyranos master branch in the next months. This project helps in stablising the CMS core, which covers a great foundation of functionality. After the conversion, it will be much easier to provide stable and extensible APIs.

Joseph Simon realised the XCPD project. It’s goal was to implement the idea of a robust, standards conform printing dialog, which is based on the Common Printing Dialog (CPD) projects code. The handled and manipulated PDF follows the PDF/X specification for embedding user side colour managed content and remote printer configuration [3] [4].

Sebastian Oliva implemented a ICC device profile data base, called taxi. It is intented to share vendor and user created ICC profiles across platforms in a automated fashion. The online data base is designed to cover meta data about the device driver calibration status alongside the characterisation information in the belonging ICC profiles. The DB is online [5] [6]. The project slot was provided by the openSUSE distribution project and mentored by a OpenICC member.

Google Summer of Code is a global program that offers student developers stipends to write code for various open source software projects.

Many thanks to all students for their great work, all people how helped in shaping the basic ideas and discussing the projects in detail and Google for providing the stipends and openSUSE for addtionally inviting their students to the europe openSUSE Conference.

[1] http://code.google.com/soc/
[2] https://github.com/yiannis/Oyranos
[3] https://www.gitorious.org/google-summer-of-code-2011/xcpd
[4] http://jsimon3.wordpress.com/2011/08/23/gsoc%E2%80%9911-update-%E2%80%93-final/
[5] http://icc.opensuse.org
[6] http://openicc.git.sourceforge.net/git/gitweb.cgi?p=openicc/taxi

Oyranos 0.3.2

The new version of the colour management system Oyranos is released.Version 0.3.2 is a feature and bug fix release.

Changes Overview:

  • prefere normal ICC profiles over automatic generated ones
  • embedd and extract JSON device calibrations for ICC profiles
  • add oyranos-profile tool
  • add oyranos-profile-install script
  • prepare CUPS module for XCPD usage
  • harmonise monitor and libraw modules device DB keys
  • add example to embedd libraw device calibration into a ICC profile
  • switch to bzip2 compressed packages for RPM and other fixes
  • add man pages to all tools
  • add automatic generated help text in oyranos-policy –help

About:
Oyranos is a colour management system allowing to share various settings
across applications and services. The provided interfaces are the
C library, native graphical front ends and partitial access through
command line tools. The library is licensed under newBSD.

Known Applications using Oyranos:
ICC Examin, the KDE Kolor Manager and Synnefo configuration dialogs, the
CompIcc colour server and more.

ChangeLog:

http://www.oyranos.org/scm?p=oyranos.git;a=commitdiff;h=refs/heads/0.3.2

Thanks to all contributors and bug reporters.

Download:
git: git://www.oyranos.org/git/oyranos
git sha1: 238cb2dd27e624ef6167313e077d3b2a18a442f3
file: http://sourceforge.net/projects/oyranos/files/Oyranos/Oyranos%200.3/oyranos-0.3.2.tar.bz2/download
size: 1247164
md5sum: c80d142741c284b06cbea29b9d859933 oyranos-0.3.2.tar.bz2
sha1sum: a216254b8d0cd2563d3781fea57169d5b70a71e8 oyranos-0.3.2.tar.bz2
Linux RPM:http://download.opensuse.org/repositories/multimedia:/color_management/

dispcalGUI 0.7 @ rwx³

My yesterday held workshop for calibrating and profiling monitors using dispcalGUI+ArgyllCMS in Nuremberg was a nice experience. Around ten people from the openSUSE Conference gathered, all being eager to do something for their health of colour vision. We wanted to create ICC device profiles and collect them for later publishing. Following is a small report and review from the workshop.

We had available a i1display, a DTP94 and a i1pro as measurement devices, which no of the attendees owned privately. Installation on openSUSE-11.4 went pretty smooth. ArgyllCMS is in the multimedia:photo repository and dispcalGUI is in multimedia:color_management. Both are easily searchable through the http://software.opensuse.org URL. One hacker had all packages installed in advance and could start instantly with the i1display. With the default settings in dispcalGUI appeared a small terminal and required to adjust native monitor settings or continue with calibration. We pressed number 7 and continued with the calibration part. This lasted relatively long. dispcalGUI and in the background ArgyllCMS took quite some work to iterate over the calibration for four times with lots of “regression getting worse” style messages. This indicated to us, that it reached an end for improvement or the application where not satisfied according to each persons like. The profiling and installation finished after quite some time and the new calibration and profile could be used in colour managed applications. In Gimp the monitor profile was not used by default. The Colour Management tab in Gimp’s preferences needs to manually enable the system profile, which is a troublesome exercise to figure out. Instead colour managed applications should look first at the system profile. Overriding the system profile by default is dangerous for a good user experience.

The next people became ready and used the DTP94 in between. Unfortunately there where four people failing to use it. What looked like a defect DTP94 device cable of my best colorimeter, was observed by three users as continuous USB device ID switching. One could spot the reason to a conflicting UDEV rule of libmtp. I filled a bug report on SourceForge including the patch of that user. With that fix the DTP94 worked again. I am happy to can use the DTP94 in the future.

I figured out how we can speed up the calibration, by reducing the calibration quality, which was nice in such a big group. One attendee had a repeated black switching of his laptop monitor during the whole process in connection with the nv driver. That persisted even during the profiling stage, which makes no sense to me. I suggested him to install the nvidia proprietary driver especially for his installed Quadro card.

In several cases dispcalGUI crashed without providing further informations on the command line. We had then to repeat the process. Luckily a longer break allowed us to finalise most ICC profiles. I requested the monitors EDID data block from all people, to have the device information available alongside the profile. This can be done by installing the oyranos-monitor package, which brings relatively many dependencies, like a default profile set and several libraries and modules. oyranos-monitor -f edid -o monitor.edid could then be used to collect that data. Some installations did not work as expected and I have to look into the issue. The new profiles need now further preparation before they can be uploaded to icc.opensuse.org.

It was fun to see how most issues where resolved in the group, and that most people could get a device profile for their usage. Registration of a smaller number of people would be helpful to ensure no overload of capacities. Between, dispcalGUI has a homepage with further detailed informations.

Taxi @ rwx³

Sebastian Oliva presented today at the openSUSE conference the results of his ICC Device Profile Repository project to his mentoring organisation as participant of the Google Summer of Code 2011 program. As profile creation is expensive to most users, he emphasised the importance to easily share ICC profiles among these users. During the summer project, which is called now taxi, he developed a generic API to store and obtain ICC profiles through JSON requests.

X Color Management 0.3 DRAFT1

The net-color spec from the libXcm repository is renamed. During a great conversation with Martin Gräßlin from KWin at oSC rwx³ he pointed out, that the _NET_ prefix is reserved inside the Xorg atom name space. So I decided to rename all atoms in the applications known to me, which use the net-color spec. The renaming went on fast and all apps work again as expected in git.

The new spec is called X Color Management. It contains the color regions in Xorg description and the used device profile for Linux desktop colour servers.
Affected are libXcm, Xcm, Oyranos, CompICC and CinePaint.