Documentation/compat
====================

This is now deprecated, use :doc:`hacking on backports <backports/hacking>`.

This page is dedicated to compat, a generic Linux kernel backport
compatibility module that has been brewing since 2009. The compat module
provides functionality introduced in newer kernels to older kernels
through a set of header files and exported symbols. The compat module
allows code from newer kernel releases to be used on older kernels
without modifications with a few exceptions. Exceptions to achieve
automatically backporting kernel code through the compat module are
dealt with through specific mechanisms documented in the :doc:`compat-drivers
project page <compat-drivers>` and most recently through the `slides
used at the 2011 Linux Plumbers presentation on automatically
backporting the kernel
<http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__
and its `follow up presentation
<https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__
at the 2012 Linux Collaboration summit in San Francisco.

By using compat and frameworks such as :doc:`compat-drivers
<compat-drivers>`, and techniques to integrate `additional patches
<http://wireless.kernel.org/en/users/Download/stable/#Additional_patches_to_stable_releases>`__
into upstream releases that prefer and push for upstream, we hope to
ensure developers can prioritize working directly upstream on the Linux
kernel and provide deliverables to end users and customers for any
target kernel.

compat-drivers today automatically backports the `Wireless
<http://wireless.kernel.org/>`__, `Bluetooth <http://www.bluez.org/>`__
and `Ethernet
<https://www.linuxfoundation.org/collaborate/workgroups/networking/ethernet>`__
drivers directly from the Linux kernel on a daily basis by using
`linux-next.git
<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__.
We also make `stable backport releases
<http://wireless.kernel.org/en/users/Download/stable/>`__ by using the
`linux-stable.git
<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__
tree. We are considering expanding this framework to automatically
backport more drivers -- patches are welcomed, otherwise, you'll have to
wait until someone decides to integrate other subsystems.

Supported kernels
-----------------

The goal is to backport new kernel functionality at least down to the
oldest stable supported kernel listed on `kernel.org
<https://kernel.org>`__, but backporting code further is certainly
welcomed. Currently this means supporting backporting down to the 2.6.27
Linux kernel release. Although our goal is to at least support backport
efforts down to 2.6.27 the compat module currently has code to support
even older kernels, and we welcome code to support older kernels as
well.

Mailing list
------------

See the :doc:`Backports mailing list <../mailing_list>` page.

Presentations
-------------

Metrics explaining the amount of code saved by automatically backporting
the kernel using compat and current research are explained.

- `2011 Linux plumbers kernel backport presentation <http://linuxplumbersconf.org/2011/ocw/proposals/771>`__ (`slides <http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__)
- `follow up presentation Automatically backporting the Linux kernel - 2012 Linux Collaboration summit <https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__

Estimating deliverables
-----------------------

Once you use compat and kernel framework integrators such as
:doc:`compat-drivers <compat-drivers>` you should have no excuse to work
outside of the upstream Linux kernel. In order to help guestimate when
your upstream code will be available through a stable release such as
the :doc:`compat-drivers stable releases <../releases>` you can rely on
the `phb-crystall-ball <http://phb-crystal-ball.org/>`__.

Tracking down when a feature went upstream
------------------------------------------

The compat module expect you to put functionality into either header
files for defines and static inlines or into c files. This functionality
must be tucked away under a file for the kernel release that introduced
the functionality. To help aid you with your scavenger hunt on tracing
down when a specific functionality was introduced into the upstream
Linux kernel you can use several techniques. This section documents
those techniques.

Git tree setup for backporting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to understand how to track a feature in the Linux kernel you
must first understand how the development of the Linux kernel happens.
Below we provide a basic description of how this works.

Greg Kroah-Hartman maintained the stable series of the Linux kernel.
Linus hands off kernel maintainership to Greg after he blesses the first
release of a stable kernel. Greg maintains the stable kernel via through
the `linux-stable.git
<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__
tree. Each stable release of the kernel has a respective
origin/linux-3.x.y branch, where x is the current release number. The
"y" here is literal, so for the 3.2 release there is an
origin/linux-3.2.y branch that keeps track of the 3.2.1, 3.2.2 and
future extra version releases of the 3.2 release. You can use the
`linux-stable.git
<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__
tree to verify if code is present on a stable release of a kernel.

Linus maintains the release candidates of the Linux kernel (RC) up to
the point he blesses away the first release of a kernel release. So for
example Linus maintained the 3.2 kernel from the inception of 3.2-r1
down to 3.2-rc7. During the 3.2-rc1 to 3.2-rc7 release Linus is supposed
to only accept regression fixes. After that he releases the first
blessed release of 3.2 and handed off maintainership to Greg. All the
magic that went into 3.2-rc1 though happens in what we call the
development cycle of the Linux kernel. The development cycle of the
Linux kernel varies subsystem to subsystem but the idea can be best
described in terms of the usage of two subsystem git trees. Subsystem
maintainers typically have to maintain two git trees, a subsystem.git
tree and a subsystem-next.git tree. The subsystem.git tree contains code
that should go into Linus' tree during the regression testing of the RC
release of the kernel. The subsystem-next.git contains code is being
queued up for the next release of the Linux kernel, hence the "-next"
postfix. All code in the subsystem-next.git trees is ephemeral, that is,
what you see there today does not mean that it will be present tomorrow,
it is code that is simply accepted on that day by the subsystem
maintainer, but a lot of it can simply be dropped, and in fact Linus can
reject merging it eventually. Linus will start merging all
subsystem-next.git trees into his own tree during what we call, the
merge window. This typically consists of a 2-3 week window period. For
example the merge window for the 3.2 kernel started after Linus released
the v3.1 kernel release on Mon Oct 24, 2011, and ended when Linus
released the v3.2-rc1 release on Mon Nov 7, 2011, the merge window in
this case lasted exactly 2 weeks.

The merge window can often be painful though and to aid with this
Stephen Rothwell has decided to take up merging all subsystem-next.git
trees into one tree, `linux-next.git
<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__
on a daily basis. He rants to each subsystem maintainer on any issues
during his merge, including build breakage and general insanity.

The implications here are that we can not only track code that is
already released, but we can also predict what likely will go into the
next release of the Linux kernel by using `linux-next.git
<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__.
This means we can backport the Linux kernel proactively through every
subsystem development cycle by using `linux-next.git
<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__.

If this is still a bit fuzzy you can review the `wireless development
process <http://wireless.kernel.org/en/developers/process>`__ for a
review of how that specific subsystem development works. The development
process there even includes a shiny diagram.

Using git grep to identify what file adds new functionality
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First you want to identify what file added the functionality you seek to
backport. Suppose you want to identify what file adds a new function,
foobar(). To do this you can use::

   git grep foobar\(

to identify where foobar() is used in the entire Linux kernel. git grep
will use the indexed data and at times can often run faster than a
regular grep, as it will also skip object file in a regular grep search.
Given that the point of compat is to backport functionality and that
this functionality is typically shared between modules and subsystems,
you can rest assured most functionality has a declaration in the header
files of the Linux kernel. You can typically save yourself git grep time
by starting to search through the include/ directory of the Linux
kernel.

::

   git grep foobar\( include/

Using git blame to identify introduction of a commit
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Typically compat will have a lot of features already backported for
older kernels, so to help backport new functionality you should be using
`linux-next.git
<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__.
To identify at what point a specific feature was added into the Linux
kernel you can use git blame to identify the sha1sum of the commit that
added the functionality you want to backport. At times though such
functionality may have not been introduced via that commit but instead
it may have simply been moved from one place to another. You can use::

   git blame sha1sum~1

to review if the functionality was present before the last commit you identified added that functionality to a file.

Using git describe to identify where an commit was introduced
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once you have identified the commit that introduced a specific
functionality you can use git describe --contains sha1sum to give you
the first commit tag that contains the commit. You can use the base of
the tag to help you figure out the first kernel release that added the
commit.

For example, commit 600715dcdf567c86f8b2c6173fcfb4b873e25a19, or short,
600715dc added to the Linux kernel the CONFIG_PHYS_ADDR_T_64BIT feature.
We can determine that it was added on the v2.6.28 release by reviewing
the results of using::

   git describe --contains 600715dc

which yields: v2.6.28-rc1~271^2^4~6. This means that in order to
backport 600715dc we can edit two files on compat:

- include/linux/compat-2.6.28.h
- compat/compat-2.6.28.c

Since this functionality can be dealt with header files the c file is
not touched to backport this functionality. You can review the
respective `compat CONFIG_PHYS_ADDR_T_64BIT backport via gitweb
<https://github.com/mcgrof/compat/commit/4ec0edbd1cc2e6165e29f8ba466324e975e1cd73#include>`__.

Testing across kernels
----------------------

The goal behind compat is to add features to provide backport but to
older kernels but to do this successfully we must also ensure that each
new patch added does not break building against a supported kernel. It
may be a daunting task to get all supported kernels to and also to build
against then, which is why we have started adding support to do all of
this for you. Under the bin/ directory you will find two scripts which
we document below:

- get-compat-kernels
- ckmake

get-compat-kernels
~~~~~~~~~~~~~~~~~~

You can run this script to install all known supported kernels.
Currently this will only work for Ubuntu, but `work is being considered
<https://lkml.org/lkml/2012/2/6/85>`__ to standardize on the same
kernels for any Linux distribution.

ckmake
~~~~~~

You can run this to test compiling compat against supported kernels
available. Although initially designed to support Ubuntu only the goal
is to make this Linux distribution agnostic.

compat modules
--------------

compat.git provides a few modules and headers to help with general
kernel compatibility.

compat module
~~~~~~~~~~~~~

Provides all exported symbols implemented in each respective kernel
compat-2.6.xy.c files. Upon module load it just initializes the Linux
kernel's *power management Quality Of Service* (aka **pm-qos**)
Interface interface added as of the 2.6.24 kernel. No other things are
initialized, the rest of the compat module just acts as a library of
exported symbols.

compat_firmware_class module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Another module which compat.git provides is a backport of the
firmware_class module which got updated recently newer with a new
request_firmware_nowait() to allow better asynchronous firmware
uploading. This was added as of the 2.6.33 kernel. The firmware_class
module has been backported into a new module called
compat_firmware_class. A separate module has been defined instead of a
direct replacement for firmware_class since your system may have old
drivers which use the old request_firmware_nowait() and would bust if
they used the new request_firmware_nowait(). The compat_firmware_class
module registers its own sysfs subsystem and as such also gets udev
events sent through a separate subsystem. Because of this a new udev
rules file is required and provided.

Research
--------

Active research is ongoing which could enhance automatically backporting
the Linux kernel further. For details please read up on:

- `Jesper Andersen & Julia L. Lawall - Generic Patch Inference <http://www.diku.dk/~julia/andersen_ase08.pdf>`__
- `Coccinelle spatch <http://coccinelle.lip6.fr/>`__

Consider one set of diffs that can be read by spdiff to produce an
spatch. This could in theory allow us to backport all functionality on a
kernel if only two or three sets of files were patched and interpreted
by spdiff.

Contributions
-------------

Contributions to compat follow the same mechanisms as used in the Linux
kernel, this means you should provide as Singed-off-by tag as documented
on the `Developer's Certificate of Origin 1.1
<http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html>`__.

Submitting patches
------------------

compat and compat-drivers contributions follow the contribution model
implemented by the Linux kernel. Patches or pull requests for compat and
compat-drivers must have be signed-offed. If you don't sign off on them
they will not accepted. This means adding a line that says
"Signed-off-by: Name email" at the end of each commit, indicating that
you wrote the code and have the right to pass it on as an open source
patch. For exact definition of what the Signed-off-by tag is you can
read the definition of the "Developer's Certificate of Origin 1.1",
which you can read here:

http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html

You can send patches as follows::

   To: mcgrof@kernel.org
   Cc: backports@vger.kernel.org
   Subject: compat: foo

License
-------

The compat module and code is all licensed under the same license as the
Linux kernel and is considered a complete derivative work of the Linux
kernel. For more information refer to the COPYRIGHT file.