diff options
Diffstat (limited to 'source/documentation/compat.rst')
| -rw-r--r-- | source/documentation/compat.rst | 336 |
1 files changed, 336 insertions, 0 deletions
diff --git a/source/documentation/compat.rst b/source/documentation/compat.rst new file mode 100644 index 0000000..b804edc --- /dev/null +++ b/source/documentation/compat.rst @@ -0,0 +1,336 @@ +Documentation/compat +==================== + +This is now deprecated, use :doc:`hacking on backports <backports/hacking>`. + +This page is dedicated to compat, a generic Linux kernel backport +compatibility module that has been brewing since 2009. The compat module +provides functionality introduced in newer kernels to older kernels +through a set of header files and exported symbols. The compat module +allows code from newer kernel releases to be used on older kernels +without modifications with a few exceptions. Exceptions to achieve +automatically backporting kernel code through the compat module are +dealt with through specific mechanisms documented in the :doc:`compat-drivers +project page <compat-drivers>` and most recently through the `slides +used at the 2011 Linux Plumbers presentation on automatically +backporting the kernel +<http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__ +and its `follow up presentation +<https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__ +at the 2012 Linux Collaboration summit in San Francisco. + +By using compat and frameworks such as :doc:`compat-drivers +<compat-drivers>`, and techniques to integrate `additional patches +<http://wireless.kernel.org/en/users/Download/stable/#Additional_patches_to_stable_releases>`__ +into upstream releases that prefer and push for upstream, we hope to +ensure developers can prioritize working directly upstream on the Linux +kernel and provide deliverables to end users and customers for any +target kernel. + +compat-drivers today automatically backports the `Wireless +<http://wireless.kernel.org/>`__, `Bluetooth <http://www.bluez.org/>`__ +and `Ethernet +<https://www.linuxfoundation.org/collaborate/workgroups/networking/ethernet>`__ +drivers directly from the Linux kernel on a daily basis by using +`linux-next.git +<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. +We also make `stable backport releases +<http://wireless.kernel.org/en/users/Download/stable/>`__ by using the +`linux-stable.git +<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ +tree. We are considering expanding this framework to automatically +backport more drivers -- patches are welcomed, otherwise, you'll have to +wait until someone decides to integrate other subsystems. + +Supported kernels +----------------- + +The goal is to backport new kernel functionality at least down to the +oldest stable supported kernel listed on `kernel.org +<https://kernel.org>`__, but backporting code further is certainly +welcomed. Currently this means supporting backporting down to the 2.6.27 +Linux kernel release. Although our goal is to at least support backport +efforts down to 2.6.27 the compat module currently has code to support +even older kernels, and we welcome code to support older kernels as +well. + +Mailing list +------------ + +See the :doc:`Backports mailing list <../mailing_list>` page. + +Presentations +------------- + +Metrics explaining the amount of code saved by automatically backporting +the kernel using compat and current research are explained. + +- `2011 Linux plumbers kernel backport presentation <http://linuxplumbersconf.org/2011/ocw/proposals/771>`__ (`slides <http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__) +- `follow up presentation Automatically backporting the Linux kernel - 2012 Linux Collaboration summit <https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__ + +Estimating deliverables +----------------------- + +Once you use compat and kernel framework integrators such as +:doc:`compat-drivers <compat-drivers>` you should have no excuse to work +outside of the upstream Linux kernel. In order to help guestimate when +your upstream code will be available through a stable release such as +the :doc:`compat-drivers stable releases <../releases>` you can rely on +the `phb-crystall-ball <http://phb-crystal-ball.org/>`__. + +Tracking down when a feature went upstream +------------------------------------------ + +The compat module expect you to put functionality into either header +files for defines and static inlines or into c files. This functionality +must be tucked away under a file for the kernel release that introduced +the functionality. To help aid you with your scavenger hunt on tracing +down when a specific functionality was introduced into the upstream +Linux kernel you can use several techniques. This section documents +those techniques. + +Git tree setup for backporting +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to understand how to track a feature in the Linux kernel you +must first understand how the development of the Linux kernel happens. +Below we provide a basic description of how this works. + +Greg Kroah-Hartman maintained the stable series of the Linux kernel. +Linus hands off kernel maintainership to Greg after he blesses the first +release of a stable kernel. Greg maintains the stable kernel via through +the `linux-stable.git +<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ +tree. Each stable release of the kernel has a respective +origin/linux-3.x.y branch, where x is the current release number. The +"y" here is literal, so for the 3.2 release there is an +origin/linux-3.2.y branch that keeps track of the 3.2.1, 3.2.2 and +future extra version releases of the 3.2 release. You can use the +`linux-stable.git +<http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ +tree to verify if code is present on a stable release of a kernel. + +Linus maintains the release candidates of the Linux kernel (RC) up to +the point he blesses away the first release of a kernel release. So for +example Linus maintained the 3.2 kernel from the inception of 3.2-r1 +down to 3.2-rc7. During the 3.2-rc1 to 3.2-rc7 release Linus is supposed +to only accept regression fixes. After that he releases the first +blessed release of 3.2 and handed off maintainership to Greg. All the +magic that went into 3.2-rc1 though happens in what we call the +development cycle of the Linux kernel. The development cycle of the +Linux kernel varies subsystem to subsystem but the idea can be best +described in terms of the usage of two subsystem git trees. Subsystem +maintainers typically have to maintain two git trees, a subsystem.git +tree and a subsystem-next.git tree. The subsystem.git tree contains code +that should go into Linus' tree during the regression testing of the RC +release of the kernel. The subsystem-next.git contains code is being +queued up for the next release of the Linux kernel, hence the "-next" +postfix. All code in the subsystem-next.git trees is ephemeral, that is, +what you see there today does not mean that it will be present tomorrow, +it is code that is simply accepted on that day by the subsystem +maintainer, but a lot of it can simply be dropped, and in fact Linus can +reject merging it eventually. Linus will start merging all +subsystem-next.git trees into his own tree during what we call, the +merge window. This typically consists of a 2-3 week window period. For +example the merge window for the 3.2 kernel started after Linus released +the v3.1 kernel release on Mon Oct 24, 2011, and ended when Linus +released the v3.2-rc1 release on Mon Nov 7, 2011, the merge window in +this case lasted exactly 2 weeks. + +The merge window can often be painful though and to aid with this +Stephen Rothwell has decided to take up merging all subsystem-next.git +trees into one tree, `linux-next.git +<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__ +on a daily basis. He rants to each subsystem maintainer on any issues +during his merge, including build breakage and general insanity. + +The implications here are that we can not only track code that is +already released, but we can also predict what likely will go into the +next release of the Linux kernel by using `linux-next.git +<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. +This means we can backport the Linux kernel proactively through every +subsystem development cycle by using `linux-next.git +<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. + +If this is still a bit fuzzy you can review the `wireless development +process <http://wireless.kernel.org/en/developers/process>`__ for a +review of how that specific subsystem development works. The development +process there even includes a shiny diagram. + +Using git grep to identify what file adds new functionality +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First you want to identify what file added the functionality you seek to +backport. Suppose you want to identify what file adds a new function, +foobar(). To do this you can use:: + + git grep foobar\( + +to identify where foobar() is used in the entire Linux kernel. git grep +will use the indexed data and at times can often run faster than a +regular grep, as it will also skip object file in a regular grep search. +Given that the point of compat is to backport functionality and that +this functionality is typically shared between modules and subsystems, +you can rest assured most functionality has a declaration in the header +files of the Linux kernel. You can typically save yourself git grep time +by starting to search through the include/ directory of the Linux +kernel. + +:: + + git grep foobar\( include/ + +Using git blame to identify introduction of a commit +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Typically compat will have a lot of features already backported for +older kernels, so to help backport new functionality you should be using +`linux-next.git +<http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. +To identify at what point a specific feature was added into the Linux +kernel you can use git blame to identify the sha1sum of the commit that +added the functionality you want to backport. At times though such +functionality may have not been introduced via that commit but instead +it may have simply been moved from one place to another. You can use:: + + git blame sha1sum~1 + +to review if the functionality was present before the last commit you identified added that functionality to a file. + +Using git describe to identify where an commit was introduced +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once you have identified the commit that introduced a specific +functionality you can use git describe --contains sha1sum to give you +the first commit tag that contains the commit. You can use the base of +the tag to help you figure out the first kernel release that added the +commit. + +For example, commit 600715dcdf567c86f8b2c6173fcfb4b873e25a19, or short, +600715dc added to the Linux kernel the CONFIG_PHYS_ADDR_T_64BIT feature. +We can determine that it was added on the v2.6.28 release by reviewing +the results of using:: + + git describe --contains 600715dc + +which yields: v2.6.28-rc1~271^2^4~6. This means that in order to +backport 600715dc we can edit two files on compat: + +- include/linux/compat-2.6.28.h +- compat/compat-2.6.28.c + +Since this functionality can be dealt with header files the c file is +not touched to backport this functionality. You can review the +respective `compat CONFIG_PHYS_ADDR_T_64BIT backport via gitweb +<https://github.com/mcgrof/compat/commit/4ec0edbd1cc2e6165e29f8ba466324e975e1cd73#include>`__. + +Testing across kernels +---------------------- + +The goal behind compat is to add features to provide backport but to +older kernels but to do this successfully we must also ensure that each +new patch added does not break building against a supported kernel. It +may be a daunting task to get all supported kernels to and also to build +against then, which is why we have started adding support to do all of +this for you. Under the bin/ directory you will find two scripts which +we document below: + +- get-compat-kernels +- ckmake + +get-compat-kernels +~~~~~~~~~~~~~~~~~~ + +You can run this script to install all known supported kernels. +Currently this will only work for Ubuntu, but `work is being considered +<https://lkml.org/lkml/2012/2/6/85>`__ to standardize on the same +kernels for any Linux distribution. + +ckmake +~~~~~~ + +You can run this to test compiling compat against supported kernels +available. Although initially designed to support Ubuntu only the goal +is to make this Linux distribution agnostic. + +compat modules +-------------- + +compat.git provides a few modules and headers to help with general +kernel compatibility. + +compat module +~~~~~~~~~~~~~ + +Provides all exported symbols implemented in each respective kernel +compat-2.6.xy.c files. Upon module load it just initializes the Linux +kernel's *power management Quality Of Service* (aka **pm-qos**) +Interface interface added as of the 2.6.24 kernel. No other things are +initialized, the rest of the compat module just acts as a library of +exported symbols. + +compat_firmware_class module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Another module which compat.git provides is a backport of the +firmware_class module which got updated recently newer with a new +request_firmware_nowait() to allow better asynchronous firmware +uploading. This was added as of the 2.6.33 kernel. The firmware_class +module has been backported into a new module called +compat_firmware_class. A separate module has been defined instead of a +direct replacement for firmware_class since your system may have old +drivers which use the old request_firmware_nowait() and would bust if +they used the new request_firmware_nowait(). The compat_firmware_class +module registers its own sysfs subsystem and as such also gets udev +events sent through a separate subsystem. Because of this a new udev +rules file is required and provided. + +Research +-------- + +Active research is ongoing which could enhance automatically backporting +the Linux kernel further. For details please read up on: + +- `Jesper Andersen & Julia L. Lawall - Generic Patch Inference <http://www.diku.dk/~julia/andersen_ase08.pdf>`__ +- `Coccinelle spatch <http://coccinelle.lip6.fr/>`__ + +Consider one set of diffs that can be read by spdiff to produce an +spatch. This could in theory allow us to backport all functionality on a +kernel if only two or three sets of files were patched and interpreted +by spdiff. + +Contributions +------------- + +Contributions to compat follow the same mechanisms as used in the Linux +kernel, this means you should provide as Singed-off-by tag as documented +on the `Developer's Certificate of Origin 1.1 +<http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html>`__. + +Submitting patches +------------------ + +compat and compat-drivers contributions follow the contribution model +implemented by the Linux kernel. Patches or pull requests for compat and +compat-drivers must have be signed-offed. If you don't sign off on them +they will not accepted. This means adding a line that says +"Signed-off-by: Name email" at the end of each commit, indicating that +you wrote the code and have the right to pass it on as an open source +patch. For exact definition of what the Signed-off-by tag is you can +read the definition of the "Developer's Certificate of Origin 1.1", +which you can read here: + +http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html + +You can send patches as follows:: + + To: mcgrof@kernel.org + Cc: backports@vger.kernel.org + Subject: compat: foo + +License +------- + +The compat module and code is all licensed under the same license as the +Linux kernel and is considered a complete derivative work of the Linux +kernel. For more information refer to the COPYRIGHT file. |