openclaw/tqma6-yocto-mirror
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Complete Yocto mirror with license table for TQMa6UL (2038-compliance)

Browse Source 
- 264 license table entries with exact download URLs (224/264 resolved)
- Complete sources/ directory with all BitBake recipes
- Build configuration: tqma6ul-multi-mba6ulx, spaetzle (musl)
- Full traceability for Softwarefreigabeantrag
- GCC 13.4.0, Linux 6.6.102, U-Boot 2023.04, musl 1.2.4
- License distribution: GPL-2.0 (24), MIT (23), GPL-2.0+ (18), BSD-3 (16)
This commit is contained in:
Siggi (OpenClaw Agent)
2026-03-01 20:58:18 +00:00
commit 16accb6b24
15086 changed files with 1292356 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
sources/poky/documentation/.gitignore vendored Normal file
View File

@@ -0,0 +1,11 @@
sphinx/__pycache__
_build/
Pipfile.lock
poky.yaml
sphinx-static/switchers.js
releases.rst
.vscode/
*/svg/*.png
*/svg/*.pdf
styles/*
!styles/config

7
sources/poky/documentation/.vale.ini Normal file
View File

@@ -0,0 +1,7 @@
StylesPath = styles
MinAlertLevel = suggestion
Packages = RedHat, proselint, write-good, alex, Readability, Joblint
Vocab = Yocto, OpenSource
[*.rst]
BasedOnStyles = Vale, RedHat, proselint, write-good, alex, Readability, Joblint

79
sources/poky/documentation/Makefile Normal file
View File

@@ -0,0 +1,79 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?= -W --keep-going -j auto
SPHINXBUILD ?= sphinx-build
# Release notes are excluded because they contain contributor names and commit messages which can't be modified
VALEOPTS ?= --no-wrap --glob '!migration-guides/release-notes-*.rst'
SOURCEDIR = .
VALEDOCS ?= $(SOURCEDIR)
SPHINXLINTDOCS ?= $(SOURCEDIR)
IMAGEDIRS = */svg
BUILDDIR = _build
DESTDIR = final
SVG2PNG = rsvg-convert
SVG2PDF = rsvg-convert
ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0)
$(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed")
endif
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: all help Makefile clean stylecheck publish epub latexpdf
publish: Makefile epub latexpdf html singlehtml
rm -rf $(BUILDDIR)/$(DESTDIR)/
mkdir -p $(BUILDDIR)/$(DESTDIR)/
cp -r $(BUILDDIR)/html/* $(BUILDDIR)/$(DESTDIR)/
mkdir -p $(BUILDDIR)/$(DESTDIR)/_static
cp $(BUILDDIR)/epub/TheYoctoProject.epub $(BUILDDIR)/latex/theyoctoproject.pdf $(BUILDDIR)/$(DESTDIR)/_static/
cp $(BUILDDIR)/singlehtml/index.html $(BUILDDIR)/$(DESTDIR)/singleindex.html
sed -i -e 's@index.html#@singleindex.html#@g' $(BUILDDIR)/$(DESTDIR)/singleindex.html
# Build a list of SVG files to convert to PDFs
PDFs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.pdf,$(wildcard $(SOURCEDIR)/$(dir)/*.svg)))
# Build a list of SVG files to convert to PNGs
PNGs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.png,$(wildcard $(SOURCEDIR)/$(dir)/*.svg)))
# Pattern rule for converting SVG to PDF
%.pdf : %.svg
$(SVG2PDF) --format=Pdf --output=$@ $<
# Pattern rule for converting SVG to PNG
%.png : %.svg
$(SVG2PNG) --format=Png --output=$@ $<
clean:
@rm -rf $(BUILDDIR) $(PNGs) $(PDFs) poky.yaml sphinx-static/switchers.js releases.rst
stylecheck:
vale sync
vale $(VALEOPTS) $(VALEDOCS)
sphinx-lint:
sphinx-lint $(SPHINXLINTDOCS)
epub: $(PNGs)
$(SOURCEDIR)/set_versions.py
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
# Note: we need to pass buf_size here (which is also configurable from
# texmf.cnf), to avoid following error:
# Unable to read an entire line---bufsize=200000. Please increase buf_size in texmf.cnf.
latexpdf: $(PDFs)
$(SOURCEDIR)/set_versions.py
buf_size=10000000 $(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
all: html epub latexpdf
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%:
$(SOURCEDIR)/set_versions.py
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

14
sources/poky/documentation/Pipfile Normal file
View File

@@ -0,0 +1,14 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
sphinx = "*"
sphinx-rtd-theme = "*"
pyyaml = "*"
[requires]
python_version = "3"

450
sources/poky/documentation/README Normal file
View File

@@ -0,0 +1,450 @@
documentation
=============
This is the directory that contains the Yocto Project documentation. The Yocto
Project source repositories at https://git.yoctoproject.org/cgit.cgi have two
instances of the "documentation" directory. You should understand each of
these instances.
poky/documentation - The directory within the poky Git repository containing
the set of Yocto Project manuals. When you clone the
poky Git repository, the documentation directory
contains the manuals. The state of the manuals in this
directory is guaranteed to reflect the latest Yocto
Project release. The manuals at the tip of this
directory will also likely contain most manual
development changes.
yocto-docs/documentation - The Git repository for the Yocto Project manuals.
This repository is where manual development
occurs. If you plan on contributing back to the
Yocto Project documentation, you should set up
a local Git repository based on this upstream
repository as follows:
git clone git://git.yoctoproject.org/yocto-docs
Changes and patches are first pushed to the
yocto-docs Git repository. Later, they make it
into the poky Git repository found at
git://git.yoctoproject.org/poky.
Manual Organization
===================
Here the folders corresponding to individual manuals:
* brief-yoctoprojectqs - Yocto Project Quick Start
* overview-manual - Yocto Project Overview and Concepts Manual
* contributor-guide - Yocto Project and OpenEmbedded Contributor Guide
* ref-manual - Yocto Project Reference Manual
* bsp-guide - Yocto Project Board Support Package (BSP) Developer's Guide
* dev-manual - Yocto Project Development Tasks Manual
* kernel-dev - Yocto Project Linux Kernel Development Manual
* profile-manual - Yocto Project Profiling and Tracing Manual
* sdk-manual - Yocto Project Software Development Kit (SDK) Developer's Guide.
* toaster-manual - Toaster User Manual
* test-manual - Yocto Project Test Environment Manual
* migration-guides - Yocto Project Release and Migration Notes
Each folder is self-contained regarding content and figures.
If you want to find HTML versions of the Yocto Project manuals on the web,
the current versions reside at https://docs.yoctoproject.org.
poky.yaml
=========
This file defines variables used for documentation production. The variables
are used to define release pathnames, URLs for the published manuals, etc.
standards.md
============
This file specifies some rules to follow when contributing to the documentation.
template/
=========
Contains a template.svg, to make it easier to create consistent
SVG diagrams.
Sphinx
======
The Yocto Project documentation was migrated from the original DocBook
format to Sphinx based documentation for the Yocto Project 3.2
release. This section will provide additional information related to
the Sphinx migration, and guidelines for developers willing to
contribute to the Yocto Project documentation.
Sphinx is a tool that makes it easy to create intelligent and
beautiful documentation, written by Georg Brandl and licensed under
the BSD license. It was originally created for the Python
documentation.
Extensive documentation is available on the Sphinx website:
https://www.sphinx-doc.org/en/master/. Sphinx is designed to be
extensible thanks to the ability to write our own custom extensions,
as Python modules, which will be executed during the generation of the
documentation.
Yocto Project documentation website
===================================
The website hosting the Yocto Project documentation, can be found
at: https://docs.yoctoproject.org/.
The entire Yocto Project documentation, as well as the BitBake manual,
is published on this website, including all previously released
versions. A version switcher was added, as a drop-down menu on the top
of the page to switch back and forth between the various versions of
the current active Yocto Project releases.
Transition pages have been added (as rst files) to show links to old
versions of the Yocto Project documentation with links to each manual
generated with DocBook.
How to build the Yocto Project documentation
============================================
Sphinx is written in Python. While it might work with Python2, for
obvious reasons, we will only support building the Yocto Project
documentation with Python3.
Sphinx might be available in your Linux distro packages repositories,
however it is not recommended to use distro packages, as they might be
old versions, especially if you are using an LTS version of your
distro. The recommended method to install the latest versions of Sphinx
and of its required dependencies is to use the Python Package Index (pip).
To install all required packages run:
$ pip3 install sphinx sphinx_rtd_theme pyyaml
To make sure you always have the latest versions of such packages, you
should regularly run the same command with an added "--upgrade" option:
$ pip3 install --upgrade sphinx sphinx_rtd_theme pyyaml
Also install the "inkscape" package from your distribution.
Inkscape is need to convert SVG graphics to PNG (for EPUB
export) and to PDF (for PDF export).
Additionally install "fncychap.sty" TeX font if you want to build PDFs. Debian
and Ubuntu have it in "texlive-latex-extra" package while RedHat distributions
and OpenSUSE have it in "texlive-fncychap" package for example.
To build the documentation locally, run:
$ cd documentation
$ make html
The resulting HTML index page will be _build/html/index.html, and you
can browse your own copy of the locally generated documentation with
your browser.
Alternatively, you can use Pipenv to automatically install all required
dependencies in a virtual environment:
$ cd documentation
$ pipenv install
$ pipenv run make html
Style checking the Yocto Project documentation
==============================================
The project is starting to use Vale (https://vale.sh/)
to validate the text style.
To install Vale:
$ pip install vale
To run Vale:
$ make stylecheck
Style checking the whole documentation might take some time and generate a
lot of warnings/errors, thus one can run Vale on a subset of files or
directories:
$ make stylecheck VALEDOCS=<file>
$ make stylecheck VALEDOCS="<file1> <file2>"
$ make stylecheck VALEDOCS=<dir>
Lint checking the Yocto Project documentation
=============================================
To fix errors which are not reported by Sphinx itself,
the project uses sphinx-lint (https://github.com/sphinx-contrib/sphinx-lint).
To install sphinx-lint:
$ pip install sphinx-lint
To run sphinx-lint:
$ make sphinx-lint
Lint checking the whole documentation might take some time and generate a
lot of warnings/errors, thus one can run sphinx-lint on a subset of files
or directories:
$ make sphinx-lint SPHINXLINTDOCS=<file>
$ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
$ make sphinx-lint SPHINXLINTDOCS=<dir>
Sphinx theme and CSS customization
==================================
The Yocto Project documentation is currently based on the "Read the
Docs" Sphinx theme, with a few changes to make sure the look and feel
of the project documentation is preserved.
Most of the theme changes can be done using the file
'sphinx-static/theme_overrides.css'. Most CSS changes in this file
were inherited from the DocBook CSS stylesheets.
Sphinx design guidelines and principles
=======================================
The initial Docbook to Sphinx migration was done with an automated
tool called Pandoc (https://pandoc.org/). The tool produced some clean
output markdown text files. After the initial automated conversion
additional changes were done to fix up headings, images and links. In
addition Sphinx has built in mechanisms (directives) which were used
to replace similar functions implemented in Docbook such as glossary,
variables substitutions, notes and references.
Headings
========
The layout of the Yocto Project manuals is organized as follows
Book
Chapter
Section
Subsection
Subsubsection
Subsubsubsection
Here are the heading styles that we use in the manuals:
Book => overline ===
Chapter => overline ***
Section => ====
Subsection => ----
Subsubsection => ~~~~
Subsubsubsection => ^^^^
With this proposal, we preserve the same TOCs between Sphinx and Docbook.
Built-in glossary
=================
Sphinx has a glossary directive. From
https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:
This directive must contain a reST definition list with terms and
definitions. It's then possible to refer to each definition through the
[https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
'term' role].
So anywhere in any of the Yocto Project manuals, :term:`VAR` can be
used to refer to an item from the glossary, and a link is created
automatically. A general index of terms is also generated by Sphinx
automatically.
Global substitutions
====================
The Yocto Project documentation makes heavy use of global
variables. In Docbook these variables are stored in the file
poky.ent. This Docbook feature is not handled automatically with
Pandoc. Sphinx has builtin support for substitutions
(https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions),
however there are important shortcomings. For example they cannot be
used/nested inside code-block sections.
A Sphinx extension was implemented to support variable substitutions
to mimic the DocBook based documentation behavior. Variable
substitutions are done while reading/parsing the .rst files. The
pattern for variables substitutions is the same as with DocBook,
e.g. `&VAR;`.
The implementation of the extension can be found here in the file
documentation/sphinx/yocto-vars.py, this extension is enabled by
default when building the Yocto Project documentation. All variables
are set in a file call poky.yaml, which was initially generated from
poky.ent. The file was converted into YAML so that it is easier to
process by the custom Sphinx extension (which is a Python module).
For example, the following .rst content will produce the 'expected'
content:
.. code-block::
$ mkdir poky-&DISTRO;
or
$ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;
Variables can be nested, like it was the case for DocBook:
YOCTO_HOME_URL : "https://www.yoctoproject.org"
YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
Note directive
==============
Sphinx has a builtin 'note' directive that produces clean Note section
in the output file. There are various types of directives such as
"attention", "caution", "danger", "error", "hint", "important", "tip",
"warning", "admonition" that are supported, and additional directives
can be added as Sphinx extension if needed.
Figures
=======
The Yocto Project documentation has many figures/images. Sphinx has a
'figure' directive which is straightforward to use. To include a
figure in the body of the documentation:
.. image:: figures/YP-flow-diagram.png
Links and References
====================
The following types of links can be used: links to other locations in
the same document, to locations in other documents and to external
websites.
More information can be found here:
https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.
For external links, we use this syntax:
`link text <link URL>`__
instead of:
`link text <link URL>`_
Both syntaxes work, but the latter also creates a "link text" reference
target which could conflict with other references with the same name.
So, only use this variant when you wish to make multiple references
to this link, reusing only the target name.
See https://stackoverflow.com/questions/27420317/restructured-text-rst-http-links-underscore-vs-use
Anchor (<#link>) links are forbidden as they are not checked by Sphinx during
the build and may be broken without knowing about it.
References
==========
The following extension is enabled by default:
sphinx.ext.autosectionlabel
(https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).
This extension allows you to refer sections by their titles. Note that
autosectionlabel_prefix_document is enabled by default, so that we can
insert references from any document.
For example, to insert an HTML link to a section from
documentation/manual/intro.rst, use:
Please check this :ref:`manual/intro:Cross-References to Locations in the Same Document`
Alternatively a custom text can be used instead of using the section
text:
Please check this :ref:`section <manual/intro:Cross-References to Locations in the Same Document>`
TIP: The following command can be used to dump all the references that
are defined in the project documentation:
python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv
This dump contains all links and for each link it shows the default
"Link Text" that Sphinx would use. If the default link text is not
appropriate, a custom link text can be used in the ':ref:' directive.
Extlinks
========
The sphinx.ext.extlinks extension is enabled by default
(https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension),
and it is configured with the 'extlinks' definitions in
the 'documentation/conf.py' file:
'yocto_home': ('https://yoctoproject.org%s', None),
'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
'yocto_lists': ('https://lists.yoctoproject.org%s', None),
'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
'yocto_docs': ('https://docs.yoctoproject.org%s', None),
'yocto_git': ('https://git.yoctoproject.org%s', None),
'oe_home': ('https://www.openembedded.org%s', None),
'oe_lists': ('https://lists.openembedded.org%s', None),
'oe_git': ('https://git.openembedded.org%s', None),
'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
'oe_layerindex': ('https://layers.openembedded.org%s', None),
'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
It creates convenient shortcuts which can be used throughout the
documentation rst files, as:
Please check this :yocto_wiki:`wiki page </Weekly_Status>`
Intersphinx links
=================
The sphinx.ext.intersphinx extension is enabled by default
(https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html),
so that we can cross reference content from other Sphinx based
documentation projects, such as the BitBake manual.
References to the BitBake manual can directly be done:
- With a specific description instead of the section name:
:ref:`Azure Storage fetcher (az://) <bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`
- With the section name:
:ref:`bitbake-user-manual/bitbake-user-manual-intro:usage and syntax` option
If you want to refer to an entire document (or chapter) in the BitBake manual,
you have to use the ":doc:" macro with the "bitbake:" prefix:
- :doc:`BitBake User Manual <bitbake:index>`
- :doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata`" chapter
Note that a reference to a variable (:term:`VARIABLE`) automatically points to
the BitBake manual if the variable is not described in the Reference Manual's Variable Glossary.
However, if you need to bypass this, you can explicitely refer to a description in the
BitBake manual as follows:
:term:`bitbake:BB_NUMBER_PARSE_THREADS`
This would be the same if we had identical document filenames in
both the Yocto Project and BitBake manuals:
:ref:`bitbake:directory/file:section title`
Submitting documentation changes
================================
Please refer to our contributor guide here: https://docs.yoctoproject.org/contributor-guide/
for full details on how to submit changes.
As a quick guide, patches should be sent to docs@lists.yoctoproject.org
The git command to do that would be:
git send-email -M -1 --to docs@lists.yoctoproject.org
The 'To' header can be set as default for this repository:
git config sendemail.to docs@lists.yoctoproject.org
Now you can just do 'git send-email origin/master..' to send all local patches.
Read the other sections in this document and documentation/standards.md for
rules to follow when contributing to the documentation.
Git repository: https://git.yoctoproject.org/yocto-docs
Mailing list: docs@lists.yoctoproject.org

19
sources/poky/documentation/bitbake.rst Normal file
View File

@@ -0,0 +1,19 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====================
BitBake Documentation
=====================
|
BitBake was originally a part of the OpenEmbedded project. It was inspired by
the Portage package management system used by the Gentoo Linux distribution. In
2004, the OpenEmbedded project was split the project into two distinct pieces:
- BitBake, a generic task executor
- OpenEmbedded, a metadata set utilized by BitBake
Today, BitBake is the primary build tool of OpenEmbedded based projects, such as
the Yocto Project.
The BitBake documentation can be found :doc:`here <bitbake:index>`.

20
sources/poky/documentation/boilerplate.rst Normal file
View File

@@ -0,0 +1,20 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. include:: <xhtml1-lat1.txt>
.. include:: <xhtml1-symbol.txt>
----
| |project_name|
| <docs@lists.yoctoproject.org>
Permission is granted to copy, distribute and/or modify this document under the
terms of the `Creative Commons Attribution-Share Alike 2.0 UK: England & Wales
<https://creativecommons.org/licenses/by-sa/2.0/uk/>`__ as published by Creative
Commons.
To report any inaccuracies or problems with this (or any other Yocto Project)
manual, or to send additions or changes, please send email/patches to the Yocto
Project documentation mailing list at ``docs@lists.yoctoproject.org`` or
log into the `Libera Chat <https://libera.chat/>`__ ``#yocto`` channel.

BIN
sources/poky/documentation/brief-yoctoprojectqs/figures/bypqs-title.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

BIN
sources/poky/documentation/brief-yoctoprojectqs/figures/yocto-project-transp.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.4 KiB

455
sources/poky/documentation/brief-yoctoprojectqs/index.rst Normal file
View File

@@ -0,0 +1,455 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=========================
Yocto Project Quick Build
=========================
Welcome!
========
This short document steps you through the process for a typical
image build using the Yocto Project. The document also introduces how to
configure a build for specific hardware. You will use Yocto Project to
build a reference embedded OS called Poky.
.. note::
- The examples in this paper assume you are using a native Linux
system running a recent Ubuntu Linux distribution. If the machine
you want to use Yocto Project on to build an image
(:term:`Build Host`) is not
a native Linux system, you can still perform these steps by using
CROss PlatformS (CROPS) and setting up a Poky container. See the
:ref:`dev-manual/start:setting up to use cross platforms (crops)`
section
in the Yocto Project Development Tasks Manual for more
information.
- You may use version 2 of Windows Subsystem For Linux (WSL 2) to set
up a build host using Windows 10 or later, Windows Server 2019 or later.
See the :ref:`dev-manual/start:setting up to use windows subsystem for
linux (wsl 2)` section in the Yocto Project Development Tasks Manual
for more information.
If you want more conceptual or background information on the Yocto
Project, see the :doc:`/overview-manual/index`.
Compatible Linux Distribution
=============================
Make sure your :term:`Build Host` meets the
following requirements:
- At least &MIN_DISK_SPACE; Gbytes of free disk space, though
much more will help to run multiple builds and increase
performance by reusing build artifacts.
- At least &MIN_RAM; Gbytes of RAM, though a modern build host with as
much RAM and as many CPU cores as possible is strongly recommended to
maximize build performance.
- Runs a supported Linux distribution (i.e. recent releases of Fedora,
openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux
distributions that support the Yocto Project, see the
:ref:`ref-manual/system-requirements:supported linux distributions`
section in the Yocto Project Reference Manual. For detailed
information on preparing your build host, see the
:ref:`dev-manual/start:preparing the build host`
section in the Yocto Project Development Tasks Manual.
- Ensure that the following utilities have these minimum version numbers:
- Git &MIN_GIT_VERSION; or greater
- tar &MIN_TAR_VERSION; or greater
- Python &MIN_PYTHON_VERSION; or greater.
- gcc &MIN_GCC_VERSION; or greater.
- GNU make &MIN_MAKE_VERSION; or greater
If your build host does not satisfy all of the above version
requirements, you can take steps to prepare the system so that you
can still use the Yocto Project. See the
:ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`
section in the Yocto Project Reference Manual for information.
Build Host Packages
===================
You must install essential host packages on your build host. The
following command installs the host packages based on an Ubuntu
distribution::
$ sudo apt install &UBUNTU_DEBIAN_HOST_PACKAGES_ESSENTIAL;
.. note::
For host package requirements on all supported Linux distributions,
see the :ref:`ref-manual/system-requirements:required packages for the build host`
section in the Yocto Project Reference Manual.
Use Git to Clone Poky
=====================
Once you complete the setup instructions for your machine, you need to
get a copy of the Poky repository on your build host. Use the following
commands to clone the Poky repository.
.. code-block:: shell
$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Counting
objects: 432160, done. remote: Compressing objects: 100%
(102056/102056), done. remote: Total 432160 (delta 323116), reused
432037 (delta 323000) Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
Resolving deltas: 100% (323116/323116), done.
Checking connectivity... done.
Go to :yocto_wiki:`Releases wiki page </Releases>`, and choose a release
codename (such as ``&DISTRO_NAME_NO_CAP;``), corresponding to either the
latest stable release or a Long Term Support release.
Then move to the ``poky`` directory and take a look at existing branches:
.. code-block:: shell
$ cd poky
$ git branch -a
.
.
.
remotes/origin/HEAD -> origin/master
remotes/origin/dunfell
remotes/origin/dunfell-next
.
.
.
remotes/origin/gatesgarth
remotes/origin/gatesgarth-next
.
.
.
remotes/origin/master
remotes/origin/master-next
.
.
.
For this example, check out the ``&DISTRO_NAME_NO_CAP;`` branch based on the
``&DISTRO_NAME;`` release:
.. code-block:: shell
$ git checkout -t origin/&DISTRO_NAME_NO_CAP; -b my-&DISTRO_NAME_NO_CAP;
Branch 'my-&DISTRO_NAME_NO_CAP;' set up to track remote branch '&DISTRO_NAME_NO_CAP;' from 'origin'.
Switched to a new branch 'my-&DISTRO_NAME_NO_CAP;'
The previous Git checkout command creates a local branch named
``my-&DISTRO_NAME_NO_CAP;``. The files available to you in that branch
exactly match the repository's files in the ``&DISTRO_NAME_NO_CAP;``
release branch.
Note that you can regularly type the following command in the same directory
to keep your local files in sync with the release branch:
.. code-block:: shell
$ git pull
For more options and information about accessing Yocto Project related
repositories, see the
:ref:`dev-manual/start:locating yocto project source files`
section in the Yocto Project Development Tasks Manual.
Building Your Image
===================
Use the following steps to build your image. The build process creates
an entire Linux distribution, including the toolchain, from source.
.. note::
- If you are working behind a firewall and your build host is not
set up for proxies, you could encounter problems with the build
process when fetching source code (e.g. fetcher failures or Git
failures).
- If you do not know your proxy settings, consult your local network
infrastructure resources and get that information. A good starting
point could also be to check your web browser settings. Finally,
you can find more information on the
":yocto_wiki:`Working Behind a Network Proxy </Working_Behind_a_Network_Proxy>`"
page of the Yocto Project Wiki.
#. **Initialize the Build Environment:** From within the ``poky``
directory, run the :ref:`ref-manual/structure:``oe-init-build-env```
environment
setup script to define Yocto Project's build environment on your
build host.
.. code-block:: shell
$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
Among other things, the script creates the :term:`Build Directory`, which is
``build`` in this case and is located in the :term:`Source Directory`. After
the script runs, your current working directory is set to the
:term:`Build Directory`. Later, when the build completes, the
:term:`Build Directory` contains all the files created during the build.
#. **Examine Your Local Configuration File:** When you set up the build
environment, a local configuration file named ``local.conf`` becomes
available in a ``conf`` subdirectory of the :term:`Build Directory`. For this
example, the defaults are set to build for a ``qemux86`` target,
which is suitable for emulation. The package manager used is set to
the RPM package manager.
.. tip::
You can significantly speed up your build and guard against fetcher
failures by using :ref:`overview-manual/concepts:shared state cache`
mirrors and enabling :ref:`overview-manual/concepts:hash equivalence`.
This way, you can use pre-built artifacts rather than building them.
This is relevant only when your network and the server that you use
can download these artifacts faster than you would be able to build them.
To use such mirrors, uncomment the below lines in your ``conf/local.conf``
file in the :term:`Build Directory`::
BB_HASHSERVE_UPSTREAM = "wss://hashserv.yoctoproject.org/ws"
SSTATE_MIRRORS ?= "file://.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"
BB_HASHSERVE = "auto"
BB_SIGNATURE_HANDLER = "OEEquivHash"
The hash equivalence server needs the websockets python module version 9.1
or later. Debian GNU/Linux 12 (Bookworm) and later, Fedora, CentOS Stream
9 and later, and Ubuntu 22.04 (LTS) and later, all have a recent enough
package. Other supported distributions need to get the module some other
place than their package feed, e.g. via ``pip``.
#. **Start the Build:** Continue with the following command to build an OS
image for the target, which is ``core-image-sato`` in this example:
.. code-block:: shell
$ bitbake core-image-sato
For information on using the ``bitbake`` command, see the
:ref:`overview-manual/concepts:bitbake` section in the Yocto Project Overview and
Concepts Manual, or see
:ref:`bitbake-user-manual/bitbake-user-manual-intro:the bitbake command`
in the BitBake User Manual.
#. **Simulate Your Image Using QEMU:** Once this particular image is
built, you can start QEMU, which is a Quick EMUlator that ships with
the Yocto Project:
.. code-block:: shell
$ runqemu qemux86-64
If you want to learn more about running QEMU, see the
:ref:`dev-manual/qemu:using the quick emulator (qemu)` chapter in
the Yocto Project Development Tasks Manual.
#. **Exit QEMU:** Exit QEMU by either clicking on the shutdown icon or by typing
``Ctrl-C`` in the QEMU transcript window from which you evoked QEMU.
Customizing Your Build for Specific Hardware
============================================
So far, all you have done is quickly built an image suitable for
emulation only. This section shows you how to customize your build for
specific hardware by adding a hardware layer into the Yocto Project
development environment.
In general, layers are repositories that contain related sets of
instructions and configurations that tell the Yocto Project what to do.
Isolating related metadata into functionally specific layers facilitates
modular development and makes it easier to reuse the layer metadata.
.. note::
By convention, layer names start with the string "meta-".
Follow these steps to add a hardware layer:
#. **Find a Layer:** Many hardware layers are available. The Yocto Project
:yocto_git:`Source Repositories <>` has many hardware layers.
This example adds the
`meta-altera <https://github.com/kraj/meta-altera>`__ hardware layer.
#. **Clone the Layer:** Use Git to make a local copy of the layer on your
machine. You can put the copy in the top level of the copy of the
Poky repository created earlier:
.. code-block:: shell
$ cd poky
$ git clone https://github.com/kraj/meta-altera.git
Cloning into 'meta-altera'...
remote: Counting objects: 25170, done.
remote: Compressing objects: 100% (350/350), done.
remote: Total 25170 (delta 645), reused 719 (delta 538), pack-reused 24219
Receiving objects: 100% (25170/25170), 41.02 MiB | 1.64 MiB/s, done.
Resolving deltas: 100% (13385/13385), done.
Checking connectivity... done.
The hardware layer is now available
next to other layers inside the Poky reference repository on your build
host as ``meta-altera`` and contains all the metadata needed to
support hardware from Altera, which is owned by Intel.
.. note::
It is recommended for layers to have a branch per Yocto Project release.
Please make sure to checkout the layer branch supporting the Yocto Project
release you're using.
#. **Change the Configuration to Build for a Specific Machine:** The
:term:`MACHINE` variable in the
``local.conf`` file specifies the machine for the build. For this
example, set the :term:`MACHINE` variable to ``cyclone5``. These
configurations are used:
https://github.com/kraj/meta-altera/blob/master/conf/machine/cyclone5.conf.
.. note::
See the "Examine Your Local Configuration File" step earlier for more
information on configuring the build.
#. **Add Your Layer to the Layer Configuration File:** Before you can use
a layer during a build, you must add it to your ``bblayers.conf``
file, which is found in the :term:`Build Directory` ``conf`` directory.
Use the ``bitbake-layers add-layer`` command to add the layer to the
configuration file:
.. code-block:: shell
$ cd poky/build
$ bitbake-layers add-layer ../meta-altera
NOTE: Starting bitbake server...
Parsing recipes: 100% |##################################################################| Time: 0:00:32
Parsing of 918 .bb files complete (0 cached, 918 parsed). 1401 targets,
123 skipped, 0 masked, 0 errors.
You can find
more information on adding layers in the
:ref:`dev-manual/layers:adding a layer using the \`\`bitbake-layers\`\` script`
section.
Completing these steps has added the ``meta-altera`` layer to your Yocto
Project development environment and configured it to build for the
``cyclone5`` machine.
.. note::
The previous steps are for demonstration purposes only. If you were
to attempt to build an image for the ``cyclone5`` machine, you should
read the Altera ``README``.
Creating Your Own General Layer
===============================
Maybe you have an application or specific set of behaviors you need to
isolate. You can create your own general layer using the
``bitbake-layers create-layer`` command. The tool automates layer
creation by setting up a subdirectory with a ``layer.conf``
configuration file, a ``recipes-example`` subdirectory that contains an
``example.bb`` recipe, a licensing file, and a ``README``.
The following commands run the tool to create a layer named
``meta-mylayer`` in the ``poky`` directory:
.. code-block:: shell
$ cd poky
$ bitbake-layers create-layer meta-mylayer
NOTE: Starting bitbake server...
Add your new layer with 'bitbake-layers add-layer meta-mylayer'
For more information
on layers and how to create them, see the
:ref:`dev-manual/layers:creating a general layer using the \`\`bitbake-layers\`\` script`
section in the Yocto Project Development Tasks Manual.
Where To Go Next
================
Now that you have experienced using the Yocto Project, you might be
asking yourself "What now?". The Yocto Project has many sources of
information including the website, wiki pages, and user manuals:
- **Website:** The :yocto_home:`Yocto Project Website <>` provides
background information, the latest builds, breaking news, full
development documentation, and access to a rich Yocto Project
Development Community into which you can tap.
- **Video Seminar:** The `Introduction to the Yocto Project and BitBake, Part 1
<https://youtu.be/yuE7my3KOpo>`__ and
`Introduction to the Yocto Project and BitBake, Part 2
<https://youtu.be/iZ05TTyzGHk>`__ videos offer a video seminar
introducing you to the most important aspects of developing a
custom embedded Linux distribution with the Yocto Project.
- **Yocto Project Overview and Concepts Manual:** The
:doc:`/overview-manual/index` is a great
place to start to learn about the Yocto Project. This manual
introduces you to the Yocto Project and its development environment.
The manual also provides conceptual information for various aspects
of the Yocto Project.
- **Yocto Project Wiki:** The :yocto_wiki:`Yocto Project Wiki <>`
provides additional information on where to go next when ramping up
with the Yocto Project, release information, project planning, and QA
information.
- **Yocto Project Mailing Lists:** Related mailing lists provide a forum
for discussion, patch submission and announcements. There are several
mailing lists grouped by topic. See the
:ref:`ref-manual/resources:mailing lists`
section in the Yocto Project Reference Manual for a complete list of
Yocto Project mailing lists.
- **Comprehensive List of Links and Other Documentation:** The
:ref:`ref-manual/resources:links and related documentation`
section in the Yocto Project Reference Manual provides a
comprehensive list of all related links and other user documentation.
.. include:: /boilerplate.rst

1491
sources/poky/documentation/bsp-guide/bsp.rst Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
sources/poky/documentation/bsp-guide/figures/bsp-dev-flow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

BIN
sources/poky/documentation/bsp-guide/figures/bsp-title.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

15
sources/poky/documentation/bsp-guide/index.rst Normal file
View File

@@ -0,0 +1,15 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====================================================
Yocto Project Board Support Package Developer's Guide
=====================================================
|
.. toctree::
:caption: Table of Contents
:numbered:
bsp
.. include:: /boilerplate.rst

197
sources/poky/documentation/conf.py Normal file
View File

@@ -0,0 +1,197 @@
# Configuration file for the Sphinx documentation builder.
#
# SPDX-License-Identifier: CC-BY-SA-2.0-UK
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import re
import sys
import datetime
try:
import yaml
except ImportError:
sys.stderr.write("The Yocto Project Sphinx documentation requires PyYAML.\
\nPlease make sure to install pyyaml Python package.\n")
sys.exit(1)
# current_version = "dev"
# bitbake_version = "" # Leave empty for development branch
# Obtain versions from poky.yaml instead
with open("poky.yaml") as data:
buff = data.read()
subst_vars = yaml.safe_load(buff)
if "DOCCONF_VERSION" not in subst_vars:
sys.stderr.write("Please set DOCCONF_VERSION in poky.yaml")
sys.exit(1)
current_version = subst_vars["DOCCONF_VERSION"]
if "BITBAKE_SERIES" not in subst_vars:
sys.stderr.write("Please set BITBAKE_SERIES in poky.yaml")
sys.exit(1)
bitbake_version = subst_vars["BITBAKE_SERIES"]
# String used in sidebar
version = 'Version: ' + current_version
if current_version == 'dev':
version = 'Version: Current Development'
# Version seen in documentation_options.js and hence in js switchers code
release = current_version
# -- Project information -----------------------------------------------------
project = 'The Yocto Project \xae'
copyright = '2010-%s, The Linux Foundation, CC-BY-SA-2.0-UK license' % datetime.datetime.now().year
author = 'The Linux Foundation'
# -- General configuration ---------------------------------------------------
# Prevent building with an outdated version of sphinx
needs_sphinx = "4.0"
# to load local extension from the folder 'sphinx'
sys.path.insert(0, os.path.abspath('sphinx'))
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autosectionlabel',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'yocto-vars'
]
autosectionlabel_prefix_document = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'boilerplate.rst']
# master document name. The default changed from contents to index. so better
# set it ourselves.
master_doc = 'index'
# create substitution for project configuration variables
rst_prolog = """
.. |project_name| replace:: %s
.. |copyright| replace:: %s
.. |author| replace:: %s
""" % (project, copyright, author)
# external links and substitutions
extlinks = {
'bitbake_git': ('https://git.openembedded.org/bitbake%s', None),
'cve_mitre': ('https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-%s', 'CVE-%s'),
'cve_nist': ('https://nvd.nist.gov/vuln/detail/CVE-%s', 'CVE-%s'),
'yocto_home': ('https://www.yoctoproject.org%s', None),
'yocto_wiki': ('https://wiki.yoctoproject.org/wiki%s', None),
'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
'yocto_lists': ('https://lists.yoctoproject.org%s', None),
'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
'yocto_docs': ('https://docs.yoctoproject.org%s', None),
'yocto_git': ('https://git.yoctoproject.org%s', None),
'yocto_sstate': ('http://sstate.yoctoproject.org%s', None),
'oe_home': ('https://www.openembedded.org%s', None),
'oe_lists': ('https://lists.openembedded.org%s', None),
'oe_git': ('https://git.openembedded.org%s', None),
'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
'oe_layerindex': ('https://layers.openembedded.org%s', None),
'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
'wikipedia': ('https://en.wikipedia.org/wiki/%s', None),
}
# To be able to use :manpage:`<something>` in the docs.
manpages_url = 'https://manpages.debian.org/{path}'
# Intersphinx config to use cross reference with BitBake user manual
intersphinx_mapping = {
'bitbake': ('https://docs.yoctoproject.org/bitbake/' + bitbake_version, None)
}
# Suppress "WARNING: unknown mimetype for ..."
suppress_warnings = ['epub.unknown_project_files']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
try:
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_options = {
'sticky_navigation': False,
}
except ImportError:
sys.stderr.write("The Sphinx sphinx_rtd_theme HTML theme was not found.\
\nPlease make sure to install the sphinx_rtd_theme Python package.\n")
sys.exit(1)
html_logo = 'sphinx-static/YoctoProject_Logo_RGB.jpg'
html_favicon = 'sphinx-static/favicon.ico'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['sphinx-static']
html_context = {
'current_version': current_version,
}
# Add customm CSS and JS files
html_css_files = ['theme_overrides.css']
html_js_files = ['switchers.js']
# Hide 'Created using Sphinx' text
html_show_sphinx = False
# Add 'Last updated' on each page
html_last_updated_fmt = '%b %d, %Y'
# Remove the trailing 'dot' in section numbers
html_secnumber_suffix = " "
# We need XeTeX to process special unicode character, sometimes the contributor
# list from the release note contains those.
# See https://docs.readthedocs.io/en/stable/guides/pdf-non-ascii-languages.html.
latex_engine = 'xelatex'
latex_use_xindy = False
latex_elements = {
'passoptionstopackages': '\\PassOptionsToPackage{bookmarksdepth=5}{hyperref}',
'preamble': '\\usepackage[UTF8]{ctex}\n\\setcounter{tocdepth}{2}',
}
from sphinx.search import SearchEnglish
from sphinx.search import languages
class DashFriendlySearchEnglish(SearchEnglish):
# Accept words that can include 'inner' hyphens or dots
_word_re = re.compile(r'[\w]+(?:[\.\-][\w]+)*')
js_splitter_code = r"""
function splitQuery(query) {
return query
.split(/[^\p{Letter}\p{Number}_\p{Emoji_Presentation}\-\.]+/gu)
.filter(term => term.length > 0);
}
"""
languages['en'] = DashFriendlySearchEnglish
# Make the EPUB builder prefer PNG to SVG because of issues rendering Inkscape SVG
from sphinx.builders.epub3 import Epub3Builder
Epub3Builder.supported_image_types = ['image/png', 'image/gif', 'image/jpeg']

31
sources/poky/documentation/contributor-guide/identify-component.rst Normal file
View File

@@ -0,0 +1,31 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Identify the component
**********************
The Yocto Project and OpenEmbedded ecosystem is built of :term:`layers <Layer>`
so the first step is to identify the component where the issue likely lies.
For example, if you have a hardware issue, it is likely related to the BSP
you are using and the best place to seek advice would be from the BSP provider
or :term:`layer`. If the issue is a build/configuration one and a distro is in
use, they would likely be the first place to ask questions. If the issue is a
generic one and/or in the core classes or metadata, the core layer or BitBake
might be the appropriate component.
Each metadata layer being used should contain a ``README`` file and that should
explain where to report issues, where to send changes and how to contact the
maintainers.
If the issue is in the core metadata layer (OpenEmbedded-Core) or in BitBake,
issues can be reported in the :yocto_bugs:`Yocto Project Bugzilla <>`. The
:yocto_lists:`yocto </g/yocto>` mailing list is a general “catch-all” location
where questions can be sent if you can’t work out where something should go.
:term:`Poky` is a commonly used “combination” repository where multiple
components have been combined (:oe_git:`bitbake </bitbake>`,
:oe_git:`openembedded-core </openembedded-core>`,
:yocto_git:`meta-yocto </meta-yocto>` and
:yocto_git:`yocto-docs </yocto-docs>`). Patches should be submitted against the
appropriate individual component rather than :term:`Poky` itself as detailed in
the appropriate ``README`` file.

26
sources/poky/documentation/contributor-guide/index.rst Normal file
View File

@@ -0,0 +1,26 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
================================================
Yocto Project and OpenEmbedded Contributor Guide
================================================
The Yocto Project and OpenEmbedded are open-source, community-based projects so
contributions are very welcome, it is how the code evolves and everyone can
effect change. Contributions take different forms, if you have a fix for an
issue you’ve run into, a patch is the most appropriate way to contribute it.
If you run into an issue but don’t have a solution, opening a defect in
:yocto_bugs:`Bugzilla <>` or asking questions on the mailing lists might be
more appropriate. This guide intends to point you in the right direction to
this.
.. toctree::
:caption: Table of Contents
:numbered:
identify-component
report-defect
recipe-style-guide
submit-changes
.. include:: /boilerplate.rst

411
sources/poky/documentation/contributor-guide/recipe-style-guide.rst Normal file
View File

@@ -0,0 +1,411 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Recipe Style Guide
******************
Recipe Naming Conventions
=========================
In general, most recipes should follow the naming convention
``recipes-category/recipename/recipename_version.bb``. Recipes for related
projects may share the same recipe directory. ``recipename`` and ``category``
may contain hyphens, but hyphens are not allowed in ``version``.
If the recipe is tracking a Git revision that does not correspond to a released
version of the software, ``version`` may be ``git`` (e.g. ``recipename_git.bb``)
and the recipe would set :term:`PV`.
Version Policy
==============
Our versions follow the form ``<epoch>:<version>-<revision>``
or in BitBake variable terms ${:term:`PE`}:${:term:`PV`}-${:term:`PR`}. We
generally follow the `Debian <https://www.debian.org/doc/debian-policy/ch-controlfields.html#version>`__
version policy which defines these terms.
In most cases the version :term:`PV` will be set automatically from the recipe
file name. It is recommended to use released versions of software as these are
revisions that upstream are expecting people to use.
Recipe versions should always compare and sort correctly so that upgrades work
as expected. With conventional versions such as ``1.4`` upgrading ``to 1.5``
this happens naturally, but some versions don't sort. For example,
``1.5 Release Candidate 2`` could be written as ``1.5rc2`` but this sorts after
``1.5``, so upgrades from feeds won't happen correctly.
Instead the tilde (``~``) operator can be used, which sorts before the empty
string so ``1.5~rc2`` comes before ``1.5``. There is a historical syntax which
may be found where :term:`PV` is set as a combination of the prior version
``+`` the pre-release version, for example ``PV=1.4+1.5rc2``. This is a valid
syntax but the tilde form is preferred.
For version comparisons, the ``opkg-compare-versions`` program from
``opkg-utils`` can be useful when attempting to determine how two version
numbers compare to each other. Our definitive version comparison algorithm is
the one within bitbake which aims to match those of the package managers and
Debian policy closely.
When a recipe references a git revision that does not correspond to a released
version of software (e.g. is not a tagged version), the :term:`PV` variable
should include the sign ``+``, so :term:`bitbake` automatically includes package
version information during the packaging phase::
PV = "<version>+git"
In this case, ``<version>`` should be the most recently released version of the
software from the current source revision (``git describe`` can be useful for
determining this). Whilst not recommended for published layers, this format is
also useful when using :term:`AUTOREV` to set the recipe to increment source
control revisions automatically, which can be useful during local development.
Version Number Changes
======================
The :term:`PR` variable is used to indicate different revisions of a recipe
that reference the same upstream source version. It can be used to force a
new version of a recipe to be installed onto a device from a package feed.
These once had to be set manually but in most cases these can now be set and
incremented automatically by a PR Server connected with a package feed.
When :term:`PV` increases, any existing :term:`PR` value can and should be
removed.
If :term:`PV` changes in such a way that it does not increase with respect to
the previous value, you need to increase :term:`PE` to ensure package managers
will upgrade it correctly. If unset you should set :term:`PE` to "1" since
the default of empty is easily confused with "0" depending on the package
manager. :term:`PE` can only have an integer value.
Recipe formatting
=================
Variable Formatting
-------------------
- Variable assignment should a space around each side of the operator, e.g.
``FOO = "bar"``, not ``FOO="bar"``.
- Double quotes should be used on the right-hand side of the assignment,
e.g. ``FOO = "bar"`` not ``FOO = 'bar'``
- Spaces should be used for indenting variables, with 4 spaces per tab
- Long variables should be split over multiple lines when possible by using
the continuation character (``\``)
- When splitting a long variable over multiple lines, all continuation lines
should be indented (with spaces) to align with the start of the quote on the
first line::
FOO = "this line is \
long \
"
Instead of::
FOO = "this line is \
long \
"
Python Function formatting
--------------------------
- Spaces must be used for indenting Python code, with 4 spaces per tab
Shell Function formatting
-------------------------
- The formatting of shell functions should be consistent within layers.
Some use tabs, some use spaces.
Recipe metadata
===============
Required Variables
------------------
The following variables should be included in all recipes:
- :term:`SUMMARY`: a one line description of the upstream project
- :term:`DESCRIPTION`: an extended description of the upstream project,
possibly with multiple lines. If no reasonable description can be written,
this may be omitted as it defaults to :term:`SUMMARY`.
- :term:`HOMEPAGE`: the URL to the upstream projects homepage.
- :term:`BUGTRACKER`: the URL upstream projects bug tracking website,
if applicable.
Recipe Ordering
---------------
When a variable is defined in recipes and classes, variables should follow the
general order when possible:
- :term:`SUMMARY`
- :term:`DESCRIPTION`
- :term:`HOMEPAGE`
- :term:`BUGTRACKER`
- :term:`SECTION`
- :term:`LICENSE`
- :term:`LIC_FILES_CHKSUM`
- :term:`DEPENDS`
- :term:`PROVIDES`
- :term:`PV`
- :term:`SRC_URI`
- :term:`SRCREV`
- :term:`S`
- ``inherit ...``
- :term:`PACKAGECONFIG`
- Build class specific variables such as ``EXTRA_QMAKEVARS_POST`` and :term:`EXTRA_OECONF`
- Tasks such as :ref:`ref-tasks-configure`
- :term:`PACKAGE_ARCH`
- :term:`PACKAGES`
- :term:`FILES`
- :term:`RDEPENDS`
- :term:`RRECOMMENDS`
- :term:`RSUGGESTS`
- :term:`RPROVIDES`
- :term:`RCONFLICTS`
- :term:`BBCLASSEXTEND`
There are some cases where ordering is important and these cases would override
this default order. Examples include:
- :term:`PACKAGE_ARCH` needing to be set before ``inherit packagegroup``
Tasks should be ordered based on the order they generally execute. For commonly
used tasks this would be:
- :ref:`ref-tasks-fetch`
- :ref:`ref-tasks-unpack`
- :ref:`ref-tasks-patch`
- :ref:`ref-tasks-prepare_recipe_sysroot`
- :ref:`ref-tasks-configure`
- :ref:`ref-tasks-compile`
- :ref:`ref-tasks-install`
- :ref:`ref-tasks-populate_sysroot`
- :ref:`ref-tasks-package`
Custom tasks should be sorted similarly.
Package specific variables are typically grouped together, e.g.::
RDEPENDS:${PN} = “foo”
RDEPENDS:${PN}-libs = “bar”
RRECOMMENDS:${PN} = “one”
RRECOMMENDS:${PN}-libs = “two”
Recipe License Fields
---------------------
Recipes need to define both the :term:`LICENSE` and
:term:`LIC_FILES_CHKSUM` variables:
- :term:`LICENSE`: This variable specifies the license for the software.
If you do not know the license under which the software you are
building is distributed, you should go to the source code and look
for that information. Typical files containing this information
include ``COPYING``, :term:`LICENSE`, and ``README`` files. You could
also find the information near the top of a source file. For example,
given a piece of software licensed under the GNU General Public
License version 2, you would set :term:`LICENSE` as follows::
LICENSE = "GPL-2.0-only"
The licenses you specify within :term:`LICENSE` can have any name as long
as you do not use spaces, since spaces are used as separators between
license names. For standard licenses, use the names of the files in
``meta/files/common-licenses/`` or the :term:`SPDXLICENSEMAP` flag names
defined in ``meta/conf/licenses.conf``.
- :term:`LIC_FILES_CHKSUM`: The OpenEmbedded build system uses this
variable to make sure the license text has not changed. If it has,
the build produces an error and it affords you the chance to figure
it out and correct the problem.
You need to specify all applicable licensing files for the software.
At the end of the configuration step, the build process will compare
the checksums of the files to be sure the text has not changed. Any
differences result in an error with the message containing the
current checksum. For more explanation and examples of how to set the
:term:`LIC_FILES_CHKSUM` variable, see the
":ref:`dev-manual/licenses:tracking license changes`" section.
To determine the correct checksum string, you can list the
appropriate files in the :term:`LIC_FILES_CHKSUM` variable with incorrect
md5 strings, attempt to build the software, and then note the
resulting error messages that will report the correct md5 strings.
See the ":ref:`dev-manual/new-recipe:fetching code`" section for
additional information.
Here is an example that assumes the software has a ``COPYING`` file::
LIC_FILES_CHKSUM = "file://COPYING;md5=xxx"
When you try to build the
software, the build system will produce an error and give you the
correct string that you can substitute into the recipe file for a
subsequent build.
License Updates
~~~~~~~~~~~~~~~
When you change the :term:`LICENSE` or :term:`LIC_FILES_CHKSUM` in the recipe
you need to briefly explain the reason for the change via a ``License-Update:``
tag. Often it's quite trivial, such as::
License-Update: copyright years refreshed
Less often, the actual licensing terms themselves will have changed. If so, do
try to link to upstream making/justifying that decision.
Tips and Guidelines for Writing Recipes
---------------------------------------
- Use :term:`BBCLASSEXTEND` instead of creating separate recipes such as ``-native``
and ``-nativesdk`` ones, whenever possible. This avoids having to maintain multiple
recipe files at the same time.
- Recipes should have tasks which are idempotent, i.e. that executing a given task
multiple times shouldn't change the end result. The build environment is built upon
this assumption and breaking it can cause obscure build failures.
- For idempotence when modifying files in tasks, it is usually best to:
- copy a file ``X`` to ``X.orig`` (only if it doesn't exist already)
- then, copy ``X.orig`` back to ``X``,
- and, finally, modify ``X``.
This ensures if rerun the task always has the same end result and the
original file can be preserved to reuse. It also guards against an
interrupted build corrupting the file.
Patch Upstream Status
=====================
In order to keep track of patches applied by recipes and ultimately reduce the
number of patches that need maintaining, the OpenEmbedded build system
requires information about the upstream status of each patch.
In its description, each patch should provide detailed information about the
bug that it addresses, such as the URL in a bug tracking system and links
to relevant mailing list archives.
Then, you should also add an ``Upstream-Status:`` tag containing one of the
following status strings:
``Pending``
No determination has been made yet, or patch has not yet been submitted to
upstream.
Keep in mind that every patch submitted upstream reduces the maintainance
burden in OpenEmbedded and Yocto Project in the long run, so this patch
status should only be used in exceptional cases if there are genuine
obstacles to submitting a patch upstream; the reason for that should be
included in the patch.
``Submitted [where]``
Submitted to upstream, waiting for approval. Optionally include where
it was submitted, such as the author, mailing list, etc.
``Backport [version]``
Accepted upstream and included in the next release, or backported from newer
upstream version, because we are at a fixed version.
Include upstream version info (e.g. commit ID or next expected version).
``Denied``
Not accepted by upstream, include reason in patch.
``Inactive-Upstream [lastcommit: when (and/or) lastrelease: when]``
The upstream is no longer available. This typically means a defunct project
where no activity has happened for a long time --- measured in years. To make
that judgement, it is recommended to look at not only when the last release
happened, but also when the last commit happened, and whether newly made bug
reports and merge requests since that time receive no reaction. It is also
recommended to add to the patch description any relevant links where the
inactivity can be clearly seen.
``Inappropriate [reason]``
The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with ``[]``. In the past, there were several different
reasons not to submit patches upstream, but we have to consider that every
non-upstreamed patch means a maintainance burden for recipe maintainers.
Currently, the only reasons to mark patches as inappropriate for upstream
submission are:
- ``oe specific``: the issue is specific to how OpenEmbedded performs builds
or sets things up at runtime, and can be resolved only with a patch that
is not however relevant or appropriate for general upstream submission.
- ``upstream ticket <link>``: the issue is not specific to Open-Embedded
and should be fixed upstream, but the patch in its current form is not
suitable for merging upstream, and the author lacks sufficient expertise
to develop a proper patch. Instead the issue is handled via a bug report
(include link).
Of course, if another person later takes care of submitting this patch upstream,
the status should be changed to ``Submitted [where]``, and an additional
``Signed-off-by:`` line should be added to the patch by the person claiming
responsibility for upstreaming.
Examples
--------
Here's an example of a patch that has been submitted upstream::
rpm: Adjusted the foo setting in bar
[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5
The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.
Upstream-Status: Submitted [rpm5-devel@rpm5.org]
Signed-off-by: Joe Developer <joe.developer@example.com>
A future update can change the value to ``Backport`` or ``Denied`` as
appropriate.
Another example of a patch that is specific to OpenEmbedded::
Do not treat warnings as errors
There are additional warnings found with musl which are
treated as errors and fails the build, we have more combinations
than upstream supports to handle.
Upstream-Status: Inappropriate [oe specific]
Here's a patch that has been backported from an upstream commit::
include missing sys/file.h for LOCK_EX
Upstream-Status: Backport [https://github.com/systemd/systemd/commit/ac8db36cbc26694ee94beecc8dca208ec4b5fd45]
CVE patches
===========
In order to have a better control of vulnerabilities, patches that fix CVEs must
contain a ``CVE:`` tag. This tag list all CVEs fixed by the patch. If more than
one CVE is fixed, separate them using spaces.
CVE Examples
------------
This should be the header of patch that fixes :cve_nist:`2015-8370` in GRUB2::
grub2: Fix CVE-2015-8370
[No upstream tracking] -- https://bugzilla.redhat.com/show_bug.cgi?id=1286966
Back to 28; Grub2 Authentication
Two functions suffer from integer underflow fault; the grub_username_get() and grub_password_get()located in
grub-core/normal/auth.c and lib/crypto.c respectively. This can be exploited to obtain a Grub rescue shell.
Upstream-Status: Backport [http://git.savannah.gnu.org/cgit/grub.git/commit/?id=451d80e52d851432e109771bb8febafca7a5f1f2]
CVE: CVE-2015-8370
Signed-off-by: Joe Developer <joe.developer@example.com>

67
sources/poky/documentation/contributor-guide/report-defect.rst Normal file
View File

@@ -0,0 +1,67 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Reporting a Defect Against the Yocto Project and OpenEmbedded
**************************************************************
You can use the Yocto Project instance of
`Bugzilla <https://www.bugzilla.org/about/>`__ to submit a defect (bug)
against BitBake, OpenEmbedded-Core, against any other Yocto Project component
or for tool issues. For additional information on this implementation of
Bugzilla see the ":ref:`Yocto Project Bugzilla <resources-bugtracker>`" section
in the Yocto Project Reference Manual. For more detail on any of the following
steps, see the Yocto Project
:yocto_wiki:`Bugzilla wiki page </Bugzilla_Configuration_and_Bug_Tracking>`.
Use the following general steps to submit a bug:
#. Open the Yocto Project implementation of :yocto_bugs:`Bugzilla <>`.
#. Click "File a Bug" to enter a new bug.
#. Choose the appropriate "Classification", "Product", and "Component"
for which the bug was found. Bugs for the Yocto Project fall into
one of several classifications, which in turn break down into
several products and components. For example, for a bug against the
``meta-intel`` layer, you would choose "Build System, Metadata &
Runtime", "BSPs", and "bsps-meta-intel", respectively.
#. Choose the "Version" of the Yocto Project for which you found the
bug (e.g. &DISTRO;).
#. Determine and select the "Severity" of the bug. The severity
indicates how the bug impacted your work.
#. Choose the "Hardware" that the bug impacts.
#. Choose the "Architecture" that the bug impacts.
#. Choose a "Documentation change" item for the bug. Fixing a bug might
or might not affect the Yocto Project documentation. If you are
unsure of the impact to the documentation, select "Don't Know".
#. Provide a brief "Summary" of the bug. Try to limit your summary to
just a line or two and be sure to capture the essence of the bug.
#. Provide a detailed "Description" of the bug. You should provide as
much detail as you can about the context, behavior, output, and so
forth that surrounds the bug. You can even attach supporting files
for output from logs by using the "Add an attachment" button.
#. Click the "Submit Bug" button submit the bug. A new Bugzilla number
is assigned to the bug and the defect is logged in the bug tracking
system.
Once you file a bug, the bug is processed by the Yocto Project Bug
Triage Team and further details concerning the bug are assigned (e.g.
priority and owner). You are the "Submitter" of the bug and any further
categorization, progress, or comments on the bug result in Bugzilla
sending you an automated email concerning the particular change or
progress to the bug.
There are no guarantees about if or when a bug might be worked on since an
open-source project has no dedicated engineering resources. However, the
project does have a good track record of resolving common issues over the
medium and long term. We do encourage people to file bugs so issues are
at least known about. It helps other users when they find somebody having
the same issue as they do, and an issue that is unknown is much less likely
to ever be fixed!

915
sources/poky/documentation/contributor-guide/submit-changes.rst Normal file
View File

@@ -0,0 +1,915 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Contributing Changes to a Component
************************************
Contributions to the Yocto Project and OpenEmbedded are very welcome.
Because the system is extremely configurable and flexible, we recognize
that developers will want to extend, configure or optimize it for their
specific uses.
.. _ref-why-mailing-lists:
Contributing through mailing lists --- Why not using web-based workflows?
=========================================================================
Both Yocto Project and OpenEmbedded have many key components that are
maintained by patches being submitted on mailing lists. We appreciate this
approach does look a little old fashioned when other workflows are available
through web technology such as GitHub, GitLab and others. Since we are often
asked this question, we’ve decided to document the reasons for using mailing
lists.
One significant factor is that we value peer review. When a change is proposed
to many of the core pieces of the project, it helps to have many eyes of review
go over them. Whilst there is ultimately one maintainer who needs to make the
final call on accepting or rejecting a patch, the review is made by many eyes
and the exact people reviewing it are likely unknown to the maintainer. It is
often the surprise reviewer that catches the most interesting issues!
This is in contrast to the "GitHub" style workflow where either just a
maintainer makes that review, or review is specifically requested from
nominated people. We believe there is significant value added to the codebase
by this peer review and that moving away from mailing lists would be to the
detriment of our code.
We also need to acknowledge that many of our developers are used to this
mailing list workflow and have worked with it for years, with tools and
processes built around it. Changing away from this would result in a loss
of key people from the project, which would again be to its detriment.
The projects are acutely aware that potential new contributors find the
mailing list approach off-putting and would prefer a web-based GUI.
Since we don’t believe that can work for us, the project is aiming to ensure
`patchwork <https://patchwork.yoctoproject.org/>`__ is available to help track
patch status and also looking at how tooling can provide more feedback to users
about patch status. We are looking at improving tools such as ``patchtest`` to
test user contributions before they hit the mailing lists and also at better
documenting how to use such workflows since we recognise that whilst this was
common knowledge a decade ago, it might not be as familiar now.
Preparing Changes for Submission
================================
Set up Git
----------
The first thing to do is to install Git packages. Here is an example
on Debian and Ubuntu::
sudo apt install git-core git-email
Then, you need to set a name and e-mail address that Git will
use to identify your commits::
git config --global user.name "Ada Lovelace"
git config --global user.email "ada.lovelace@gmail.com"
By default, Git adds a signature line at the end of patches containing the Git
version. We suggest to remove it as it doesn't add useful information.
Remove it with the following command::
git config --global format.signature ""
Clone the Git repository for the component to modify
----------------------------------------------------
After identifying the component to modify as described in the
":doc:`../contributor-guide/identify-component`" section, clone the
corresponding Git repository. Here is an example for OpenEmbedded-Core::
git clone https://git.openembedded.org/openembedded-core
cd openembedded-core
Create a new branch
-------------------
Then, create a new branch in your local Git repository
for your changes, starting from the reference branch in the upstream
repository (often called ``master``)::
$ git checkout <ref-branch>
$ git checkout -b my-changes
If you have completely unrelated sets of changes to submit, you should even
create one branch for each set.
Implement and commit changes
----------------------------
In each branch, you should group your changes into small, controlled and
isolated ones. Keeping changes small and isolated aids review, makes
merging/rebasing easier and keeps the change history clean should anyone need
to refer to it in future.
To this purpose, you should create *one Git commit per change*,
corresponding to each of the patches you will eventually submit.
See `further guidance <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#separate-your-changes>`__
in the Linux kernel documentation if needed.
For example, when you intend to add multiple new recipes, each recipe
should be added in a separate commit. For upgrades to existing recipes,
the previous version should usually be deleted as part of the same commit
to add the upgraded version.
#. *Stage Your Changes:* Stage your changes by using the ``git add``
command on each file you modified. If you want to stage all the
files you modified, you can even use the ``git add -A`` command.
#. *Commit Your Changes:* This is when you can create separate commits. For
each commit to create, use the ``git commit -s`` command with the files
or directories you want to include in the commit::
$ git commit -s file1 file2 dir1 dir2 ...
To include **a**\ ll staged files::
$ git commit -sa
- The ``-s`` option of ``git commit`` adds a "Signed-off-by:" line
to your commit message. There is the same requirement for contributing
to the Linux kernel. Adding such a line signifies that you, the
submitter, have agreed to the `Developer's Certificate of Origin 1.1
<https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>`__
as follows:
.. code-block:: none
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
- Provide a single-line summary of the change and, if more
explanation is needed, provide more detail in the body of the
commit. This summary is typically viewable in the "shortlist" of
changes. Thus, providing something short and descriptive that
gives the reader a summary of the change is useful when viewing a
list of many commits. You should prefix this short description
with the recipe name (if changing a recipe), or else with the
short form path to the file being changed.
.. note::
To find a suitable prefix for the commit summary, a good idea
is to look for prefixes used in previous commits touching the
same files or directories::
git log --oneline <paths>
- For the body of the commit message, provide detailed information
that describes what you changed, why you made the change, and the
approach you used. It might also be helpful if you mention how you
tested the change. Provide as much detail as you can in the body
of the commit message.
.. note::
If the single line summary is enough to describe a simple
change, the body of the commit message can be left empty.
- If the change addresses a specific bug or issue that is associated
with a bug-tracking ID, include a reference to that ID in your
detailed description. For example, the Yocto Project uses a
specific convention for bug references --- any commit that addresses
a specific bug should use the following form for the detailed
description. Be sure to use the actual bug-tracking ID from
Bugzilla for bug-id::
Fixes [YOCTO #bug-id]
detailed description of change
#. *Crediting contributors:* By using the ``git commit --amend`` command,
you can add some tags to the commit description to credit other contributors
to the change:
- ``Reported-by``: name and email of a person reporting a bug
that your commit is trying to fix. This is a good practice
to encourage people to go on reporting bugs and let them
know that their reports are taken into account.
- ``Suggested-by``: name and email of a person to credit for the
idea of making the change.
- ``Tested-by``, ``Reviewed-by``: name and email for people having
tested your changes or reviewed their code. These fields are
usually added by the maintainer accepting a patch, or by
yourself if you submitted your patches to early reviewers,
or are submitting an unmodified patch again as part of a
new iteration of your patch series.
- ``CC:`` Name and email of people you want to send a copy
of your changes to. This field will be used by ``git send-email``.
See `more guidance about using such tags
<https://www.kernel.org/doc/html/latest/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes>`__
in the Linux kernel documentation.
Test your changes
-----------------
For each contributions you make, you should test your changes as well.
For this the Yocto Project offers several types of tests. Those tests cover
different areas and it depends on your changes which are feasible. For example run:
- For changes that affect the build environment:
- ``bitbake-selftest``: for changes within BitBake
- ``oe-selftest``: to test combinations of BitBake runs
- ``oe-build-perf-test``: to test the performance of common build scenarios
- For changes in a recipe:
- ``ptest``: run package specific tests, if they exist
- ``testimage``: build an image, boot it and run testcases on it
- If applicable, ensure also the ``native`` and ``nativesdk`` variants builds
- For changes relating to the SDK:
- ``testsdk``: to build, install and run tests against a SDK
- ``testsdk_ext``: to build, install and run tests against an extended SDK
Note that this list just gives suggestions and is not exhaustive. More details can
be found here: :ref:`test-manual/intro:Yocto Project Tests --- Types of Testing Overview`.
Creating Patches
================
Here is the general procedure on how to create patches to be sent through email:
#. *Describe the Changes in your Branch:* If you have more than one commit
in your branch, it's recommended to provide a cover letter describing
the series of patches you are about to send.
For this purpose, a good solution is to store the cover letter contents
in the branch itself::
git branch --edit-description
This will open a text editor to fill in the description for your
changes. This description can be updated when necessary and will
be used by Git to create the cover letter together with the patches.
It is recommended to start this description with a title line which
will serve a the subject line for the cover letter.
#. *Generate Patches for your Branch:* The ``git format-patch`` command will
generate patch files for each of the commits in your branch. You need
to pass the reference branch your branch starts from.
If you branch didn't need a description in the previous step::
$ git format-patch <ref-branch>
If you filled a description for your branch, you will want to generate
a cover letter too::
$ git format-patch --cover-letter --cover-from-description=auto <ref-branch>
After the command is run, the current directory contains numbered
``.patch`` files for the commits in your branch. If you have a cover
letter, it will be in the ``0000-cover-letter.patch``.
.. note::
The ``--cover-from-description=auto`` option makes ``git format-patch``
use the first paragraph of the branch description as the cover
letter title. Another possibility, which is easier to remember, is to pass
only the ``--cover-letter`` option, but you will have to edit the
subject line manually every time you generate the patches.
See the `git format-patch manual page <https://git-scm.com/docs/git-format-patch>`__
for details.
#. *Review each of the Patch Files:* This final review of the patches
before sending them often allows to view your changes from a different
perspective and discover defects such as typos, spacing issues or lines
or even files that you didn't intend to modify. This review should
include the cover letter patch too.
If necessary, rework your commits as described in
":ref:`contributor-guide/submit-changes:taking patch review into account`".
Validating Patches with Patchtest
=================================
``patchtest`` is available in ``openembedded-core`` as a tool for making
sure that your patches are well-formatted and contain important info for
maintenance purposes, such as ``Signed-off-by`` and ``Upstream-Status``
tags. Note that no functional testing of the changes will be performed by ``patchtest``.
Currently, it only supports testing patches for ``openembedded-core`` branches.
To setup, perform the following::
pip install -r meta/lib/patchtest/requirements.txt
source oe-init-build-env
bitbake-layers add-layer ../meta-selftest
Once these steps are complete and you have generated your patch files,
you can run ``patchtest`` like so::
patchtest --patch <patch_name>
Alternatively, if you want ``patchtest`` to iterate over and test
multiple patches stored in a directory, you can use::
patchtest --directory <directory_name>
By default, ``patchtest`` uses its own modules' file paths to determine what
repository and test suite to check patches against. If you wish to test
patches against a repository other than ``openembedded-core`` and/or use
a different set of tests, you can use the ``--repodir`` and ``--testdir``
flags::
patchtest --patch <patch_name> --repodir <path/to/repo> --testdir <path/to/testdir>
Finally, note that ``patchtest`` is designed to test patches in a standalone
way, so if your patches are meant to apply on top of changes made by
previous patches in a series, it is possible that ``patchtest`` will report
false failures regarding the "merge on head" test.
Using ``patchtest`` in this manner provides a final check for the overall
quality of your changes before they are submitted for review by the
maintainers.
Sending the Patches via Email
=============================
Using Git to Send Patches
-------------------------
To submit patches through email, it is very important that you send them
without any whitespace or HTML formatting that either you or your mailer
introduces. The maintainer that receives your patches needs to be able
to save and apply them directly from your emails, using the ``git am``
command.
Using the ``git send-email`` command is the only error-proof way of sending
your patches using email since there is no risk of compromising whitespace
in the body of the message, which can occur when you use your own mail
client. It will also properly include your patches as *inline attachments*,
which is not easy to do with standard e-mail clients without breaking lines.
If you used your regular e-mail client and shared your patches as regular
attachments, reviewers wouldn't be able to quote specific sections of your
changes and make comments about them.
Setting up Git to Send Email
----------------------------
The ``git send-email`` command can send email by using a local or remote
Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or
through a direct SMTP configuration in your Git ``~/.gitconfig`` file.
Here are the settings for letting ``git send-email`` send e-mail through your
regular STMP server, using a Google Mail account as an example::
git config --global sendemail.smtpserver smtp.gmail.com
git config --global sendemail.smtpserverport 587
git config --global sendemail.smtpencryption tls
git config --global sendemail.smtpuser ada.lovelace@gmail.com
git config --global sendemail.smtppass = XXXXXXXX
These settings will appear in the ``.gitconfig`` file in your home directory.
If you neither can use a local MTA nor SMTP, make sure you use an email client
that does not touch the message (turning spaces in tabs, wrapping lines, etc.).
A good mail client to do so is Pine (or Alpine) or Mutt. For more
information about suitable clients, see `Email clients info for Linux
<https://www.kernel.org/doc/html/latest/process/email-clients.html>`__
in the Linux kernel sources.
If you use such clients, just include the patch in the body of your email.
Finding a Suitable Mailing List
-------------------------------
You should send patches to the appropriate mailing list so that they can be
reviewed by the right contributors and merged by the appropriate maintainer.
The specific mailing list you need to use depends on the location of the code
you are changing.
If people have concerns with any of the patches, they will usually voice
their concern over the mailing list. If patches do not receive any negative
reviews, the maintainer of the affected layer typically takes them, tests them,
and then based on successful testing, merges them.
In general, each component (e.g. layer) should have a ``README`` file
that indicates where to send the changes and which process to follow.
The "poky" repository, which is the Yocto Project's reference build
environment, is a hybrid repository that contains several individual
pieces (e.g. BitBake, Metadata, documentation, and so forth) built using
the combo-layer tool. The upstream location used for submitting changes
varies by component:
- *Core Metadata:* Send your patches to the
:oe_lists:`openembedded-core </g/openembedded-core>`
mailing list. For example, a change to anything under the ``meta`` or
``scripts`` directories should be sent to this mailing list.
- *BitBake:* For changes to BitBake (i.e. anything under the
``bitbake`` directory), send your patches to the
:oe_lists:`bitbake-devel </g/bitbake-devel>`
mailing list.
- *meta-poky* and *meta-yocto-bsp* trees: These trees contain Metadata. Use the
:yocto_lists:`poky </g/poky>` mailing list.
- *Documentation*: For changes to the Yocto Project documentation, use the
:yocto_lists:`docs </g/docs>` mailing list.
For changes to other layers and tools hosted in the Yocto Project source
repositories (i.e. :yocto_git:`git.yoctoproject.org <>`), use the
:yocto_lists:`yocto-patches </g/yocto-patches/>` general mailing list.
For changes to other layers hosted in the OpenEmbedded source
repositories (i.e. :oe_git:`git.openembedded.org <>`), use
the :oe_lists:`openembedded-devel </g/openembedded-devel>`
mailing list, unless specified otherwise in the layer's ``README`` file.
If you intend to submit a new recipe that neither fits into the core Metadata,
nor into :oe_git:`meta-openembedded </meta-openembedded/>`, you should
look for a suitable layer in https://layers.openembedded.org. If similar
recipes can be expected, you may consider :ref:`dev-manual/layers:creating your own layer`.
If in doubt, please ask on the :yocto_lists:`yocto </g/yocto/>` general mailing list
or on the :oe_lists:`openembedded-devel </g/openembedded-devel>` mailing list.
Subscribing to the Mailing List
-------------------------------
After identifying the right mailing list to use, you will have to subscribe to
it if you haven't done it yet.
If you attempt to send patches to a list you haven't subscribed to, your email
will be returned as undelivered.
However, if you don't want to be receive all the messages sent to a mailing list,
you can set your subscription to "no email". You will still be a subscriber able
to send messages, but you won't receive any e-mail. If people reply to your message,
their e-mail clients will default to including your email address in the
conversation anyway.
Anyway, you'll also be able to access the new messages on mailing list archives,
either through a web browser, or for the lists archived on https://lore.kernel.org,
through an individual newsgroup feed or a git repository.
Sending Patches via Email
-------------------------
At this stage, you are ready to send your patches via email. Here's the
typical usage of ``git send-email``::
git send-email --to <mailing-list-address> *.patch
Then, review each subject line and list of recipients carefully, and then
allow the command to send each message.
You will see that ``git send-email`` will automatically copy the people listed
in any commit tags such as ``Signed-off-by`` or ``Reported-by``.
In case you are sending patches for :oe_git:`meta-openembedded </meta-openembedded/>`
or any layer other than :oe_git:`openembedded-core </openembedded-core/>`,
please add the appropriate prefix so that it is clear which layer the patch is intended
to be applied to::
git format-patch --subject-prefix="meta-oe][PATCH" ...
.. note::
It is actually possible to send patches without generating them
first. However, make sure you have reviewed your changes carefully
because ``git send-email`` will just show you the title lines of
each patch.
Here's a command you can use if you just have one patch in your
branch::
git send-email --to <mailing-list-address> -1
If you have multiple patches and a cover letter, you can send
patches for all the commits between the reference branch
and the tip of your branch::
git send-email --cover-letter --cover-from-description=auto --to <mailing-list-address> -M <ref-branch>
See the `git send-email manual page <https://git-scm.com/docs/git-send-email>`__
for details.
Troubleshooting Email Issues
----------------------------
Fixing your From identity
~~~~~~~~~~~~~~~~~~~~~~~~~
We have a frequent issue with contributors whose patches are received through
a ``From`` field which doesn't match the ``Signed-off-by`` information. Here is
a typical example for people sending from a domain name with :wikipedia:`DMARC`::
From: "Linus Torvalds via lists.openembedded.org <linus.torvalds=kernel.org@lists.openembedded.org>"
This ``From`` field is used by ``git am`` to recreate commits with the right
author name. The following will ensure that your e-mails have an additional
``From`` field at the beginning of the Email body, and therefore that
maintainers accepting your patches don't have to fix commit author information
manually::
git config --global sendemail.from "linus.torvalds@kernel.org"
The ``sendemail.from`` should match your ``user.email`` setting,
which appears in the ``Signed-off-by`` line of your commits.
Streamlining git send-email usage
---------------------------------
If you want to save time and not be forced to remember the right options to use
with ``git send-email``, you can use Git configuration settings.
- To set the right mailing list address for a given repository::
git config --local sendemail.to openembedded-devel@lists.openembedded.org
- If the mailing list requires a subject prefix for the layer
(this only works when the repository only contains one layer)::
git config --local format.subjectprefix "meta-something][PATCH"
Using Scripts to Push a Change Upstream and Request a Pull
==========================================================
For larger patch series it is preferable to send a pull request which not
only includes the patch but also a pointer to a branch that can be pulled
from. This involves making a local branch for your changes, pushing this
branch to an accessible repository and then using the ``create-pull-request``
and ``send-pull-request`` scripts from openembedded-core to create and send a
patch series with a link to the branch for review.
Follow this procedure to push a change to an upstream "contrib" Git
repository once the steps in
":ref:`contributor-guide/submit-changes:preparing changes for submission`"
have been followed:
.. note::
You can find general Git information on how to push a change upstream
in the
`Git Community Book <https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows>`__.
#. *Request Push Access to an "Upstream" Contrib Repository:* Send an email to
``helpdesk@yoctoproject.org``:
- Attach your SSH public key which usually named ``id_rsa.pub.``.
If you don't have one generate it by running ``ssh-keygen -t rsa -b 4096 -C "your_email@example.com"``.
- List the repositories you're planning to contribute to.
- Include your preferred branch prefix for ``-contrib`` repositories.
#. *Push Your Commits to the "Contrib" Upstream:* Push your
changes to that repository::
$ git push upstream_remote_repo local_branch_name
For example, suppose you have permissions to push
into the upstream ``meta-intel-contrib`` repository and you are
working in a local branch named `your_name`\ ``/README``. The following
command pushes your local commits to the ``meta-intel-contrib``
upstream repository and puts the commit in a branch named
`your_name`\ ``/README``::
$ git push meta-intel-contrib your_name/README
#. *Determine Who to Notify:* Determine the maintainer or the mailing
list that you need to notify for the change.
Before submitting any change, you need to be sure who the maintainer
is or what mailing list that you need to notify. Use either these
methods to find out:
- *Maintenance File:* Examine the ``maintainers.inc`` file, which is
located in the :term:`Source Directory` at
``meta/conf/distro/include``, to see who is responsible for code.
- *Search by File:* Using :ref:`overview-manual/development-environment:git`, you can
enter the following command to bring up a short list of all
commits against a specific file::
git shortlog -- filename
Just provide the name of the file for which you are interested. The
information returned is not ordered by history but does include a
list of everyone who has committed grouped by name. From the list,
you can see who is responsible for the bulk of the changes against
the file.
- *Find the Mailing List to Use:* See the
":ref:`contributor-guide/submit-changes:finding a suitable mailing list`"
section above.
#. *Make a Pull Request:* Notify the maintainer or the mailing list that
you have pushed a change by making a pull request.
The Yocto Project provides two scripts that conveniently let you
generate and send pull requests to the Yocto Project. These scripts
are ``create-pull-request`` and ``send-pull-request``. You can find
these scripts in the ``scripts`` directory within the
:term:`Source Directory` (e.g.
``poky/scripts``).
Using these scripts correctly formats the requests without
introducing any whitespace or HTML formatting. The maintainer that
receives your patches either directly or through the mailing list
needs to be able to save and apply them directly from your emails.
Using these scripts is the preferred method for sending patches.
First, create the pull request. For example, the following command
runs the script, specifies the upstream repository in the contrib
directory into which you pushed the change, and provides a subject
line in the created patch files::
$ poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README"
Running this script forms ``*.patch`` files in a folder named
``pull-``\ `PID` in the current directory. One of the patch files is a
cover letter.
Before running the ``send-pull-request`` script, you must edit the
cover letter patch to insert information about your change. After
editing the cover letter, send the pull request. For example, the
following command runs the script and specifies the patch directory
and email address. In this example, the email address is a mailing
list::
$ poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@lists.yoctoproject.org
You need to follow the prompts as the script is interactive.
.. note::
For help on using these scripts, simply provide the ``-h``
argument as follows::
$ poky/scripts/create-pull-request -h
$ poky/scripts/send-pull-request -h
Submitting Changes to Stable Release Branches
=============================================
The process for proposing changes to a Yocto Project stable branch differs
from the steps described above. Changes to a stable branch must address
identified bugs or CVEs and should be made carefully in order to avoid the
risk of introducing new bugs or breaking backwards compatibility. Typically
bug fixes must already be accepted into the master branch before they can be
backported to a stable branch unless the bug in question does not affect the
master branch or the fix on the master branch is unsuitable for backporting.
The list of stable branches along with the status and maintainer for each
branch can be obtained from the
:yocto_wiki:`Releases wiki page </Releases>`.
.. note::
Changes will not typically be accepted for branches which are marked as
End-Of-Life (EOL).
With this in mind, the steps to submit a change for a stable branch are as
follows:
#. *Identify the bug or CVE to be fixed:* This information should be
collected so that it can be included in your submission.
See :ref:`dev-manual/vulnerabilities:checking for vulnerabilities`
for details about CVE tracking.
#. *Check if the fix is already present in the master branch:* This will
result in the most straightforward path into the stable branch for the
fix.
#. *If the fix is present in the master branch --- submit a backport request
by email:* You should send an email to the relevant stable branch
maintainer and the mailing list with details of the bug or CVE to be
fixed, the commit hash on the master branch that fixes the issue and
the stable branches which you would like this fix to be backported to.
#. *If the fix is not present in the master branch --- submit the fix to the
master branch first:* This will ensure that the fix passes through the
project's usual patch review and test processes before being accepted.
It will also ensure that bugs are not left unresolved in the master
branch itself. Once the fix is accepted in the master branch a backport
request can be submitted as above.
#. *If the fix is unsuitable for the master branch --- submit a patch
directly for the stable branch:* This method should be considered as a
last resort. It is typically necessary when the master branch is using
a newer version of the software which includes an upstream fix for the
issue or when the issue has been fixed on the master branch in a way
that introduces backwards incompatible changes. In this case follow the
steps in ":ref:`contributor-guide/submit-changes:preparing changes for submission`"
and in the following sections but modify the subject header of your patch
email to include the name of the stable branch which you are
targetting. This can be done using the ``--subject-prefix`` argument to
``git format-patch``, for example to submit a patch to the
"&DISTRO_NAME_NO_CAP_MINUS_ONE;" branch use::
git format-patch --subject-prefix='&DISTRO_NAME_NO_CAP_MINUS_ONE;][PATCH' ...
Taking Patch Review into Account
================================
You may get feedback on your submitted patches from other community members
or from the automated patchtest service. If issues are identified in your
patches then it is usually necessary to address these before the patches are
accepted into the project. In this case you should your commits according
to the feedback and submit an updated version to the relevant mailing list.
In any case, never fix reported issues by fixing them in new commits
on the tip of your branch. Always come up with a new series of commits
without the reported issues.
.. note::
It is a good idea to send a copy to the reviewers who provided feedback
to the previous version of the patch. You can make sure this happens
by adding a ``CC`` tag to the commit description::
CC: William Shakespeare <bill@yoctoproject.org>
A single patch can be amended using ``git commit --amend``, and multiple
patches can be easily reworked and reordered through an interactive Git rebase::
git rebase -i <ref-branch>
See `this tutorial <https://hackernoon.com/beginners-guide-to-interactive-rebasing-346a3f9c3a6d>`__
for practical guidance about using Git interactive rebasing.
You should also modify the ``[PATCH]`` tag in the email subject line when
sending the revised patch to mark the new iteration as ``[PATCH v2]``,
``[PATCH v3]``, etc as appropriate. This can be done by passing the ``-v``
argument to ``git format-patch`` with a version number::
git format-patch -v2 <ref-branch>
After generating updated patches (v2, v3, and so on) via ``git
format-patch``, ideally developers will add a patch version changelog
to each patch that describes what has changed between each revision of
the patch. Add patch version changelogs after the ``---`` marker in the
patch, indicating that this information is part of this patch, but is not
suitable for inclusion in the commit message (i.e. the git history) itself.
Providing a patch version changelog makes it easier for maintainers and
reviewers to succinctly understand what changed in all versions of the
patch, without having to consult alternate sources of information, such as
searching through messages on a mailing list. For example::
<patch title>
<commit message>
<Signed-off-by/other trailers>
---
changes in v4:
- provide a clearer commit message
- fix spelling mistakes
changes in v3:
- replace func() to use other_func() instead
changes in v2:
- this patch was added in v2
---
<diffstat output>
<unified diff>
Lastly please ensure that you also test your revised changes. In particular
please don't just edit the patch file written out by ``git format-patch`` and
resend it.
Tracking the Status of Patches
==============================
The Yocto Project uses a `Patchwork instance <https://patchwork.yoctoproject.org/>`__
to track the status of patches submitted to the various mailing lists and to
support automated patch testing. Each submitted patch is checked for common
mistakes and deviations from the expected patch format and submitters are
notified by ``patchtest`` if such mistakes are found. This process helps to
reduce the burden of patch review on maintainers.
.. note::
This system is imperfect and changes can sometimes get lost in the flow.
Asking about the status of a patch or change is reasonable if the change
has been idle for a while with no feedback.
If your patches have not had any feedback in a few days, they may have already
been merged. You can run ``git pull`` branch to check this. Note that many if
not most layer maintainers do not send out acknowledgement emails when they
accept patches. Alternatively, if there is no response or merge after a few days
the patch may have been missed or the appropriate reviewers may not currently be
around. It is then perfectly fine to reply to it yourself with a reminder asking
for feedback.
.. note::
Patch reviews for feature and recipe upgrade patches are likely be delayed
during a feature freeze because these types of patches aren't merged during
at that time --- you may have to wait until after the freeze is lifted.
Maintainers also commonly use ``-next`` branches to test submissions prior to
merging patches. Thus, you can get an idea of the status of a patch based on
whether the patch has been merged into one of these branches. The commonly
used testing branches for OpenEmbedded-Core are as follows:
- *openembedded-core "master-next" branch:* This branch is part of the
:oe_git:`openembedded-core </openembedded-core/>` repository and contains
proposed changes to the core metadata.
- *poky "master-next" branch:* This branch is part of the
:yocto_git:`poky </poky/>` repository and combines proposed
changes to BitBake, the core metadata and the poky distro.
Similarly, stable branches maintained by the project may have corresponding
``-next`` branches which collect proposed changes. For example,
``&DISTRO_NAME_NO_CAP;-next`` and ``&DISTRO_NAME_NO_CAP_MINUS_ONE;-next``
branches in both the "openembdedded-core" and "poky" repositories.
Other layers may have similar testing branches but there is no formal
requirement or standard for these so please check the documentation for the
layers you are contributing to.
Acceptance of AI Generated Code
===============================
The Yocto Project and OpenEmbedded follow the guidance of the Linux Foundation
in regards to the use of generative AI tools. See:
https://www.linuxfoundation.org/legal/generative-ai.
All of the existing guidelines in this document are expected to be followed,
including in the :doc:`recipe-style-guide`, and contributing the changes with
additional requirements to the items in section
:ref:`contributor-guide/submit-changes:Implement and commit changes`.
All AI Generated Code must be labeled as such in the commit message,
prior to your ``Signed-off-by`` line. It is also strongly recommended,
that any patches or code within the commit also have a comment or other
indication that this code was AI generated.
For example, here is a properly formatted commit message::
component: Add the ability to ...
AI-Generated: Uses GitHub Copilot
Signed-off-by: Your Name <your.name@domain>
The ``Signed-off-by`` line must be written by you, and not the AI helper.
As a reminder, when contributing a change, your ``Signed-off-by`` line is
required and the stipulations in the `Developer's Statement of Origin
1.1 <https://developercertificate.org/>`__ still apply.
Additionally, you must stipulate AI contributions conform to the Linux
Foundation policy, specifically:
#. Contributors should ensure that the terms and conditions of the generative AI
tool do not place any contractual restrictions on how the tool's output can
be used that are inconsistent with the project's open source software
license, the project's intellectual property policies, or the Open Source
Definition.
#. If any pre-existing copyrighted materials (including pre-existing open
source code) authored or owned by third parties are included in the AI tool's
output, prior to contributing such output to the project, the Contributor
should confirm that they have permission from the third party
owners -- such as the form of an open source license or public domain
declaration that complies with the project's licensing policies -- to use and
modify such pre-existing materials and contribute them to the project.
Additionally, the contributor should provide notice and attribution of such
third party rights, along with information about the applicable license
terms, with their contribution.

129
sources/poky/documentation/dev-manual/bblock.rst Normal file
View File

@@ -0,0 +1,129 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Locking and Unlocking Recipes Using ``bblock``
**********************************************
By design, the OpenEmbedded build system builds everything from scratch
unless BitBake determines that specific tasks do not require rebuilding.
At startup, it computes a signature for all tasks, based on the task's input.
Then, it compares these signatures with the ones from the sstate cache (if they
exist). Any changes cause the task to rerun.
During development, changes might trigger BitBake to rebuild certain
recipes, even when we know they do not require rebuilding at that stage.
For example, modifying a recipe can lead to rebuilding its native
counterpart, which might prove unnecessary. Editing the ``python3`` recipe,
for instance, can prompt BitBake to rebuild ``python3-native`` along with any
recipes that depend on it.
To prevent this, use ``bblock`` to lock specific tasks or recipes to
specific signatures, forcing BitBake to use the sstate cache for them.
.. warning::
Use ``bblock`` only during the development phase.
Forcing BitBake to use the sstate cache, regardless of input changes, means
the recipe metadata no longer directly reflect the output. Use this feature
with caution. If you do not understand why signatures change, see the section
on :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`.
Locking tasks and recipes
-------------------------
To lock a recipe, use::
$ bblock recipe
You can also use a space-separated list of recipes to lock multiple recipes::
$ bblock recipe1 recipe2
Locking a recipe means locking all tasks of the recipe. If you need to
lock only particular tasks, use the `-t` option with a comma-separated
list of tasks::
$ bblock -t task1,task2 recipe
Unlocking tasks and recipes
---------------------------
To unlock a recipe, use the ``-r`` option::
$ bblock -r recipe
You can also use a space-separated list of recipes to unlock multiple recipes::
$ bblock -r recipe1 recipe2
Unlocking a recipe means unlocking all tasks of the recipe. If you need to
unlock only particular tasks use the ``-t`` option with a comma-separated
list of tasks::
$ bblock -r -t task1,task2 recipe
To unlock all recipes, do not specify any recipe::
$ bblock -r
Configuration file
------------------
``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
included by default in :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`.
To dump the file, use the ``-d`` option::
$ bblock -d
Locking mechanism
-----------------
``bblock`` computes the signature(s) of the task(s) and sets the 3 following
variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
In particular, ``bblock`` sets::
SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
This produces architecture specific locks and reminds user that some tasks
have locked signatures.
Example
-------
When working on the ``python3`` recipe, we can lock ``python3-native`` with
the following::
$ bblock python3-native
$ bblock -d
# Generated by bblock
SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"

58
sources/poky/documentation/dev-manual/bmaptool.rst Normal file
View File

@@ -0,0 +1,58 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Flashing Images Using `bmaptool`
********************************
A fast and easy way to flash an image to a bootable device is to use
`bmaptool`, which is integrated into the OpenEmbedded build system.
`bmaptool` is a generic tool that creates a file's block map (bmap) and
then uses that map to copy the file. As compared to traditional tools
such as `dd` or `cp`, `bmaptool` can copy (or flash) large files like raw
system image files much faster.
.. note::
- If you are using Ubuntu or Debian distributions, you can install
the ``bmap-tools`` package using the following command and then
use the tool without specifying ``PATH`` even from the root
account::
$ sudo apt install bmap-tools
- If you are unable to install the ``bmap-tools`` package, you will
need to build `bmaptool` before using it. Use the following command::
$ bitbake bmaptool-native -caddto_recipe_sysroot
Following, is an example that shows how to flash a Wic image. Realize
that while this example uses a Wic image, you can use `bmaptool` to flash
any type of image. Use these steps to flash an image using `bmaptool`:
#. *Update your local.conf File:* You need to have the following set
in your ``local.conf`` file before building your image::
IMAGE_FSTYPES += "wic wic.bmap"
#. *Get Your Image:* Either have your image ready (pre-built with the
:term:`IMAGE_FSTYPES`
setting previously mentioned) or take the step to build the image::
$ bitbake image
#. *Flash the Device:* Flash the device with the image by using `bmaptool`
depending on your particular setup. The following commands assume the
image resides in the :term:`Build Directory`'s ``deploy/images/`` area:
- If you installed the package for `bmaptool`, you can directly run::
$ sudo bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
- Otherwise, if you built `bmaptool` with BitBake, run::
$ sudo chmod a+w /dev/sdX # get write access to the media, needed only once after booting
$ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
For help on the ``bmaptool`` command, use the following command::
$ bmaptool --help

409
sources/poky/documentation/dev-manual/build-quality.rst Normal file
View File

@@ -0,0 +1,409 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Maintaining Build Output Quality
********************************
Many factors can influence the quality of a build. For example, if you
upgrade a recipe to use a new version of an upstream software package or
you experiment with some new configuration options, subtle changes can
occur that you might not detect until later. Consider the case where
your recipe is using a newer version of an upstream package. In this
case, a new version of a piece of software might introduce an optional
dependency on another library, which is auto-detected. If that library
has already been built when the software is building, the software will
link to the built library and that library will be pulled into your
image along with the new software even if you did not want the library.
The :ref:`ref-classes-buildhistory` class helps you maintain the quality of
your build output. You can use the class to highlight unexpected and possibly
unwanted changes in the build output. When you enable build history, it records
information about the contents of each package and image and then commits that
information to a local Git repository where you can examine the information.
The remainder of this section describes the following:
- :ref:`How you can enable and disable build history <dev-manual/build-quality:enabling and disabling build history>`
- :ref:`How to understand what the build history contains <dev-manual/build-quality:understanding what the build history contains>`
- :ref:`How to limit the information used for build history <dev-manual/build-quality:using build history to gather image information only>`
- :ref:`How to examine the build history from both a command-line and web interface <dev-manual/build-quality:examining build history information>`
Enabling and Disabling Build History
====================================
Build history is disabled by default. To enable it, add the following
:term:`INHERIT` statement and set the :term:`BUILDHISTORY_COMMIT` variable to
"1" at the end of your ``conf/local.conf`` file found in the
:term:`Build Directory`::
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"
Enabling build history as
previously described causes the OpenEmbedded build system to collect
build output information and commit it as a single commit to a local
:ref:`overview-manual/development-environment:git` repository.
.. note::
Enabling build history increases your build times slightly,
particularly for images, and increases the amount of disk space used
during the build.
You can disable build history by removing the previous statements from
your ``conf/local.conf`` file.
Understanding What the Build History Contains
=============================================
Build history information is kept in ``${``\ :term:`TOPDIR`\ ``}/buildhistory``
in the :term:`Build Directory` as defined by the :term:`BUILDHISTORY_DIR`
variable. Here is an example abbreviated listing:
.. image:: figures/buildhistory.png
:align: center
:width: 50%
At the top level, there is a ``metadata-revs`` file that lists the
revisions of the repositories for the enabled layers when the build was
produced. The rest of the data splits into separate ``packages``,
``images`` and ``sdk`` directories, the contents of which are described
as follows.
Build History Package Information
---------------------------------
The history for each package contains a text file that has name-value
pairs with information about the package. For example,
``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
contains the following:
.. code-block:: none
PV = 1.22.1
PR = r32
RPROVIDES =
RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
PKGSIZE = 540168
FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
/etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
/usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
/usr/share/pixmaps /usr/share/applications /usr/share/idl \
/usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
/etc/busybox.links.nosuid /etc/busybox.links.suid
Most of these
name-value pairs correspond to variables used to produce the package.
The exceptions are ``FILELIST``, which is the actual list of files in
the package, and ``PKGSIZE``, which is the total size of files in the
package in bytes.
There is also a file that corresponds to the recipe from which the package
came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``):
.. code-block:: none
PV = 1.22.1
PR = r32
DEPENDS = initscripts kern-tools-native update-rc.d-native \
virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
virtual/libc virtual/update-alternatives
PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
Finally, for those recipes fetched from a version control system (e.g.,
Git), there is a file that lists source revisions that are specified in
the recipe and the actual revisions used during the build. Listed
and actual revisions might differ when
:term:`SRCREV` is set to
${:term:`AUTOREV`}. Here is an
example assuming
``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``)::
# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
# SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f"
You can use the
``buildhistory-collect-srcrevs`` command with the ``-a`` option to
collect the stored :term:`SRCREV` values from build history and report them
in a format suitable for use in global configuration (e.g.,
``local.conf`` or a distro include file) to override floating
:term:`AUTOREV` values to a fixed set of revisions. Here is some example
output from this command::
$ buildhistory-collect-srcrevs -a
# all-poky-linux
SRCREV:pn-ca-certificates = "07de54fdcc5806bde549e1edf60738c6bccf50e8"
SRCREV:pn-update-rc.d = "8636cf478d426b568c1be11dbd9346f67e03adac"
# core2-64-poky-linux
SRCREV:pn-binutils = "87d4632d36323091e731eb07b8aa65f90293da66"
SRCREV:pn-btrfs-tools = "8ad326b2f28c044cb6ed9016d7c3285e23b673c8"
SRCREV_bzip2-tests:pn-bzip2 = "f9061c030a25de5b6829e1abf373057309c734c0"
SRCREV:pn-e2fsprogs = "02540dedd3ddc52c6ae8aaa8a95ce75c3f8be1c0"
SRCREV:pn-file = "504206e53a89fd6eed71aeaf878aa3512418eab1"
SRCREV_glibc:pn-glibc = "24962427071fa532c3c48c918e9d64d719cc8a6c"
SRCREV:pn-gnome-desktop-testing = "e346cd4ed2e2102c9b195b614f3c642d23f5f6e7"
SRCREV:pn-init-system-helpers = "dbd9197569c0935029acd5c9b02b84c68fd937ee"
SRCREV:pn-kmod = "b6ecfc916a17eab8f93be5b09f4e4f845aabd3d1"
SRCREV:pn-libnsl2 = "82245c0c58add79a8e34ab0917358217a70e5100"
SRCREV:pn-libseccomp = "57357d2741a3b3d3e8425889a6b79a130e0fa2f3"
SRCREV:pn-libxcrypt = "50cf2b6dd4fdf04309445f2eec8de7051d953abf"
SRCREV:pn-ncurses = "51d0fd9cc3edb975f04224f29f777f8f448e8ced"
SRCREV:pn-procps = "19a508ea121c0c4ac6d0224575a036de745eaaf8"
SRCREV:pn-psmisc = "5fab6b7ab385080f1db725d6803136ec1841a15f"
SRCREV:pn-ptest-runner = "bcb82804daa8f725b6add259dcef2067e61a75aa"
SRCREV:pn-shared-mime-info = "18e558fa1c8b90b86757ade09a4ba4d6a6cf8f70"
SRCREV:pn-zstd = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
# qemux86_64-poky-linux
SRCREV_machine:pn-linux-yocto = "20301aeb1a64164b72bc72af58802b315e025c9c"
SRCREV_meta:pn-linux-yocto = "2d38a472b21ae343707c8bd64ac68a9eaca066a0"
# x86_64-linux
SRCREV:pn-binutils-cross-x86_64 = "87d4632d36323091e731eb07b8aa65f90293da66"
SRCREV_glibc:pn-cross-localedef-native = "24962427071fa532c3c48c918e9d64d719cc8a6c"
SRCREV_localedef:pn-cross-localedef-native = "794da69788cbf9bf57b59a852f9f11307663fa87"
SRCREV:pn-debianutils-native = "de14223e5bffe15e374a441302c528ffc1cbed57"
SRCREV:pn-libmodulemd-native = "ee80309bc766d781a144e6879419b29f444d94eb"
SRCREV:pn-virglrenderer-native = "363915595e05fb252e70d6514be2f0c0b5ca312b"
SRCREV:pn-zstd-native = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
.. note::
Here are some notes on using the ``buildhistory-collect-srcrevs`` command:
- By default, only values where the :term:`SRCREV` was not hardcoded
(usually when :term:`AUTOREV` is used) are reported. Use the ``-a``
option to see all :term:`SRCREV` values.
- The output statements might not have any effect if overrides are
applied elsewhere in the build system configuration. Use the
``-f`` option to add the ``forcevariable`` override to each output
line if you need to work around this restriction.
- The script does apply special handling when building for multiple
machines. However, the script does place a comment before each set
of values that specifies which triplet to which they belong as
previously shown (e.g., ``i586-poky-linux``).
Build History Image Information
-------------------------------
The files produced for each image are as follows:
- ``image-files:`` A directory containing selected files from the root
filesystem. The files are defined by
:term:`BUILDHISTORY_IMAGE_FILES`.
- ``build-id.txt:`` Human-readable information about the build
configuration and metadata source revisions. This file contains the
full build header as printed by BitBake.
- ``*.dot:`` Dependency graphs for the image that are compatible with
``graphviz``.
- ``files-in-image.txt:`` A list of files in the image with
permissions, owner, group, size, and symlink information.
- ``image-info.txt:`` A text file containing name-value pairs with
information about the image. See the following listing example for
more information.
- ``installed-package-names.txt:`` A list of installed packages by name
only.
- ``installed-package-sizes.txt:`` A list of installed packages ordered
by size.
- ``installed-packages.txt:`` A list of installed packages with full
package filenames.
.. note::
Installed package information is able to be gathered and produced
even if package management is disabled for the final image.
Here is an example of ``image-info.txt``:
.. code-block:: none
DISTRO = poky
DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be
USER_CLASSES = buildstats
IMAGE_CLASSES = qemuboot qemuboot license_image
IMAGE_FEATURES = debug-tweaks
IMAGE_LINGUAS =
IMAGE_INSTALL = packagegroup-core-boot speex speexdsp
BAD_RECOMMENDATIONS =
NO_RECOMMENDATIONS =
PACKAGE_EXCLUDE =
ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; cve_check_write_rootfs_manifest; ssh_allow_empty_password; ssh_allow_root_login; postinst_enable_logging; rootfs_update_timestamp; write_image_test_data; empty_var_volatile; sort_passwd; rootfs_reproducible;
IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
IMAGESIZE = 9265
Other than ``IMAGESIZE``,
which is the total size of the files in the image in Kbytes, the
name-value pairs are variables that may have influenced the content of
the image. This information is often useful when you are trying to
determine why a change in the package or file listings has occurred.
Using Build History to Gather Image Information Only
----------------------------------------------------
As you can see, build history produces image information, including
dependency graphs, so you can see why something was pulled into the
image. If you are just interested in this information and not interested
in collecting specific package or SDK information, you can enable
writing only image information without any history by adding the
following to your ``conf/local.conf`` file found in the
:term:`Build Directory`::
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "0"
BUILDHISTORY_FEATURES = "image"
Here, you set the
:term:`BUILDHISTORY_FEATURES`
variable to use the image feature only.
Build History SDK Information
-----------------------------
Build history collects similar information on the contents of SDKs (e.g.
``bitbake -c populate_sdk imagename``) as compared to information it
collects for images. Furthermore, this information differs depending on
whether an extensible or standard SDK is being produced.
The following list shows the files produced for SDKs:
- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
owner, group, size, and symlink information. This list includes both
the host and target parts of the SDK.
- ``sdk-info.txt:`` A text file containing name-value pairs with
information about the SDK. See the following listing example for more
information.
- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
with information about task group sizes (e.g. :ref:`ref-tasks-populate_sysroot`
tasks have a total size). The ``sstate-task-sizes.txt`` file exists
only when an extensible SDK is created.
- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
with information for the shared-state packages and sizes in the SDK.
The ``sstate-package-sizes.txt`` file exists only when an extensible
SDK is created.
- ``sdk-files:`` A folder that contains copies of the files mentioned
in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
specific to the extensible SDK although you can set it differently if
you would like to pull in specific files from the standard SDK.
The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
``conf/auto.conf``, ``conf/locked-sigs.inc``, and
``conf/devtool.conf``. Thus, for an extensible SDK, these files get
copied into the ``sdk-files`` directory.
- The following information appears under each of the ``host`` and
``target`` directories for the portions of the SDK that run on the
host and on the target, respectively:
.. note::
The following files for the most part are empty when producing an
extensible SDK because this type of SDK is not constructed from
packages as is the standard SDK.
- ``depends.dot:`` Dependency graph for the SDK that is compatible
with ``graphviz``.
- ``installed-package-names.txt:`` A list of installed packages by
name only.
- ``installed-package-sizes.txt:`` A list of installed packages
ordered by size.
- ``installed-packages.txt:`` A list of installed packages with full
package filenames.
Here is an example of ``sdk-info.txt``:
.. code-block:: none
DISTRO = poky
DISTRO_VERSION = 1.3+snapshot-20130327
SDK_NAME = poky-glibc-i686-arm
SDK_VERSION = 1.3+snapshot
SDKMACHINE =
SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
BAD_RECOMMENDATIONS =
SDKSIZE = 352712
Other than ``SDKSIZE``, which is
the total size of the files in the SDK in Kbytes, the name-value pairs
are variables that might have influenced the content of the SDK. This
information is often useful when you are trying to determine why a
change in the package or file listings has occurred.
Examining Build History Information
-----------------------------------
You can examine build history output from the command line or from a web
interface.
To see any changes that have occurred (assuming you have
:term:`BUILDHISTORY_COMMIT` = "1"),
you can simply use any Git command that allows you to view the history
of a repository. Here is one method::
$ git log -p
You need to realize,
however, that this method does show changes that are not significant
(e.g. a package's size changing by a few bytes).
There is a command-line tool called ``buildhistory-diff``, though,
that queries the Git repository and prints just the differences that
might be significant in human-readable form. Here is an example::
$ poky/poky/scripts/buildhistory-diff . HEAD^
Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
/etc/anotherpkg.conf was added
/sbin/anotherpkg was added
* (installed-package-names.txt):
* anotherpkg was added
Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
anotherpkg was added
packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
.. note::
The ``buildhistory-diff`` tool requires the ``GitPython``
package. Be sure to install it using Pip3 as follows::
$ pip3 install GitPython --user
Alternatively, you can install ``python3-git`` using the appropriate
distribution package manager (e.g. ``apt``, ``dnf``, or ``zipper``).
To see changes to the build history using a web interface, follow the
instruction in the ``README`` file
:yocto_git:`here </buildhistory-web/>`.
Here is a sample screenshot of the interface:
.. image:: figures/buildhistory-web.png
:width: 100%

1024
sources/poky/documentation/dev-manual/building.rst Normal file
View File

File diff suppressed because it is too large Load Diff

135
sources/poky/documentation/dev-manual/custom-distribution.rst Normal file
View File

@@ -0,0 +1,135 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating Your Own Distribution
******************************
When you build an image using the Yocto Project and do not alter any
distribution :term:`Metadata`, you are using the Poky distribution.
Poky is explicitly a *reference* distribution for testing and
development purposes. It enables most hardware and software features
so that they can be tested, but this also means that from a security
point of view the attack surface is very large. Additionally, at some
point it is likely that you will want to gain more control over package
alternative selections, compile-time options, and other low-level
configurations. For both of these reasons, if you are using the Yocto
Project for production use then you are strongly encouraged to create
your own distribution.
To create your own distribution, the basic steps consist of creating
your own distribution layer, creating your own distribution
configuration file, and then adding any needed code and Metadata to the
layer. The following steps provide some more detail:
- *Create a layer for your new distro:* Create your distribution layer
so that you can keep your Metadata and code for the distribution
separate. It is strongly recommended that you create and use your own
layer for configuration and code. Using your own layer as compared to
just placing configurations in a ``local.conf`` configuration file
makes it easier to reproduce the same build configuration when using
multiple build machines. See the
":ref:`dev-manual/layers:creating a general layer using the \`\`bitbake-layers\`\` script`"
section for information on how to quickly set up a layer.
- *Create the distribution configuration file:* The distribution
configuration file needs to be created in the ``conf/distro``
directory of your layer. You need to name it using your distribution
name (e.g. ``mydistro.conf``).
.. note::
The :term:`DISTRO` variable in your ``local.conf`` file determines the
name of your distribution.
You can split out parts of your configuration file into include files
and then "require" them from within your distribution configuration
file. Be sure to place the include files in the
``conf/distro/include`` directory of your layer. A common example
usage of include files would be to separate out the selection of
desired version and revisions for individual recipes.
Your configuration file needs to set the following required
variables:
- :term:`DISTRO_NAME`
- :term:`DISTRO_VERSION`
These following variables are optional and you typically set them
from the distribution configuration file:
- :term:`DISTRO_FEATURES`
- :term:`DISTRO_EXTRA_RDEPENDS`
- :term:`DISTRO_EXTRA_RRECOMMENDS`
- :term:`TCLIBC`
.. tip::
If you want to base your distribution configuration file on the
very basic configuration from OE-Core, you can use
``conf/distro/defaultsetup.conf`` as a reference and just include
variables that differ as compared to ``defaultsetup.conf``.
Alternatively, you can create a distribution configuration file
from scratch using the ``defaultsetup.conf`` file or configuration files
from another distribution such as Poky as a reference.
- *Provide miscellaneous variables:* Be sure to define any other
variables for which you want to create a default or enforce as part
of the distribution configuration. You can include nearly any
variable from the ``local.conf`` file. The variables you use are not
limited to the list in the previous bulleted item.
- *Point to Your distribution configuration file:* In your ``local.conf``
file in the :term:`Build Directory`, set your :term:`DISTRO` variable to
point to your distribution's configuration file. For example, if your
distribution's configuration file is named ``mydistro.conf``, then
you point to it as follows::
DISTRO = "mydistro"
- *Add more to the layer if necessary:* Use your layer to hold other
information needed for the distribution:
- Add recipes for installing distro-specific configuration files
that are not already installed by another recipe. If you have
distro-specific configuration files that are included by an
existing recipe, you should add an append file (``.bbappend``) for
those. For general information and recommendations on how to add
recipes to your layer, see the
":ref:`dev-manual/layers:creating your own layer`" and
":ref:`dev-manual/layers:following best practices when creating layers`"
sections.
- Add any image recipes that are specific to your distribution.
- Add a ``psplash`` append file for a branded splash screen, using
the :term:`SPLASH_IMAGES` variable.
- Add any other append files to make custom changes that are
specific to individual recipes.
For information on append files, see the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section.
Copying and modifying the Poky distribution
===========================================
Instead of creating a custom distribution from scratch as per above, you may
wish to start your custom distribution configuration by copying the Poky
distribution provided within the ``meta-poky`` layer and then modifying it.
This is fine, however if you do this you should keep the following in mind:
- Every reference to Poky needs to be updated in your copy so that it
will still apply. This includes override usage within files (e.g. ``:poky``)
and in directory names. This is a good opportunity to evaluate each one of
these customizations to see if they are needed for your use case.
- Unless you also intend to use them, the ``poky-tiny``, ``poky-altcfg`` and
``poky-bleeding`` variants and any references to them can be removed.
- More generally, the Poky distribution configuration enables a lot more
than you likely need for your production use case. You should evaluate *every*
configuration choice made in your copy to determine if it is needed.

52
sources/poky/documentation/dev-manual/custom-template-configuration-directory.rst Normal file
View File

@@ -0,0 +1,52 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Custom Template Configuration Directory
**************************************************
If you are producing your own customized version of the build system for
use by other users, you might want to provide a custom build configuration
that includes all the necessary settings and layers (i.e. ``local.conf`` and
``bblayers.conf`` that are created in a new :term:`Build Directory`) and a custom
message that is shown when setting up the build. This can be done by
creating one or more template configuration directories in your
custom distribution layer.
This can be done by using ``bitbake-layers save-build-conf``::
$ bitbake-layers save-build-conf ../../meta-alex/ test-1
NOTE: Starting bitbake server...
NOTE: Configuration template placed into /srv/work/alex/meta-alex/conf/templates/test-1
Please review the files in there, and particularly provide a configuration description in /srv/work/alex/meta-alex/conf/templates/test-1/conf-notes.txt
You can try out the configuration with
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
The above command takes the config files from the currently active :term:`Build Directory` under ``conf``,
replaces site-specific paths in ``bblayers.conf`` with ``##OECORE##``-relative paths, and copies
the config files into a specified layer under a specified template name.
To use those saved templates as a starting point for a build, users should point
to one of them with :term:`TEMPLATECONF` environment variable::
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
The OpenEmbedded build system uses the environment variable
:term:`TEMPLATECONF` to locate the directory from which it gathers
configuration information that ultimately ends up in the
:term:`Build Directory` ``conf`` directory.
If :term:`TEMPLATECONF` is not set, the default value is obtained
from ``.templateconf`` file that is read from the same directory as
``oe-init-build-env`` script. For the Poky reference distribution this
would be::
TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf/templates/default}
If you look at a configuration template directory, you will
see the ``bblayers.conf.sample``, ``local.conf.sample``, ``conf-summary.txt`` and
``conf-notes.txt`` files. The build system uses these files to form the
respective ``bblayers.conf`` file, ``local.conf`` file, and show
users usage information about the build they're setting up
when running the ``oe-init-build-env`` setup script. These can be
edited further if needed to improve or change the build configurations
available to the users, and provide useful summaries and detailed usage notes.

222
sources/poky/documentation/dev-manual/customizing-images.rst Normal file
View File

@@ -0,0 +1,222 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Customizing Images
******************
You can customize images to satisfy particular requirements. This
section describes several methods and provides guidelines for each.
Customizing Images Using ``local.conf``
=======================================
Probably the easiest way to customize an image is to add a package by
way of the ``local.conf`` configuration file. Because it is limited to
local use, this method generally only allows you to add packages and is
not as flexible as creating your own customized image. When you add
packages using local variables this way, you need to realize that these
variable changes are in effect for every build and consequently affect
all images, which might not be what you require.
To add a package to your image using the local configuration file, use
the :term:`IMAGE_INSTALL` variable with the ``:append`` operator::
IMAGE_INSTALL:append = " strace"
Use of the syntax is important; specifically, the leading space
after the opening quote and before the package name, which is
``strace`` in this example. This space is required since the ``:append``
operator does not add the space.
Furthermore, you must use ``:append`` instead of the ``+=`` operator if
you want to avoid ordering issues. The reason for this is because doing
so unconditionally appends to the variable and avoids ordering problems
due to the variable being set in image recipes and ``.bbclass`` files
with operators like ``?=``. Using ``:append`` ensures the operation
takes effect.
As shown in its simplest use, ``IMAGE_INSTALL:append`` affects all
images. It is possible to extend the syntax so that the variable applies
to a specific image only. Here is an example::
IMAGE_INSTALL:append:pn-core-image-minimal = " strace"
This example adds ``strace`` to the ``core-image-minimal`` image only.
You can add packages using a similar approach through the
:term:`CORE_IMAGE_EXTRA_INSTALL` variable. If you use this variable, only
``core-image-*`` images are affected.
Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES``
===============================================================================
Another method for customizing your image is to enable or disable
high-level image features by using the
:term:`IMAGE_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES`
variables. Although the functions for both variables are nearly
equivalent, best practices dictate using :term:`IMAGE_FEATURES` from within
a recipe and using :term:`EXTRA_IMAGE_FEATURES` from within your
``local.conf`` file, which is found in the :term:`Build Directory`.
To understand how these features work, the best reference is
:ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`.
This class lists out the available
:term:`IMAGE_FEATURES` of which most map to package groups while some, such
as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general
configuration settings.
In summary, the file looks at the contents of the :term:`IMAGE_FEATURES`
variable and then maps or configures the feature accordingly. Based on
this information, the build system automatically adds the appropriate
packages or configurations to the
:term:`IMAGE_INSTALL` variable.
Effectively, you are enabling extra features by extending the class or
creating a custom class for use with specialized image ``.bb`` files.
Use the :term:`EXTRA_IMAGE_FEATURES` variable from within your local
configuration file. Using a separate area from which to enable features
with this variable helps you avoid overwriting the features in the image
recipe that are enabled with :term:`IMAGE_FEATURES`. The value of
:term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within
``meta/conf/bitbake.conf``.
To illustrate how you can use these variables to modify your image, consider an
example that selects the SSH server. The Yocto Project ships with two SSH
servers you can use with your images: Dropbear and OpenSSH. Dropbear is a
minimal SSH server appropriate for resource-constrained environments, while
OpenSSH is a well-known standard SSH server implementation. By default, the
``core-image-sato`` image is configured to use Dropbear. The
``core-image-full-cmdline`` image includes OpenSSH. The ``core-image-minimal``
image does not contain an SSH server.
You can customize your image and change these defaults. Edit the
:term:`IMAGE_FEATURES` variable in your recipe or use the
:term:`EXTRA_IMAGE_FEATURES` in your ``local.conf`` file so that it
configures the image you are working with to include
``ssh-server-dropbear`` or ``ssh-server-openssh``.
.. note::
See the ":ref:`ref-manual/features:image features`" section in the Yocto
Project Reference Manual for a complete list of image features that ship
with the Yocto Project.
Customizing Images Using Custom .bb Files
=========================================
You can also customize an image by creating a custom recipe that defines
additional software as part of the image. The following example shows
the form for the two lines you need::
IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
inherit core-image
Defining the software using a custom recipe gives you total control over
the contents of the image. It is important to use the correct names of
packages in the :term:`IMAGE_INSTALL` variable. You must use the
OpenEmbedded notation and not the Debian notation for the names (e.g.
``glibc-dev`` instead of ``libc6-dev``).
The other method for creating a custom image is to base it on an
existing image. For example, if you want to create an image based on
``core-image-sato`` but add the additional package ``strace`` to the
image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new
``.bb`` and add the following line to the end of the copy::
IMAGE_INSTALL += "strace"
Customizing Images Using Custom Package Groups
==============================================
For complex custom images, the best approach for customizing an image is
to create a custom package group recipe that is used to build the image
or images. A good example of a package group recipe is
``meta/recipes-core/packagegroups/packagegroup-base.bb``.
If you examine that recipe, you see that the :term:`PACKAGES` variable lists
the package group packages to produce. The ``inherit packagegroup``
statement sets appropriate default values and automatically adds
``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each
package specified in the :term:`PACKAGES` statement.
.. note::
The ``inherit packagegroup`` line should be located near the top of the
recipe, certainly before the :term:`PACKAGES` statement.
For each package you specify in :term:`PACKAGES`, you can use :term:`RDEPENDS`
and :term:`RRECOMMENDS` entries to provide a list of packages the parent
task package should contain. You can see examples of these further down
in the ``packagegroup-base.bb`` recipe.
Here is a short, fabricated example showing the same basic pieces for a
hypothetical packagegroup defined in ``packagegroup-custom.bb``, where
the variable :term:`PN` is the standard way to abbreviate the reference to
the full packagegroup name ``packagegroup-custom``::
DESCRIPTION = "My Custom Package Groups"
inherit packagegroup
PACKAGES = "\
${PN}-apps \
${PN}-tools \
"
RDEPENDS:${PN}-apps = "\
dropbear \
portmap \
psplash"
RDEPENDS:${PN}-tools = "\
oprofile \
oprofileui-server \
lttng-tools"
RRECOMMENDS:${PN}-tools = "\
kernel-module-oprofile"
In the previous example, two package group packages are created with
their dependencies and their recommended package dependencies listed:
``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To
build an image using these package group packages, you need to add
``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to
:term:`IMAGE_INSTALL`. For other forms of image dependencies see the other
areas of this section.
Customizing an Image Hostname
=============================
By default, the configured hostname (i.e. ``/etc/hostname``) in an image
is the same as the machine name. For example, if
:term:`MACHINE` equals "qemux86", the
configured hostname written to ``/etc/hostname`` is "qemux86".
You can customize this name by altering the value of the "hostname"
variable in the ``base-files`` recipe using either an append file or a
configuration file. Use the following in an append file::
hostname = "myhostname"
Use the following in a configuration file::
hostname:pn-base-files = "myhostname"
Changing the default value of the variable "hostname" can be useful in
certain situations. For example, suppose you need to do extensive
testing on an image and you would like to easily identify the image
under test from existing images with typical default hostnames. In this
situation, you could change the default hostname to "testme", which
results in all the images using the name "testme". Once testing is
complete and you do not need to rebuild the image for test any longer,
you can easily reset the default hostname.
Another point of interest is that if you unset the variable, the image
will have no default hostname in the filesystem. Here is an example that
unsets the variable in a configuration file::
hostname:pn-base-files = ""
Having no default hostname in the filesystem is suitable for
environments that use dynamic hostnames such as virtual machines.

1271
sources/poky/documentation/dev-manual/debugging.rst Normal file
View File

File diff suppressed because it is too large Load Diff

82
sources/poky/documentation/dev-manual/development-shell.rst Normal file
View File

@@ -0,0 +1,82 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using a Development Shell
*************************
When debugging certain commands or even when just editing packages,
``devshell`` can be a useful tool. When you invoke ``devshell``, all
tasks up to and including
:ref:`ref-tasks-patch` are run for the
specified target. Then, a new terminal is opened and you are placed in
``${``\ :term:`S`\ ``}``, the source
directory. In the new terminal, all the OpenEmbedded build-related
environment variables are still defined so you can use commands such as
``configure`` and ``make``. The commands execute just as if the
OpenEmbedded build system were executing them. Consequently, working
this way can be helpful when debugging a build or preparing software to
be used with the OpenEmbedded build system.
Here is an example that uses ``devshell`` on a target named
``matchbox-desktop``::
$ bitbake matchbox-desktop -c devshell
This command spawns a terminal with a shell prompt within the
OpenEmbedded build environment. The
:term:`OE_TERMINAL` variable
controls what type of shell is opened.
For spawned terminals, the following occurs:
- The ``PATH`` variable includes the cross-toolchain.
- The ``pkgconfig`` variables find the correct ``.pc`` files.
- The ``configure`` command finds the Yocto Project site files as well
as any other necessary files.
Within this environment, you can run configure or compile commands as if
they were being run by the OpenEmbedded build system itself. As noted
earlier, the working directory also automatically changes to the Source
Directory (:term:`S`).
To manually run a specific task using ``devshell``, run the
corresponding ``run.*`` script in the
``${``\ :term:`WORKDIR`\ ``}/temp``
directory (e.g., ``run.do_configure.``\ `pid`). If a task's script does
not exist, which would be the case if the task was skipped by way of the
sstate cache, you can create the task by first running it outside of the
``devshell``::
$ bitbake -c task
.. note::
- Execution of a task's ``run.*`` script and BitBake's execution of
a task are identical. In other words, running the script re-runs
the task just as it would be run using the ``bitbake -c`` command.
- Any ``run.*`` file that does not have a ``.pid`` extension is a
symbolic link (symlink) to the most recent version of that file.
Remember, that the ``devshell`` is a mechanism that allows you to get
into the BitBake task execution environment. And as such, all commands
must be called just as BitBake would call them. That means you need to
provide the appropriate options for cross-compilation and so forth as
applicable.
When you are finished using ``devshell``, exit the shell or close the
terminal window.
.. note::
- It is worth remembering that when using ``devshell`` you need to
use the full compiler name such as ``arm-poky-linux-gnueabi-gcc``
instead of just using ``gcc``. The same applies to other
applications such as ``binutils``, ``libtool`` and so forth.
BitBake sets up environment variables such as :term:`CC` to assist
applications, such as ``make`` to find the correct tools.
- It is also worth noting that ``devshell`` still works over X11
forwarding and similar situations.

74
sources/poky/documentation/dev-manual/device-manager.rst Normal file
View File

@@ -0,0 +1,74 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. _device-manager:
Selecting a Device Manager
**************************
The Yocto Project provides multiple ways to manage the device manager
(``/dev``):
- Persistent and Pre-Populated ``/dev``: For this case, the ``/dev``
directory is persistent and the required device nodes are created
during the build.
- Use ``devtmpfs`` with a Device Manager: For this case, the ``/dev``
directory is provided by the kernel as an in-memory file system and
is automatically populated by the kernel at runtime. Additional
configuration of device nodes is done in user space by a device
manager like ``udev`` or ``busybox-mdev``.
Using Persistent and Pre-Populated ``/dev``
===========================================
To use the static method for device population, you need to set the
:term:`USE_DEVFS` variable to "0"
as follows::
USE_DEVFS = "0"
The content of the resulting ``/dev`` directory is defined in a Device
Table file. The
:term:`IMAGE_DEVICE_TABLES`
variable defines the Device Table to use and should be set in the
machine or distro configuration file. Alternatively, you can set this
variable in your ``local.conf`` configuration file.
If you do not define the :term:`IMAGE_DEVICE_TABLES` variable, the default
``device_table-minimal.txt`` is used::
IMAGE_DEVICE_TABLES = "device_table-mymachine.txt"
The population is handled by the ``makedevs`` utility during image
creation:
Using ``devtmpfs`` and a Device Manager
=======================================
To use the dynamic method for device population, you need to use (or be
sure to set) the :term:`USE_DEVFS`
variable to "1", which is the default::
USE_DEVFS = "1"
With this
setting, the resulting ``/dev`` directory is populated by the kernel
using ``devtmpfs``. Make sure the corresponding kernel configuration
variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux
kernel.
All devices created by ``devtmpfs`` will be owned by ``root`` and have
permissions ``0600``.
To have more control over the device nodes, you can use a device manager like
``udev`` or ``busybox-mdev``. You choose the device manager by defining the
:term:`VIRTUAL-RUNTIME_dev_manager <VIRTUAL-RUNTIME>` variable in your machine
or distro configuration file. Alternatively, you can set this variable in
your ``local.conf`` configuration file::
VIRTUAL-RUNTIME_dev_manager = "udev"
# Some alternative values
# VIRTUAL-RUNTIME_dev_manager = "busybox-mdev"
# VIRTUAL-RUNTIME_dev_manager = "systemd"

61
sources/poky/documentation/dev-manual/disk-space.rst Normal file
View File

@@ -0,0 +1,61 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Conserving Disk Space
*********************
Conserving Disk Space During Builds
===================================
To help conserve disk space during builds, you can add the following
statement to your project's ``local.conf`` configuration file found in
the :term:`Build Directory`::
INHERIT += "rm_work"
Adding this statement deletes the work directory used for
building a recipe once the recipe is built. For more information on
"rm_work", see the :ref:`ref-classes-rm-work` class in the
Yocto Project Reference Manual.
When you inherit this class and build a ``core-image-sato`` image for a
``qemux86-64`` machine from an Ubuntu 22.04 x86-64 system, you end up with a
final disk usage of 22 Gbytes instead of &MIN_DISK_SPACE; Gbytes. However,
&MIN_DISK_SPACE_RM_WORK; Gbytes of initial free disk space are still needed to
create temporary files before they can be deleted.
Purging Obsolete Shared State Cache Files
=========================================
After multiple build iterations, the Shared State (sstate) cache can contain
multiple cache files for a given package, consuming a substantial amount of
disk space. However, only the most recent ones are likely to be reused.
The following command is a quick way to purge all the cache files which
haven't been used for a least a specified number of days::
find build/sstate-cache -type f -mtime +$DAYS -delete
The above command relies on the fact that BitBake touches the sstate cache
files as it accesses them, when it has write access to the cache.
You could use ``-atime`` instead of ``-mtime`` if the partition isn't mounted
with the ``noatime`` option for a read only cache.
For more advanced needs, OpenEmbedded-Core also offers a more elaborate
command. It has the ability to purge all but the newest cache files on each
architecture, and also to remove files that it considers unreachable by
exploring a set of build configurations. However, this command
requires a full build environment to be available and doesn't work well
covering multiple releases. It won't work either on limited environments
such as BSD based NAS::
sstate-cache-management.py --remove-duplicated --cache-dir=sstate-cache
This command will ask you to confirm the deletions it identifies.
Run ``sstate-cache-management.sh`` for more details about this script.
.. note::
As this command is much more cautious and selective, removing only cache files,
it will execute much slower than the simple ``find`` command described above.
Therefore, it may not be your best option to trim huge cache directories.

68
sources/poky/documentation/dev-manual/efficiently-fetching-sources.rst Normal file
View File

@@ -0,0 +1,68 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Efficiently Fetching Source Files During a Build
************************************************
The OpenEmbedded build system works with source files located through
the :term:`SRC_URI` variable. When
you build something using BitBake, a big part of the operation is
locating and downloading all the source tarballs. For images,
downloading all the source for various packages can take a significant
amount of time.
This section shows you how you can use mirrors to speed up fetching
source files and how you can pre-fetch files all of which leads to more
efficient use of resources and time.
Setting up Effective Mirrors
============================
A good deal that goes into a Yocto Project build is simply downloading
all of the source tarballs. Maybe you have been working with another
build system for which you have built up a
sizable directory of source tarballs. Or, perhaps someone else has such
a directory for which you have read access. If so, you can save time by
adding statements to your configuration file so that the build process
checks local directories first for existing tarballs before checking the
Internet.
Here is an efficient way to set it up in your ``local.conf`` file::
SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"
# BB_NO_NETWORK = "1"
In the previous example, the
:term:`BB_GENERATE_MIRROR_TARBALLS`
variable causes the OpenEmbedded build system to generate tarballs of
the Git repositories and store them in the
:term:`DL_DIR` directory. Due to
performance reasons, generating and storing these tarballs is not the
build system's default behavior.
You can also use the
:term:`PREMIRRORS` variable. For
an example, see the variable's glossary entry in the Yocto Project
Reference Manual.
Getting Source Files and Suppressing the Build
==============================================
Another technique you can use to ready yourself for a successive string
of build operations, is to pre-fetch all the source files without
actually starting a build. This technique lets you work through any
download issues and ultimately gathers all the source files into your
download directory :ref:`structure-build-downloads`,
which is located with :term:`DL_DIR`.
Use the following BitBake command form to fetch all the necessary
sources without starting the build::
$ bitbake target --runall=fetch
This
variation of the BitBake command guarantees that you have all the
sources for that BitBake target should you disconnect from the Internet
and want to do the build later offline.

84
sources/poky/documentation/dev-manual/error-reporting-tool.rst Normal file
View File

@@ -0,0 +1,84 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using the Error Reporting Tool
******************************
The error reporting tool allows you to submit errors encountered during
builds to a central database. Outside of the build environment, you can
use a web interface to browse errors, view statistics, and query for
errors. The tool works using a client-server system where the client
portion is integrated with the installed Yocto Project
:term:`Source Directory` (e.g. ``poky``).
The server receives the information collected and saves it in a
database.
There is a live instance of the error reporting server at
https://errors.yoctoproject.org.
When you want to get help with build failures, you can submit all of the
information on the failure easily and then point to the URL in your bug
report or send an email to the mailing list.
.. note::
If you send error reports to this server, the reports become publicly
visible.
Enabling and Using the Tool
===========================
By default, the error reporting tool is disabled. You can enable it by
inheriting the :ref:`ref-classes-report-error` class by adding the
following statement to the end of your ``local.conf`` file in your
:term:`Build Directory`::
INHERIT += "report-error"
By default, the error reporting feature stores information in
``${``\ :term:`LOG_DIR`\ ``}/error-report``.
However, you can specify a directory to use by adding the following to
your ``local.conf`` file::
ERR_REPORT_DIR = "path"
Enabling error
reporting causes the build process to collect the errors and store them
in a file as previously described. When the build system encounters an
error, it includes a command as part of the console output. You can run
the command to send the error file to the server. For example, the
following command sends the errors to an upstream server::
$ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
In the previous example, the errors are sent to a public database
available at https://errors.yoctoproject.org, which is used by the
entire community. If you specify a particular server, you can send the
errors to a different database. Use the following command for more
information on available options::
$ send-error-report --help
When sending the error file, you are prompted to review the data being
sent as well as to provide a name and optional email address. Once you
satisfy these prompts, the command returns a link from the server that
corresponds to your entry in the database. For example, here is a
typical link: https://errors.yoctoproject.org/Errors/Details/9522/
Following the link takes you to a web interface where you can browse,
query the errors, and view statistics.
Disabling the Tool
==================
To disable the error reporting feature, simply remove or comment out the
following statement from the end of your ``local.conf`` file in your
:term:`Build Directory`::
INHERIT += "report-error"
Setting Up Your Own Error Reporting Server
==========================================
If you want to set up your own error reporting server, you can obtain
the code from the Git repository at :yocto_git:`/error-report-web/`.
Instructions on how to set it up are in the README document.

70
sources/poky/documentation/dev-manual/external-scm.rst Normal file
View File

@@ -0,0 +1,70 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using an External SCM
*********************
If you're working on a recipe that pulls from an external Source Code
Manager (SCM), it is possible to have the OpenEmbedded build system
notice new recipe changes added to the SCM and then build the resulting
packages that depend on the new recipes by using the latest versions.
This only works for SCMs from which it is possible to get a sensible
revision number for changes. Currently, you can do this with Apache
Subversion (SVN), Git, and Bazaar (BZR) repositories.
To enable this behavior, the :term:`PV` of
the recipe needs to include a ``+`` sign in its assignment.
Here is an example::
PV = "1.2.3+git"
:term:`Bitbake` later includes the source control information in :term:`PKGV`
during the packaging phase.
Then, you can add the following to your
``local.conf``::
SRCREV:pn-PN = "${AUTOREV}"
:term:`PN` is the name of the recipe for
which you want to enable automatic source revision updating.
If you do not want to update your local configuration file, you can add
the following directly to the recipe to finish enabling the feature::
SRCREV = "${AUTOREV}"
The Yocto Project provides a distribution named ``poky-bleeding``, whose
configuration file contains the line::
require conf/distro/include/poky-floating-revisions.inc
This line pulls in the
listed include file that contains numerous lines of exactly that form::
#SRCREV:pn-opkg-native ?= "${AUTOREV}"
#SRCREV:pn-opkg-sdk ?= "${AUTOREV}"
#SRCREV:pn-opkg ?= "${AUTOREV}"
#SRCREV:pn-opkg-utils-native ?= "${AUTOREV}"
#SRCREV:pn-opkg-utils ?= "${AUTOREV}"
SRCREV:pn-gconf-dbus ?= "${AUTOREV}"
SRCREV:pn-matchbox-common ?= "${AUTOREV}"
SRCREV:pn-matchbox-config-gtk ?= "${AUTOREV}"
SRCREV:pn-matchbox-desktop ?= "${AUTOREV}"
SRCREV:pn-matchbox-keyboard ?= "${AUTOREV}"
SRCREV:pn-matchbox-panel-2 ?= "${AUTOREV}"
SRCREV:pn-matchbox-themes-extra ?= "${AUTOREV}"
SRCREV:pn-matchbox-terminal ?= "${AUTOREV}"
SRCREV:pn-matchbox-wm ?= "${AUTOREV}"
SRCREV:pn-settings-daemon ?= "${AUTOREV}"
SRCREV:pn-screenshot ?= "${AUTOREV}"
. . .
These lines allow you to
experiment with building a distribution that tracks the latest
development source for numerous packages.
.. note::
The ``poky-bleeding`` distribution is not tested on a regular basis. Keep
this in mind if you use it.

40
sources/poky/documentation/dev-manual/external-toolchain.rst Normal file
View File

@@ -0,0 +1,40 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Optionally Using an External Toolchain
**************************************
You might want to use an external toolchain as part of your development.
If this is the case, the fundamental steps you need to accomplish are as
follows:
- Understand where the installed toolchain resides. For cases where you
need to build the external toolchain, you would need to take separate
steps to build and install the toolchain.
- Make sure you add the layer that contains the toolchain to your
``bblayers.conf`` file through the
:term:`BBLAYERS` variable.
- Set the :term:`EXTERNAL_TOOLCHAIN` variable in your ``local.conf`` file
to the location in which you installed the toolchain.
The toolchain configuration is very flexible and customizable. It
is primarily controlled with the :term:`TCMODE` variable. This variable
controls which ``tcmode-*.inc`` file to include from the
``meta/conf/distro/include`` directory within the :term:`Source Directory`.
The default value of :term:`TCMODE` is "default", which tells the
OpenEmbedded build system to use its internally built toolchain (i.e.
``tcmode-default.inc``). However, other patterns are accepted. In
particular, "external-\*" refers to external toolchains. One example is
the Mentor Graphics Sourcery G++ Toolchain. Support for this toolchain resides
in the separate ``meta-sourcery`` layer at
https://github.com/MentorEmbedded/meta-sourcery/.
See its ``README`` file for details about how to use this layer.
Another example of external toolchain layer is
:yocto_git:`meta-arm-toolchain </meta-arm/tree/meta-arm-toolchain/>`
supporting GNU toolchains released by ARM.
You can find further information by reading about the :term:`TCMODE` variable
in the Yocto Project Reference Manual's variable glossary.

BIN
sources/poky/documentation/dev-manual/figures/bitbake-build-flow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

BIN
sources/poky/documentation/dev-manual/figures/buildhistory-web.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

BIN
sources/poky/documentation/dev-manual/figures/buildhistory.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

BIN
sources/poky/documentation/dev-manual/figures/cute-files-npm-example.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 72 KiB

BIN
sources/poky/documentation/dev-manual/figures/dev-title.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
sources/poky/documentation/dev-manual/figures/multiconfig_files.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
sources/poky/documentation/dev-manual/figures/recipe-workflow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB

155
sources/poky/documentation/dev-manual/gobject-introspection.rst Normal file
View File

@@ -0,0 +1,155 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Enabling GObject Introspection Support
**************************************
`GObject introspection <https://gi.readthedocs.io/en/latest/>`__
is the standard mechanism for accessing GObject-based software from
runtime environments. GObject is a feature of the GLib library that
provides an object framework for the GNOME desktop and related software.
GObject Introspection adds information to GObject that allows objects
created within it to be represented across different programming
languages. If you want to construct GStreamer pipelines using Python, or
control UPnP infrastructure using Javascript and GUPnP, GObject
introspection is the only way to do it.
This section describes the Yocto Project support for generating and
packaging GObject introspection data. GObject introspection data is a
description of the API provided by libraries built on top of the GLib
framework, and, in particular, that framework's GObject mechanism.
GObject Introspection Repository (GIR) files go to ``-dev`` packages,
``typelib`` files go to main packages as they are packaged together with
libraries that are introspected.
The data is generated when building such a library, by linking the
library with a small executable binary that asks the library to describe
itself, and then executing the binary and processing its output.
Generating this data in a cross-compilation environment is difficult
because the library is produced for the target architecture, but its
code needs to be executed on the build host. This problem is solved with
the OpenEmbedded build system by running the code through QEMU, which
allows precisely that. Unfortunately, QEMU does not always work
perfectly as mentioned in the ":ref:`dev-manual/gobject-introspection:known issues`"
section.
Enabling the Generation of Introspection Data
=============================================
Enabling the generation of introspection data (GIR files) in your
library package involves the following:
#. Inherit the :ref:`ref-classes-gobject-introspection` class.
#. Make sure introspection is not disabled anywhere in the recipe or
from anything the recipe includes. Also, make sure that
"gobject-introspection-data" is not in
:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
and that "qemu-usermode" is not in
:term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
In either of these conditions, nothing will happen.
#. Try to build the recipe. If you encounter build errors that look like
something is unable to find ``.so`` libraries, check where these
libraries are located in the source tree and add the following to the
recipe::
GIR_EXTRA_LIBS_PATH = "${B}/something/.libs"
.. note::
See recipes in the ``oe-core`` repository that use that
:term:`GIR_EXTRA_LIBS_PATH` variable as an example.
#. Look for any other errors, which probably mean that introspection
support in a package is not entirely standard, and thus breaks down
in a cross-compilation environment. For such cases, custom-made fixes
are needed. A good place to ask and receive help in these cases is
the :ref:`Yocto Project mailing
lists <resources-mailinglist>`.
.. note::
Using a library that no longer builds against the latest Yocto
Project release and prints introspection related errors is a good
candidate for the previous procedure.
Disabling the Generation of Introspection Data
==============================================
You might find that you do not want to generate introspection data. Or,
perhaps QEMU does not work on your build host and target architecture
combination. If so, you can use either of the following methods to
disable GIR file generations:
- Add the following to your distro configuration::
DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
Adding this statement disables generating introspection data using
QEMU but will still enable building introspection tools and libraries
(i.e. building them does not require the use of QEMU).
- Add the following to your machine configuration::
MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode"
Adding this statement disables the use of QEMU when building packages for your
machine. Currently, this feature is used only by introspection
recipes and has the same effect as the previously described option.
.. note::
Future releases of the Yocto Project might have other features
affected by this option.
If you disable introspection data, you can still obtain it through other
means such as copying the data from a suitable sysroot, or by generating
it on the target hardware. The OpenEmbedded build system does not
currently provide specific support for these techniques.
Testing that Introspection Works in an Image
============================================
Use the following procedure to test if generating introspection data is
working in an image:
#. Make sure that "gobject-introspection-data" is not in
:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
and that "qemu-usermode" is not in
:term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
#. Build ``core-image-sato``.
#. Launch a Terminal and then start Python in the terminal.
#. Enter the following in the terminal::
>>> from gi.repository import GLib
>>> GLib.get_host_name()
#. For something a little more advanced, enter the following see:
https://python-gtk-3-tutorial.readthedocs.io/en/latest/introduction.html
Known Issues
============
Here are know issues in GObject Introspection Support:
- ``qemu-ppc64`` immediately crashes. Consequently, you cannot build
introspection data on that architecture.
- x32 is not supported by QEMU. Consequently, introspection data is
disabled.
- musl causes transient GLib binaries to crash on assertion failures.
Consequently, generating introspection data is disabled.
- Because QEMU is not able to run the binaries correctly, introspection
is disabled for some specific packages under specific architectures
(e.g. ``gcr``, ``libsecret``, and ``webkit``).
- QEMU usermode might not work properly when running 64-bit binaries
under 32-bit host machines. In particular, "qemumips64" is known to
not work under i686.

52
sources/poky/documentation/dev-manual/index.rst Normal file
View File

@@ -0,0 +1,52 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
======================================
Yocto Project Development Tasks Manual
======================================
.. toctree::
:caption: Table of Contents
:numbered:
intro
start
layers
customizing-images
new-recipe
new-machine
upgrading-recipes
temporary-source-code
quilt.rst
development-shell
python-development-shell
building
speeding-up-build
libraries
prebuilt-libraries
x32-psabi
gobject-introspection
external-toolchain
wic
bmaptool
securing-images
custom-distribution
custom-template-configuration-directory
disk-space
packages
efficiently-fetching-sources
init-manager
device-manager
external-scm
read-only-rootfs
build-quality
debugging
licenses
security-subjects
vulnerabilities
sbom
error-reporting-tool
wayland
qemu
bblock
.. include:: /boilerplate.rst

162
sources/poky/documentation/dev-manual/init-manager.rst Normal file
View File

@@ -0,0 +1,162 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. _init-manager:
Selecting an Initialization Manager
***********************************
By default, the Yocto Project uses :wikipedia:`SysVinit <Init#SysV-style>` as
the initialization manager. There is also support for BusyBox init, a simpler
implementation, as well as support for :wikipedia:`systemd <Systemd>`, which
is a full replacement for init with parallel starting of services, reduced
shell overhead, increased security and resource limits for services, and other
features that are used by many distributions.
Within the system, SysVinit and BusyBox init treat system components as
services. These services are maintained as shell scripts stored in the
``/etc/init.d/`` directory.
SysVinit is more elaborate than BusyBox init and organizes services in
different run levels. This organization is maintained by putting links
to the services in the ``/etc/rcN.d/`` directories, where `N/` is one
of the following options: "S", "0", "1", "2", "3", "4", "5", or "6".
.. note::
Each runlevel has a dependency on the previous runlevel. This
dependency allows the services to work properly.
Both SysVinit and BusyBox init are configured through the ``/etc/inittab``
file, with a very similar syntax, though of course BusyBox init features
are more limited.
In comparison, systemd treats components as units. Using units is a
broader concept as compared to using a service. A unit includes several
different types of entities. ``Service`` is one of the types of entities.
The runlevel concept in SysVinit corresponds to the concept of a target
in systemd, where target is also a type of supported unit.
In systems with SysVinit or BusyBox init, services load sequentially (i.e. one
by one) during init and parallelization is not supported. With systemd, services
start in parallel. This method can have an impact on the startup performance
of a given service, though systemd will also provide more services by default,
therefore increasing the total system boot time. systemd also substantially
increases system size because of its multiple components and the extra
dependencies it pulls.
On the contrary, BusyBox init is the simplest and the lightest solution and
also comes with BusyBox mdev as device manager, a lighter replacement to
:wikipedia:`udev <Udev>`, which SysVinit and systemd both use.
The ":ref:`device-manager`" chapter has more details about device managers.
Using SysVinit with udev
=========================
SysVinit with the udev device manager corresponds to the
default setting in Poky. This corresponds to setting::
INIT_MANAGER = "sysvinit"
Using BusyBox init with BusyBox mdev
====================================
BusyBox init with BusyBox mdev is the simplest and lightest solution
for small root filesystems. All you need is BusyBox, which most systems
have anyway::
INIT_MANAGER = "mdev-busybox"
Using systemd
=============
The last option is to use systemd together with the udev device
manager. This is the most powerful and versatile solution, especially
for more complex systems::
INIT_MANAGER = "systemd"
This will enable systemd and remove sysvinit components from the image.
See :yocto_git:`meta/conf/distro/include/init-manager-systemd.inc
</poky/tree/meta/conf/distro/include/init-manager-systemd.inc>` for exact
details on what this does.
Controling systemd from the target command line
-----------------------------------------------
Here is a quick reference for controling systemd from the command line on the
target. Instead of opening and sometimes modifying files, most interaction
happens through the ``systemctl`` and ``journalctl`` commands:
- ``systemctl status``: show the status of all services
- ``systemctl status <service>``: show the status of one service
- ``systemctl [start|stop] <service>``: start or stop a service
- ``systemctl [enable|disable] <service>``: enable or disable a service at boot time
- ``systemctl list-units``: list all available units
- ``journalctl -a``: show all logs for all services
- ``journalctl -f``: show only the last log entries, and keep printing updates as they arrive
- ``journalctl -u``: show only logs from a particular service
Using systemd-journald without a traditional syslog daemon
----------------------------------------------------------
Counter-intuitively, ``systemd-journald`` is not a syslog runtime or provider,
and the proper way to use ``systemd-journald`` as your sole logging mechanism is to
effectively disable syslog entirely by setting these variables in your distribution
configuration file::
VIRTUAL-RUNTIME_syslog = ""
VIRTUAL-RUNTIME_base-utils-syslog = ""
Doing so will prevent ``rsyslog`` / ``busybox-syslog`` from being pulled in by
default, leaving only ``systemd-journald``.
Summary
-------
The Yocto Project supports three different initialization managers, offering
increasing levels of complexity and functionality:
.. list-table::
:widths: 40 20 20 20
:header-rows: 1
* -
- BusyBox init
- SysVinit
- systemd
* - Size
- Small
- Small
- Big [#footnote-systemd-size]_
* - Complexity
- Small
- Medium
- High
* - Support for boot profiles
- No
- Yes ("runlevels")
- Yes ("targets")
* - Services defined as
- Shell scripts
- Shell scripts
- Description files
* - Starting services in parallel
- No
- No
- Yes
* - Setting service resource limits
- No
- No
- Yes
* - Support service isolation
- No
- No
- Yes
* - Integrated logging
- No
- No
- Yes
.. [#footnote-systemd-size] Using systemd increases the ``core-image-minimal``
image size by 160\% for ``qemux86-64`` on Mickledore (4.2), compared to SysVinit.

59
sources/poky/documentation/dev-manual/intro.rst Normal file
View File

@@ -0,0 +1,59 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
******************************************
The Yocto Project Development Tasks Manual
******************************************
Welcome
=======
Welcome to the Yocto Project Development Tasks Manual. This manual
provides relevant procedures necessary for developing in the Yocto
Project environment (i.e. developing embedded Linux images and
user-space applications that run on targeted devices). This manual groups
related procedures into higher-level sections. Procedures can consist of
high-level steps or low-level steps depending on the topic.
This manual provides the following:
- Procedures that help you get going with the Yocto Project; for
example, procedures that show you how to set up a build host and work
with the Yocto Project source repositories.
- Procedures that show you how to submit changes to the Yocto Project.
Changes can be improvements, new features, or bug fixes.
- Procedures related to "everyday" tasks you perform while developing
images and applications using the Yocto Project, such as
creating a new layer, customizing an image, writing a new recipe,
and so forth.
This manual does not provide the following:
- Redundant step-by-step instructions: For example, the
:doc:`/sdk-manual/index` manual contains detailed
instructions on how to install an SDK, which is used to develop
applications for target hardware.
- Reference or conceptual material: This type of material resides in an
appropriate reference manual. As an example, system variables are
documented in the :doc:`/ref-manual/index`.
- Detailed public information not specific to the Yocto Project: For
example, exhaustive information on how to use the Git version
control system is better covered with Internet searches and official Git
documentation than through the Yocto Project documentation.
Other Information
=================
Because this manual presents information for many different topics,
supplemental information is recommended for full comprehension. For
introductory information on the Yocto Project, see the
:yocto_home:`Yocto Project Website <>`. If you want to build an image with no
knowledge of Yocto Project as a way of quickly testing it out, see the
:doc:`/brief-yoctoprojectqs/index` document.
For a comprehensive list of links and other documentation, see the
":ref:`ref-manual/resources:links and related documentation`"
section in the Yocto Project Reference Manual.

1009
sources/poky/documentation/dev-manual/layers.rst Normal file
View File

File diff suppressed because it is too large Load Diff

267
sources/poky/documentation/dev-manual/libraries.rst Normal file
View File

@@ -0,0 +1,267 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working With Libraries
**********************
Libraries are an integral part of your system. This section describes
some common practices you might find helpful when working with libraries
to build your system:
- :ref:`How to include static library files
<dev-manual/libraries:including static library files>`
- :ref:`How to use the Multilib feature to combine multiple versions of
library files into a single image
<dev-manual/libraries:combining multiple versions of library files into one image>`
- :ref:`How to install multiple versions of the same library in parallel on
the same system
<dev-manual/libraries:installing multiple versions of the same library>`
Including Static Library Files
==============================
If you are building a library and the library offers static linking, you
can control which static library files (``*.a`` files) get included in
the built library.
The :term:`PACKAGES` and
:term:`FILES:* <FILES>` variables in the
``meta/conf/bitbake.conf`` configuration file define how files installed
by the :ref:`ref-tasks-install` task are packaged. By default, the :term:`PACKAGES`
variable includes ``${PN}-staticdev``, which represents all static
library files.
.. note::
Some previously released versions of the Yocto Project defined the
static library files through ``${PN}-dev``.
Here is the part of the BitBake configuration file, where you can see
how the static library files are defined::
PACKAGE_BEFORE_PN ?= ""
PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
PACKAGES_DYNAMIC = "^${PN}-locale-.*"
FILES = ""
FILES:${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
${sysconfdir} ${sharedstatedir} ${localstatedir} \
${base_bindir}/* ${base_sbindir}/* \
${base_libdir}/*${SOLIBS} \
${base_prefix}/lib/udev ${prefix}/lib/udev \
${base_libdir}/udev ${libdir}/udev \
${datadir}/${BPN} ${libdir}/${BPN}/* \
${datadir}/pixmaps ${datadir}/applications \
${datadir}/idl ${datadir}/omf ${datadir}/sounds \
${libdir}/bonobo/servers"
FILES:${PN}-bin = "${bindir}/* ${sbindir}/*"
FILES:${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
${datadir}/gnome/help"
SECTION:${PN}-doc = "doc"
FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
FILES:${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
${datadir}/aclocal ${base_libdir}/*.o \
${libdir}/${BPN}/*.la ${base_libdir}/*.la \
${libdir}/cmake ${datadir}/cmake"
SECTION:${PN}-dev = "devel"
ALLOW_EMPTY:${PN}-dev = "1"
RDEPENDS:${PN}-dev = "${PN} (= ${EXTENDPKGV})"
FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
SECTION:${PN}-staticdev = "devel"
RDEPENDS:${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
Combining Multiple Versions of Library Files into One Image
===========================================================
The build system offers the ability to build libraries with different
target optimizations or architecture formats and combine these together
into one system image. You can link different binaries in the image
against the different libraries as needed for specific use cases. This
feature is called "Multilib".
An example would be where you have most of a system compiled in 32-bit
mode using 32-bit libraries, but you have something large, like a
database engine, that needs to be a 64-bit application and uses 64-bit
libraries. Multilib allows you to get the best of both 32-bit and 64-bit
libraries.
While the Multilib feature is most commonly used for 32 and 64-bit
differences, the approach the build system uses facilitates different
target optimizations. You could compile some binaries to use one set of
libraries and other binaries to use a different set of libraries. The
libraries could differ in architecture, compiler options, or other
optimizations.
There are several examples in the ``meta-skeleton`` layer found in the
:term:`Source Directory`:
- :oe_git:`conf/multilib-example.conf </openembedded-core/tree/meta-skeleton/conf/multilib-example.conf>`
configuration file.
- :oe_git:`conf/multilib-example2.conf </openembedded-core/tree/meta-skeleton/conf/multilib-example2.conf>`
configuration file.
- :oe_git:`recipes-multilib/images/core-image-multilib-example.bb </openembedded-core/tree/meta-skeleton/recipes-multilib/images/core-image-multilib-example.bb>`
recipe
Preparing to Use Multilib
-------------------------
User-specific requirements drive the Multilib feature. Consequently,
there is no one "out-of-the-box" configuration that would
meet your needs.
In order to enable Multilib, you first need to ensure your recipe is
extended to support multiple libraries. Many standard recipes are
already extended and support multiple libraries. You can check in the
``meta/conf/multilib.conf`` configuration file in the
:term:`Source Directory` to see how this is
done using the
:term:`BBCLASSEXTEND` variable.
Eventually, all recipes will be covered and this list will not be
needed.
For the most part, the :ref:`Multilib <ref-classes-multilib*>`
class extension works automatically to
extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where
:term:`MLPREFIX` is the particular multilib (e.g. "lib32-" or "lib64-").
Standard variables such as
:term:`DEPENDS`,
:term:`RDEPENDS`,
:term:`RPROVIDES`,
:term:`RRECOMMENDS`,
:term:`PACKAGES`, and
:term:`PACKAGES_DYNAMIC` are
automatically extended by the system. If you are extending any manual
code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure
those names are extended correctly.
Using Multilib
--------------
After you have set up the recipes, you need to define the actual
combination of multiple libraries you want to build. You accomplish this
through your ``local.conf`` configuration file in the
:term:`Build Directory`. An example configuration would be as follows::
MACHINE = "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 = "x86"
IMAGE_INSTALL:append = " lib32-glib-2.0"
This example enables an additional library named
``lib32`` alongside the normal target packages. When combining these
"lib32" alternatives, the example uses "x86" for tuning. For information
on this particular tuning, see
``meta/conf/machine/include/ia32/arch-ia32.inc``.
The example then includes ``lib32-glib-2.0`` in all the images, which
illustrates one method of including a multiple library dependency. You
can use a normal image build to include this dependency, for example::
$ bitbake core-image-sato
You can also build Multilib packages
specifically with a command like this::
$ bitbake lib32-glib-2.0
Additional Implementation Details
---------------------------------
There are generic implementation details as well as details that are specific to
package management systems. Here are implementation details
that exist regardless of the package management system:
- The typical convention used for the class extension code as used by
Multilib assumes that all package names specified in
:term:`PACKAGES` that contain
``${PN}`` have ``${PN}`` at the start of the name. When that
convention is not followed and ``${PN}`` appears at the middle or the
end of a name, problems occur.
- The :term:`TARGET_VENDOR`
value under Multilib will be extended to "-vendormlmultilib" (e.g.
"-pokymllib32" for a "lib32" Multilib with Poky). The reason for this
slightly unwieldy contraction is that any "-" characters in the
vendor string presently break Autoconf's ``config.sub``, and other
separators are problematic for different reasons.
Here are the implementation details for the RPM Package Management System:
- A unique architecture is defined for the Multilib packages, along
with creating a unique deploy folder under ``tmp/deploy/rpm`` in the
:term:`Build Directory`. For example, consider ``lib32`` in a
``qemux86-64`` image. The possible architectures in the system are "all",
"qemux86_64", "lib32:qemux86_64", and "lib32:x86".
- The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM
packaging. The naming for a normal RPM package and a Multilib RPM
package in a ``qemux86-64`` system resolves to something similar to
``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``,
respectively.
- When installing a Multilib image, the RPM backend first installs the
base image and then installs the Multilib libraries.
- The build system relies on RPM to resolve the identical files in the
two (or more) Multilib packages.
Here are the implementation details for the IPK Package Management System:
- The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK
packaging. The naming for a normal RPM package and a Multilib IPK
package in a ``qemux86-64`` system resolves to something like
``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw:x86.ipk``,
respectively.
- The IPK deploy folder is not modified with ``${MLPREFIX}`` because
packages with and without the Multilib feature can exist in the same
folder due to the ``${PN}`` differences.
- IPK defines a sanity check for Multilib installation using certain
rules for file comparison, overridden, etc.
Installing Multiple Versions of the Same Library
================================================
There are be situations where you need to install and use multiple versions
of the same library on the same system at the same time. This
almost always happens when a library API changes and you have
multiple pieces of software that depend on the separate versions of the
library. To accommodate these situations, you can install multiple
versions of the same library in parallel on the same system.
The process is straightforward as long as the libraries use proper
versioning. With properly versioned libraries, all you need to do to
individually specify the libraries is create separate, appropriately
named recipes where the :term:`PN` part of
the name includes a portion that differentiates each library version
(e.g. the major part of the version number). Thus, instead of having a
single recipe that loads one version of a library (e.g. ``clutter``),
you provide multiple recipes that result in different versions of the
libraries you want. As an example, the following two recipes would allow
the two separate versions of the ``clutter`` library to co-exist on the
same system:
.. code-block:: none
clutter-1.6_1.6.20.bb
clutter-1.8_1.8.4.bb
Additionally, if
you have other recipes that depend on a given library, you need to use
the :term:`DEPENDS` variable to
create the dependency. Continuing with the same example, if you want to
have a recipe depend on the 1.8 version of the ``clutter`` library, use
the following in your recipe::
DEPENDS = "clutter-1.8"

544
sources/poky/documentation/dev-manual/licenses.rst Normal file
View File

@@ -0,0 +1,544 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working With Licenses
*********************
As mentioned in the ":ref:`overview-manual/development-environment:licensing`"
section in the Yocto Project Overview and Concepts Manual, open source
projects are open to the public and they consequently have different
licensing structures in place. This section describes the mechanism by
which the :term:`OpenEmbedded Build System`
tracks changes to
licensing text and covers how to maintain open source license compliance
during your project's lifecycle. The section also describes how to
enable commercially licensed recipes, which by default are disabled.
Tracking License Changes
========================
The license of an upstream project might change in the future. In order
to prevent these changes going unnoticed, the
:term:`LIC_FILES_CHKSUM`
variable tracks changes to the license text. The checksums are validated
at the end of the configure step, and if the checksums do not match, the
build will fail.
Specifying the ``LIC_FILES_CHKSUM`` Variable
--------------------------------------------
The :term:`LIC_FILES_CHKSUM` variable contains checksums of the license text
in the source code for the recipe. Here is an example of how to
specify :term:`LIC_FILES_CHKSUM`::
LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
file://licfile2.txt;endline=50;md5=zzzz \
..."
.. note::
- When using "beginline" and "endline", realize that line numbering
begins with one and not zero. Also, the included lines are
inclusive (i.e. lines five through and including 29 in the
previous example for ``licfile1.txt``).
- When a license check fails, the selected license text is included
as part of the QA message. Using this output, you can determine
the exact start and finish for the needed license text.
The build system uses the :term:`S`
variable as the default directory when searching files listed in
:term:`LIC_FILES_CHKSUM`. The previous example employs the default
directory.
Consider this next example::
LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
md5=bb14ed3c4cda583abc85401304b5cd4e"
LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
The first line locates a file in ``${S}/src/ls.c`` and isolates lines
five through 16 as license text. The second line refers to a file in
:term:`WORKDIR`.
Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes,
unless the :term:`LICENSE` variable is set to "CLOSED".
Explanation of Syntax
---------------------
As mentioned in the previous section, the :term:`LIC_FILES_CHKSUM` variable
lists all the important files that contain the license text for the
source code. It is possible to specify a checksum for an entire file, or
a specific section of a file (specified by beginning and ending line
numbers with the "beginline" and "endline" parameters, respectively).
The latter is useful for source files with a license notice header,
README documents, and so forth. If you do not use the "beginline"
parameter, then it is assumed that the text begins on the first line of
the file. Similarly, if you do not use the "endline" parameter, it is
assumed that the license text ends with the last line of the file.
The "md5" parameter stores the md5 checksum of the license text. If the
license text changes in any way as compared to this parameter then a
mismatch occurs. This mismatch triggers a build failure and notifies the
developer. Notification allows the developer to review and address the
license text changes. Also note that if a mismatch occurs during the
build, the correct md5 checksum is placed in the build log and can be
easily copied to the recipe.
There is no limit to how many files you can specify using the
:term:`LIC_FILES_CHKSUM` variable. Generally, however, every project
requires a few specifications for license tracking. Many projects have a
"COPYING" file that stores the license information for all the source
code files. This practice allows you to just track the "COPYING" file as
long as it is kept up to date.
.. note::
- If you specify an empty or invalid "md5" parameter,
:term:`BitBake` returns an md5
mis-match error and displays the correct "md5" parameter value
during the build. The correct parameter is also captured in the
build log.
- If the whole file contains only license text, you do not need to
use the "beginline" and "endline" parameters.
Enabling Commercially Licensed Recipes
======================================
By default, the OpenEmbedded build system disables components that have
commercial or other special licensing requirements. Such requirements
are defined on a recipe-by-recipe basis through the
:term:`LICENSE_FLAGS` variable
definition in the affected recipe. For instance, the
``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
contains the following statement::
LICENSE_FLAGS = "commercial"
Here is a
slightly more complicated example that contains both an explicit recipe
name and version (after variable expansion)::
LICENSE_FLAGS = "license_${PN}_${PV}"
It is possible to give more details about a specific license
using flags on the :term:`LICENSE_FLAGS_DETAILS` variable::
LICENSE_FLAGS_DETAILS[my-eula-license] = "For further details, see https://example.com/eula."
If set, this will be displayed to the user if the license hasn't been accepted.
In order for a component restricted by a
:term:`LICENSE_FLAGS` definition to be enabled and included in an image, it
needs to have a matching entry in the global
:term:`LICENSE_FLAGS_ACCEPTED`
variable, which is a variable typically defined in your ``local.conf``
file. For example, to enable the
``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
could add either the string "commercial_gst-plugins-ugly" or the more
general string "commercial" to :term:`LICENSE_FLAGS_ACCEPTED`. See the
":ref:`dev-manual/licenses:license flag matching`" section for a full
explanation of how :term:`LICENSE_FLAGS` matching works. Here is the
example::
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly"
Likewise, to additionally enable the package built from the recipe
containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
the actual recipe name was ``emgd_1.10.bb``, the following string would
enable that package as well as the original ``gst-plugins-ugly``
package::
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly license_emgd_1.10"
As a convenience, you do not need to specify the
complete license string for every package. You can use
an abbreviated form, which consists of just the first portion or
portions of the license string before the initial underscore character
or characters. A partial string will match any license that contains the
given string as the first portion of its license. For example, the
following value will also match both of the packages
previously mentioned as well as any other packages that have licenses
starting with "commercial" or "license"::
LICENSE_FLAGS_ACCEPTED = "commercial license"
License Flag Matching
---------------------
License flag matching allows you to control what recipes the
OpenEmbedded build system includes in the build. Fundamentally, the
build system attempts to match :term:`LICENSE_FLAGS` strings found in
recipes against strings found in :term:`LICENSE_FLAGS_ACCEPTED`.
A match causes the build system to include a recipe in the
build, while failure to find a match causes the build system to exclude
a recipe.
In general, license flag matching is simple. However, understanding some
concepts will help you correctly and effectively use matching.
Before a flag defined by a particular recipe is tested against the
entries of :term:`LICENSE_FLAGS_ACCEPTED`, the expanded
string ``_${PN}`` is appended to the flag. This expansion makes each
:term:`LICENSE_FLAGS` value recipe-specific. After expansion, the
string is then matched against the entries. Thus, specifying
``LICENSE_FLAGS = "commercial"`` in recipe "foo", for example, results
in the string ``"commercial_foo"``. And, to create a match, that string
must appear among the entries of :term:`LICENSE_FLAGS_ACCEPTED`.
Judicious use of the :term:`LICENSE_FLAGS` strings and the contents of the
:term:`LICENSE_FLAGS_ACCEPTED` variable allows you a lot of flexibility for
including or excluding recipes based on licensing. For example, you can
broaden the matching capabilities by using license flags string subsets
in :term:`LICENSE_FLAGS_ACCEPTED`.
.. note::
When using a string subset, be sure to use the part of the expanded
string that precedes the appended underscore character (e.g.
``usethispart_1.3``, ``usethispart_1.4``, and so forth).
For example, simply specifying the string "commercial" in the
:term:`LICENSE_FLAGS_ACCEPTED` variable matches any expanded
:term:`LICENSE_FLAGS` definition that starts with the string
"commercial" such as "commercial_foo" and "commercial_bar", which
are the strings the build system automatically generates for
hypothetical recipes named "foo" and "bar" assuming those recipes simply
specify the following::
LICENSE_FLAGS = "commercial"
Thus, you can choose to exhaustively enumerate each license flag in the
list and allow only specific recipes into the image, or you can use a
string subset that causes a broader range of matches to allow a range of
recipes into the image.
This scheme works even if the :term:`LICENSE_FLAGS` string already has
``_${PN}`` appended. For example, the build system turns the license
flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
both the general "commercial" and the specific "commercial_1.2_foo"
strings found in the :term:`LICENSE_FLAGS_ACCEPTED` variable, as expected.
Here are some other scenarios:
- You can specify a versioned string in the recipe such as
"commercial_foo_1.2" in a "foo" recipe. The build system expands this
string to "commercial_foo_1.2_foo". Combine this license flag with a
:term:`LICENSE_FLAGS_ACCEPTED` variable that has the string
"commercial" and you match the flag along with any other flag that
starts with the string "commercial".
- Under the same circumstances, you can add "commercial_foo" in the
:term:`LICENSE_FLAGS_ACCEPTED` variable and the build system not only
matches "commercial_foo_1.2" but also matches any license flag with
the string "commercial_foo", regardless of the version.
- You can be very specific and use both the package and version parts
in the :term:`LICENSE_FLAGS_ACCEPTED` list (e.g.
"commercial_foo_1.2") to specifically match a versioned recipe.
Other Variables Related to Commercial Licenses
----------------------------------------------
There are other helpful variables related to commercial license handling,
defined in the
``poky/meta/conf/distro/include/default-distrovars.inc`` file::
COMMERCIAL_AUDIO_PLUGINS ?= ""
COMMERCIAL_VIDEO_PLUGINS ?= ""
If you want to enable these components, you can do so by making sure you have
statements similar to the following in your ``local.conf`` configuration file::
COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
gst-plugins-ugly-mpegaudioparse"
COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
Of course, you could also create a matching list for those components using the
more general "commercial" string in the :term:`LICENSE_FLAGS_ACCEPTED` variable,
but that would also enable all the other packages with :term:`LICENSE_FLAGS`
containing "commercial", which you may or may not want::
LICENSE_FLAGS_ACCEPTED = "commercial"
Specifying audio and video plugins as part of the
:term:`COMMERCIAL_AUDIO_PLUGINS` and :term:`COMMERCIAL_VIDEO_PLUGINS` statements
(along with :term:`LICENSE_FLAGS_ACCEPTED`) includes the plugins or
components into built images, thus adding support for media formats or
components.
.. note::
GStreamer "ugly" and "bad" plugins are actually available through
open source licenses. However, the "ugly" ones can be subject to software
patents in some countries, making it necessary to pay licensing fees
to distribute them. The "bad" ones are just deemed unreliable by the
GStreamer community and should therefore be used with care.
Maintaining Open Source License Compliance During Your Product's Lifecycle
==========================================================================
One of the concerns for a development organization using open source
software is how to maintain compliance with various open source
licensing during the lifecycle of the product. While this section does
not provide legal advice or comprehensively cover all scenarios, it does
present methods that you can use to assist you in meeting the compliance
requirements during a software release.
With hundreds of different open source licenses that the Yocto Project
tracks, it is difficult to know the requirements of each and every
license. However, the requirements of the major FLOSS licenses can begin
to be covered by assuming that there are three main areas of concern:
- Source code must be provided.
- License text for the software must be provided.
- Compilation scripts and modifications to the source code must be
provided.
There are other requirements beyond the scope of these three and the
methods described in this section (e.g. the mechanism through which
source code is distributed).
As different organizations have different ways of releasing software,
there can be multiple ways of meeting license obligations. At
least, we describe here two methods for achieving compliance:
- The first method is to use OpenEmbedded's ability to provide
the source code, provide a list of licenses, as well as
compilation scripts and source code modifications.
The remainder of this section describes supported methods to meet
the previously mentioned three requirements.
- The second method is to generate a *Software Bill of Materials*
(:term:`SBoM`), as described in the ":doc:`/dev-manual/sbom`" section.
Not only do you generate :term:`SPDX` output which can be used meet
license compliance requirements (except for sharing the build system
and layers sources for the time being), but this output also includes
component version and patch information which can be used
for vulnerability assessment.
Whatever method you choose, prior to releasing images, sources,
and the build system, you should audit all artifacts to ensure
completeness.
.. note::
The Yocto Project generates a license manifest during image creation
that is located in
``${DEPLOY_DIR}/licenses/${SSTATE_PKGARCH}/<image-name>-<machine>.rootfs-<datestamp>/``
to assist with any audits.
Providing the Source Code
-------------------------
Compliance activities should begin before you generate the final image.
The first thing you should look at is the requirement that tops the list
for most compliance groups --- providing the source. The Yocto Project has
a few ways of meeting this requirement.
One of the easiest ways to meet this requirement is to provide the
entire :term:`DL_DIR` used by the
build. This method, however, has a few issues. The most obvious is the
size of the directory since it includes all sources used in the build
and not just the source used in the released image. It will include
toolchain source, and other artifacts, which you would not generally
release. However, the more serious issue for most companies is
accidental release of proprietary software. The Yocto Project provides
an :ref:`ref-classes-archiver` class to help avoid some of these concerns.
Before you employ :term:`DL_DIR` or the :ref:`ref-classes-archiver` class, you
need to decide how you choose to provide source. The source
:ref:`ref-classes-archiver` class can generate tarballs and SRPMs and can
create them with various levels of compliance in mind.
One way of doing this (but certainly not the only way) is to release
just the source as a tarball. You can do this by adding the following to
the ``local.conf`` file found in the :term:`Build Directory`::
INHERIT += "archiver"
ARCHIVER_MODE[src] = "original"
During the creation of your
image, the source from all recipes that deploy packages to the image is
placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
:term:`LICENSE` for each recipe.
Releasing the entire directory enables you to comply with requirements
concerning providing the unmodified source. It is important to note that
the size of the directory can get large.
A way to help mitigate the size issue is to only release tarballs for
licenses that require the release of source. Let us assume you are only
concerned with GPL code as identified by running the following script:
.. code-block:: shell
# Script to archive a subset of packages matching specific license(s)
# Source and license files are copied into sub folders of package folder
# Must be run from build folder
#!/bin/bash
src_release_dir="source-release"
mkdir -p $src_release_dir
for a in tmp/deploy/sources/*; do
for d in $a/*; do
# Get package name from path
p=`basename $d`
p=${p%-*}
p=${p%-*}
# Only archive GPL packages (update *GPL* regex for your license check)
numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
if [ $numfiles -ge 1 ]; then
echo Archiving $p
mkdir -p $src_release_dir/$p/source
cp $d/* $src_release_dir/$p/source 2> /dev/null
mkdir -p $src_release_dir/$p/license
cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
fi
done
done
At this point, you
could create a tarball from the ``gpl_source_release`` directory and
provide that to the end user. This method would be a step toward
achieving compliance with section 3a of GPLv2 and with section 6 of
GPLv3.
Providing License Text
----------------------
One requirement that is often overlooked is inclusion of license text.
This requirement also needs to be dealt with prior to generating the
final image. Some licenses require the license text to accompany the
binary. You can achieve this by adding the following to your
``local.conf`` file::
COPY_LIC_MANIFEST = "1"
COPY_LIC_DIRS = "1"
LICENSE_CREATE_PACKAGE = "1"
Adding these statements to the
configuration file ensures that the licenses collected during package
generation are included on your image.
.. note::
Setting all three variables to "1" results in the image having two
copies of the same license file. One copy resides in
``/usr/share/common-licenses`` and the other resides in
``/usr/share/license``.
The reason for this behavior is because
:term:`COPY_LIC_DIRS` and
:term:`COPY_LIC_MANIFEST`
add a copy of the license when the image is built but do not offer a
path for adding licenses for newly installed packages to an image.
:term:`LICENSE_CREATE_PACKAGE`
adds a separate package and an upgrade path for adding licenses to an
image.
As the source :ref:`ref-classes-archiver` class has already archived the
original unmodified source that contains the license files, you would have
already met the requirements for inclusion of the license information
with source as defined by the GPL and other open source licenses.
Providing Compilation Scripts and Source Code Modifications
-----------------------------------------------------------
At this point, we have addressed all we need prior to generating the
image. The next two requirements are addressed during the final
packaging of the release.
By releasing the version of the OpenEmbedded build system and the layers
used during the build, you will be providing both compilation scripts
and the source code modifications in one step.
If the deployment team has a :ref:`overview-manual/concepts:bsp layer`
and a distro layer, and those
those layers are used to patch, compile, package, or modify (in any way)
any open source software included in your released images, you might be
required to release those layers under section 3 of GPLv2 or section 1
of GPLv3. One way of doing that is with a clean checkout of the version
of the Yocto Project and layers used during your build. Here is an
example:
.. code-block:: shell
# We built using the dunfell branch of the poky repo
$ git clone -b dunfell git://git.yoctoproject.org/poky
$ cd poky
# We built using the release_branch for our layers
$ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
$ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
# clean up the .git repos
$ find . -name ".git" -type d -exec rm -rf {} \;
One thing a development organization might want to consider for end-user
convenience is to modify
``meta-poky/conf/templates/default/bblayers.conf.sample`` to ensure that when
the end user utilizes the released build system to build an image, the
development organization's layers are included in the ``bblayers.conf`` file
automatically::
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
##OEROOT##/meta \
##OEROOT##/meta-poky \
##OEROOT##/meta-yocto-bsp \
##OEROOT##/meta-mylayer \
"
Creating and
providing an archive of the :term:`Metadata`
layers (recipes, configuration files, and so forth) enables you to meet
your requirements to include the scripts to control compilation as well
as any modifications to the original source.
Compliance Limitations with Executables Built from Static Libraries
-------------------------------------------------------------------
When package A is added to an image via the :term:`RDEPENDS` or :term:`RRECOMMENDS`
mechanisms as well as explicitly included in the image recipe with
:term:`IMAGE_INSTALL`, and depends on a static linked library recipe B
(``DEPENDS += "B"``), package B will neither appear in the generated license
manifest nor in the generated source tarballs. This occurs as the
:ref:`ref-classes-license` and :ref:`ref-classes-archiver` classes assume that
only packages included via :term:`RDEPENDS` or :term:`RRECOMMENDS`
end up in the image.
As a result, potential obligations regarding license compliance for package B
may not be met.
The Yocto Project doesn't enable static libraries by default, in part because
of this issue. Before a solution to this limitation is found, you need to
keep in mind that if your root filesystem is built from static libraries,
you will need to manually ensure that your deliveries are compliant
with the licenses of these libraries.
Copying Non Standard Licenses
=============================
Some packages, such as the linux-firmware package, have many licenses
that are not in any way common. You can avoid adding a lot of these
types of common license files, which are only applicable to a specific
package, by using the
:term:`NO_GENERIC_LICENSE`
variable. Using this variable also avoids QA errors when you use a
non-common, non-CLOSED license in a recipe.
Here is an example that uses the ``LICENSE.Abilis.txt`` file as
the license from the fetched source::
NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"

118
sources/poky/documentation/dev-manual/new-machine.rst Normal file
View File

@@ -0,0 +1,118 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Adding a New Machine
********************
Adding a new machine to the Yocto Project is a straightforward process.
This section describes how to add machines that are similar to those
that the Yocto Project already supports.
.. note::
Although well within the capabilities of the Yocto Project, adding a
totally new architecture might require changes to ``gcc``/``glibc``
and to the site information, which is beyond the scope of this
manual.
For a complete example that shows how to add a new machine, see the
":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
section in the Yocto Project Board Support Package (BSP) Developer's
Guide.
Adding the Machine Configuration File
=====================================
To add a new machine, you need to add a new machine configuration file
to the layer's ``conf/machine`` directory. This configuration file
provides details about the device you are adding.
The OpenEmbedded build system uses the root name of the machine
configuration file to reference the new machine. For example, given a
machine configuration file named ``crownbay.conf``, the build system
recognizes the machine as "crownbay".
The most important variables you must set in your machine configuration
file or include from a lower-level configuration file are as follows:
- :term:`TARGET_ARCH` (e.g. "arm")
- ``PREFERRED_PROVIDER_virtual/kernel``
- :term:`MACHINE_FEATURES` (e.g. "screen wifi")
You might also need these variables:
- :term:`SERIAL_CONSOLES` (e.g. "115200;ttyS0 115200;ttyS1")
- :term:`KERNEL_IMAGETYPE` (e.g. "zImage")
- :term:`IMAGE_FSTYPES` (e.g. "tar.gz jffs2")
You can find full details on these variables in the reference section.
You can leverage existing machine ``.conf`` files from
``meta-yocto-bsp/conf/machine/``.
Adding a Kernel for the Machine
===============================
The OpenEmbedded build system needs to be able to build a kernel for the
machine. You need to either create a new kernel recipe for this machine,
or extend an existing kernel recipe. You can find several kernel recipe
examples in the Source Directory at ``meta/recipes-kernel/linux`` that
you can use as references.
If you are creating a new kernel recipe, normal recipe-writing rules
apply for setting up a :term:`SRC_URI`. Thus, you need to specify any
necessary patches and set :term:`S` to point at the source code. You need to
create a :ref:`ref-tasks-configure` task that configures the unpacked kernel with
a ``defconfig`` file. You can do this by using a ``make defconfig``
command or, more commonly, by copying in a suitable ``defconfig`` file
and then running ``make oldconfig``. By making use of ``inherit kernel``
and potentially some of the ``linux-*.inc`` files, most other
functionality is centralized and the defaults of the class normally work
well.
If you are extending an existing kernel recipe, it is usually a matter
of adding a suitable ``defconfig`` file. The file needs to be added into
a location similar to ``defconfig`` files used for other machines in a
given kernel recipe. A possible way to do this is by listing the file in
the :term:`SRC_URI` and adding the machine to the expression in
:term:`COMPATIBLE_MACHINE`::
COMPATIBLE_MACHINE = '(qemux86|qemumips)'
For more information on ``defconfig`` files, see the
":ref:`kernel-dev/common:changing the configuration`"
section in the Yocto Project Linux Kernel Development Manual.
Adding a Formfactor Configuration File
======================================
A formfactor configuration file provides information about the target
hardware for which the image is being built and information that the
build system cannot obtain from other sources such as the kernel. Some
examples of information contained in a formfactor configuration file
include framebuffer orientation, whether or not the system has a
keyboard, the positioning of the keyboard in relation to the screen, and
the screen resolution.
The build system uses reasonable defaults in most cases. However, if
customization is necessary, you need to create a ``machconfig`` file in
the ``meta/recipes-bsp/formfactor/files`` directory. This directory
contains directories for specific machines such as ``qemuarm`` and
``qemux86``. For information about the settings available and the
defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file
found in the same area.
Here is an example for "qemuarm" machine::
HAVE_TOUCHSCREEN=1
HAVE_KEYBOARD=1
DISPLAY_CAN_ROTATE=0
DISPLAY_ORIENTATION=0
#DISPLAY_WIDTH_PIXELS=640
#DISPLAY_HEIGHT_PIXELS=480
#DISPLAY_BPP=16
DISPLAY_DPI=150
DISPLAY_SUBPIXEL_ORDER=vrgb

1639
sources/poky/documentation/dev-manual/new-recipe.rst Normal file
View File

File diff suppressed because it is too large Load Diff

1139
sources/poky/documentation/dev-manual/packages.rst Normal file
View File

File diff suppressed because it is too large Load Diff

209
sources/poky/documentation/dev-manual/prebuilt-libraries.rst Normal file
View File

@@ -0,0 +1,209 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working with Pre-Built Libraries
********************************
Introduction
============
Some library vendors do not release source code for their software but do
release pre-built binaries. When shared libraries are built, they should
be versioned (see `this article
<https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html>`__
for some background), but sometimes this is not done.
To summarize, a versioned library must meet two conditions:
#. The filename must have the version appended, for example: ``libfoo.so.1.2.3``.
#. The library must have the ELF tag ``SONAME`` set to the major version
of the library, for example: ``libfoo.so.1``. You can check this by
running ``readelf -d filename | grep SONAME``.
This section shows how to deal with both versioned and unversioned
pre-built libraries.
Versioned Libraries
===================
In this example we work with pre-built libraries for the FT4222H USB I/O chip.
Libraries are built for several target architecture variants and packaged in
an archive as follows::
├── build-arm-hisiv300
│   └── libft4222.so.1.4.4.44
├── build-arm-v5-sf
│   └── libft4222.so.1.4.4.44
├── build-arm-v6-hf
│   └── libft4222.so.1.4.4.44
├── build-arm-v7-hf
│   └── libft4222.so.1.4.4.44
├── build-arm-v8
│   └── libft4222.so.1.4.4.44
├── build-i386
│   └── libft4222.so.1.4.4.44
├── build-i486
│   └── libft4222.so.1.4.4.44
├── build-mips-eglibc-hf
│   └── libft4222.so.1.4.4.44
├── build-pentium
│   └── libft4222.so.1.4.4.44
├── build-x86_64
│   └── libft4222.so.1.4.4.44
├── examples
│   ├── get-version.c
│   ├── i2cm.c
│   ├── spim.c
│   └── spis.c
├── ftd2xx.h
├── install4222.sh
├── libft4222.h
├── ReadMe.txt
└── WinTypes.h
To write a recipe to use such a library in your system:
- The vendor will probably have a proprietary licence, so set
:term:`LICENSE_FLAGS` in your recipe.
- The vendor provides a tarball containing libraries so set :term:`SRC_URI`
appropriately.
- Set :term:`COMPATIBLE_HOST` so that the recipe cannot be used with an
unsupported architecture. In the following example, we only support the 32
and 64 bit variants of the ``x86`` architecture.
- As the vendor provides versioned libraries, we can use ``oe_soinstall``
from :ref:`ref-classes-utils` to install the shared library and create
symbolic links. If the vendor does not do this, we need to follow the
non-versioned library guidelines in the next section.
- As the vendor likely used :term:`LDFLAGS` different from those in your Yocto
Project build, disable the corresponding checks by adding ``ldflags``
to :term:`INSANE_SKIP`.
- The vendor will typically ship release builds without debugging symbols.
Avoid errors by preventing the packaging task from stripping out the symbols
and adding them to a separate debug package. This is done by setting the
``INHIBIT_`` flags shown below.
The complete recipe would look like this::
SUMMARY = "FTDI FT4222H Library"
SECTION = "libs"
LICENSE_FLAGS = "ftdi"
LICENSE = "CLOSED"
COMPATIBLE_HOST = "(i.86|x86_64).*-linux"
# Sources available in a .tgz file in .zip archive
# at https://ftdichip.com/wp-content/uploads/2021/01/libft4222-linux-1.4.4.44.zip
# Found on https://ftdichip.com/software-examples/ft4222h-software-examples/
# Since dealing with this particular type of archive is out of topic here,
# we use a local link.
SRC_URI = "file://libft4222-linux-${PV}.tgz"
S = "${WORKDIR}"
ARCH_DIR:x86-64 = "build-x86_64"
ARCH_DIR:i586 = "build-i386"
ARCH_DIR:i686 = "build-i386"
INSANE_SKIP:${PN} = "ldflags"
INHIBIT_PACKAGE_STRIP = "1"
INHIBIT_SYSROOT_STRIP = "1"
INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
do_install () {
install -m 0755 -d ${D}${libdir}
oe_soinstall ${S}/${ARCH_DIR}/libft4222.so.${PV} ${D}${libdir}
install -d ${D}${includedir}
install -m 0755 ${S}/*.h ${D}${includedir}
}
If the precompiled binaries are not statically linked and have dependencies on
other libraries, then by adding those libraries to :term:`DEPENDS`, the linking
can be examined and the appropriate :term:`RDEPENDS` automatically added.
Non-Versioned Libraries
=======================
Some Background
---------------
Libraries in Linux systems are generally versioned so that it is possible
to have multiple versions of the same library installed, which eases upgrades
and support for older software. For example, suppose that in a versioned
library, an actual library is called ``libfoo.so.1.2``, a symbolic link named
``libfoo.so.1`` points to ``libfoo.so.1.2``, and a symbolic link named
``libfoo.so`` points to ``libfoo.so.1.2``. Given these conditions, when you
link a binary against a library, you typically provide the unversioned file
name (i.e. ``-lfoo`` to the linker). However, the linker follows the symbolic
link and actually links against the versioned filename. The unversioned symbolic
link is only used at development time. Consequently, the library is packaged
along with the headers in the development package ``${PN}-dev`` along with the
actual library and versioned symbolic links in ``${PN}``. Because versioned
libraries are far more common than unversioned libraries, the default packaging
rules assume versioned libraries.
Yocto Library Packaging Overview
--------------------------------
It follows that packaging an unversioned library requires a bit of work in the
recipe. By default, ``libfoo.so`` gets packaged into ``${PN}-dev``, which
triggers a QA warning that a non-symlink library is in a ``-dev`` package,
and binaries in the same recipe link to the library in ``${PN}-dev``,
which triggers more QA warnings. To solve this problem, you need to package the
unversioned library into ``${PN}`` where it belongs. The abridged
default :term:`FILES` variables in ``bitbake.conf`` are::
SOLIBS = ".so.*"
SOLIBSDEV = ".so"
FILES:${PN} = "... ${libdir}/lib*${SOLIBS} ..."
FILES_SOLIBSDEV ?= "... ${libdir}/lib*${SOLIBSDEV} ..."
FILES:${PN}-dev = "... ${FILES_SOLIBSDEV} ..."
:term:`SOLIBS` defines a pattern that matches real shared object libraries.
:term:`SOLIBSDEV` matches the development form (unversioned symlink). These two
variables are then used in ``FILES:${PN}`` and ``FILES:${PN}-dev``, which puts
the real libraries into ``${PN}`` and the unversioned symbolic link into ``${PN}-dev``.
To package unversioned libraries, you need to modify the variables in the recipe
as follows::
SOLIBS = ".so"
FILES_SOLIBSDEV = ""
The modifications cause the ``.so`` file to be the real library
and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into
``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed,
``${PN}-dev`` collects files before `${PN}`. ``${PN}-dev`` must not collect any of
the files you want in ``${PN}``.
Finally, loadable modules, essentially unversioned libraries that are linked
at runtime using ``dlopen()`` instead of at build time, should generally be
installed in a private directory. However, if they are installed in ``${libdir}``,
then the modules can be treated as unversioned libraries.
Example
-------
The example below installs an unversioned x86-64 pre-built library named
``libfoo.so``. The :term:`COMPATIBLE_HOST` variable limits recipes to the
x86-64 architecture while the :term:`INSANE_SKIP`, :term:`INHIBIT_PACKAGE_STRIP`
and :term:`INHIBIT_SYSROOT_STRIP` variables are all set as in the above
versioned library example. The "magic" is setting the :term:`SOLIBS` and
:term:`FILES_SOLIBSDEV` variables as explained above::
SUMMARY = "libfoo sample recipe"
SECTION = "libs"
LICENSE = "CLOSED"
SRC_URI = "file://libfoo.so"
COMPATIBLE_HOST = "x86_64.*-linux"
INSANE_SKIP:${PN} = "ldflags"
INHIBIT_PACKAGE_STRIP = "1"
INHIBIT_SYSROOT_STRIP = "1"
SOLIBS = ".so"
FILES_SOLIBSDEV = ""
do_install () {
install -d ${D}${libdir}
install -m 0755 ${WORKDIR}/libfoo.so ${D}${libdir}
}

50
sources/poky/documentation/dev-manual/python-development-shell.rst Normal file
View File

@@ -0,0 +1,50 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using a Python Development Shell
********************************
Similar to working within a development shell as described in the
previous section, you can also spawn and work within an interactive
Python development shell. When debugging certain commands or even when
just editing packages, ``pydevshell`` can be a useful tool. When you
invoke the ``pydevshell`` task, all tasks up to and including
:ref:`ref-tasks-patch` are run for the
specified target. Then a new terminal is opened. Additionally, key
Python objects and code are available in the same way they are to
BitBake tasks, in particular, the data store 'd'. So, commands such as
the following are useful when exploring the data store and running
functions::
pydevshell> d.getVar("STAGING_DIR")
'/media/build1/poky/build/tmp/sysroots'
pydevshell> d.getVar("STAGING_DIR", False)
'${TMPDIR}/sysroots'
pydevshell> d.setVar("FOO", "bar")
pydevshell> d.getVar("FOO")
'bar'
pydevshell> d.delVar("FOO")
pydevshell> d.getVar("FOO")
pydevshell> bb.build.exec_func("do_unpack", d)
pydevshell>
See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`"
section in the BitBake User Manual for details about available functions.
The commands execute just as if the OpenEmbedded build
system were executing them. Consequently, working this way can be
helpful when debugging a build or preparing software to be used with the
OpenEmbedded build system.
Here is an example that uses ``pydevshell`` on a target named
``matchbox-desktop``::
$ bitbake matchbox-desktop -c pydevshell
This command spawns a terminal and places you in an interactive Python
interpreter within the OpenEmbedded build environment. The
:term:`OE_TERMINAL` variable
controls what type of shell is opened.
When you are finished using ``pydevshell``, you can exit the shell
either by using Ctrl+d or closing the terminal window.

470
sources/poky/documentation/dev-manual/qemu.rst Normal file
View File

@@ -0,0 +1,470 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
*******************************
Using the Quick EMUlator (QEMU)
*******************************
The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
Open Source project as part of the Yocto Project development "tool set".
This chapter provides both procedures that show you how to use the Quick
EMUlator (QEMU) and other QEMU information helpful for development
purposes.
Overview
========
Within the context of the Yocto Project, QEMU is an emulator and
virtualization machine that allows you to run a complete image you have
built using the Yocto Project as just another task on your build system.
QEMU is useful for running and testing images and applications on
supported Yocto Project architectures without having actual hardware.
Among other things, the Yocto Project uses QEMU to run automated Quality
Assurance (QA) tests on final images shipped with each release.
.. note::
This implementation is not the same as QEMU in general.
This section provides a brief reference for the Yocto Project
implementation of QEMU.
For official information and documentation on QEMU in general, see the
following references:
- `QEMU Website <https://wiki.qemu.org/Main_Page>`__\ *:* The official
website for the QEMU Open Source project.
- `Documentation <https://wiki.qemu.org/Manual>`__\ *:* The QEMU user
manual.
Running QEMU
============
To use QEMU, you need to have QEMU installed and initialized as well as
have the proper artifacts (i.e. image files and root filesystems)
available. Follow these general steps to run QEMU:
#. *Install QEMU:* QEMU is made available with the Yocto Project a
number of ways. One method is to install a Software Development Kit
(SDK). See ":ref:`sdk-manual/intro:the qemu emulator`" section in the
Yocto Project Application Development and the Extensible Software
Development Kit (eSDK) manual for information on how to install QEMU.
#. *Setting Up the Environment:* How you set up the QEMU environment
depends on how you installed QEMU:
- If you cloned the ``poky`` repository or you downloaded and
unpacked a Yocto Project release tarball, you can source the build
environment script (i.e. :ref:`structure-core-script`)::
$ cd poky
$ source oe-init-build-env
- If you installed a cross-toolchain, you can run the script that
initializes the toolchain. For example, the following commands run
the initialization script from the default ``poky_sdk`` directory::
. poky_sdk/environment-setup-core2-64-poky-linux
#. *Ensure the Artifacts are in Place:* You need to be sure you have a
pre-built kernel that will boot in QEMU. You also need the target
root filesystem for your target machine's architecture:
- If you have previously built an image for QEMU (e.g. ``qemux86``,
``qemuarm``, and so forth), then the artifacts are in place in
your :term:`Build Directory`.
- If you have not built an image, you can go to the
:yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu/>` area and download a
pre-built image that matches your architecture and can be run on
QEMU.
See the ":ref:`sdk-manual/appendix-obtain:extracting the root filesystem`"
section in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual for information on
how to extract a root filesystem.
#. *Run QEMU:* The basic ``runqemu`` command syntax is as follows::
$ runqemu [option ] [...]
Based on what you provide on the command
line, ``runqemu`` does a good job of figuring out what you are trying
to do. For example, by default, QEMU looks for the most recently
built image according to the timestamp when it needs to look for an
image. Minimally, through the use of options, you must provide either
a machine name, a virtual machine image (``*wic.vmdk``), or a kernel
image (``*.bin``).
Here are some additional examples to help illustrate further QEMU:
- This example starts QEMU with MACHINE set to "qemux86-64".
Assuming a standard :term:`Build Directory`, ``runqemu``
automatically finds the ``bzImage-qemux86-64.bin`` image file and
the ``core-image-minimal-qemux86-64-20200218002850.rootfs.ext4``
(assuming the current build created a ``core-image-minimal``
image)::
$ runqemu qemux86-64
.. note::
When more than one image with the same name exists, QEMU finds
and uses the most recently built image according to the
timestamp.
- This example produces the exact same results as the previous
example. This command, however, specifically provides the image
and root filesystem type::
$ runqemu qemux86-64 core-image-minimal ext4
- This example specifies to boot an :term:`Initramfs` image and to
enable audio in QEMU. For this case, ``runqemu`` sets the internal
variable ``FSTYPE`` to ``cpio.gz``. Also, for audio to be enabled,
an appropriate driver must be installed (see the ``audio`` option
in :ref:`dev-manual/qemu:\`\`runqemu\`\` command-line options`
for more information)::
$ runqemu qemux86-64 ramfs audio
- This example does not provide enough information for QEMU to
launch. While the command does provide a root filesystem type, it
must also minimally provide a `MACHINE`, `KERNEL`, or `VM` option::
$ runqemu ext4
- This example specifies to boot a virtual machine image
(``.wic.vmdk`` file). From the ``.wic.vmdk``, ``runqemu``
determines the QEMU architecture (`MACHINE`) to be "qemux86-64" and
the root filesystem type to be "vmdk"::
$ runqemu /home/scott-lenovo/vm/core-image-minimal-qemux86-64.wic.vmdk
Switching Between Consoles
==========================
When booting or running QEMU, you can switch between supported consoles
by using Ctrl+Alt+number. For example, Ctrl+Alt+3 switches you to the
serial console as long as that console is enabled. Being able to switch
consoles is helpful, for example, if the main QEMU console breaks for
some reason.
.. note::
Usually, "2" gets you to the main console and "3" gets you to the
serial console.
Removing the Splash Screen
==========================
You can remove the splash screen when QEMU is booting by using Alt+left.
Removing the splash screen allows you to see what is happening in the
background.
Disabling the Cursor Grab
=========================
The default QEMU integration captures the cursor within the main window.
It does this since standard mouse devices only provide relative input
and not absolute coordinates. You then have to break out of the grab
using the "Ctrl+Alt" key combination. However, the Yocto Project's
integration of QEMU enables the wacom USB touch pad driver by default to
allow input of absolute coordinates. This default means that the mouse
can enter and leave the main window without the grab taking effect
leading to a better user experience.
Running Under a Network File System (NFS) Server
================================================
One method for running QEMU is to run it on an NFS server. This is
useful when you need to access the same file system from both the build
and the emulated system at the same time. It is also worth noting that
the system does not need root privileges to run. It uses a user space
NFS server to avoid that. Follow these steps to set up for running QEMU
using an NFS server.
#. *Extract a Root Filesystem:* Once you are able to run QEMU in your
environment, you can use the ``runqemu-extract-sdk`` script, which is
located in the ``scripts`` directory along with the ``runqemu``
script.
The ``runqemu-extract-sdk`` takes a root filesystem tarball and
extracts it into a location that you specify. Here is an example that
takes a file system and extracts it to a directory named
``test-nfs``:
.. code-block:: none
runqemu-extract-sdk ./tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.tar.bz2 test-nfs
#. *Start QEMU:* Once you have extracted the file system, you can run
``runqemu`` normally with the additional location of the file system.
You can then also make changes to the files within ``./test-nfs`` and
see those changes appear in the image in real time. Here is an
example using the ``qemux86`` image:
.. code-block:: none
runqemu qemux86-64 ./test-nfs
.. note::
Should you need to start, stop, or restart the NFS share, you can use
the following commands:
- To start the NFS share::
runqemu-export-rootfs start file-system-location
- To stop the NFS share::
runqemu-export-rootfs stop file-system-location
- To restart the NFS share::
runqemu-export-rootfs restart file-system-location
QEMU CPU Compatibility Under KVM
================================
By default, the QEMU build compiles for and targets 64-bit and x86 Intel
Core2 Duo processors and 32-bit x86 Intel Pentium II processors. QEMU
builds for and targets these CPU types because they display a broad
range of CPU feature compatibility with many commonly used CPUs.
Despite this broad range of compatibility, the CPUs could support a
feature that your host CPU does not support. Although this situation is
not a problem when QEMU uses software emulation of the feature, it can
be a problem when QEMU is running with KVM enabled. Specifically,
software compiled with a certain CPU feature crashes when run on a CPU
under KVM that does not support that feature. To work around this
problem, you can override QEMU's runtime CPU setting by changing the
``QB_CPU_KVM`` variable in ``qemuboot.conf`` in the :term:`Build Directory`
``deploy/image`` directory. This setting specifies a ``-cpu`` option passed
into QEMU in the ``runqemu`` script. Running ``qemu -cpu help`` returns a
list of available supported CPU types.
QEMU Performance
================
Using QEMU to emulate your hardware can result in speed issues depending
on the target and host architecture mix. For example, using the
``qemux86`` image in the emulator on an Intel-based 32-bit (x86) host
machine is fast because the target and host architectures match. On the
other hand, using the ``qemuarm`` image on the same Intel-based host can
be slower. But, you still achieve faithful emulation of ARM-specific
issues.
To speed things up, the QEMU images support using ``distcc`` to call a
cross-compiler outside the emulated system. If you used ``runqemu`` to
start QEMU, and the ``distccd`` application is present on the host
system, any BitBake cross-compiling toolchain available from the build
system is automatically used from within QEMU simply by calling
``distcc``. You can accomplish this by defining the cross-compiler
variable (e.g. ``export CC="distcc"``). Alternatively, if you are using
a suitable SDK image or the appropriate stand-alone toolchain is
present, the toolchain is also automatically used.
.. note::
There are several mechanisms to connect to the system running
on the QEMU emulator:
- QEMU provides a framebuffer interface that makes standard consoles
available.
- Generally, headless embedded devices have a serial port. If so,
you can configure the operating system of the running image to use
that port to run a console. The connection uses standard IP
networking.
- SSH servers are available in some QEMU images. The ``core-image-sato``
QEMU image has a Dropbear secure shell (SSH) server that runs with the
root password disabled. The ``core-image-full-cmdline`` QEMU image has
OpenSSH instead of Dropbear. Including these SSH servers allow you to use
standard ``ssh`` and ``scp`` commands. The ``core-image-minimal`` QEMU
image, however, contains no SSH server.
- You can use a provided, user-space NFS server to boot the QEMU
session using a local copy of the root filesystem on the host. In
order to make this connection, you must extract a root filesystem
tarball by using the ``runqemu-extract-sdk`` command. After
running the command, you must then point the ``runqemu`` script to
the extracted directory instead of a root filesystem image file.
See the
":ref:`dev-manual/qemu:running under a network file system (nfs) server`"
section for more information.
QEMU Command-Line Syntax
========================
The basic ``runqemu`` command syntax is as follows::
$ runqemu [option ] [...]
Based on what you provide on the command line, ``runqemu`` does a
good job of figuring out what you are trying to do. For example, by
default, QEMU looks for the most recently built image according to the
timestamp when it needs to look for an image. Minimally, through the use
of options, you must provide either a machine name, a virtual machine
image (``*wic.vmdk``), or a kernel image (``*.bin``).
Here is the command-line help output for the ``runqemu`` command::
$ runqemu --help
Usage: you can run this script with any valid combination
of the following environment variables (in any order):
KERNEL - the kernel image file to use
ROOTFS - the rootfs image file or nfsroot directory to use
MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
Simplified QEMU command-line options can be passed with:
nographic - disable video console
serial - enable a serial console on /dev/ttyS0
slirp - enable user networking, no root privileges required
kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
publicvnc - enable a VNC server open to all hosts
audio - enable audio
[*/]ovmf* - OVMF firmware file or base name for booting with UEFI
tcpserial=<port> - specify tcp serial port number
biosdir=<dir> - specify custom bios dir
biosfilename=<filename> - specify bios filename
qemuparams=<xyz> - specify custom parameters to QEMU
bootparams=<xyz> - specify custom kernel parameters during boot
help, -h, --help: print this text
Examples:
runqemu
runqemu qemuarm
runqemu tmp/deploy/images/qemuarm
runqemu tmp/deploy/images/qemux86/<qemuboot.conf>
runqemu qemux86-64 core-image-sato ext4
runqemu qemux86-64 wic-image-minimal wic
runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
runqemu qemux86 qemuparams="-m 256"
runqemu qemux86 bootparams="psplash=false"
runqemu path/to/<image>-<machine>.wic
runqemu path/to/<image>-<machine>.wic.vmdk
``runqemu`` Command-Line Options
================================
Here is a description of ``runqemu`` options you can provide on the
command line:
.. note::
If you do provide some "illegal" option combination or perhaps you do
not provide enough in the way of options, ``runqemu``
provides appropriate error messaging to help you correct the problem.
- `QEMUARCH`: The QEMU machine architecture, which must be "qemuarm",
"qemuarm64", "qemumips", "qemumips64", "qemuppc", "qemux86", or
"qemux86-64".
- `VM`: The virtual machine image, which must be a ``.wic.vmdk``
file. Use this option when you want to boot a ``.wic.vmdk`` image.
The image filename you provide must contain one of the following
strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64",
"qemumips", "qemuppc", or "qemush4".
- `ROOTFS`: A root filesystem that has one of the following filetype
extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If
the filename you provide for this option uses "nfs", it must provide
an explicit root filesystem path.
- `KERNEL`: A kernel image, which is a ``.bin`` file. When you provide a
``.bin`` file, ``runqemu`` detects it and assumes the file is a
kernel image.
- `MACHINE`: The architecture of the QEMU machine, which must be one of
the following: "qemux86", "qemux86-64", "qemuarm", "qemuarm64",
"qemumips", "qemumips64", or "qemuppc". The MACHINE and QEMUARCH
options are basically identical. If you do not provide a MACHINE
option, ``runqemu`` tries to determine it based on other options.
- ``ramfs``: Indicates you are booting an :term:`Initramfs`
image, which means the ``FSTYPE`` is ``cpio.gz``.
- ``iso``: Indicates you are booting an ISO image, which means the
``FSTYPE`` is ``.iso``.
- ``nographic``: Disables the video console, which sets the console to
"ttys0". This option is useful when you have logged into a server and
you do not want to disable forwarding from the X Window System (X11)
to your workstation or laptop.
- ``serial``: Enables a serial console on ``/dev/ttyS0``.
- ``biosdir``: Establishes a custom directory for BIOS, VGA BIOS and
keymaps.
- ``biosfilename``: Establishes a custom BIOS name.
- ``qemuparams=\"xyz\"``: Specifies custom QEMU parameters. Use this
option to pass options other than the simple "kvm" and "serial"
options.
- ``bootparams=\"xyz\"``: Specifies custom boot parameters for the
kernel.
- ``audio``: Enables audio in QEMU. The MACHINE option must be either
"qemux86" or "qemux86-64" in order for audio to be enabled.
Additionally, the ``snd_intel8x0`` or ``snd_ens1370`` driver must be
installed in linux guest.
- ``slirp``: Enables "slirp" networking, which is a different way of
networking that does not need root access but also is not as easy to
use or comprehensive as the default.
Using ``slirp`` by default will forward the guest machine's
22 and 23 TCP ports to host machine's 2222 and 2323 ports
(or the next free ports). Specific forwarding rules can be configured
by setting ``QB_SLIRP_OPT`` as environment variable or in ``qemuboot.conf``
in the :term:`Build Directory` ``deploy/image`` directory.
Examples::
QB_SLIRP_OPT="-netdev user,id=net0,hostfwd=tcp::8080-:80"
QB_SLIRP_OPT="-netdev user,id=net0,hostfwd=tcp::8080-:80,hostfwd=tcp::2222-:22"
The first example forwards TCP port 80 from the emulated system to
port 8080 (or the next free port) on the host system,
allowing access to an http server running in QEMU from
``http://<host ip>:8080/``.
The second example does the same, but also forwards TCP port 22 on the
guest system to 2222 (or the next free port) on the host system,
allowing ssh access to the emulated system using
``ssh -P 2222 <user>@<host ip>``.
Keep in mind that proper configuration of firewall software is required.
- ``kvm``: Enables KVM when running "qemux86" or "qemux86-64" QEMU
architectures. For KVM to work, all the following conditions must be
met:
- Your MACHINE must be either qemux86" or "qemux86-64".
- Your build host has to have the KVM modules installed, which are
``/dev/kvm``.
- The build host ``/dev/kvm`` directory has to be both writable and
readable.
- ``kvm-vhost``: Enables KVM with VHOST support when running "qemux86"
or "qemux86-64" QEMU architectures. For KVM with VHOST to work, the
following conditions must be met:
- ``kvm`` option conditions defined above must be met.
- Your build host has to have virtio net device, which are
``/dev/vhost-net``.
- The build host ``/dev/vhost-net`` directory has to be either
readable or writable and "slirp-enabled".
- ``publicvnc``: Enables a VNC server open to all hosts.

89
sources/poky/documentation/dev-manual/quilt.rst Normal file
View File

@@ -0,0 +1,89 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using Quilt in Your Workflow
****************************
`Quilt <https://savannah.nongnu.org/projects/quilt>`__ is a powerful tool
that allows you to capture source code changes without having a clean
source tree. This section outlines the typical workflow you can use to
modify source code, test changes, and then preserve the changes in the
form of a patch all using Quilt.
.. note::
With regard to preserving changes to source files, if you clean a
recipe or have :ref:`ref-classes-rm-work` enabled, the
:ref:`devtool workflow <sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow>`
as described in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual is a safer
development flow than the flow that uses Quilt.
Follow these general steps:
#. *Find the Source Code:* Temporary source code used by the
OpenEmbedded build system is kept in the :term:`Build Directory`. See the
":ref:`dev-manual/temporary-source-code:finding temporary source code`" section to
learn how to locate the directory that has the temporary source code for a
particular package.
#. *Change Your Working Directory:* You need to be in the directory that
has the temporary source code. That directory is defined by the
:term:`S` variable.
#. *Create a New Patch:* Before modifying source code, you need to
create a new patch. To create a new patch file, use ``quilt new`` as
below::
$ quilt new my_changes.patch
#. *Notify Quilt and Add Files:* After creating the patch, you need to
notify Quilt about the files you plan to edit. You notify Quilt by
adding the files to the patch you just created::
$ quilt add file1.c file2.c file3.c
#. *Edit the Files:* Make your changes in the source code to the files
you added to the patch.
#. *Test Your Changes:* Once you have modified the source code, the
easiest way to test your changes is by calling the :ref:`ref-tasks-compile`
task as shown in the following example::
$ bitbake -c compile -f package
The ``-f`` or ``--force`` option forces the specified task to
execute. If you find problems with your code, you can just keep
editing and re-testing iteratively until things work as expected.
.. note::
All the modifications you make to the temporary source code disappear
once you run the :ref:`ref-tasks-clean` or :ref:`ref-tasks-cleanall`
tasks using BitBake (i.e. ``bitbake -c clean package`` and
``bitbake -c cleanall package``). Modifications will also disappear if
you use the :ref:`ref-classes-rm-work` feature as described in
the ":ref:`dev-manual/disk-space:conserving disk space during builds`"
section.
#. *Generate the Patch:* Once your changes work as expected, you need to
use Quilt to generate the final patch that contains all your
modifications::
$ quilt refresh
At this point, the
``my_changes.patch`` file has all your edits made to the ``file1.c``,
``file2.c``, and ``file3.c`` files.
You can find the resulting patch file in the ``patches/``
subdirectory of the source (:term:`S`) directory.
#. *Copy the Patch File:* For simplicity, copy the patch file into a
directory named ``files``, which you can create in the same directory
that holds the recipe (``.bb``) file or the append (``.bbappend``)
file. Placing the patch here guarantees that the OpenEmbedded build
system will find the patch. Next, add the patch into the :term:`SRC_URI`
of the recipe. Here is an example::
SRC_URI += "file://my_changes.patch"

89
sources/poky/documentation/dev-manual/read-only-rootfs.rst Normal file
View File

@@ -0,0 +1,89 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Read-Only Root Filesystem
************************************
Suppose, for security reasons, you need to disable your target device's
root filesystem's write permissions (i.e. you need a read-only root
filesystem). Or, perhaps you are running the device's operating system
from a read-only storage device. For either case, you can customize your
image for that behavior.
.. note::
Supporting a read-only root filesystem requires that the system and
applications do not try to write to the root filesystem. You must
configure all parts of the target system to write elsewhere, or to
gracefully fail in the event of attempting to write to the root
filesystem.
Creating the Root Filesystem
============================
To create the read-only root filesystem, simply add the
"read-only-rootfs" feature to your image, normally in one of two ways.
The first way is to add the "read-only-rootfs" image feature in the
image's recipe file via the :term:`IMAGE_FEATURES` variable::
IMAGE_FEATURES += "read-only-rootfs"
As an alternative, you can add the same feature
from within your :term:`Build Directory`'s ``local.conf`` file with the
associated :term:`EXTRA_IMAGE_FEATURES` variable, as in::
EXTRA_IMAGE_FEATURES = "read-only-rootfs"
For more information on how to use these variables, see the
":ref:`dev-manual/customizing-images:Customizing Images Using Custom \`\`IMAGE_FEATURES\`\` and \`\`EXTRA_IMAGE_FEATURES\`\``"
section. For information on the variables, see
:term:`IMAGE_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES`.
Post-Installation Scripts and Read-Only Root Filesystem
=======================================================
It is very important that you make sure all post-Installation
(``pkg_postinst``) scripts for packages that are installed into the
image can be run at the time when the root filesystem is created during
the build on the host system. These scripts cannot attempt to run during
the first boot on the target device. With the "read-only-rootfs" feature
enabled, the build system makes sure that all post-installation scripts
succeed at file system creation time. If any of these scripts
still need to be run after the root filesystem is created, the build
immediately fails. These build-time checks ensure that the build fails
rather than the target device fails later during its initial boot
operation.
Most of the common post-installation scripts generated by the build
system for the out-of-the-box Yocto Project are engineered so that they
can run during root filesystem creation (e.g. post-installation scripts
for caching fonts). However, if you create and add custom scripts, you
need to be sure they can be run during this file system creation.
Here are some common problems that prevent post-installation scripts
from running during root filesystem creation:
- *Not using $D in front of absolute paths:* The build system defines
``$``\ :term:`D` when the root
filesystem is created. Furthermore, ``$D`` is blank when the script
is run on the target device. This implies two purposes for ``$D``:
ensuring paths are valid in both the host and target environments,
and checking to determine which environment is being used as a method
for taking appropriate actions.
- *Attempting to run processes that are specific to or dependent on the
target architecture:* You can work around these attempts by using
native tools, which run on the host system, to accomplish the same
tasks, or by alternatively running the processes under QEMU, which
has the ``qemu_run_binary`` function. For more information, see the
:ref:`ref-classes-qemu` class.
Areas With Write Access
=======================
With the "read-only-rootfs" feature enabled, any attempt by the target
to write to the root filesystem at runtime fails. Consequently, you must
make sure that you configure processes and applications that attempt
these types of writes do so to directories with write access (e.g.
``/tmp`` or ``/var/run``).

76
sources/poky/documentation/dev-manual/sbom.rst Normal file
View File

@@ -0,0 +1,76 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Software Bill of Materials
*************************************
Once you are able to build an image for your project, once the licenses for
each software component are all identified (see
":ref:`dev-manual/licenses:working with licenses`") and once vulnerability
fixes are applied (see ":ref:`dev-manual/vulnerabilities:checking
for vulnerabilities`"), the OpenEmbedded build system can generate
a description of all the components you used, their licenses, their dependencies,
their sources, the changes that were applied to them and the known
vulnerabilities that were fixed.
This description is generated in the form of a *Software Bill of Materials*
(:term:`SBOM`), using the :term:`SPDX` standard.
When you release software, this is the most standard way to provide information
about the Software Supply Chain of your software image and SDK. The
:term:`SBOM` tooling is often used to ensure open source license compliance by
providing the license texts used in the product which legal departments and end
users can read in standardized format.
:term:`SBOM` information is also critical to performing vulnerability exposure
assessments, as all the components used in the Software Supply Chain are listed.
The OpenEmbedded build system doesn't generate such information by default.
To make this happen, you must inherit the
:ref:`ref-classes-create-spdx` class from a configuration file::
INHERIT += "create-spdx"
Upon building an image, you will then get the compressed archive
``IMAGE-MACHINE.spdx.tar.zst`` contains the index and the files for the single
recipes.
The :ref:`ref-classes-create-spdx` class offers options to include
more information in the output :term:`SPDX` data:
- Make the json files more human readable by setting (:term:`SPDX_PRETTY`).
- Add compressed archives of the files in the generated target packages by
setting (:term:`SPDX_ARCHIVE_PACKAGED`).
- Add a description of the source files used to generate host tools and target
packages (:term:`SPDX_INCLUDE_SOURCES`)
- Add archives of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`).
Though the toplevel :term:`SPDX` output is available in
``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary
generated files are available in ``tmp/deploy/spdx`` too, such as:
- The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst``
archive.
- Compressed archives of the files in the generated target packages,
in ``packages/packagename.tar.zst`` (when :term:`SPDX_ARCHIVE_PACKAGED`
is set).
- Compressed archives of the source files used to build the host tools
and the target packages in ``recipes/recipe-packagename.tar.zst``
(when :term:`SPDX_ARCHIVE_SOURCES` is set). Those are needed to fulfill
"source code access" license requirements.
See also the :term:`SPDX_CUSTOM_ANNOTATION_VARS` variable which allows
to associate custom notes to a recipe.
See the `tools page <https://spdx.dev/resources/tools/>`__ on the :term:`SPDX`
project website for a list of tools to consume and transform the :term:`SPDX`
data generated by the OpenEmbedded build system.
See also Joshua Watt's presentations
`Automated SBoM generation with OpenEmbedded and the Yocto Project <https://youtu.be/Q5UQUM6zxVU>`__
at FOSDEM 2023 and
`SPDX in the Yocto Project <https://fosdem.org/2024/schedule/event/fosdem-2024-3318-spdx-in-the-yocto-project/>`__
at FOSDEM 2024.

156
sources/poky/documentation/dev-manual/securing-images.rst Normal file
View File

@@ -0,0 +1,156 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Making Images More Secure
*************************
Security is of increasing concern for embedded devices. Consider the
issues and problems discussed in just this sampling of work found across
the Internet:
- *"*\ `Security Risks of Embedded
Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"*
by Bruce Schneier
- *"*\ `Internet Census
2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna
Botnet
- *"*\ `Security Issues for Embedded
Devices <https://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"*
by Jake Edge
When securing your image is of concern, there are steps, tools, and
variables that you can consider to help you reach the security goals you
need for your particular device. Not all situations are identical when
it comes to making an image secure. Consequently, this section provides
some guidance and suggestions for consideration when you want to make
your image more secure.
.. note::
Because the security requirements and risks are different for every
type of device, this section cannot provide a complete reference on
securing your custom OS. It is strongly recommended that you also
consult other sources of information on embedded Linux system
hardening and on security.
General Considerations
======================
There are general considerations that help you create more secure images.
You should consider the following suggestions to make your device
more secure:
- Scan additional code you are adding to the system (e.g. application
code) by using static analysis tools. Look for buffer overflows and
other potential security problems.
- Pay particular attention to the security for any web-based
administration interface.
Web interfaces typically need to perform administrative functions and
tend to need to run with elevated privileges. Thus, the consequences
resulting from the interface's security becoming compromised can be
serious. Look for common web vulnerabilities such as
cross-site-scripting (XSS), unvalidated inputs, and so forth.
As with system passwords, the default credentials for accessing a
web-based interface should not be the same across all devices. This
is particularly true if the interface is enabled by default as it can
be assumed that many end-users will not change the credentials.
- Ensure you can update the software on the device to mitigate
vulnerabilities discovered in the future. This consideration
especially applies when your device is network-enabled.
- Regularly scan and apply fixes for CVE security issues affecting
all software components in the product, see ":ref:`dev-manual/vulnerabilities:checking for vulnerabilities`".
- Regularly update your version of Poky and OE-Core from their upstream
developers, e.g. to apply updates and security fixes from stable
and :term:`LTS` branches.
- Ensure you remove or disable debugging functionality before producing
the final image. For information on how to do this, see the
":ref:`dev-manual/securing-images:considerations specific to the openembedded build system`"
section.
- Ensure you have no network services listening that are not needed.
- Remove any software from the image that is not needed.
- Enable hardware support for secure boot functionality when your
device supports this functionality.
Security Flags
==============
The Yocto Project has security flags that you can enable that help make
your build output more secure. The security flags are in the
``meta/conf/distro/include/security_flags.inc`` file in your
:term:`Source Directory` (e.g. ``poky``).
.. note::
Depending on the recipe, certain security flags are enabled and
disabled by default.
Use the following line in your ``local.conf`` file or in your custom
distribution configuration file to enable the security compiler and
linker flags for your build::
require conf/distro/include/security_flags.inc
Considerations Specific to the OpenEmbedded Build System
========================================================
You can take some steps that are specific to the OpenEmbedded build
system to make your images more secure:
- Ensure "debug-tweaks" is not one of your selected
:term:`IMAGE_FEATURES`.
When creating a new project, the default is to provide you with an
initial ``local.conf`` file that enables this feature using the
:term:`EXTRA_IMAGE_FEATURES`
variable with the line::
EXTRA_IMAGE_FEATURES = "debug-tweaks"
To disable that feature, simply comment out that line in your
``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain
"debug-tweaks" before producing your final image. Among other things,
leaving this in place sets the root password as blank, which makes
logging in for debugging or inspection easy during development but
also means anyone can easily log in during production.
- It is possible to set a root password for the image and also to set
passwords for any extra users you might add (e.g. administrative or
service type users). When you set up passwords for multiple images or
users, you should not duplicate passwords.
To set up passwords, use the :ref:`ref-classes-extrausers` class, which
is the preferred method. For an example on how to set up both root and
user passwords, see the ":ref:`ref-classes-extrausers`" section.
.. note::
When adding extra user accounts or setting a root password, be
cautious about setting the same password on every device. If you
do this, and the password you have set is exposed, then every
device is now potentially compromised. If you need this access but
want to ensure security, consider setting a different, random
password for each device. Typically, you do this as a separate
step after you deploy the image onto the device.
- Consider enabling a Mandatory Access Control (MAC) framework such as
SMACK or SELinux and tuning it appropriately for your device's usage.
You can find more information in the
:yocto_git:`meta-selinux </meta-selinux/>` layer.
Tools for Hardening Your Image
==============================
The Yocto Project provides tools for making your image more secure. You
can find these tools in the ``meta-security`` layer of the
:yocto_git:`Yocto Project Source Repositories <>`.

189
sources/poky/documentation/dev-manual/security-subjects.rst Normal file
View File

@@ -0,0 +1,189 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Dealing with Vulnerability Reports
**********************************
The Yocto Project and OpenEmbedded are open-source, community-based projects
used in numerous products. They assemble multiple other open-source projects,
and need to handle security issues and practices both internal (in the code
maintained by both projects), and external (maintained by other projects and
organizations).
This manual assembles security-related information concerning the whole
ecosystem. It includes information on reporting a potential security issue,
the operation of the YP Security team and how to contribute in the
related code. It is written to be useful for both security researchers and
YP developers.
How to report a potential security vulnerability?
=================================================
If you would like to report a public issue (for example, one with a released
CVE number), please report it using the
:yocto_bugs:`Security Bugzilla </enter_bug.cgi?product=Security>`.
If you are dealing with a not-yet-released issue, or an urgent one, please send
a message to security AT yoctoproject DOT org, including as many details as
possible: the layer or software module affected, the recipe and its version,
and any example code, if available. This mailing list is monitored by the
Yocto Project Security team.
For each layer, you might also look for specific instructions (if any) for
reporting potential security issues in the specific ``SECURITY.md`` file at the
root of the repository. Instructions on how and where submit a patch are
usually available in ``README.md``. If this is your first patch to the
Yocto Project/OpenEmbedded, you might want to have a look into the
Contributor's Manual section
":ref:`contributor-guide/submit-changes:preparing changes for submission`".
Branches maintained with security fixes
---------------------------------------
See the
:ref:`Release process <ref-manual/release-process:Stable Release Process>`
documentation for details regarding the policies and maintenance of stable
branches.
The :yocto_wiki:`Releases page </Releases>` contains a list
of all releases of the Yocto Project. Versions in gray are no longer actively
maintained with security patches, but well-tested patches may still be accepted
for them for significant issues.
Security-related discussions at the Yocto Project
-------------------------------------------------
We have set up two security-related mailing lists:
- Public List: yocto [dash] security [at] yoctoproject[dot] org
This is a public mailing list for anyone to subscribe to. This list is an
open list to discuss public security issues/patches and security-related
initiatives. For more information, including subscription information,
please see the :yocto_lists:`yocto-security mailing list info page </g/yocto-security>`.
- Private List: security [at] yoctoproject [dot] org
This is a private mailing list for reporting non-published potential
vulnerabilities. The list is monitored by the Yocto Project Security team.
What you should do if you find a security vulnerability
-------------------------------------------------------
If you find a security flaw: a crash, an information leakage, or anything that
can have a security impact if exploited in any Open Source software built or
used by the Yocto Project, please report this to the Yocto Project Security
Team. If you prefer to contact the upstream project directly, please send a
copy to the security team at the Yocto Project as well. If you believe this is
highly sensitive information, please report the vulnerability in a secure way,
i.e. encrypt the email and send it to the private list. This ensures that
the exploit is not leaked and exploited before a response/fix has been generated.
Security team
=============
The Yocto Project/OpenEmbedded security team coordinates the work on security
subjects in the project. All general discussion takes place publicly. The
Security Team only uses confidential communication tools to deal with private
vulnerability reports before they are released.
Security team appointment
-------------------------
The Yocto Project Security Team consists of at least three members. When new
members are needed, the Yocto Project Technical Steering Committee (YP TSC)
asks for nominations by public channels including a nomination deadline.
Self-nominations are possible. When the limit time is
reached, the YP TSC posts the list of candidates for the comments of project
participants and developers. Comments may be sent publicly or privately to the
YP and OE TSCs. The candidates are approved by both YP TSC and OpenEmbedded
Technical Steering Committee (OE TSC) and the final list of the team members
is announced publicly. The aim is to have people representing technical
leadership, security knowledge and infrastructure present with enough people
to provide backup/coverage but keep the notification list small enough to
minimize information risk and maintain trust.
YP Security Team members may resign at any time.
Security Team Operations
------------------------
The work of the Security Team might require high confidentiality. Team members
are individuals selected by merit and do not represent the companies they work
for. They do not share information about confidential issues outside of the team
and do not hint about ongoing embargoes.
Team members can bring in domain experts as needed. Those people should be
added to individual issues only and adhere to the same standards as the YP
Security Team.
The YP security team organizes its meetings and communication as needed.
When the YP Security team receives a report about a potential security
vulnerability, they quickly analyze and notify the reporter of the result.
They might also request more information.
If the issue is confirmed and affects the code maintained by the YP, they
confidentially notify maintainers of that code and work with them to prepare
a fix.
If the issue is confirmed and affects an upstream project, the YP security team
notifies the project. Usually, the upstream project analyzes the problem again.
If they deem it a real security problem in their software, they develop and
release a fix following their security policy. They may want to include the
original reporter in the loop. There is also sometimes some coordination for
handling patches, backporting patches etc, or just understanding the problem
or what caused it.
When the fix is publicly available, the YP security team member or the
package maintainer sends patches against the YP code base, following usual
procedures, including public code review.
What Yocto Security Team does when it receives a security vulnerability
-----------------------------------------------------------------------
The YP Security Team team performs a quick analysis and would usually report
the flaw to the upstream project. Normally the upstream project analyzes the
problem. If they deem it a real security problem in their software, they
develop and release a fix following their own security policy. They may want
to include the original reporter in the loop. There is also sometimes some
coordination for handling patches, backporting patches etc, or just
understanding the problem or what caused it.
The security policy of the upstream project might include a notification to
Linux distributions or other important downstream projects in advance to
discuss coordinated disclosure. These mailing lists are normally non-public.
When the upstream project releases a version with the fix, they are responsible
for contacting `Mitre <https://www.cve.org/>`__ to get a CVE number assigned and
the CVE record published.
If an upstream project does not respond quickly
-----------------------------------------------
If an upstream project does not fix the problem in a reasonable time,
the Yocto's Security Team will contact other interested parties (usually
other distributions) in the community and together try to solve the
vulnerability as quickly as possible.
The Yocto Project Security team adheres to the 90 days disclosure policy
by default. An increase of the embargo time is possible when necessary.
Current Security Team members
-----------------------------
For secure communications, please send your messages encrypted using the GPG
keys. Remember, message headers are not encrypted so do not include sensitive
information in the subject line.
- Ross Burton: <ross@burtonini.com> `Public key <https://keys.openpgp.org/search?q=ross%40burtonini.com>`__
- Michael Halstead: <mhalstead [at] linuxfoundation [dot] org>
`Public key <https://pgp.mit.edu/pks/lookup?op=vindex&search=0x3373170601861969>`__
or `Public key <https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xd1f2407285e571ed12a407a73373170601861969>`__
- Richard Purdie: <richard.purdie@linuxfoundation.org> `Public key <https://keys.openpgp.org/search?q=richard.purdie%40linuxfoundation.org>`__
- Marta Rybczynska: <marta DOT rybczynska [at] syslinbit [dot] com> `Public key <https://keys.openpgp.org/search?q=marta.rybczynska@syslinbit.com>`__
- Steve Sakoman: <steve [at] sakoman [dot] com> `Public key <https://keys.openpgp.org/search?q=steve%40sakoman.com>`__

109
sources/poky/documentation/dev-manual/speeding-up-build.rst Normal file
View File

@@ -0,0 +1,109 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Speeding Up a Build
*******************
Build time can be an issue. By default, the build system uses simple
controls to try and maximize build efficiency. In general, the default
settings for all the following variables result in the most efficient
build times when dealing with single socket systems (i.e. a single CPU).
If you have multiple CPUs, you might try increasing the default values
to gain more speed. See the descriptions in the glossary for each
variable for more information:
- :term:`BB_NUMBER_THREADS`:
The maximum number of threads BitBake simultaneously executes.
- :term:`BB_NUMBER_PARSE_THREADS`:
The number of threads BitBake uses during parsing.
- :term:`PARALLEL_MAKE`: Extra
options passed to the ``make`` command during the
:ref:`ref-tasks-compile` task in
order to specify parallel compilation on the local build host.
- :term:`PARALLEL_MAKEINST`:
Extra options passed to the ``make`` command during the
:ref:`ref-tasks-install` task in
order to specify parallel installation on the local build host.
As mentioned, these variables all scale to the number of processor cores
available on the build system. For single socket systems, this
auto-scaling ensures that the build system fundamentally takes advantage
of potential parallel operations during the build based on the build
machine's capabilities.
Additional factors that can affect build speed are:
- File system type: The file system type that the build is being
performed on can also influence performance. Using ``ext4`` is
recommended as compared to ``ext2`` and ``ext3`` due to ``ext4``
improved features such as extents.
- Disabling the updating of access time using ``noatime``: The
``noatime`` mount option prevents the build system from updating file
and directory access times.
- Setting a longer commit: Using the "commit=" mount option increases
the interval in seconds between disk cache writes. Changing this
interval from the five second default to something longer increases
the risk of data loss but decreases the need to write to the disk,
thus increasing the build performance.
- Choosing the packaging backend: Of the available packaging backends,
IPK is the fastest. Additionally, selecting a singular packaging
backend also helps.
- Using ``tmpfs`` for :term:`TMPDIR`
as a temporary file system: While this can help speed up the build,
the benefits are limited due to the compiler using ``-pipe``. The
build system goes to some lengths to avoid ``sync()`` calls into the
file system on the principle that if there was a significant failure,
the :term:`Build Directory` contents could easily be rebuilt.
- Inheriting the :ref:`ref-classes-rm-work` class:
Inheriting this class has shown to speed up builds due to
significantly lower amounts of data stored in the data cache as well
as on disk. Inheriting this class also makes cleanup of
:term:`TMPDIR` faster, at the
expense of being easily able to dive into the source code. File
system maintainers have recommended that the fastest way to clean up
large numbers of files is to reformat partitions rather than delete
files due to the linear nature of partitions. This, of course,
assumes you structure the disk partitions and file systems in a way
that this is practical.
Aside from the previous list, you should keep some trade offs in mind
that can help you speed up the build:
- Remove items from
:term:`DISTRO_FEATURES`
that you might not need.
- Exclude debug symbols and other debug information: If you do not need
these symbols and other debug information, disabling the ``*-dbg``
package generation can speed up the build. You can disable this
generation by setting the
:term:`INHIBIT_PACKAGE_DEBUG_SPLIT`
variable to "1".
- Disable static library generation for recipes derived from
``autoconf`` or ``libtool``: Here is an example showing how to
disable static libraries and still provide an override to handle
exceptions::
STATICLIBCONF = "--disable-static"
STATICLIBCONF:sqlite3-native = ""
EXTRA_OECONF += "${STATICLIBCONF}"
.. note::
- Some recipes need static libraries in order to work correctly
(e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides,
as in the previous example, account for these kinds of
exceptions.
- Some packages have packaging code that assumes the presence of
the static libraries. If so, you might need to exclude them as
well.

856
sources/poky/documentation/dev-manual/start.rst Normal file
View File

@@ -0,0 +1,856 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
***********************************
Setting Up to Use the Yocto Project
***********************************
This chapter provides guidance on how to prepare to use the Yocto
Project. You can learn about creating a team environment to develop
using the Yocto Project, how to set up a :ref:`build
host <dev-manual/start:preparing the build host>`, how to locate
Yocto Project source repositories, and how to create local Git
repositories.
Creating a Team Development Environment
=======================================
It might not be immediately clear how you can use the Yocto Project in a
team development environment, or how to scale it for a large team of
developers. You can adapt the Yocto Project to many different use cases
and scenarios; however, this flexibility could cause difficulties if you
are trying to create a working setup that scales effectively.
To help you understand how to set up this type of environment, this
section presents a procedure that gives you information that can help
you get the results you want. The procedure is high-level and presents
some of the project's most successful experiences, practices, solutions,
and available technologies that have proved to work well in the past;
however, keep in mind, the procedure here is simply a starting point.
You can build off these steps and customize the procedure to fit any
particular working environment and set of practices.
#. *Determine Who is Going to be Developing:* You first need to
understand who is going to be doing anything related to the Yocto
Project and determine their roles. Making this determination is
essential to completing subsequent steps, which are to get your
equipment together and set up your development environment's
hardware topology.
Possible roles are:
- *Application Developer:* This type of developer does application
level work on top of an existing software stack.
- *Core System Developer:* This type of developer works on the
contents of the operating system image itself.
- *Build Engineer:* This type of developer manages Autobuilders and
releases. Depending on the specifics of the environment, not all
situations might need a Build Engineer.
- *Test Engineer:* This type of developer creates and manages
automated tests that are used to ensure all application and core
system development meets desired quality standards.
#. *Gather the Hardware:* Based on the size and make-up of the team,
get the hardware together. Ideally, any development, build, or test
engineer uses a system that runs a supported Linux distribution.
These systems, in general, should be high performance (e.g. dual,
six-core Xeons with 24 Gbytes of RAM and plenty of disk space). You
can help ensure efficiency by having any machines used for testing
or that run Autobuilders be as high performance as possible.
.. note::
Given sufficient processing power, you might also consider
building Yocto Project development containers to be run under
Docker, which is described later.
#. *Understand the Hardware Topology of the Environment:* Once you
understand the hardware involved and the make-up of the team, you
can understand the hardware topology of the development environment.
You can get a visual idea of the machines and their roles across the
development environment.
#. *Use Git as Your Source Control Manager (SCM):* Keeping your
:term:`Metadata` (i.e. recipes,
configuration files, classes, and so forth) and any software you are
developing under the control of an SCM system that is compatible
with the OpenEmbedded build system is advisable. Of all of the SCMs
supported by BitBake, the Yocto Project team strongly recommends using
:ref:`overview-manual/development-environment:git`.
Git is a distributed system
that is easy to back up, allows you to work remotely, and then
connects back to the infrastructure.
.. note::
For information about BitBake, see the
:doc:`bitbake:index`.
It is relatively easy to set up Git services and create infrastructure like
:yocto_git:`/`, which is based on server software called
`Gitolite <https://gitolite.com>`__
with `cgit <https://git.zx2c4.com/cgit/about/>`__ being used to
generate the web interface that lets you view the repositories.
``gitolite`` identifies users using SSH keys and allows
branch-based access controls to repositories that you can control as
little or as much as necessary.
#. *Set up the Application Development Machines:* As mentioned earlier,
application developers are creating applications on top of existing
software stacks. Here are some best practices for setting up
machines used for application development:
- Use a pre-built toolchain that contains the software stack
itself. Then, develop the application code on top of the stack.
This method works well for small numbers of relatively isolated
applications.
- Keep your cross-development toolchains updated. You can do this
through provisioning either as new toolchain downloads or as
updates through a package update mechanism using ``opkg`` to
provide updates to an existing toolchain. The exact mechanics of
how and when to do this depend on local policy.
- Use multiple toolchains installed locally into different
locations to allow development across versions.
#. *Set up the Core Development Machines:* As mentioned earlier, core
developers work on the contents of the operating system itself.
Here are some best practices for setting up machines used for
developing images:
- Have the :term:`OpenEmbedded Build System` available on
the developer workstations so developers can run their own builds
and directly rebuild the software stack.
- Keep the core system unchanged as much as possible and do your
work in layers on top of the core system. Doing so gives you a
greater level of portability when upgrading to new versions of
the core system or Board Support Packages (BSPs).
- Share layers amongst the developers of a particular project and
contain the policy configuration that defines the project.
#. *Set up an Autobuilder:* Autobuilders are often the core of the
development environment. It is here that changes from individual
developers are brought together and centrally tested. Based on this
automated build and test environment, subsequent decisions about
releases can be made. Autobuilders also allow for "continuous
integration" style testing of software components and regression
identification and tracking.
See ":yocto_ab:`Yocto Project Autobuilder <>`" for more
information and links to buildbot. The Yocto Project team has found
this implementation works well in this role. A public example of
this is the Yocto Project Autobuilders, which the Yocto Project team
uses to test the overall health of the project.
The features of this system are:
- Highlights when commits break the build.
- Populates an :ref:`sstate
cache <overview-manual/concepts:shared state cache>` from which
developers can pull rather than requiring local builds.
- Allows commit hook triggers, which trigger builds when commits
are made.
- Allows triggering of automated image booting and testing under
the QuickEMUlator (QEMU).
- Supports incremental build testing and from-scratch builds.
- Shares output that allows developer testing and historical
regression investigation.
- Creates output that can be used for releases.
- Allows scheduling of builds so that resources can be used
efficiently.
#. *Set up Test Machines:* Use a small number of shared, high
performance systems for testing purposes. Developers can use these
systems for wider, more extensive testing while they continue to
develop locally using their primary development system.
#. *Document Policies and Change Flow:* The Yocto Project uses a
hierarchical structure and a pull model. There are scripts to create and
send pull requests (i.e. ``create-pull-request`` and
``send-pull-request``). This model is in line with other open source
projects where maintainers are responsible for specific areas of the
project and a single maintainer handles the final "top-of-tree"
merges.
.. note::
You can also use a more collective push model. The ``gitolite``
software supports both the push and pull models quite easily.
As with any development environment, it is important to document the
policy used as well as any main project guidelines so they are
understood by everyone. It is also a good idea to have
well-structured commit messages, which are usually a part of a
project's guidelines. Good commit messages are essential when
looking back in time and trying to understand why changes were made.
If you discover that changes are needed to the core layer of the
project, it is worth sharing those with the community as soon as
possible. Chances are if you have discovered the need for changes,
someone else in the community needs them also.
#. *Development Environment Summary:* Aside from the previous steps,
here are best practices within the Yocto Project development
environment:
- Use :ref:`overview-manual/development-environment:git` as the source control
system.
- Maintain your Metadata in layers that make sense for your
situation. See the ":ref:`overview-manual/yp-intro:the yocto project layer model`"
section in the Yocto Project Overview and Concepts Manual and the
":ref:`dev-manual/layers:understanding and creating layers`"
section for more information on layers.
- Separate the project's Metadata and code by using separate Git
repositories. See the ":ref:`overview-manual/development-environment:yocto project source repositories`"
section in the Yocto Project Overview and Concepts Manual for
information on these repositories. See the
":ref:`dev-manual/start:locating yocto project source files`"
section for information on how to set up local Git repositories
for related upstream Yocto Project Git repositories.
- Set up the directory for the shared state cache
(:term:`SSTATE_DIR`) where
it makes sense. For example, set up the sstate cache on a system
used by developers in the same organization and share the same
source directories on their machines.
- Set up an Autobuilder and have it populate the sstate cache and
source directories.
- The Yocto Project community encourages you to send patches to the
project to fix bugs or add features. If you do submit patches,
follow the project commit guidelines for writing good commit
messages. See the ":doc:`../contributor-guide/submit-changes`"
section in the Yocto Project and OpenEmbedded Contributor Guide.
- Send changes to the core sooner than later as others are likely
to run into the same issues. For some guidance on mailing lists
to use, see the lists in the
":ref:`contributor-guide/submit-changes:finding a suitable mailing list`"
section. For a description
of the available mailing lists, see the ":ref:`resources-mailinglist`" section in
the Yocto Project Reference Manual.
Preparing the Build Host
========================
This section provides procedures to set up a system to be used as your
:term:`Build Host` for
development using the Yocto Project. Your build host can be a native
Linux machine (recommended), it can be a machine (Linux, Mac, or
Windows) that uses `CROPS <https://github.com/crops/poky-container>`__,
which leverages `Docker Containers <https://www.docker.com/>`__ or it
can be a Windows machine capable of running version 2 of Windows Subsystem
For Linux (WSL 2).
.. note::
The Yocto Project is not compatible with version 1 of
:wikipedia:`Windows Subsystem for Linux <Windows_Subsystem_for_Linux>`.
It is compatible but neither officially supported nor validated with
WSL 2. If you still decide to use WSL please upgrade to
`WSL 2 <https://learn.microsoft.com/en-us/windows/wsl/install>`__.
Once your build host is set up to use the Yocto Project, further steps
are necessary depending on what you want to accomplish. See the
following references for information on how to prepare for Board Support
Package (BSP) development and kernel development:
- *BSP Development:* See the ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
section in the Yocto Project Board Support Package (BSP) Developer's
Guide.
- *Kernel Development:* See the ":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section in the Yocto Project Linux Kernel Development Manual.
Setting Up a Native Linux Host
------------------------------
Follow these steps to prepare a native Linux machine as your Yocto
Project Build Host:
#. *Use a Supported Linux Distribution:* You should have a reasonably
current Linux-based host system. You will have the best results with
a recent release of Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS
as these releases are frequently tested against the Yocto Project and
officially supported. For a list of the distributions under
validation and their status, see the ":ref:`Supported Linux
Distributions <system-requirements-supported-distros>`"
section in the Yocto Project Reference Manual and the wiki page at
:yocto_wiki:`Distribution Support </Distribution_Support>`.
#. *Have Enough Free Memory:* Your system should have at least 50 Gbytes
of free disk space for building images.
#. *Meet Minimal Version Requirements:* The OpenEmbedded build system
should be able to run on any modern distribution that has the
following versions for Git, tar, Python, gcc and make.
- Git &MIN_GIT_VERSION; or greater
- tar &MIN_TAR_VERSION; or greater
- Python &MIN_PYTHON_VERSION; or greater.
- gcc &MIN_GCC_VERSION; or greater.
- GNU make &MIN_MAKE_VERSION; or greater
If your build host does not meet any of these listed version
requirements, you can take steps to prepare the system so that you
can still use the Yocto Project. See the
":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
section in the Yocto Project Reference Manual for information.
#. *Install Development Host Packages:* Required development host
packages vary depending on your build host and what you want to do
with the Yocto Project. Collectively, the number of required packages
is large if you want to be able to cover all cases.
For lists of required packages for all scenarios, see the
":ref:`ref-manual/system-requirements:required packages for the build host`"
section in the Yocto Project Reference Manual.
Once you have completed the previous steps, you are ready to continue
using a given development path on your native Linux machine. If you are
going to use BitBake, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section. If you are going
to use the Extensible SDK, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you want to work on the kernel, see the :doc:`/kernel-dev/index`. If you are going to use
Toaster, see the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual. If you are a VSCode user, you can configure
the `Yocto Project BitBake
<https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`__
extension accordingly.
Setting Up to Use CROss PlatformS (CROPS)
-----------------------------------------
With `CROPS <https://github.com/crops/poky-container>`__, which
leverages `Docker Containers <https://www.docker.com/>`__, you can
create a Yocto Project development environment that is operating system
agnostic. You can set up a container in which you can develop using the
Yocto Project on a Windows, Mac, or Linux machine.
Follow these general steps to prepare a Windows, Mac, or Linux machine
as your Yocto Project build host:
#. *Determine What Your Build Host Needs:*
`Docker <https://www.docker.com/what-docker>`__ is a software
container platform that you need to install on the build host.
Depending on your build host, you might have to install different
software to support Docker containers. Go to the Docker installation
page and read about the platform requirements in "`Supported
Platforms <https://docs.docker.com/engine/install/#supported-platforms>`__"
your build host needs to run containers.
#. *Choose What To Install:* Depending on whether or not your build host
meets system requirements, you need to install "Docker CE Stable" or
the "Docker Toolbox". Most situations call for Docker CE. However, if
you have a build host that does not meet requirements (e.g.
Pre-Windows 10 or Windows 10 "Home" version), you must install Docker
Toolbox instead.
#. *Go to the Install Site for Your Platform:* Click the link for the
Docker edition associated with your build host's native software. For
example, if your build host is running Microsoft Windows Version 10
and you want the Docker CE Stable edition, click that link under
"Supported Platforms".
#. *Install the Software:* Once you have understood all the
pre-requisites, you can download and install the appropriate
software. Follow the instructions for your specific machine and the
type of the software you need to install:
- Install `Docker Desktop on
Windows <https://docs.docker.com/docker-for-windows/install/#install-docker-desktop-on-windows>`__
for Windows build hosts that meet requirements.
- Install `Docker Desktop on
MacOs <https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-desktop-on-mac>`__
for Mac build hosts that meet requirements.
- Install `Docker Engine on
CentOS <https://docs.docker.com/engine/install/centos/>`__
for Linux build hosts running the CentOS distribution.
- Install `Docker Engine on
Debian <https://docs.docker.com/engine/install/debian/>`__
for Linux build hosts running the Debian distribution.
- Install `Docker Engine for
Fedora <https://docs.docker.com/engine/install/fedora/>`__
for Linux build hosts running the Fedora distribution.
- Install `Docker Engine for
Ubuntu <https://docs.docker.com/engine/install/ubuntu/>`__
for Linux build hosts running the Ubuntu distribution.
#. *Optionally Orient Yourself With Docker:* If you are unfamiliar with
Docker and the container concept, you can learn more here -
https://docs.docker.com/get-started/.
#. *Launch Docker or Docker Toolbox:* You should be able to launch
Docker or the Docker Toolbox and have a terminal shell on your
development host.
#. *Set Up the Containers to Use the Yocto Project:* Go to
https://github.com/crops/docker-win-mac-docs/wiki and follow
the directions for your particular build host (i.e. Linux, Mac, or
Windows).
Once you complete the setup instructions for your machine, you have
the Poky, Extensible SDK, and Toaster containers available. You can
click those links from the page and learn more about using each of
those containers.
Once you have a container set up, everything is in place to develop just
as if you were running on a native Linux machine. If you are going to
use the Poky container, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section. If you are going to use the Extensible SDK container, see the
":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you are going to use the Toaster container, see
the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual. If you are a VSCode user, you can configure
the `Yocto Project BitBake
<https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`__
extension accordingly.
Setting Up to Use Windows Subsystem For Linux (WSL 2)
-----------------------------------------------------
With `Windows Subsystem for Linux (WSL 2)
<https://learn.microsoft.com/en-us/windows/wsl/>`__,
you can create a Yocto Project development environment that allows you
to build on Windows. You can set up a Linux distribution inside Windows
in which you can develop using the Yocto Project.
Follow these general steps to prepare a Windows machine using WSL 2 as
your Yocto Project build host:
#. *Make sure your Windows machine is capable of running WSL 2:*
While all Windows 11 and Windows Server 2022 builds support WSL 2,
the first versions of Windows 10 and Windows Server 2019 didn't.
Check the minimum build numbers for `Windows 10
<https://learn.microsoft.com/en-us/windows/wsl/install-manual#step-2---check-requirements-for-running-wsl-2>`__
and for `Windows Server 2019
<https://learn.microsoft.com/en-us/windows/wsl/install-on-server>`__.
To check which build version you are running, you may open a command
prompt on Windows and execute the command "ver"::
C:\Users\myuser> ver
Microsoft Windows [Version 10.0.19041.153]
#. *Install the Linux distribution of your choice inside WSL 2:*
Once you know your version of Windows supports WSL 2, you can
install the distribution of your choice from the Microsoft Store.
Open the Microsoft Store and search for Linux. While there are
several Linux distributions available, the assumption is that your
pick will be one of the distributions supported by the Yocto Project
as stated on the instructions for using a native Linux host. After
making your selection, simply click "Get" to download and install the
distribution.
#. *Check which Linux distribution WSL 2 is using:* Open a Windows
PowerShell and run::
C:\WINDOWS\system32> wsl -l -v
NAME STATE VERSION
*Ubuntu Running 2
Note that WSL 2 supports running as many different Linux distributions
as you want to install.
#. *Optionally Get Familiar with WSL:* You can learn more on
https://docs.microsoft.com/en-us/windows/wsl/wsl2-about.
#. *Launch your WSL Distibution:* From the Windows start menu simply
launch your WSL distribution just like any other application.
#. *Optimize your WSL 2 storage often:* Due to the way storage is
handled on WSL 2, the storage space used by the underlying Linux
distribution is not reflected immediately, and since BitBake heavily
uses storage, after several builds, you may be unaware you are
running out of space. As WSL 2 uses a VHDX file for storage, this issue
can be easily avoided by regularly optimizing this file in a manual way:
1. *Find the location of your VHDX file:*
First you need to find the distro app package directory, to achieve this
open a Windows Powershell as Administrator and run::
C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
PackageFamilyName
-----------------
CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
You should now
replace the PackageFamilyName and your user on the following path
to find your VHDX file::
ls C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
Mode LastWriteTime Length Name
-a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
Your VHDX file path is:
``C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx``
2a. *Optimize your VHDX file using Windows Powershell:*
To use the ``optimize-vhd`` cmdlet below, first install the Hyper-V
option on Windows. Then, open a Windows Powershell as Administrator to
optimize your VHDX file, shutting down WSL first::
C:\WINDOWS\system32> wsl --shutdown
C:\WINDOWS\system32> optimize-vhd -Path C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
A progress bar should be shown while optimizing the
VHDX file, and storage should now be reflected correctly on the
Windows Explorer.
2b. *Optimize your VHDX file using DiskPart:*
The ``optimize-vhd`` cmdlet noted in step 2a above is provided by
Hyper-V. Not all SKUs of Windows can install Hyper-V. As an alternative,
use the DiskPart tool. To start, open a Windows command prompt as
Administrator to optimize your VHDX file, shutting down WSL first::
C:\WINDOWS\system32> wsl --shutdown
C:\WINDOWS\system32> diskpart
DISKPART> select vdisk file="<path_to_VHDX_file>"
DISKPART> attach vdisk readonly
DISKPART> compact vdisk
DISKPART> detach
DISKPART> exit
.. note::
The current implementation of WSL 2 does not have out-of-the-box
access to external devices such as those connected through a USB
port, but it automatically mounts your ``C:`` drive on ``/mnt/c/``
(and others), which you can use to share deploy artifacts to be later
flashed on hardware through Windows, but your :term:`Build Directory`
should not reside inside this mountpoint.
Once you have WSL 2 set up, everything is in place to develop just as if
you were running on a native Linux machine. If you are going to use the
Extensible SDK container, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you are going to use the Toaster container, see
the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual. If you are a VSCode user, you can configure
the `Yocto Project BitBake
<https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`__
extension accordingly.
Locating Yocto Project Source Files
===================================
This section shows you how to locate, fetch and configure the source
files you'll need to work with the Yocto Project.
.. note::
- For concepts and introductory information about Git as it is used
in the Yocto Project, see the ":ref:`overview-manual/development-environment:git`"
section in the Yocto Project Overview and Concepts Manual.
- For concepts on Yocto Project source repositories, see the
":ref:`overview-manual/development-environment:yocto project source repositories`"
section in the Yocto Project Overview and Concepts Manual."
Accessing Source Repositories
-----------------------------
Working from a copy of the upstream :ref:`dev-manual/start:accessing source repositories` is the
preferred method for obtaining and using a Yocto Project release. You
can view the Yocto Project Source Repositories at
:yocto_git:`/`. In particular, you can find the ``poky``
repository at :yocto_git:`/poky`.
Use the following procedure to locate the latest upstream copy of the
``poky`` Git repository:
#. *Access Repositories:* Open a browser and go to
:yocto_git:`/` to access the GUI-based interface into the
Yocto Project source repositories.
#. *Select the Repository:* Click on the repository in which you are
interested (e.g. ``poky``).
#. *Find the URL Used to Clone the Repository:* At the bottom of the
page, note the URL used to clone that repository
(e.g. :yocto_git:`/poky`).
.. note::
For information on cloning a repository, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`" section.
Accessing Source Archives
-------------------------
The Yocto Project also provides source archives of its releases, which
are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory
containing the release you wish to use, for example
:yocto_dl:`&DISTRO_REL_LATEST_TAG; </releases/yocto/&DISTRO_REL_LATEST_TAG;/>`.
You will find there source archives of individual components (if you wish
to use them individually), and of the corresponding Poky release bundling
a selection of these components.
.. note::
The recommended method for accessing Yocto Project components is to
use Git to clone the upstream repository and work from within that
locally cloned repository.
Using the Downloads Page
------------------------
The :yocto_home:`Yocto Project Website <>` uses a "RELEASES" page
from which you can locate and download tarballs of any Yocto Project
release. Rather than Git repositories, these files represent snapshot
tarballs similar to the tarballs located in the Index of Releases
described in the ":ref:`dev-manual/start:accessing source archives`" section.
#. *Go to the Yocto Project Website:* Open The
:yocto_home:`Yocto Project Website <>` in your browser.
#. *Get to the Downloads Area:* Select the "RELEASES" item from the
pull-down "DEVELOPMENT" tab menu near the top of the page.
#. *Select a Yocto Project Release:* On the top of the "RELEASE" page currently
supported releases are displayed, further down past supported Yocto Project
releases are visible. The "Download" links in the rows of the table there
will lead to the download tarballs for the release.
.. note::
For a "map" of Yocto Project releases to version numbers, see the
:yocto_wiki:`Releases </Releases>` wiki page.
You can use the "RELEASE ARCHIVE" link to reveal a menu of all Yocto
Project releases.
#. *Download Tools or Board Support Packages (BSPs):* Next to the tarballs you
will find download tools or BSPs as well. Just select a Yocto Project
release and look for what you need.
Cloning and Checking Out Branches
=================================
To use the Yocto Project for development, you need a release locally
installed on your development system. This locally installed set of
files is referred to as the :term:`Source Directory`
in the Yocto Project documentation.
The preferred method of creating your Source Directory is by using
:ref:`overview-manual/development-environment:git` to clone a local copy of the upstream
``poky`` repository. Working from a cloned copy of the upstream
repository allows you to contribute back into the Yocto Project or to
simply work with the latest software on a development branch. Because
Git maintains and creates an upstream repository with a complete history
of changes and you are working with a local clone of that repository,
you have access to all the Yocto Project development branches and tag
names used in the upstream repository.
Cloning the ``poky`` Repository
-------------------------------
Follow these steps to create a local version of the upstream
:term:`Poky` Git repository.
#. *Set Your Directory:* Change your working directory to where you want
to create your local copy of ``poky``.
#. *Clone the Repository:* The following example command clones the
``poky`` repository and uses the default name "poky" for your local
repository::
$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Counting objects: 432160, done.
remote: Compressing objects: 100% (102056/102056), done.
remote: Total 432160 (delta 323116), reused 432037 (delta 323000)
Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
Resolving deltas: 100% (323116/323116), done.
Checking connectivity... done.
Unless you
specify a specific development branch or tag name, Git clones the
"master" branch, which results in a snapshot of the latest
development changes for "master". For information on how to check out
a specific development branch or on how to check out a local branch
based on a tag name, see the
":ref:`dev-manual/start:checking out by branch in poky`" and
":ref:`dev-manual/start:checking out by tag in poky`" sections, respectively.
Once the local repository is created, you can change to that
directory and check its status. The ``master`` branch is checked out
by default::
$ cd poky
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
$ git branch
* master
Your local repository of poky is identical to the
upstream poky repository at the time from which it was cloned. As you
work with the local branch, you can periodically use the
``git pull --rebase`` command to be sure you are up-to-date
with the upstream branch.
Checking Out by Branch in Poky
------------------------------
When you clone the upstream poky repository, you have access to all its
development branches. Each development branch in a repository is unique
as it forks off the "master" branch. To see and use the files of a
particular development branch locally, you need to know the branch name
and then specifically check out that development branch.
.. note::
Checking out an active development branch by branch name gives you a
snapshot of that particular branch at the time you check it out.
Further development on top of the branch that occurs after check it
out can occur.
#. *Switch to the Poky Directory:* If you have a local poky Git
repository, switch to that directory. If you do not have the local
copy of poky, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section.
#. *Determine Existing Branch Names:*
::
$ git branch -a
* master
remotes/origin/1.1_M1
remotes/origin/1.1_M2
remotes/origin/1.1_M3
remotes/origin/1.1_M4
remotes/origin/1.2_M1
remotes/origin/1.2_M2
remotes/origin/1.2_M3
. . .
remotes/origin/thud
remotes/origin/thud-next
remotes/origin/warrior
remotes/origin/warrior-next
remotes/origin/zeus
remotes/origin/zeus-next
... and so on ...
#. *Check out the Branch:* Check out the development branch in which you
want to work. For example, to access the files for the Yocto Project
&DISTRO; Release (&DISTRO_NAME;), use the following command::
$ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
Branch &DISTRO_NAME_NO_CAP; set up to track remote branch &DISTRO_NAME_NO_CAP; from origin.
Switched to a new branch '&DISTRO_NAME_NO_CAP;'
The previous command checks out the "&DISTRO_NAME_NO_CAP;" development
branch and reports that the branch is tracking the upstream
"origin/&DISTRO_NAME_NO_CAP;" branch.
The following command displays the branches that are now part of your
local poky repository. The asterisk character indicates the branch
that is currently checked out for work::
$ git branch
master
* &DISTRO_NAME_NO_CAP;
Checking Out by Tag in Poky
---------------------------
Similar to branches, the upstream repository uses tags to mark specific
commits associated with significant points in a development branch (i.e.
a release point or stage of a release). You might want to set up a local
branch based on one of those points in the repository. The process is
similar to checking out by branch name except you use tag names.
.. note::
Checking out a branch based on a tag gives you a stable set of files
not affected by development on the branch above the tag.
#. *Switch to the Poky Directory:* If you have a local poky Git
repository, switch to that directory. If you do not have the local
copy of poky, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section.
#. *Fetch the Tag Names:* To checkout the branch based on a tag name,
you need to fetch the upstream tags into your local repository::
$ git fetch --tags
$
#. *List the Tag Names:* You can list the tag names now::
$ git tag
1.1_M1.final
1.1_M1.rc1
1.1_M1.rc2
1.1_M2.final
1.1_M2.rc1
.
.
.
yocto-2.5
yocto-2.5.1
yocto-2.5.2
yocto-2.5.3
yocto-2.6
yocto-2.6.1
yocto-2.6.2
yocto-2.7
yocto_1.5_M5.rc8
#. *Check out the Branch:*
::
$ git checkout tags/yocto-&DISTRO; -b my_yocto_&DISTRO;
Switched to a new branch 'my_yocto_&DISTRO;'
$ git branch
master
* my_yocto_&DISTRO;
The previous command creates and
checks out a local branch named "my_yocto_&DISTRO;", which is based on
the commit in the upstream poky repository that has the same tag. In
this example, the files you have available locally as a result of the
``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;"
development branch at the point where Yocto Project &DISTRO; was
released.

66
sources/poky/documentation/dev-manual/temporary-source-code.rst Normal file
View File

@@ -0,0 +1,66 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Finding Temporary Source Code
*****************************
You might find it helpful during development to modify the temporary
source code used by recipes to build packages. For example, suppose you
are developing a patch and you need to experiment a bit to figure out
your solution. After you have initially built the package, you can
iteratively tweak the source code, which is located in the
:term:`Build Directory`, and then you can force a re-compile and quickly
test your altered code. Once you settle on a solution, you can then preserve
your changes in the form of patches.
During a build, the unpacked temporary source code used by recipes to
build packages is available in the :term:`Build Directory` as defined by the
:term:`S` variable. Below is the default value for the :term:`S` variable as
defined in the ``meta/conf/bitbake.conf`` configuration file in the
:term:`Source Directory`::
S = "${WORKDIR}/${BP}"
You should be aware that many recipes override the
:term:`S` variable. For example, recipes that fetch their source from Git
usually set :term:`S` to ``${WORKDIR}/git``.
.. note::
The :term:`BP` represents the base recipe name, which consists of the name
and version::
BP = "${BPN}-${PV}"
The path to the work directory for the recipe
(:term:`WORKDIR`) is defined as
follows::
${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}
The actual directory depends on several things:
- :term:`TMPDIR`: The top-level build
output directory.
- :term:`MULTIMACH_TARGET_SYS`:
The target system identifier.
- :term:`PN`: The recipe name.
- :term:`EXTENDPE`: The epoch --- if
:term:`PE` is not specified, which is
usually the case for most recipes, then :term:`EXTENDPE` is blank.
- :term:`PV`: The recipe version.
- :term:`PR`: The recipe revision.
As an example, assume a Source Directory top-level folder named
``poky``, a default :term:`Build Directory` at ``poky/build``, and a
``qemux86-poky-linux`` machine target system. Furthermore, suppose your
recipe is named ``foo_1.3.0.bb``. In this case, the work directory the
build system uses to build the package would be as follows::
poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0

397
sources/poky/documentation/dev-manual/upgrading-recipes.rst Normal file
View File

@@ -0,0 +1,397 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Upgrading Recipes
*****************
Over time, upstream developers publish new versions for software built
by layer recipes. It is recommended to keep recipes up-to-date with
upstream version releases.
While there are several methods to upgrade a recipe, you might
consider checking on the upgrade status of a recipe first. You can do so
using the ``devtool check-upgrade-status`` command. See the
":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
section in the Yocto Project Reference Manual for more information.
The remainder of this section describes three ways you can upgrade a
recipe. You can use the Automated Upgrade Helper (AUH) to set up
automatic version upgrades. Alternatively, you can use
``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
you can manually upgrade a recipe by editing the recipe itself.
Using the Auto Upgrade Helper (AUH)
===================================
The AUH utility works in conjunction with the OpenEmbedded build system
in order to automatically generate upgrades for recipes based on new
versions being published upstream. Use AUH when you want to create a
service that performs the upgrades automatically and optionally sends
you an email with the results.
AUH allows you to update several recipes with a single use. You can also
optionally perform build and integration tests using images with the
results saved to your hard drive and emails of results optionally sent
to recipe maintainers. Finally, AUH creates Git commits with appropriate
commit messages in the layer's tree for the changes made to recipes.
.. note::
In some conditions, you should not use AUH to upgrade recipes
and should instead use either ``devtool upgrade`` or upgrade your
recipes manually:
- When AUH cannot complete the upgrade sequence. This situation
usually results because custom patches carried by the recipe
cannot be automatically rebased to the new version. In this case,
``devtool upgrade`` allows you to manually resolve conflicts.
- When for any reason you want fuller control over the upgrade
process. For example, when you want special arrangements for
testing.
The following steps describe how to set up the AUH utility:
#. *Be Sure the Development Host is Set Up:* You need to be sure that
your development host is set up to use the Yocto Project. For
information on how to set up your host, see the
":ref:`dev-manual/start:Preparing the Build Host`" section.
#. *Make Sure Git is Configured:* The AUH utility requires Git to be
configured because AUH uses Git to save upgrades. Thus, you must have
Git user and email configured. The following command shows your
configurations::
$ git config --list
If you do not have the user and
email configured, you can use the following commands to do so::
$ git config --global user.name some_name
$ git config --global user.email username@domain.com
#. *Clone the AUH Repository:* To use AUH, you must clone the repository
onto your development host. The following command uses Git to create
a local copy of the repository on your system::
$ git clone git://git.yoctoproject.org/auto-upgrade-helper
Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done.
remote: Compressing objects: 100% (300/300), done.
remote: Total 768 (delta 499), reused 703 (delta 434)
Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
Resolving deltas: 100% (499/499), done.
Checking connectivity... done.
AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or
:term:`Poky` repositories.
#. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script`
script to create a fresh :term:`Build Directory` that you use exclusively
for running the AUH utility::
$ cd poky
$ source oe-init-build-env your_AUH_build_directory
Re-using an existing :term:`Build Directory` and its configurations is not
recommended as existing settings could cause AUH to fail or behave
undesirably.
#. *Make Configurations in Your Local Configuration File:* Several
settings are needed in the ``local.conf`` file in the build
directory you just created for AUH. Make these following
configurations:
- If you want to enable :ref:`Build
History <dev-manual/build-quality:maintaining build output quality>`,
which is optional, you need the following lines in the
``conf/local.conf`` file::
INHERIT =+ "buildhistory"
BUILDHISTORY_COMMIT = "1"
With this configuration and a successful
upgrade, a build history "diff" file appears in the
``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
your :term:`Build Directory`.
- If you want to enable testing through the :ref:`ref-classes-testimage`
class, which is optional, you need to have the following set in
your ``conf/local.conf`` file::
IMAGE_CLASSES += "testimage"
.. note::
If your distro does not enable by default ptest, which Poky
does, you need the following in your ``local.conf`` file::
DISTRO_FEATURES:append = " ptest"
#. *Optionally Start a vncserver:* If you are running in a server
without an X11 session, you need to start a vncserver::
$ vncserver :1
$ export DISPLAY=:1
#. *Create and Edit an AUH Configuration File:* You need to have the
``upgrade-helper/upgrade-helper.conf`` configuration file in your
:term:`Build Directory`. You can find a sample configuration file in the
:yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`.
Read through the sample file and make configurations as needed. For
example, if you enabled build history in your ``local.conf`` as
described earlier, you must enable it in ``upgrade-helper.conf``.
Also, if you are using the default ``maintainers.inc`` file supplied
with Poky and located in ``meta-yocto`` and you do not set a
"maintainers_whitelist" or "global_maintainer_override" in the
``upgrade-helper.conf`` configuration, and you specify "-e all" on
the AUH command-line, the utility automatically sends out emails to
all the default maintainers. Please avoid this.
This next set of examples describes how to use the AUH:
- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
following form::
$ upgrade-helper.py recipe_name
For example, this command upgrades the ``xmodmap`` recipe::
$ upgrade-helper.py xmodmap
- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
specific recipe to a particular version, use the following form::
$ upgrade-helper.py recipe_name -t version
For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3::
$ upgrade-helper.py xmodmap -t 1.2.3
- *Upgrading all Recipes to the Latest Versions and Suppressing Email
Notifications:* To upgrade all recipes to their most recent versions
and suppress the email notifications, use the following command::
$ upgrade-helper.py all
- *Upgrading all Recipes to the Latest Versions and Send Email
Notifications:* To upgrade all recipes to their most recent versions
and send email messages to maintainers for each attempted recipe as
well as a status email, use the following command::
$ upgrade-helper.py -e all
Once you have run the AUH utility, you can find the results in the AUH
:term:`Build Directory`::
${BUILDDIR}/upgrade-helper/timestamp
The AUH utility
also creates recipe update commits from successful upgrade attempts in
the layer tree.
You can easily set up to run the AUH utility on a regular basis by using
a cron job. See the
:yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>`
file distributed with the utility for an example.
Using ``devtool upgrade``
=========================
As mentioned earlier, an alternative method for upgrading recipes to
newer versions is to use
:doc:`devtool upgrade </ref-manual/devtool-reference>`.
You can read about ``devtool upgrade`` in general in the
":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`"
section in the Yocto Project Application Development and the Extensible
Software Development Kit (eSDK) Manual.
To see all the command-line options available with ``devtool upgrade``,
use the following help command::
$ devtool upgrade -h
If you want to find out what version a recipe is currently at upstream
without any attempt to upgrade your local version of the recipe, you can
use the following command::
$ devtool latest-version recipe_name
As mentioned in the previous section describing AUH, ``devtool upgrade``
works in a less-automated manner than AUH. Specifically,
``devtool upgrade`` only works on a single recipe that you name on the
command line, cannot perform build and integration testing using images,
and does not automatically generate commits for changes in the source
tree. Despite all these "limitations", ``devtool upgrade`` updates the
recipe file to the new upstream version and attempts to rebase custom
patches contained by the recipe as needed.
.. note::
AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat
of a "wrapper" application for ``devtool upgrade``.
A typical scenario involves having used Git to clone an upstream
repository that you use during build operations. Because you have built the
recipe in the past, the layer is likely added to your
configuration already. If for some reason, the layer is not added, you
could add it easily using the
":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`"
script. For example, suppose you use the ``nano.bb`` recipe from the
``meta-oe`` layer in the ``meta-openembedded`` repository. For this
example, assume that the layer has been cloned into following area::
/home/scottrif/meta-openembedded
The following command from your :term:`Build Directory` adds the layer to
your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``)::
$ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
NOTE: Starting bitbake server...
Parsing recipes: 100% |##########################################| Time: 0:00:55
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
For this example, assume that the ``nano.bb`` recipe that
is upstream has a 2.9.3 version number. However, the version in the
local repository is 2.7.4. The following command from your build
directory automatically upgrades the recipe for you::
$ devtool upgrade nano -V 2.9.3
NOTE: Starting bitbake server...
NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
Parsing recipes: 100% |##########################################| Time: 0:00:46
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Extracting current version source...
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
Adding changed files: 100% |#####################################| Time: 0:00:00
NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
.. note::
Using the ``-V`` option is not necessary. Omitting the version number causes
``devtool upgrade`` to upgrade the recipe to the most recent version.
Continuing with this example, you can use ``devtool build`` to build the
newly upgraded recipe::
$ devtool build nano
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:01
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
Within the ``devtool upgrade`` workflow, you can
deploy and test your rebuilt software. For this example,
however, running ``devtool finish`` cleans up the workspace once the
source in your workspace is clean. This usually means using Git to stage
and submit commits for the changes generated by the upgrade process.
Once the tree is clean, you can clean things up in this example with the
following command from the ``${BUILDDIR}/workspace/sources/nano``
directory::
$ devtool finish nano meta-oe
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:00
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
NOTE: Updating recipe nano_2.9.3.bb
NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
Using the ``devtool finish`` command cleans up the workspace and creates a patch
file based on your commits. The tool puts all patch files back into the
source directory in a sub-directory named ``nano`` in this case.
Manually Upgrading a Recipe
===========================
If for some reason you choose not to upgrade recipes using
:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or
by :ref:`dev-manual/upgrading-recipes:Using ``devtool upgrade```,
you can manually edit the recipe files to upgrade the versions.
.. note::
Manually updating multiple recipes scales poorly and involves many
steps. The recommendation to upgrade recipe versions is through AUH
or ``devtool upgrade``, both of which automate some steps and provide
guidance for others needed for the manual process.
To manually upgrade recipe versions, follow these general steps:
#. *Change the Version:* Rename the recipe such that the version (i.e.
the :term:`PV` part of the recipe name)
changes appropriately. If the version is not part of the recipe name,
change the value as it is set for :term:`PV` within the recipe itself.
#. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds
is fetched from Git or some other version control system, update
:term:`SRCREV` to point to the
commit hash that matches the new version.
#. *Build the Software:* Try to build the recipe using BitBake. Typical
build failures include the following:
- License statements were updated for the new version. For this
case, you need to review any changes to the license and update the
values of :term:`LICENSE` and
:term:`LIC_FILES_CHKSUM`
as needed.
.. note::
License changes are often inconsequential. For example, the
license text's copyright year might have changed.
- Custom patches carried by the older version of the recipe might
fail to apply to the new version. For these cases, you need to
review the failures. Patches might not be necessary for the new
version of the software if the upgraded version has fixed those
issues. If a patch is necessary and failing, you need to rebase it
into the new version.
#. *Optionally Attempt to Build for Several Architectures:* Once you
successfully build the new software for a given architecture, you
could test the build for other architectures by changing the
:term:`MACHINE` variable and
rebuilding the software. This optional step is especially important
if the recipe is to be released publicly.
#. *Check the Upstream Change Log or Release Notes:* Checking both these
reveals if there are new features that could break
backwards-compatibility. If so, you need to take steps to mitigate or
eliminate that situation.
#. *Optionally Create a Bootable Image and Test:* If you want, you can
test the new software by booting it onto actual hardware.
#. *Create a Commit with the Change in the Layer Repository:* After all
builds work and any testing is successful, you can create commits for
any changes in the layer holding your upgraded recipe.

333
sources/poky/documentation/dev-manual/vulnerabilities.rst Normal file
View File

@@ -0,0 +1,333 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Checking for Vulnerabilities
****************************
Vulnerabilities in Poky and OE-Core
===================================
The Yocto Project has an infrastructure to track and address unfixed
known security vulnerabilities, as tracked by the public
:wikipedia:`Common Vulnerabilities and Exposures (CVE) <Common_Vulnerabilities_and_Exposures>`
database.
The Yocto Project maintains a `list of known vulnerabilities
<https://autobuilder.yocto.io/pub/non-release/patchmetrics/>`__
for packages in Poky and OE-Core, tracking the evolution of the number of
unpatched CVEs and the status of patches. Such information is available for
the current development version and for each supported release.
Security is a process, not a product, and thus at any time, a number of security
issues may be impacting Poky and OE-Core. It is up to the maintainers, users,
contributors and anyone interested in the issues to investigate and possibly fix them by
updating software components to newer versions or by applying patches to address them.
It is recommended to work with Poky and OE-Core upstream maintainers and submit
patches to fix them, see ":doc:`../contributor-guide/submit-changes`" for details.
Vulnerability check at build time
=================================
To enable a check for CVE security vulnerabilities using
:ref:`ref-classes-cve-check` in the specific image or target you are building,
add the following setting to your configuration::
INHERIT += "cve-check"
The CVE database contains some old incomplete entries which have been
deemed not to impact Poky or OE-Core. These CVE entries can be excluded from the
check using build configuration::
include conf/distro/include/cve-extra-exclusions.inc
With this CVE check enabled, BitBake build will try to map each compiled software component
recipe name and version information to the CVE database and generate recipe and
image specific reports. These reports will contain:
- metadata about the software component like names and versions
- metadata about the CVE issue such as description and NVD link
- for each software component, a list of CVEs which are possibly impacting this version
- status of each CVE: ``Patched``, ``Unpatched`` or ``Ignored``
The status ``Patched`` means that a patch file to address the security issue has been
applied. ``Unpatched`` status means that no patches to address the issue have been
applied and that the issue needs to be investigated. ``Ignored`` means that after
analysis, it has been deemed to ignore the issue as it for example affects
the software component on a different operating system platform.
After a build with CVE check enabled, reports for each compiled source recipe will be
found in ``build/tmp/deploy/cve``.
For example the CVE check report for the ``flex-native`` recipe looks like::
$ cat ./tmp/deploy/cve/flex-native_cve.json
{
"version": "1",
"package": [
{
"name": "flex-native",
"layer": "meta",
"version": "2.6.4",
"products": [
{
"product": "flex",
"cvesInRecord": "No"
},
{
"product": "flex",
"cvesInRecord": "Yes"
}
],
"issue": [
{
"id": "CVE-2006-0459",
"status": "Patched",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2006-0459",
"summary": "flex.skl in Will Estes and John Millaway Fast Lexical Analyzer Generator (flex) before 2.5.33 does not allocate enough memory for grammars containing (1) REJECT statements or (2) trailing context rules, which causes flex to generate code that contains a buffer overflow that might allow context-dependent attackers to execute arbitrary code.",
"scorev2": "7.5",
"scorev3": "0.0",
"scorev4": "0.0",
"modified": "2024-11-21T00:06Z",
"vector": "NETWORK",
"vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P",
"detail": "version-not-in-range"
},
{
"id": "CVE-2016-6354",
"status": "Patched",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2016-6354",
"summary": "Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.",
"scorev2": "7.5",
"scorev3": "9.8",
"scorev4": "0.0",
"modified": "2024-11-21T02:55Z",
"vector": "NETWORK",
"vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P",
"detail": "version-not-in-range"
},
{
"id": "CVE-2019-6293",
"status": "Ignored",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2019-6293",
"summary": "An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.",
"scorev2": "4.3",
"scorev3": "5.5",
"scorev4": "0.0",
"modified": "2024-11-21T04:46Z",
"vector": "NETWORK",
"vectorString": "AV:N/AC:M/Au:N/C:N/I:N/A:P",
"detail": "upstream-wontfix",
"description": "there is stack exhaustion but no bug and it is building the parser, not running it, effectively similar to a compiler ICE. Upstream no plans to address this."
}
]
}
]
}
For images, a summary of all recipes included in the image and their CVEs is also
generated in the JSON format. These ``.json`` reports can be found
in the ``tmp/deploy/images`` directory for each compiled image.
At build time CVE check will also throw warnings about ``Unpatched`` CVEs::
WARNING: qemu-native-9.2.0-r0 do_cve_check: Found unpatched CVE (CVE-2023-1386)
It is also possible to check the CVE status of individual packages as follows::
bitbake -c cve_check flex libarchive
Fixing CVE product name and version mappings
============================================
By default, :ref:`ref-classes-cve-check` uses the recipe name :term:`BPN` as CVE
product name when querying the CVE database. If this mapping contains false positives, e.g.
some reported CVEs are not for the software component in question, or false negatives like
some CVEs are not found to impact the recipe when they should, then the problems can be
in the recipe name to CVE product mapping. These mapping issues can be fixed by setting
the :term:`CVE_PRODUCT` variable inside the recipe. This defines the name of the software component in the
upstream `NIST CVE database <https://nvd.nist.gov/>`__.
The variable supports using vendor and product names like this::
CVE_PRODUCT = "flex_project:flex westes:flex"
In this example we have two possible vendors names, ``flex_project`` and ``westes``,
with the product name ``flex``. With this setting the ``flex`` recipe only maps to this specific
product and not products from other vendors with same name ``flex``.
Similarly, when the recipe version :term:`PV` is not compatible with software versions used by
the upstream software component releases and the CVE database, these can be fixed using
the :term:`CVE_VERSION` variable.
Note that if the CVE entries in the NVD database contain bugs or have missing or incomplete
information, it is recommended to fix the information there directly instead of working
around the issues possibly for a long time in Poky and OE-Core side recipes. Feedback to
NVD about CVE entries can be provided through the `NVD contact form <https://nvd.nist.gov/info/contact-form>`__.
Fixing vulnerabilities in recipes
=================================
Suppose a CVE security issue impacts a software component. In that case, it can
be fixed by updating to a newer version, by applying a patch, or by marking it
as patched via :term:`CVE_STATUS` variable flag. For Poky and OE-Core master
branches, updating to a more recent software component release with fixes is
the best option, but patches can be applied if releases are not yet available.
For stable branches, we want to avoid API (Application Programming Interface)
or ABI (Application Binary Interface) breakages. When submitting an update,
a minor version update of a component is preferred if the version is
backward-compatible. Many software components have backward-compatible stable
versions, with a notable example of the Linux kernel. However, if the new
version does or likely might introduce incompatibilities, extracting and
backporting patches is preferred.
Here is an example of fixing CVE security issues with patch files,
an example from the :oe_layerindex:`ffmpeg recipe for dunfell </layerindex/recipe/122174>`::
SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \
file://mips64_cpu_detection.patch \
file://CVE-2020-12284.patch \
file://0001-libavutil-include-assembly-with-full-path-from-sourc.patch \
file://CVE-2021-3566.patch \
file://CVE-2021-38291.patch \
file://CVE-2022-1475.patch \
file://CVE-2022-3109.patch \
file://CVE-2022-3341.patch \
file://CVE-2022-48434.patch \
"
The recipe has both generic and security-related fixes. The CVE patch files are named
according to the CVE they fix.
When preparing the patch file, take the original patch from the upstream repository.
Do not use patches from different distributions, except if it is the only available source.
Modify the patch adding OE-related metadata. We will follow the example of the
``CVE-2022-3341.patch``.
The original `commit message <https://github.com/FFmpeg/FFmpeg/commit/9cf652cef49d74afe3d454f27d49eb1a1394951e.patch/>`__
is::
From 9cf652cef49d74afe3d454f27d49eb1a1394951e Mon Sep 17 00:00:00 2001
From: Jiasheng Jiang <jiasheng@iscas.ac.cn>
Date: Wed, 23 Feb 2022 10:31:59 +0800
Subject: [PATCH] avformat/nutdec: Add check for avformat_new_stream
Check for failure of avformat_new_stream() and propagate
the error code.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
libavformat/nutdec.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
For the correct operations of the ``cve-check``, it requires the CVE
identification in a ``CVE:`` tag of the patch file commit message using
the format::
CVE: CVE-2022-3341
It is also recommended to add the ``Upstream-Status:`` tag with a link
to the original patch and sign-off by people working on the backport.
If there are any modifications to the original patch, note them in
the ``Comments:`` tag.
With the additional information, the header of the patch file in OE-core becomes::
From 9cf652cef49d74afe3d454f27d49eb1a1394951e Mon Sep 17 00:00:00 2001
From: Jiasheng Jiang <jiasheng@iscas.ac.cn>
Date: Wed, 23 Feb 2022 10:31:59 +0800
Subject: [PATCH] avformat/nutdec: Add check for avformat_new_stream
Check for failure of avformat_new_stream() and propagate
the error code.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
CVE: CVE-2022-3341
Upstream-Status: Backport [https://github.com/FFmpeg/FFmpeg/commit/9cf652cef49d74afe3d454f27d49eb1a1394951e]
Comments: Refreshed Hunk
Signed-off-by: Narpat Mali <narpat.mali@windriver.com>
Signed-off-by: Bhabu Bindu <bhabu.bindu@kpit.com>
---
libavformat/nutdec.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
A good practice is to include the CVE identifier in the patch file name, the patch file
commit message and optionally in the recipe commit message.
CVE checker will then capture this information and change the CVE status to ``Patched``
in the generated reports.
If analysis shows that the CVE issue does not impact the recipe due to configuration, platform,
version or other reasons, the CVE can be marked as ``Ignored`` by using
the :term:`CVE_STATUS` variable flag with appropriate reason which is mapped to ``Ignored``.
The entry should have the format like::
CVE_STATUS[CVE-2016-10642] = "cpe-incorrect: This is specific to the npm package that installs cmake, so isn't relevant to OpenEmbedded"
As mentioned previously, if data in the CVE database is wrong, it is recommended
to fix those issues in the CVE database (NVD in the case of OE-core and Poky)
directly.
Note that if there are many CVEs with the same status and reason, those can be
shared by using the :term:`CVE_STATUS_GROUPS` variable.
Recipes can be completely skipped by CVE check by including the recipe name in
the :term:`CVE_CHECK_SKIP_RECIPE` variable.
Implementation details
======================
Here's what the :ref:`ref-classes-cve-check` class does to find unpatched CVE IDs.
First the code goes through each patch file provided by a recipe. If a valid CVE ID
is found in the name of the file, the corresponding CVE is considered as patched.
Don't forget that if multiple CVE IDs are found in the filename, only the last
one is considered. Then, the code looks for ``CVE: CVE-ID`` lines in the patch
file. The found CVE IDs are also considered as patched.
Additionally ``CVE_STATUS`` variable flags are parsed for reasons mapped to ``Patched``
and these are also considered as patched.
Then, the code looks up all the CVE IDs in the NIST database for all the
products defined in :term:`CVE_PRODUCT`. Then, for each found CVE:
- If the package name (:term:`PN`) is part of
:term:`CVE_CHECK_SKIP_RECIPE`, it is considered as ``Patched``.
- If the CVE ID has status ``CVE_STATUS[<CVE ID>] = "ignored"`` or if it's set to
any reason which is mapped to status ``Ignored`` via ``CVE_CHECK_STATUSMAP``,
it is set as ``Ignored``.
- If the CVE ID is part of the patched CVE for the recipe, it is
already considered as ``Patched``.
- Otherwise, the code checks whether the recipe version (:term:`PV`)
is within the range of versions impacted by the CVE. If so, the CVE
is considered as ``Unpatched``.
The CVE database is stored in :term:`DL_DIR` and can be inspected using
``sqlite3`` command as follows::
sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462
When analyzing CVEs, it is recommended to:
- study the latest information in `CVE database <https://nvd.nist.gov/vuln/search>`__.
- check how upstream developers of the software component addressed the issue, e.g.
what patch was applied, which upstream release contains the fix.
- check what other Linux distributions like `Debian <https://security-tracker.debian.org/tracker/>`__
did to analyze and address the issue.
- follow security notices from other Linux distributions.
- follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for
discussions and advance notifications of CVE bugs and software releases with fixes.

90
sources/poky/documentation/dev-manual/wayland.rst Normal file
View File

@@ -0,0 +1,90 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using Wayland and Weston
************************
:wikipedia:`Wayland <Wayland_(display_server_protocol)>`
is a computer display server protocol that provides a method for
compositing window managers to communicate directly with applications
and video hardware and expects them to communicate with input hardware
using other libraries. Using Wayland with supporting targets can result
in better control over graphics frame rendering than an application
might otherwise achieve.
The Yocto Project provides the Wayland protocol libraries and the
reference :wikipedia:`Weston <Wayland_(display_server_protocol)#Weston>`
compositor as part of its release. You can find the integrated packages
in the ``meta`` layer of the :term:`Source Directory`.
Specifically, you
can find the recipes that build both Wayland and Weston at
``meta/recipes-graphics/wayland``.
You can build both the Wayland and Weston packages for use only with targets
that accept the :wikipedia:`Mesa 3D and Direct Rendering Infrastructure
<Mesa_(computer_graphics)>`, which is also known as Mesa DRI. This implies that
you cannot build and use the packages if your target uses, for example, the
Intel Embedded Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI.
.. note::
Due to lack of EGL support, Weston 1.0.3 will not run directly on the
emulated QEMU hardware. However, this version of Weston will run
under X emulation without issues.
This section describes what you need to do to implement Wayland and use
the Weston compositor when building an image for a supporting target.
Enabling Wayland in an Image
============================
To enable Wayland, you need to enable it to be built and enable it to be
included (installed) in the image.
Building Wayland
----------------
To cause Mesa to build the ``wayland-egl`` platform and Weston to build
Wayland with Kernel Mode Setting
(`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__)
support, include the "wayland" flag in the
:term:`DISTRO_FEATURES`
statement in your ``local.conf`` file::
DISTRO_FEATURES:append = " wayland"
.. note::
If X11 has been enabled elsewhere, Weston will build Wayland with X11
support
Installing Wayland and Weston
-----------------------------
To install the Wayland feature into an image, you must include the
following
:term:`CORE_IMAGE_EXTRA_INSTALL`
statement in your ``local.conf`` file::
CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
Running Weston
==============
To run Weston inside X11, enabling it as described earlier and building
a Sato image is sufficient. If you are running your image under Sato, a
Weston Launcher appears in the "Utility" category.
Alternatively, you can run Weston through the command-line interpretor
(CLI), which is better suited for development work. To run Weston under
the CLI, you need to do the following after your image is built:
#. Run these commands to export ``XDG_RUNTIME_DIR``::
mkdir -p /tmp/$USER-weston
chmod 0700 /tmp/$USER-weston
export XDG_RUNTIME_DIR=/tmp/$USER-weston
#. Launch Weston in the shell::
weston

731
sources/poky/documentation/dev-manual/wic.rst Normal file
View File

@@ -0,0 +1,731 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating Partitioned Images Using Wic
*************************************
Creating an image for a particular hardware target using the
OpenEmbedded build system does not necessarily mean you can boot that
image as is on your device. Physical devices accept and boot images in
various ways depending on the specifics of the device. Usually,
information about the hardware can tell you what image format the device
requires. Should your device require multiple partitions on an SD card,
flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to
create the properly partitioned image.
The ``wic`` command generates partitioned images from existing
OpenEmbedded build artifacts. Image generation is driven by partitioning
commands contained in an OpenEmbedded kickstart file (``.wks``)
specified either directly on the command line or as one of a selection
of canned kickstart files as shown with the ``wic list images`` command
in the
":ref:`dev-manual/wic:generate an image using an existing kickstart file`"
section. When you apply the command to a given set of build artifacts, the
result is an image or set of images that can be directly written onto media and
used on a particular system.
.. note::
For a kickstart file reference, see the
":ref:`ref-manual/kickstart:openembedded kickstart (\`\`.wks\`\`) reference`"
Chapter in the Yocto Project Reference Manual.
The ``wic`` command and the infrastructure it is based on is by
definition incomplete. The purpose of the command is to allow the
generation of customized images, and as such, was designed to be
completely extensible through a plugin interface. See the
":ref:`dev-manual/wic:using the wic plugin interface`" section
for information on these plugins.
This section provides some background information on Wic, describes what
you need to have in place to run the tool, provides instruction on how
to use the Wic utility, provides information on using the Wic plugins
interface, and provides several examples that show how to use Wic.
Background
==========
This section provides some background on the Wic utility. While none of
this information is required to use Wic, you might find it interesting.
- The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The
"oe" diphthong in "oeic" was promoted to the letter "w", because
"oeic" is both difficult to remember and to pronounce.
- Wic is loosely based on the Meego Image Creator (``mic``) framework.
The Wic implementation has been heavily modified to make direct use
of OpenEmbedded build artifacts instead of package installation and
configuration, which are already incorporated within the OpenEmbedded
artifacts.
- Wic is a completely independent standalone utility that initially
provides easier-to-use and more flexible replacements for an existing
functionality in OE-Core's :ref:`ref-classes-image-live`
class. The difference between Wic and those examples is that with Wic
the functionality of those scripts is implemented by a
general-purpose partitioning language, which is based on Redhat
kickstart syntax.
Requirements
============
In order to use the Wic utility with the OpenEmbedded Build system, your
system needs to meet the following requirements:
- The Linux distribution on your development host must support the
Yocto Project. See the ":ref:`system-requirements-supported-distros`"
section in the Yocto Project Reference Manual for the list of
distributions that support the Yocto Project.
- The standard system utilities, such as ``cp``, must be installed on
your development host system.
- You must have sourced the build environment setup script (i.e.
:ref:`structure-core-script`) found in the :term:`Build Directory`.
- You need to have the build artifacts already available, which
typically means that you must have already created an image using the
OpenEmbedded build system (e.g. ``core-image-minimal``). While it
might seem redundant to generate an image in order to create an image
using Wic, the current version of Wic requires the artifacts in the
form generated by the OpenEmbedded build system.
- You must build several native tools, which are built to run on the
build system::
$ bitbake wic-tools
- Include "wic" as part of the
:term:`IMAGE_FSTYPES`
variable.
- Include the name of the :ref:`wic kickstart file <openembedded-kickstart-wks-reference>`
as part of the :term:`WKS_FILE` variable. If multiple candidate files can
be provided by different layers, specify all the possible names through the
:term:`WKS_FILES` variable instead.
Getting Help
============
You can get general help for the ``wic`` command by entering the ``wic``
command by itself or by entering the command with a help argument as
follows::
$ wic -h
$ wic --help
$ wic help
Currently, Wic supports seven commands: ``cp``, ``create``, ``help``,
``list``, ``ls``, ``rm``, and ``write``. You can get help for all these
commands except "help" by using the following form::
$ wic help command
For example, the following command returns help for the ``write``
command::
$ wic help write
Wic supports help for three topics: ``overview``, ``plugins``, and
``kickstart``. You can get help for any topic using the following form::
$ wic help topic
For example, the following returns overview help for Wic::
$ wic help overview
There is one additional level of help for Wic. You can get help on
individual images through the ``list`` command. You can use the ``list``
command to return the available Wic images as follows::
$ wic list images
genericx86 Create an EFI disk image for genericx86*
beaglebone-yocto Create SD card image for Beaglebone
qemuriscv Create qcow2 image for RISC-V QEMU machines
mkefidisk Create an EFI disk image
qemuloongarch Create qcow2 image for LoongArch QEMU machines
directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
directdisk Create a 'pcbios' direct disk image
efi-bootdisk
mkhybridiso Create a hybrid ISO image
directdisk-gpt Create a 'pcbios' direct disk image
systemd-bootdisk Create an EFI disk image with systemd-boot
sdimage-bootpart Create SD card image with a boot partition
qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
Once you know the list of available
Wic images, you can use ``help`` with the command to get help on a
particular image. For example, the following command returns help on the
"beaglebone-yocto" image::
$ wic list beaglebone-yocto help
Creates a partitioned SD card image for Beaglebone.
Boot files are located in the first vfat partition.
Operational Modes
=================
You can use Wic in two different modes, depending on how much control
you need for specifying the OpenEmbedded build artifacts that are used
for creating the image: Raw and Cooked:
- *Raw Mode:* You explicitly specify build artifacts through Wic
command-line arguments.
- *Cooked Mode:* The current
:term:`MACHINE` setting and image
name are used to automatically locate and provide the build
artifacts. You just supply a kickstart file and the name of the image
from which to use artifacts.
Regardless of the mode you use, you need to have the build artifacts
ready and available.
Raw Mode
--------
Running Wic in raw mode allows you to specify all the partitions through
the ``wic`` command line. The primary use for raw mode is if you have
built your kernel outside of the Yocto Project :term:`Build Directory`.
In other words, you can point to arbitrary kernel, root filesystem locations,
and so forth. Contrast this behavior with cooked mode where Wic looks in the
:term:`Build Directory` (e.g. ``tmp/deploy/images/``\ machine).
The general form of the ``wic`` command in raw mode is::
$ wic create wks_file options ...
Where:
wks_file:
An OpenEmbedded kickstart file. You can provide
your own custom file or use a file from a set of
existing files as described by further options.
optional arguments:
-h, --help show this help message and exit
-o OUTDIR, --outdir OUTDIR
name of directory to create image in
-e IMAGE_NAME, --image-name IMAGE_NAME
name of the image to use the artifacts from e.g. core-
image-sato
-r ROOTFS_DIR, --rootfs-dir ROOTFS_DIR
path to the /rootfs dir to use as the .wks rootfs
source
-b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR
path to the dir containing the boot artifacts (e.g.
/EFI or /syslinux dirs) to use as the .wks bootimg
source
-k KERNEL_DIR, --kernel-dir KERNEL_DIR
path to the dir containing the kernel to use in the
.wks bootimg
-n NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT
path to the native sysroot containing the tools to use
to build the image
-s, --skip-build-check
skip the build check
-f, --build-rootfs build rootfs
-c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz}
compress image with specified compressor
-m, --bmap generate .bmap
--no-fstab-update Do not change fstab file.
-v VARS_DIR, --vars VARS_DIR
directory with <image>.env files that store bitbake
variables
-D, --debug output debug information
.. note::
You do not need root privileges to run Wic. In fact, you should not
run as root when using the utility.
Cooked Mode
-----------
Running Wic in cooked mode leverages off artifacts in the
:term:`Build Directory`. In other words, you do not have to specify kernel or
root filesystem locations as part of the command. All you need to provide is
a kickstart file and the name of the image from which to use artifacts
by using the "-e" option. Wic looks in the :term:`Build Directory` (e.g.
``tmp/deploy/images/``\ machine) for artifacts.
The general form of the ``wic`` command using Cooked Mode is as follows::
$ wic create wks_file -e IMAGE_NAME
Where:
wks_file:
An OpenEmbedded kickstart file. You can provide
your own custom file or use a file from a set of
existing files provided with the Yocto Project
release.
required argument:
-e IMAGE_NAME, --image-name IMAGE_NAME
name of the image to use the artifacts from e.g. core-
image-sato
Using an Existing Kickstart File
================================
If you do not want to create your own kickstart file, you can use an
existing file provided by the Wic installation. As shipped, kickstart
files can be found in the :ref:`overview-manual/development-environment:yocto project source repositories` in the
following two locations::
poky/meta-yocto-bsp/wic
poky/scripts/lib/wic/canned-wks
Use the following command to list the available kickstart files::
$ wic list images
genericx86 Create an EFI disk image for genericx86*
beaglebone-yocto Create SD card image for Beaglebone
qemuriscv Create qcow2 image for RISC-V QEMU machines
mkefidisk Create an EFI disk image
qemuloongarch Create qcow2 image for LoongArch QEMU machines
directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
directdisk Create a 'pcbios' direct disk image
efi-bootdisk
mkhybridiso Create a hybrid ISO image
directdisk-gpt Create a 'pcbios' direct disk image
systemd-bootdisk Create an EFI disk image with systemd-boot
sdimage-bootpart Create SD card image with a boot partition
qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
When you use an existing file, you
do not have to use the ``.wks`` extension. Here is an example in Raw
Mode that uses the ``directdisk`` file::
$ wic create directdisk -r rootfs_dir -b bootimg_dir \
-k kernel_dir -n native_sysroot
Here are the actual partition language commands used in the
``genericx86.wks`` file to generate an image::
# short-description: Create an EFI disk image for genericx86*
# long-description: Creates a partitioned EFI disk image for genericx86* machines
part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
part swap --ondisk sda --size 44 --label swap1 --fstype=swap
bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
Using the Wic Plugin Interface
==============================
You can extend and specialize Wic functionality by using Wic plugins.
This section explains the Wic plugin interface.
.. note::
Wic plugins consist of "source" and "imager" plugins. Imager plugins
are beyond the scope of this section.
Source plugins provide a mechanism to customize partition content during
the Wic image generation process. You can use source plugins to map
values that you specify using ``--source`` commands in kickstart files
(i.e. ``*.wks``) to a plugin implementation used to populate a given
partition.
.. note::
If you use plugins that have build-time dependencies (e.g. native
tools, bootloaders, and so forth) when building a Wic image, you need
to specify those dependencies using the :term:`WKS_FILE_DEPENDS`
variable.
Source plugins are subclasses defined in plugin files. As shipped, the
Yocto Project provides several plugin files. You can see the source
plugin files that ship with the Yocto Project
:yocto_git:`here </poky/tree/scripts/lib/wic/plugins/source>`.
Each of these plugin files contains source plugins that are designed to
populate a specific Wic image partition.
Source plugins are subclasses of the ``SourcePlugin`` class, which is
defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example,
the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py``
file is a subclass of the ``SourcePlugin`` class, which is found in the
``pluginbase.py`` file.
You can also implement source plugins in a layer outside of the Source
Repositories (external layer). To do so, be sure that your plugin files
are located in a directory whose path is
``scripts/lib/wic/plugins/source/`` within your external layer. When the
plugin files are located there, the source plugins they contain are made
available to Wic.
When the Wic implementation needs to invoke a partition-specific
implementation, it looks for the plugin with the same name as the
``--source`` parameter used in the kickstart file given to that
partition. For example, if the partition is set up using the following
command in a kickstart file::
part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
The methods defined as class
members of the matching source plugin (i.e. ``bootimg-pcbios``) in the
``bootimg-pcbios.py`` plugin file are used.
To be more concrete, here is the corresponding plugin definition from
the ``bootimg-pcbios.py`` file for the previous command along with an
example method called by the Wic implementation when it needs to prepare
a partition using an implementation-specific function::
.
.
.
class BootimgPcbiosPlugin(SourcePlugin):
"""
Create MBR boot partition and install syslinux on it.
"""
name = 'bootimg-pcbios'
.
.
.
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
oe_builddir, bootimg_dir, kernel_dir,
rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
In this case, prepare content for legacy bios boot partition.
"""
.
.
.
If a
subclass (plugin) itself does not implement a particular function, Wic
locates and uses the default version in the superclass. It is for this
reason that all source plugins are derived from the ``SourcePlugin``
class.
The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines
a set of methods that source plugins can implement or override. Any
plugins (subclass of ``SourcePlugin``) that do not implement a
particular method inherit the implementation of the method from the
``SourcePlugin`` class. For more information, see the ``SourcePlugin``
class in the ``pluginbase.py`` file for details:
The following list describes the methods implemented in the
``SourcePlugin`` class:
- ``do_prepare_partition()``: Called to populate a partition with
actual content. In other words, the method prepares the final
partition image that is incorporated into the disk image.
- ``do_configure_partition()``: Called before
``do_prepare_partition()`` to create custom configuration files for a
partition (e.g. syslinux or grub configuration files).
- ``do_install_disk()``: Called after all partitions have been
prepared and assembled into a disk image. This method provides a hook
to allow finalization of a disk image (e.g. writing an MBR).
- ``do_stage_partition()``: Special content-staging hook called
before ``do_prepare_partition()``. This method is normally empty.
Typically, a partition just uses the passed-in parameters (e.g. the
unmodified value of ``bootimg_dir``). However, in some cases, things
might need to be more tailored. As an example, certain files might
additionally need to be taken from ``bootimg_dir + /boot``. This hook
allows those files to be staged in a customized fashion.
.. note::
``get_bitbake_var()`` allows you to access non-standard variables that
you might want to use for this behavior.
You can extend the source plugin mechanism. To add more hooks, create
more source plugin methods within ``SourcePlugin`` and the corresponding
derived subclasses. The code that calls the plugin methods uses the
``plugin.get_source_plugin_methods()`` function to find the method or
methods needed by the call. Retrieval of those methods is accomplished
by filling up a dict with keys that contain the method names of
interest. On success, these will be filled in with the actual methods.
See the Wic implementation for examples and details.
Wic Examples
============
This section provides several examples that show how to use the Wic
utility. All the examples assume the list of requirements in the
":ref:`dev-manual/wic:requirements`" section have been met. The
examples assume the previously generated image is
``core-image-minimal``.
Generate an Image using an Existing Kickstart File
--------------------------------------------------
This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart
file::
$ wic create mkefidisk -e core-image-minimal
INFO: Building wic-tools...
.
.
.
INFO: The new image(s) can be found here:
./mkefidisk-201804191017-sda.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
/home/stephano/yocto/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
The previous example shows the easiest way to create an image by running
in cooked mode and supplying a kickstart file and the "-e" option to
point to the existing build artifacts. Your ``local.conf`` file needs to
have the :term:`MACHINE` variable set
to the machine you are using, which is "qemux86" in this example.
Once the image builds, the output provides image location, artifact use,
and kickstart file information.
.. note::
You should always verify the details provided in the output to make
sure that the image was indeed created exactly as expected.
Continuing with the example, you can now write the image from the
:term:`Build Directory` onto a USB stick, or whatever media for which you
built your image, and boot from the media. You can write the image by using
``bmaptool`` or ``dd``::
$ oe-run-native bmaptool-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sdX
or ::
$ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sdX
.. note::
For more information on how to use the ``bmaptool``
to flash a device with an image, see the
":ref:`dev-manual/bmaptool:flashing images using \`bmaptool\``"
section.
Using a Modified Kickstart File
-------------------------------
Because partitioned image creation is driven by the kickstart file, it
is easy to affect image creation by changing the parameters in the file.
This next example demonstrates that through modification of the
``directdisk-gpt`` kickstart file.
As mentioned earlier, you can use the command ``wic list images`` to
show the list of existing kickstart files. The directory in which the
``directdisk-gpt.wks`` file resides is
``scripts/lib/image/canned-wks/``, which is located in the
:term:`Source Directory` (e.g. ``poky``).
Because available files reside in this directory, you can create and add
your own custom files to the directory. Subsequent use of the
``wic list images`` command would then include your kickstart files.
In this example, the existing ``directdisk-gpt`` file already does most
of what is needed. However, for the hardware in this example, the image
will need to boot from ``sdb`` instead of ``sda``, which is what the
``directdisk-gpt`` kickstart file uses.
The example begins by making a copy of the ``directdisk-gpt.wks`` file
in the ``scripts/lib/image/canned-wks`` directory and then by changing
the lines that specify the target disk from which to boot::
$ cp /home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \
/home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
Next, the example modifies the ``directdisksdb-gpt.wks`` file and
changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The
example changes the following two lines and leaves the remaining lines
untouched::
part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024
part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid
Once the lines are changed, the
example generates the ``directdisksdb-gpt`` image. The command points
the process at the ``core-image-minimal`` artifacts for the Next Unit of
Computing (nuc) :term:`MACHINE` the
``local.conf``::
$ wic create directdisksdb-gpt -e core-image-minimal
INFO: Building wic-tools...
.
.
.
Initialising tasks: 100% |#######################################| Time: 0:00:01
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded.
INFO: Creating image(s)...
INFO: The new image(s) can be found here:
./directdisksdb-gpt-201710090938-sdb.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
/home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
Continuing with the example, you can now directly ``dd`` the image to a
USB stick, or whatever media for which you built your image, and boot
the resulting media::
$ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb
140966+0 records in
140966+0 records out
72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s
$ sudo eject /dev/sdb
Using a Modified Kickstart File and Running in Raw Mode
-------------------------------------------------------
This next example manually specifies each build artifact (runs in Raw
Mode) and uses a modified kickstart file. The example also uses the
``-o`` option to cause Wic to create the output somewhere other than the
default output directory, which is the current directory::
$ wic create test.wks -o /home/stephano/testwic \
--rootfs-dir /home/stephano/yocto/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \
--bootimg-dir /home/stephano/yocto/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \
--kernel-dir /home/stephano/yocto/build/tmp/deploy/images/qemux86 \
--native-sysroot /home/stephano/yocto/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: Creating image(s)...
INFO: The new image(s) can be found here:
/home/stephano/testwic/test-201710091445-sdb.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
test.wks
For this example,
:term:`MACHINE` did not have to be
specified in the ``local.conf`` file since the artifact is manually
specified.
Using Wic to Manipulate an Image
--------------------------------
Wic image manipulation allows you to shorten turnaround time during
image development. For example, you can use Wic to delete the kernel
partition of a Wic image and then insert a newly built kernel. This
saves you time from having to rebuild the entire image each time you
modify the kernel.
.. note::
In order to use Wic to manipulate a Wic image as in this example,
your development machine must have the ``mtools`` package installed.
The following example examines the contents of the Wic image, deletes
the existing kernel, and then inserts a new kernel:
#. *List the Partitions:* Use the ``wic ls`` command to list all the
partitions in the Wic image::
$ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic
Num Start End Size Fstype
1 1048576 25041919 23993344 fat16
2 25165824 72157183 46991360 ext4
The previous output shows two partitions in the
``core-image-minimal-qemux86.wic`` image.
#. *Examine a Particular Partition:* Use the ``wic ls`` command again
but in a different form to examine a particular partition.
.. note::
You can get command usage on any Wic command using the following
form::
$ wic help command
For example, the following command shows you the various ways to
use the
wic ls
command::
$ wic help ls
The following command shows what is in partition one::
$ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1
Volume in drive : is boot
Volume Serial Number is E894-1809
Directory for ::/
libcom32 c32 186500 2017-10-09 16:06
libutil c32 24148 2017-10-09 16:06
syslinux cfg 220 2017-10-09 16:06
vesamenu c32 27104 2017-10-09 16:06
vmlinuz 6904608 2017-10-09 16:06
5 files 7 142 580 bytes
16 582 656 bytes free
The previous output shows five files, with the
``vmlinuz`` being the kernel.
.. note::
If you see the following error, you need to update or create a
``~/.mtoolsrc`` file and be sure to have the line "mtools_skip_check=1"
in the file. Then, run the Wic command again::
ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
output: Total number of sectors (47824) not a multiple of sectors per track (32)!
Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
#. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the
``vmlinuz`` file (kernel)::
$ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
#. *Add In the New Kernel:* Use the ``wic cp`` command to add the
updated kernel to the Wic image. Depending on how you built your
kernel, it could be in different places. If you used ``devtool`` and
an SDK to build your kernel, it resides in the ``tmp/work`` directory
of the extensible SDK. If you used ``make`` to build the kernel, the
kernel will be in the ``workspace/sources`` area.
The following example assumes ``devtool`` was used to build the
kernel::
$ wic cp poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \
poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
Once the new kernel is added back into the image, you can use the
``dd`` command or :ref:`bmaptool
<dev-manual/bmaptool:flashing images using \`bmaptool\`>` commands
to flash your wic image onto an SD card or USB stick and test your
target.
.. note::
Using ``bmaptool`` is generally 10 to 20 times faster than using ``dd``.

54
sources/poky/documentation/dev-manual/x32-psabi.rst Normal file
View File

@@ -0,0 +1,54 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using x32 psABI
***************
x32 processor-specific Application Binary Interface (`x32
psABI <https://software.intel.com/en-us/node/628948>`__) is a native
32-bit processor-specific ABI for Intel 64 (x86-64) architectures. An
ABI defines the calling conventions between functions in a processing
environment. The interface determines what registers are used and what
the sizes are for various C data types.
Some processing environments prefer using 32-bit applications even when
running on Intel 64-bit platforms. Consider the i386 psABI, which is a
very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not
provide efficient use and access of the Intel 64-bit processor
resources, leaving the system underutilized. Now consider the x86_64
psABI. This ABI is newer and uses 64-bits for data sizes and program
pointers. The extra bits increase the footprint size of the programs,
libraries, and also increases the memory and file system size
requirements. Executing under the x32 psABI enables user programs to
utilize CPU and system resources more efficiently while keeping the
memory footprint of the applications low. Extra bits are used for
registers but not for addressing mechanisms.
The Yocto Project supports the final specifications of x32 psABI as
follows:
- You can create packages and images in x32 psABI format on x86_64
architecture targets.
- You can successfully build recipes with the x32 toolchain.
- You can create and boot ``core-image-minimal`` and
``core-image-sato`` images.
- There is RPM Package Manager (RPM) support for x32 binaries.
- There is support for large images.
To use the x32 psABI, you need to edit your ``conf/local.conf``
configuration file as follows::
MACHINE = "qemux86-64"
DEFAULTTUNE = "x86-64-x32"
baselib = "${@d.getVar('BASE_LIB:tune-' + (d.getVar('DEFAULTTUNE') \
or 'INVALID')) or 'lib'}"
Once you have set
up your configuration file, use BitBake to build an image that supports
the x32 psABI. Here is an example::
$ bitbake core-image-sato

11
sources/poky/documentation/downloads.rst Normal file
View File

@@ -0,0 +1,11 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=======================
Documentation Downloads
=======================
The documentation can be downloaded in file formats to be read offline or on
another device. The currently supported formats are linked below:
- `EPub <_static/TheYoctoProject.epub>`_
- `PDF <_static/theyoctoproject.pdf>`_

BIN
sources/poky/documentation/figures/yp-how-it-works-new-diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 214 KiB

5
sources/poky/documentation/genindex.rst Normal file
View File

@@ -0,0 +1,5 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====
Index
=====

60
sources/poky/documentation/index.rst Normal file
View File

@@ -0,0 +1,60 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. The Yocto Project documentation master file, created by
sphinx-quickstart on Mon Apr 13 09:38:33 2020.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to the Yocto Project Documentation
==========================================
|
.. toctree::
:maxdepth: 1
:caption: Introduction and Overview
Quick Build <brief-yoctoprojectqs/index>
what-i-wish-id-known
transitioning-to-a-custom-environment
Yocto Project Software Overview <https://www.yoctoproject.org/software-overview/>
Tips and Tricks Wiki <https://wiki.yoctoproject.org/wiki/TipsAndTricks>
.. toctree::
:maxdepth: 1
:caption: Manuals
Overview and Concepts Manual <overview-manual/index>
Contributor Guide <contributor-guide/index>
Reference Manual <ref-manual/index>
Board Support Package (BSP) Developer's guide <bsp-guide/index>
Development Tasks Manual <dev-manual/index>
Linux Kernel Development Manual <kernel-dev/index>
Profile and Tracing Manual <profile-manual/index>
Application Development and the Extensible SDK (eSDK) <sdk-manual/index>
Toaster Manual <toaster-manual/index>
Test Environment Manual <test-manual/index>
bitbake
.. toctree::
:maxdepth: 1
:caption: Release Manuals
:hidden:
Release Information <migration-guides/index>
releases
.. toctree::
:maxdepth: 1
:caption: Documentation Index
:hidden:
genindex
.. toctree::
:maxdepth: 1
:caption: Documentation Downloads
:hidden:
downloads

912
sources/poky/documentation/kernel-dev/advanced.rst Normal file
View File

@@ -0,0 +1,912 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
*******************************************************
Working with Advanced Metadata (``yocto-kernel-cache``)
*******************************************************
Overview
========
In addition to supporting configuration fragments and patches, the Yocto
Project kernel tools also support rich
:term:`Metadata` that you can use to define
complex policies and Board Support Package (BSP) support. The purpose of
the Metadata and the tools that manage it is to help you manage the
complexity of the configuration and sources used to support multiple
BSPs and Linux kernel types.
Kernel Metadata exists in many places. One area in the
:ref:`overview-manual/development-environment:yocto project source repositories`
is the ``yocto-kernel-cache`` Git repository. You can find this repository
grouped under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
Kernel development tools ("kern-tools") are also available in the Yocto Project
Source Repositories under the "Yocto Linux Kernel" heading in the
``yocto-kernel-tools`` Git repository. The recipe that builds these
tools is ``meta/recipes-kernel/kern-tools/kern-tools-native_git.bb`` in
the :term:`Source Directory` (e.g.
``poky``).
Using Kernel Metadata in a Recipe
=================================
As mentioned in the introduction, the Yocto Project contains kernel
Metadata, which is located in the ``yocto-kernel-cache`` Git repository.
This Metadata defines Board Support Packages (BSPs) that correspond to
definitions in linux-yocto recipes for corresponding BSPs. A BSP
consists of an aggregation of kernel policy and enabled
hardware-specific features. The BSP can be influenced from within the
linux-yocto recipe.
.. note::
A Linux kernel recipe that contains kernel Metadata (e.g. inherits
from the ``linux-yocto.inc`` file) is said to be a "linux-yocto style" recipe.
Every linux-yocto style recipe must define the
:term:`KMACHINE` variable. This
variable is typically set to the same value as the :term:`MACHINE` variable,
which is used by :term:`BitBake`.
However, in some cases, the variable might instead refer to the
underlying platform of the :term:`MACHINE`.
Multiple BSPs can reuse the same :term:`KMACHINE` name if they are built
using the same BSP description. Multiple Corei7-based BSPs could share
the same "intel-corei7-64" value for :term:`KMACHINE`. It is important to
realize that :term:`KMACHINE` is just for kernel mapping, while :term:`MACHINE`
is the machine type within a BSP Layer. Even with this distinction,
however, these two variables can hold the same value. See the
":ref:`kernel-dev/advanced:bsp descriptions`" section for more information.
Every linux-yocto style recipe must also indicate the Linux kernel
source repository branch used to build the Linux kernel. The
:term:`KBRANCH` variable must be set
to indicate the branch.
.. note::
You can use the :term:`KBRANCH` value to define an alternate branch typically
with a machine override as shown here from the ``meta-yocto-bsp`` layer::
KBRANCH:beaglebone-yocto = "standard/beaglebone"
The linux-yocto style recipes can optionally define the following
variables:
- :term:`KERNEL_FEATURES`
- :term:`LINUX_KERNEL_TYPE`
:term:`LINUX_KERNEL_TYPE`
defines the kernel type to be used in assembling the configuration. If
you do not specify a :term:`LINUX_KERNEL_TYPE`, it defaults to "standard".
Together with :term:`KMACHINE`, :term:`LINUX_KERNEL_TYPE` defines the search
arguments used by the kernel tools to find the appropriate description
within the kernel Metadata with which to build out the sources and
configuration. The linux-yocto recipes define "standard", "tiny", and
"preempt-rt" kernel types. See the ":ref:`kernel-dev/advanced:kernel types`"
section for more information on kernel types.
During the build, the kern-tools search for the BSP description file
that most closely matches the :term:`KMACHINE` and :term:`LINUX_KERNEL_TYPE`
variables passed in from the recipe. The tools use the first BSP
description they find that matches both variables. If the tools cannot find
a match, they issue a warning.
The tools first search for the :term:`KMACHINE` and then for the
:term:`LINUX_KERNEL_TYPE`. If the tools cannot find a partial match, they
will use the sources from the :term:`KBRANCH` and any configuration
specified in the :term:`SRC_URI`.
You can use the
:term:`KERNEL_FEATURES`
variable to include features (configuration fragments, patches, or both)
that are not already included by the :term:`KMACHINE` and
:term:`LINUX_KERNEL_TYPE` variable combination. For example, to include a
feature specified as "features/netfilter/netfilter.scc", specify::
KERNEL_FEATURES += "features/netfilter/netfilter.scc"
To include a
feature called "cfg/sound.scc" just for the ``qemux86`` machine,
specify::
KERNEL_FEATURES:append:qemux86 = " cfg/sound.scc"
The value of
the entries in :term:`KERNEL_FEATURES` are dependent on their location
within the kernel Metadata itself. The examples here are taken from the
``yocto-kernel-cache`` repository. Each branch of this repository
contains "features" and "cfg" subdirectories at the top-level. For more
information, see the ":ref:`kernel-dev/advanced:kernel metadata syntax`"
section.
Kernel Metadata Syntax
======================
The kernel Metadata consists of three primary types of files: ``scc``
[1]_ description files, configuration fragments, and patches. The
``scc`` files define variables and include or otherwise reference any of
the three file types. The description files are used to aggregate all
types of kernel Metadata into what ultimately describes the sources and
the configuration required to build a Linux kernel tailored to a
specific machine.
The ``scc`` description files are used to define two fundamental types
of kernel Metadata:
- Features
- Board Support Packages (BSPs)
Features aggregate sources in the form of patches and configuration
fragments into a modular reusable unit. You can use features to
implement conceptually separate kernel Metadata descriptions such as
pure configuration fragments, simple patches, complex features, and
kernel types. :ref:`kernel-dev/advanced:kernel types` define general kernel
features and policy to be reused in the BSPs.
BSPs define hardware-specific features and aggregate them with kernel
types to form the final description of what will be assembled and built.
While the kernel Metadata syntax does not enforce any logical separation
of configuration fragments, patches, features or kernel types, best
practices dictate a logical separation of these types of Metadata. The
following Metadata file hierarchy is recommended::
base/
bsp/
cfg/
features/
ktypes/
patches/
The ``bsp`` directory contains the :ref:`kernel-dev/advanced:bsp descriptions`.
The remaining directories all contain "features". Separating ``bsp`` from the
rest of the structure aids conceptualizing intended usage.
Use these guidelines to help place your ``scc`` description files within
the structure:
- If your file contains only configuration fragments, place the file in
the ``cfg`` directory.
- If your file contains only source-code fixes, place the file in the
``patches`` directory.
- If your file encapsulates a major feature, often combining sources
and configurations, place the file in ``features`` directory.
- If your file aggregates non-hardware configuration and patches in
order to define a base kernel policy or major kernel type to be
reused across multiple BSPs, place the file in ``ktypes`` directory.
These distinctions can easily become blurred --- especially as out-of-tree
features slowly merge upstream over time. Also, remember that how the
description files are placed is a purely logical organization and has no
impact on the functionality of the kernel Metadata. There is no impact
because all of ``cfg``, ``features``, ``patches``, and ``ktypes``,
contain "features" as far as the kernel tools are concerned.
Paths used in kernel Metadata files are relative to base, which is
either
:term:`FILESEXTRAPATHS` if
you are creating Metadata in
:ref:`recipe-space <kernel-dev/advanced:recipe-space metadata>`,
or the top level of
:yocto_git:`yocto-kernel-cache </yocto-kernel-cache/tree/>`
if you are creating
:ref:`kernel-dev/advanced:metadata outside the recipe-space`.
.. [1]
``scc`` stands for Series Configuration Control, but the naming has
less significance in the current implementation of the tooling than
it had in the past. Consider ``scc`` files to be description files.
Configuration
-------------
The simplest unit of kernel Metadata is the configuration-only feature.
This feature consists of one or more Linux kernel configuration
parameters in a configuration fragment file (``.cfg``) and a ``.scc``
file that describes the fragment.
As an example, consider the Symmetric Multi-Processing (SMP) fragment
used with the ``linux-yocto-4.12`` kernel as defined outside of the
recipe space (i.e. ``yocto-kernel-cache``). This Metadata consists of
two files: ``smp.scc`` and ``smp.cfg``. You can find these files in the
``cfg`` directory of the ``yocto-4.12`` branch in the
``yocto-kernel-cache`` Git repository::
cfg/smp.scc:
define KFEATURE_DESCRIPTION "Enable SMP for 32 bit builds"
define KFEATURE_COMPATIBILITY all
kconf hardware smp.cfg
cfg/smp.cfg:
CONFIG_SMP=y
CONFIG_SCHED_SMT=y
# Increase default NR_CPUS from 8 to 64 so that platform with
# more than 8 processors can be all activated at boot time
CONFIG_NR_CPUS=64
# The following is needed when setting NR_CPUS to something
# greater than 8 on x86 architectures, it should be automatically
# disregarded by Kconfig when using a different arch
CONFIG_X86_BIGSMP=y
You can find general information on configuration
fragment files in the ":ref:`kernel-dev/common:creating configuration fragments`" section.
Within the ``smp.scc`` file, the
:term:`KFEATURE_DESCRIPTION`
statement provides a short description of the fragment. Higher level
kernel tools use this description.
Also within the ``smp.scc`` file, the ``kconf`` command includes the
actual configuration fragment in an ``.scc`` file, and the "hardware"
keyword identifies the fragment as being hardware enabling, as opposed
to general policy, which would use the "non-hardware" keyword. The
distinction is made for the benefit of the configuration validation
tools, which warn you if a hardware fragment overrides a policy set by a
non-hardware fragment.
.. note::
The description file can include multiple ``kconf`` statements, one per
fragment.
As described in the
":ref:`kernel-dev/common:validating configuration`" section, you can
use the following BitBake command to audit your configuration::
$ bitbake linux-yocto -c kernel_configcheck -f
Patches
-------
Patch descriptions are very similar to configuration fragment
descriptions, which are described in the previous section. However,
instead of a ``.cfg`` file, these descriptions work with source patches
(i.e. ``.patch`` files).
A typical patch includes a description file and the patch itself. As an
example, consider the build patches used with the ``linux-yocto-4.12``
kernel as defined outside of the recipe space (i.e.
``yocto-kernel-cache``). This Metadata consists of several files:
``build.scc`` and a set of ``*.patch`` files. You can find these files
in the ``patches/build`` directory of the ``yocto-4.12`` branch in the
``yocto-kernel-cache`` Git repository.
The following listings show the ``build.scc`` file and part of the
``modpost-mask-trivial-warnings.patch`` file::
patches/build/build.scc:
patch arm-serialize-build-targets.patch
patch powerpc-serialize-image-targets.patch
patch kbuild-exclude-meta-directory-from-distclean-processi.patch
# applied by kgit
# patch kbuild-add-meta-files-to-the-ignore-li.patch
patch modpost-mask-trivial-warnings.patch
patch menuconfig-check-lxdiaglog.sh-Allow-specification-of.patch
patches/build/modpost-mask-trivial-warnings.patch:
From bd48931bc142bdd104668f3a062a1f22600aae61 Mon Sep 17 00:00:00 2001
From: Paul Gortmaker <paul.gortmaker@windriver.com>
Date: Sun, 25 Jan 2009 17:58:09 -0500
Subject: [PATCH] modpost: mask trivial warnings
Newer HOSTCC will complain about various stdio fcns because
.
.
.
char *dump_write = NULL, *files_source = NULL;
int opt;
--
2.10.1
generated by cgit v0.10.2 at 2017-09-28 15:23:23 (GMT)
The description file can
include multiple patch statements where each statement handles a single
patch. In the example ``build.scc`` file, there are five patch statements
for the five patches in the directory.
You can create a typical ``.patch`` file using ``diff -Nurp`` or
``git format-patch`` commands. For information on how to create patches,
see the ":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
and ":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
sections.
Features
--------
Features are complex kernel Metadata types that consist of configuration
fragments, patches, and possibly other feature description files. As an
example, consider the following generic listing::
features/myfeature.scc
define KFEATURE_DESCRIPTION "Enable myfeature"
patch 0001-myfeature-core.patch
patch 0002-myfeature-interface.patch
include cfg/myfeature_dependency.scc
kconf non-hardware myfeature.cfg
This example shows how the ``patch`` and ``kconf`` commands are used as well
as how an additional feature description file is included with the
``include`` command.
Typically, features are less granular than configuration fragments and
are more likely than configuration fragments and patches to be the types
of things you want to specify in the :term:`KERNEL_FEATURES` variable of the
Linux kernel recipe. See the
":ref:`kernel-dev/advanced:using kernel metadata in a recipe`" section earlier
in the manual.
Kernel Types
------------
A kernel type defines a high-level kernel policy by aggregating non-hardware
configuration fragments with patches you want to use when building a Linux
kernel of a specific type (e.g. a real-time kernel). Syntactically, kernel
types are no different than features as described in the
":ref:`kernel-dev/advanced:features`" section. The :term:`LINUX_KERNEL_TYPE`
variable in the kernel recipe selects the kernel type. For example, in the
``linux-yocto_4.12.bb`` kernel recipe found in ``poky/meta/recipes-kernel/linux``, a
:ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>`
directive includes the ``poky/meta/recipes-kernel/linux/linux-yocto.inc`` file,
which has the following statement that defines the default kernel type::
LINUX_KERNEL_TYPE ??= "standard"
Another example would be the real-time kernel (i.e.
``linux-yocto-rt_4.12.bb``). This kernel recipe directly sets the kernel
type as follows::
LINUX_KERNEL_TYPE = "preempt-rt"
.. note::
You can find kernel recipes in the ``meta/recipes-kernel/linux`` directory
of the :ref:`overview-manual/development-environment:yocto project source repositories`
(e.g. ``poky/meta/recipes-kernel/linux/linux-yocto_4.12.bb``). See the
":ref:`kernel-dev/advanced:using kernel metadata in a recipe`"
section for more information.
Three kernel types ("standard", "tiny", and "preempt-rt") are supported
for Linux Yocto kernels:
- "standard": Includes the generic Linux kernel policy of the Yocto
Project linux-yocto kernel recipes. This policy includes, among other
things, which file systems, networking options, core kernel features,
and debugging and tracing options are supported.
- "preempt-rt": Applies the ``PREEMPT_RT`` patches and the
configuration options required to build a real-time Linux kernel.
This kernel type inherits from the "standard" kernel type.
- "tiny": Defines a bare minimum configuration meant to serve as a base
for very small Linux kernels. The "tiny" kernel type is independent
from the "standard" configuration. Although the "tiny" kernel type
does not currently include any source changes, it might in the
future.
For any given kernel type, the Metadata is defined by the ``.scc`` (e.g.
``standard.scc``). Here is a partial listing for the ``standard.scc``
file, which is found in the ``ktypes/standard`` directory of the
``yocto-kernel-cache`` Git repository::
# Include this kernel type fragment to get the standard features and
# configuration values.
# Note: if only the features are desired, but not the configuration
# then this should be included as:
# include ktypes/standard/standard.scc nocfg
# if no chained configuration is desired, include it as:
# include ktypes/standard/standard.scc nocfg inherit
include ktypes/base/base.scc
branch standard
kconf non-hardware standard.cfg
include features/kgdb/kgdb.scc
.
.
.
include cfg/net/ip6_nf.scc
include cfg/net/bridge.scc
include cfg/systemd.scc
include features/rfkill/rfkill.scc
As with any ``.scc`` file, a kernel type definition can aggregate other
``.scc`` files with ``include`` commands. These definitions can also
directly pull in configuration fragments and patches with the ``kconf``
and ``patch`` commands, respectively.
.. note::
It is not strictly necessary to create a kernel type ``.scc``
file. The Board Support Package (BSP) file can implicitly define the
kernel type using a ``define`` :term:`KTYPE` ``myktype`` line. See the
":ref:`kernel-dev/advanced:bsp descriptions`" section for more
information.
BSP Descriptions
----------------
BSP descriptions (i.e. ``*.scc`` files) combine kernel types with
hardware-specific features. The hardware-specific Metadata is typically
defined independently in the BSP layer, and then aggregated with each
supported kernel type.
.. note::
For BSPs supported by the Yocto Project, the BSP description files
are located in the ``bsp`` directory of the ``yocto-kernel-cache``
repository organized under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
This section overviews the BSP description structure, the aggregation
concepts, and presents a detailed example using a BSP supported by the
Yocto Project (i.e. BeagleBone Board). For complete information on BSP
layer file hierarchy, see the :doc:`/bsp-guide/index`.
Description Overview
~~~~~~~~~~~~~~~~~~~~
For simplicity, consider the following root BSP layer description files
for the BeagleBone board. These files employ both a structure and naming
convention for consistency. The naming convention for the file is as
follows::
bsp_root_name-kernel_type.scc
Here are some example root layer
BSP filenames for the BeagleBone Board BSP, which is supported by the
Yocto Project::
beaglebone-standard.scc
beaglebone-preempt-rt.scc
Each file uses the root name (i.e "beaglebone") BSP name followed by the
kernel type.
Examine the ``beaglebone-standard.scc`` file::
define KMACHINE beaglebone
define KTYPE standard
define KARCH arm
include ktypes/standard/standard.scc
branch beaglebone
include beaglebone.scc
# default policy for standard kernels
include features/latencytop/latencytop.scc
include features/profiling/profiling.scc
Every top-level BSP description file
should define the :term:`KMACHINE`,
:term:`KTYPE`, and
:term:`KARCH` variables. These
variables allow the OpenEmbedded build system to identify the
description as meeting the criteria set by the recipe being built. This
example supports the "beaglebone" machine for the "standard" kernel and
the "arm" architecture.
Be aware that there is no hard link between the :term:`KTYPE` variable and a kernel
type description file. Thus, if you do not have the
kernel type defined in your kernel Metadata as it is here, you only need
to ensure that the
:term:`LINUX_KERNEL_TYPE`
variable in the kernel recipe and the :term:`KTYPE` variable in the BSP
description file match.
To separate your kernel policy from your hardware configuration, you
include a kernel type (``ktype``), such as "standard". In the previous
example, this is done using the following::
include ktypes/standard/standard.scc
This file aggregates all the configuration
fragments, patches, and features that make up your standard kernel
policy. See the ":ref:`kernel-dev/advanced:kernel types`" section for more
information.
To aggregate common configurations and features specific to the kernel
for `mybsp`, use the following::
include mybsp.scc
You can see that in the BeagleBone example with the following::
include beaglebone.scc
For information on how to break a complete ``.config`` file into the various
configuration fragments, see the ":ref:`kernel-dev/common:creating configuration fragments`" section.
Finally, if you have any configurations specific to the hardware that
are not in a ``*.scc`` file, you can include them as follows::
kconf hardware mybsp-extra.cfg
The BeagleBone example does not include these
types of configurations. However, the Malta 32-bit board does
("mti-malta32"). Here is the ``mti-malta32-le-standard.scc`` file::
define KMACHINE mti-malta32-le
define KMACHINE qemumipsel
define KTYPE standard
define KARCH mips
include ktypes/standard/standard.scc
branch mti-malta32
include mti-malta32.scc
kconf hardware mti-malta32-le.cfg
Example
~~~~~~~
Many real-world examples are more complex. Like any other ``.scc`` file,
BSP descriptions can aggregate features. Consider the Minnow BSP
definition given the ``linux-yocto-4.4`` branch of the
``yocto-kernel-cache`` (i.e. ``yocto-kernel-cache/bsp/minnow/minnow.scc``)::
include cfg/x86.scc
include features/eg20t/eg20t.scc
include cfg/dmaengine.scc
include features/power/intel.scc
include cfg/efi.scc
include features/usb/ehci-hcd.scc
include features/usb/ohci-hcd.scc
include features/usb/usb-gadgets.scc
include features/usb/touchscreen-composite.scc
include cfg/timer/hpet.scc
include features/leds/leds.scc
include features/spi/spidev.scc
include features/i2c/i2cdev.scc
include features/mei/mei-txe.scc
# Earlyprintk and port debug requires 8250
kconf hardware cfg/8250.cfg
kconf hardware minnow.cfg
kconf hardware minnow-dev.cfg
.. note::
Although the Minnow Board BSP is unused, the Metadata remains and is
being used here just as an example.
The ``minnow.scc`` description file includes a hardware configuration
fragment (``minnow.cfg``) specific to the Minnow BSP as well as several
more general configuration fragments and features enabling hardware
found on the machine. This ``minnow.scc`` description file is then
included in each of the three "minnow" description files for the
supported kernel types (i.e. "standard", "preempt-rt", and "tiny").
Consider the "minnow" description for the "standard" kernel type (i.e.
``minnow-standard.scc``)::
define KMACHINE minnow
define KTYPE standard
define KARCH i386
include ktypes/standard
include minnow.scc
# Extra minnow configs above the minimal defined in minnow.scc
include cfg/efi-ext.scc
include features/media/media-all.scc
include features/sound/snd_hda_intel.scc
# The following should really be in standard.scc
# USB live-image support
include cfg/usb-mass-storage.scc
include cfg/boot-live.scc
# Basic profiling
include features/latencytop/latencytop.scc
include features/profiling/profiling.scc
# Requested drivers that don't have an existing scc
kconf hardware minnow-drivers-extra.cfg
The ``include`` command midway through the file includes the ``minnow.scc`` description
that defines all enabled hardware for the BSP that is common to all
kernel types. Using this command significantly reduces duplication.
Now consider the "minnow" description for the "tiny" kernel type (i.e.
``minnow-tiny.scc``)::
define KMACHINE minnow
define KTYPE tiny
define KARCH i386
include ktypes/tiny
include minnow.scc
As you might expect,
the "tiny" description includes quite a bit less. In fact, it includes
only the minimal policy defined by the "tiny" kernel type and the
hardware-specific configuration required for booting the machine along
with the most basic functionality of the system as defined in the base
"minnow" description file.
Notice again the three critical variables:
:term:`KMACHINE`,
:term:`KTYPE`, and
:term:`KARCH`. Of these variables, only
:term:`KTYPE` has changed to specify the "tiny" kernel type.
Kernel Metadata Location
========================
Kernel Metadata always exists outside of the kernel tree either defined
in a kernel recipe (recipe-space) or outside of the recipe. Where you
choose to define the Metadata depends on what you want to do and how you
intend to work. Regardless of where you define the kernel Metadata, the
syntax used applies equally.
If you are unfamiliar with the Linux kernel and only wish to apply a
configuration and possibly a couple of patches provided to you by
others, the recipe-space method is recommended. This method is also a
good approach if you are working with Linux kernel sources you do not
control or if you just do not want to maintain a Linux kernel Git
repository on your own. For partial information on how you can define
kernel Metadata in the recipe-space, see the
":ref:`kernel-dev/common:modifying an existing recipe`" section.
Conversely, if you are actively developing a kernel and are already
maintaining a Linux kernel Git repository of your own, you might find it
more convenient to work with kernel Metadata kept outside the
recipe-space. Working with Metadata in this area can make iterative
development of the Linux kernel more efficient outside of the BitBake
environment.
Recipe-Space Metadata
---------------------
When stored in recipe-space, the kernel Metadata files reside in a
directory hierarchy below :term:`FILESEXTRAPATHS`. For
a linux-yocto recipe or for a Linux kernel recipe derived by copying
:oe_git:`meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
</openembedded-core/tree/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb>`
into your layer and modifying it, :term:`FILESEXTRAPATHS` is typically set to
``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``.
See the ":ref:`kernel-dev/common:modifying an existing recipe`"
section for more information.
Here is an example that shows a trivial tree of kernel Metadata stored
in recipe-space within a BSP layer::
meta-my_bsp_layer/
`-- recipes-kernel
`-- linux
`-- linux-yocto
|-- bsp-standard.scc
|-- bsp.cfg
`-- standard.cfg
When the Metadata is stored in recipe-space, you must take steps to
ensure BitBake has the necessary information to decide what files to
fetch and when they need to be fetched again. It is only necessary to
specify the ``.scc`` files on the
:term:`SRC_URI`. BitBake parses them
and fetches any files referenced in the ``.scc`` files by the
``include``, ``patch``, or ``kconf`` commands. Because of this, it is
necessary to bump the recipe :term:`PR`
value when changing the content of files not explicitly listed in the
:term:`SRC_URI`.
If the BSP description is in recipe space, you cannot simply list the
``*.scc`` in the :term:`SRC_URI` statement. You need to use the following
form from your kernel append file::
SRC_URI:append:myplatform = " \
file://myplatform;type=kmeta;destsuffix=myplatform \
"
Metadata Outside the Recipe-Space
---------------------------------
When stored outside of the recipe-space, the kernel Metadata files
reside in a separate repository. The OpenEmbedded build system adds the
Metadata to the build as a "type=kmeta" repository through the
:term:`SRC_URI` variable. As an
example, consider the following :term:`SRC_URI` statement from the
``linux-yocto_5.15.bb`` kernel recipe::
SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH};protocol=https \
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.15;destsuffix=${KMETA};protocol=https"
``${KMETA}``, in this context, is simply used to name the directory into
which the Git fetcher places the Metadata. This behavior is no different
than any multi-repository :term:`SRC_URI` statement used in a recipe (e.g.
see the previous section).
You can keep kernel Metadata in a "kernel-cache", which is a directory
containing configuration fragments. As with any Metadata kept outside
the recipe-space, you simply need to use the :term:`SRC_URI` statement with
the "type=kmeta" attribute. Doing so makes the kernel Metadata available
during the configuration phase.
If you modify the Metadata, you must not forget to update the :term:`SRCREV`
statements in the kernel's recipe. In particular, you need to update the
``SRCREV_meta`` variable to match the commit in the ``KMETA`` branch you
wish to use. Changing the data in these branches and not updating the
:term:`SRCREV` statements to match will cause the build to fetch an older
commit.
Organizing Your Source
======================
Many recipes based on the ``linux-yocto-custom.bb`` recipe use Linux
kernel sources that have only a single branch. This type of
repository structure is fine for linear development supporting a single
machine and architecture. However, if you work with multiple boards and
architectures, a kernel source repository with multiple branches is more
efficient. For example, suppose you need a series of patches for one
board to boot. Sometimes, these patches are works-in-progress or
fundamentally wrong, yet they are still necessary for specific boards.
In these situations, you most likely do not want to include these
patches in every kernel you build (i.e. have the patches as part of the
default branch). It is situations like these that give rise to
multiple branches used within a Linux kernel sources Git repository.
Here are repository organization strategies maximizing source reuse,
removing redundancy, and logically ordering your changes. This section
presents strategies for the following cases:
- Encapsulating patches in a feature description and only including the
patches in the BSP descriptions of the applicable boards.
- Creating a machine branch in your kernel source repository and
applying the patches on that branch only.
- Creating a feature branch in your kernel source repository and
merging that branch into your BSP when needed.
The approach you take is entirely up to you and depends on what works
best for your development model.
Encapsulating Patches
---------------------
If you are reusing patches from an external tree and are not working on
the patches, you might find the encapsulated feature to be appropriate.
Given this scenario, you do not need to create any branches in the
source repository. Rather, you just take the static patches you need and
encapsulate them within a feature description. Once you have the feature
description, you simply include that into the BSP description as
described in the ":ref:`kernel-dev/advanced:bsp descriptions`" section.
You can find information on how to create patches and BSP descriptions
in the ":ref:`kernel-dev/advanced:patches`" and
":ref:`kernel-dev/advanced:bsp descriptions`" sections.
Machine Branches
----------------
When you have multiple machines and architectures to support, or you are
actively working on board support, it is more efficient to create
branches in the repository based on individual machines. Having machine
branches allows common source to remain in the development branch with any
features specific to a machine stored in the appropriate machine branch.
This organization method frees you from continually reintegrating your
patches into a feature.
Once you have a new branch, you can set up your kernel Metadata to use
the branch a couple different ways. In the recipe, you can specify the
new branch as the :term:`KBRANCH` to use for the board as follows::
KBRANCH = "mynewbranch"
Another method is to use the ``branch`` command in the BSP
description::
mybsp.scc:
define KMACHINE mybsp
define KTYPE standard
define KARCH i386
include standard.scc
branch mynewbranch
include mybsp-hw.scc
If you find yourself with numerous branches, you might consider using a
hierarchical branching system similar to what the Yocto Linux Kernel Git
repositories use::
common/kernel_type/machine
If you had two kernel types, "standard" and "small" for instance, three
machines, and common as ``mydir``, the branches in your Git repository
might look like this::
mydir/base
mydir/standard/base
mydir/standard/machine_a
mydir/standard/machine_b
mydir/standard/machine_c
mydir/small/base
mydir/small/machine_a
This organization can help clarify the branch relationships. In this
case, ``mydir/standard/machine_a`` includes everything in ``mydir/base``
and ``mydir/standard/base``. The "standard" and "small" branches add
sources specific to those kernel types that for whatever reason are not
appropriate for the other branches.
.. note::
The "base" branches are an artifact of the way Git manages its data
internally on the filesystem: Git will not allow you to use
``mydir/standard`` and ``mydir/standard/machine_a`` because it would have to
create a file and a directory named "standard".
Feature Branches
----------------
When you are actively developing new features, it can be more efficient
to work with that feature as a branch, rather than as a set of patches
that have to be regularly updated. The Yocto Project Linux kernel tools
provide for this with the ``git merge`` command.
To merge a feature branch into a BSP, insert the ``git merge`` command
after any ``branch`` commands::
mybsp.scc:
define KMACHINE mybsp
define KTYPE standard
define KARCH i386
include standard.scc
branch mynewbranch
git merge myfeature
include mybsp-hw.scc
SCC Description File Reference
==============================
This section provides a brief reference for the commands you can use
within an SCC description file (``.scc``):
- ``branch [ref]``: Creates a new branch relative to the current branch
(typically ``${KTYPE}``) using the currently checked-out branch, or
"ref" if specified.
- ``define``: Defines variables, such as
:term:`KMACHINE`,
:term:`KTYPE`,
:term:`KARCH`, and
:term:`KFEATURE_DESCRIPTION`.
- ``include SCC_FILE``: Includes an SCC file in the current file. The
file is parsed as if you had inserted it inline.
- ``kconf [hardware|non-hardware] CFG_FILE``: Queues a configuration
fragment for merging into the final Linux ``.config`` file.
- ``git merge GIT_BRANCH``: Merges the feature branch into the current
branch.
- ``patch PATCH_FILE``: Applies the patch to the current Git branch.

1859
sources/poky/documentation/kernel-dev/common.rst Normal file
View File

File diff suppressed because it is too large Load Diff

420
sources/poky/documentation/kernel-dev/concepts-appx.rst Normal file
View File

@@ -0,0 +1,420 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
************************
Advanced Kernel Concepts
************************
Yocto Project Kernel Development and Maintenance
================================================
Kernels available through the Yocto Project (Yocto Linux kernels), like
other kernels, are based off the Linux kernel releases from
https://www.kernel.org. At the beginning of a major Linux kernel
development cycle, the Yocto Project team chooses a Linux kernel based
on factors such as release timing, the anticipated release timing of
final upstream ``kernel.org`` versions, and Yocto Project feature
requirements. Typically, the Linux kernel chosen is in the final stages
of development by the Linux community. In other words, the Linux kernel
is in the release candidate or "rc" phase and has yet to reach final
release. But, by being in the final stages of external development, the
team knows that the ``kernel.org`` final release will clearly be within
the early stages of the Yocto Project development window.
This balance allows the Yocto Project team to deliver the most
up-to-date Yocto Linux kernel possible, while still ensuring that the
team has a stable official release for the baseline Linux kernel
version.
As implied earlier, the ultimate source for Yocto Linux kernels are
released kernels from ``kernel.org``. In addition to a foundational
kernel from ``kernel.org``, the available Yocto Linux kernels contain a
mix of important new mainline developments, non-mainline developments
(when no alternative exists), Board Support Package (BSP) developments,
and custom features. These additions result in a commercially released
Yocto Project Linux kernel that caters to specific embedded designer
needs for targeted hardware.
You can find a web interface to the Yocto Linux kernels in the
:ref:`overview-manual/development-environment:yocto project source repositories`
at :yocto_git:`/`. If you look at the interface, you will see to
the left a grouping of Git repositories titled "Yocto Linux Kernel".
Within this group, you will find several Linux Yocto kernels developed
and included with Yocto Project releases:
- *linux-yocto-4.1:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.0. This kernel is based on the Linux 4.1
released kernel.
- *linux-yocto-4.4:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.1. This kernel is based on the Linux 4.4
released kernel.
- *linux-yocto-4.6:* A temporary kernel that is not tied to any
Yocto Project release.
- *linux-yocto-4.8:* The stable yocto Project kernel to use with
the Yocto Project Release 2.2.
- *linux-yocto-4.9:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.3. This kernel is based on the Linux 4.9
released kernel.
- *linux-yocto-4.10:* The default stable Yocto Project kernel to
use with the Yocto Project Release 2.3. This kernel is based on the
Linux 4.10 released kernel.
- *linux-yocto-4.12:* The default stable Yocto Project kernel to
use with the Yocto Project Release 2.4. This kernel is based on the
Linux 4.12 released kernel.
- *yocto-kernel-cache:* The ``linux-yocto-cache`` contains patches
and configurations for the linux-yocto kernel tree. This repository
is useful when working on the linux-yocto kernel. For more
information on this "Advanced Kernel Metadata", see the
":doc:`/kernel-dev/advanced`" Chapter.
- *linux-yocto-dev:* A development kernel based on the latest
upstream release candidate available.
.. note::
Long Term Support Initiative (LTSI) for Yocto Linux kernels is as
follows:
- For Yocto Project releases 1.7, 1.8, and 2.0, the LTSI kernel is
``linux-yocto-3.14``.
- For Yocto Project releases 2.1, 2.2, and 2.3, the LTSI kernel is
``linux-yocto-4.1``.
- For Yocto Project release 2.4, the LTSI kernel is
``linux-yocto-4.9``
- ``linux-yocto-4.4`` is an LTS kernel.
Once a Yocto Linux kernel is officially released, the Yocto Project team
goes into their next development cycle, or upward revision (uprev)
cycle, while still continuing maintenance on the released kernel. It is
important to note that the most sustainable and stable way to include
feature development upstream is through a kernel uprev process.
Back-porting hundreds of individual fixes and minor features from
various kernel versions is not sustainable and can easily compromise
quality.
During the uprev cycle, the Yocto Project team uses an ongoing analysis
of Linux kernel development, BSP support, and release timing to select
the best possible ``kernel.org`` Linux kernel version on which to base
subsequent Yocto Linux kernel development. The team continually monitors
Linux community kernel development to look for significant features of
interest. The team does consider back-porting large features if they
have a significant advantage. User or community demand can also trigger
a back-port or creation of new functionality in the Yocto Project
baseline kernel during the uprev cycle.
Generally speaking, every new Linux kernel both adds features and
introduces new bugs. These consequences are the basic properties of
upstream Linux kernel development and are managed by the Yocto Project
team's Yocto Linux kernel development strategy. It is the Yocto Project
team's policy to not back-port minor features to the released Yocto
Linux kernel. They only consider back-porting significant technological
jumps --- and, that is done after a complete gap analysis. The reason
for this policy is that back-porting any small to medium sized change
from an evolving Linux kernel can easily create mismatches,
incompatibilities and very subtle errors.
The policies described in this section result in both a stable and a
cutting edge Yocto Linux kernel that mixes forward ports of existing
Linux kernel features and significant and critical new functionality.
Forward porting Linux kernel functionality into the Yocto Linux kernels
available through the Yocto Project can be thought of as a "micro
uprev". The many "micro uprevs" produce a Yocto Linux kernel version
with a mix of important new mainline, non-mainline, BSP developments and
feature integrations. This Yocto Linux kernel gives insight into new
features and allows focused amounts of testing to be done on the kernel,
which prevents surprises when selecting the next major uprev. The
quality of these cutting edge Yocto Linux kernels is evolving and the
kernels are used in leading edge feature and BSP development.
Yocto Linux Kernel Architecture and Branching Strategies
========================================================
As mentioned earlier, a key goal of the Yocto Project is to present the
developer with a kernel that has a clear and continuous history that is
visible to the user. The architecture and mechanisms, in particular the
branching strategies, used achieve that goal in a manner similar to
upstream Linux kernel development in ``kernel.org``.
You can think of a Yocto Linux kernel as consisting of a baseline Linux
kernel with added features logically structured on top of the baseline.
The features are tagged and organized by way of a branching strategy
implemented by the Yocto Project team using the Source Code Manager
(SCM) Git.
.. note::
- Git is the obvious SCM for meeting the Yocto Linux kernel
organizational and structural goals described in this section. Not
only is Git the SCM for Linux kernel development in ``kernel.org``
but, Git continues to grow in popularity and supports many
different work flows, front-ends and management techniques.
- You can find documentation on Git at https://git-scm.com/doc. You can
also get an introduction to Git as it applies to the Yocto Project in the
":ref:`overview-manual/development-environment:git`" section in the Yocto Project
Overview and Concepts Manual. The latter reference provides an
overview of Git and presents a minimal set of Git commands that
allows you to be functional using Git. You can use as much, or as
little, of what Git has to offer to accomplish what you need for
your project. You do not have to be a "Git Expert" in order to use
it with the Yocto Project.
Using Git's tagging and branching features, the Yocto Project team
creates kernel branches at points where functionality is no longer
shared and thus, needs to be isolated. For example, board-specific
incompatibilities would require different functionality and would
require a branch to separate the features. Likewise, for specific kernel
features, the same branching strategy is used.
This "tree-like" architecture results in a structure that has features
organized to be specific for particular functionality, single kernel
types, or a subset of kernel types. Thus, the user has the ability to
see the added features and the commits that make up those features. In
addition to being able to see added features, the user can also view the
history of what made up the baseline Linux kernel.
Another consequence of this strategy results in not having to store the
same feature twice internally in the tree. Rather, the kernel team
stores the unique differences required to apply the feature onto the
kernel type in question.
.. note::
The Yocto Project team strives to place features in the tree such
that features can be shared by all boards and kernel types where
possible. However, during development cycles or when large features
are merged, the team cannot always follow this practice. In those
cases, the team uses isolated branches to merge features.
BSP-specific code additions are handled in a similar manner to
kernel-specific additions. Some BSPs only make sense given certain
kernel types. So, for these types, the team creates branches off the end
of that kernel type for all of the BSPs that are supported on that
kernel type. From the perspective of the tools that create the BSP
branch, the BSP is really no different than a feature. Consequently, the
same branching strategy applies to BSPs as it does to kernel features.
So again, rather than store the BSP twice, the team only stores the
unique differences for the BSP across the supported multiple kernels.
While this strategy can result in a tree with a significant number of
branches, it is important to realize that from the developer's point of
view, there is a linear path that travels from the baseline
``kernel.org``, through a select group of features and ends with their
BSP-specific commits. In other words, the divisions of the kernel are
transparent and are not relevant to the developer on a day-to-day basis.
From the developer's perspective, this path is the development branch.
The developer does not need to be aware of the existence of
any other branches at all. Of course, it can make sense to have these
branches in the tree, should a person decide to explore them. For
example, a comparison between two BSPs at either the commit level or at
the line-by-line code ``diff`` level is now a trivial operation.
The following illustration shows the conceptual Yocto Linux kernel.
.. image:: figures/kernel-architecture-overview.png
:width: 100%
In the illustration, the "Kernel.org Branch Point" marks the specific
spot (or Linux kernel release) from which the Yocto Linux kernel is
created. From this point forward in the tree, features and differences
are organized and tagged.
The "Yocto Project Baseline Kernel" contains functionality that is
common to every kernel type and BSP that is organized further along in
the tree. Placing these common features in the tree this way means
features do not have to be duplicated along individual branches of the
tree structure.
From the "Yocto Project Baseline Kernel", branch points represent
specific functionality for individual Board Support Packages (BSPs) as
well as real-time kernels. The illustration represents this through
three BSP-specific branches and a real-time kernel branch. Each branch
represents some unique functionality for the BSP or for a real-time
Yocto Linux kernel.
In this example structure, the "Real-time (rt) Kernel" branch has common
features for all real-time Yocto Linux kernels and contains more
branches for individual BSP-specific real-time kernels. The illustration
shows three branches as an example. Each branch points the way to
specific, unique features for a respective real-time kernel as they
apply to a given BSP.
The resulting tree structure presents a clear path of markers (or
branches) to the developer that, for all practical purposes, is the
Yocto Linux kernel needed for any given set of requirements.
.. note::
Keep in mind the figure does not take into account all the supported
Yocto Linux kernels, but rather shows a single generic kernel just
for conceptual purposes. Also keep in mind that this structure
represents the
:ref:`overview-manual/development-environment:yocto project source repositories`
that are either pulled from during the build or established on the
host development system prior to the build by either cloning a
particular kernel's Git repository or by downloading and unpacking a
tarball.
Working with the kernel as a structured tree follows recognized
community best practices. In particular, the kernel as shipped with the
product, should be considered an "upstream source" and viewed as a
series of historical and documented modifications (commits). These
modifications represent the development and stabilization done by the
Yocto Project kernel development team.
Because commits only change at significant release points in the product
life cycle, developers can work on a branch created from the last
relevant commit in the shipped Yocto Project Linux kernel. As mentioned
previously, the structure is transparent to the developer because the
kernel tree is left in this state after cloning and building the kernel.
Kernel Build File Hierarchy
===========================
Upstream storage of all the available kernel source code is one thing,
while representing and using the code on your host development system is
another. Conceptually, you can think of the kernel source repositories
as all the source files necessary for all the supported Yocto Linux
kernels. As a developer, you are just interested in the source files for
the kernel on which you are working. And, furthermore, you need them
available on your host system.
Kernel source code is available on your host system several different
ways:
- *Files Accessed While using devtool:* ``devtool``, which is
available with the Yocto Project, is the preferred method by which to
modify the kernel. See the ":ref:`kernel-dev/intro:kernel modification workflow`" section.
- *Cloned Repository:* If you are working in the kernel all the time,
you probably would want to set up your own local Git repository of
the Yocto Linux kernel tree. For information on how to clone a Yocto
Linux kernel Git repository, see the
":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section.
- *Temporary Source Files from a Build:* If you just need to make some
patches to the kernel using a traditional BitBake workflow (i.e. not
using the ``devtool``), you can access temporary kernel source files
that were extracted and used during a kernel build.
The temporary kernel source files resulting from a build using BitBake
have a particular hierarchy. When you build the kernel on your
development system, all files needed for the build are taken from the
source repositories pointed to by the
:term:`SRC_URI` variable and gathered
in a temporary work area where they are subsequently used to create the
unique kernel. Thus, in a sense, the process constructs a local source
tree specific to your kernel from which to generate the new kernel
image.
The following figure shows the temporary file structure created on your
host system when you build the kernel using BitBake. This
:term:`Build Directory` contains all the source files used during the build.
.. image:: figures/kernel-overview-2-generic.png
:align: center
:width: 70%
Again, for additional information on the Yocto Project kernel's
architecture and its branching strategy, see the
":ref:`kernel-dev/concepts-appx:yocto linux kernel architecture and branching strategies`"
section. You can also reference the
":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
and
":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
sections for detailed example that modifies the kernel.
Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase
=======================================================================================
This section describes part of the kernel configuration audit phase that
most developers can ignore. For general information on kernel
configuration including ``menuconfig``, ``defconfig`` files, and
configuration fragments, see the
":ref:`kernel-dev/common:configuring the kernel`" section.
During this part of the audit phase, the contents of the final
``.config`` file are compared against the fragments specified by the
system. These fragments can be system fragments, distro fragments, or
user-specified configuration elements. Regardless of their origin, the
OpenEmbedded build system warns the user if a specific option is not
included in the final kernel configuration.
By default, in order to not overwhelm the user with configuration
warnings, the system only reports missing "hardware" options as they
could result in a boot failure or indicate that important hardware is
not available.
To determine whether or not a given option is "hardware" or
"non-hardware", the kernel Metadata in ``yocto-kernel-cache`` contains
files that classify individual or groups of options as either hardware
or non-hardware. To better show this, consider a situation where the
``yocto-kernel-cache`` contains the following files::
yocto-kernel-cache/features/drm-psb/hardware.cfg
yocto-kernel-cache/features/kgdb/hardware.cfg
yocto-kernel-cache/ktypes/base/hardware.cfg
yocto-kernel-cache/bsp/mti-malta32/hardware.cfg
yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg
yocto-kernel-cache/bsp/qemuarma9/hardware.cfg
yocto-kernel-cache/bsp/mti-malta64/hardware.cfg
yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
yocto-kernel-cache/bsp/common-pc/hardware.cfg
yocto-kernel-cache/bsp/common-pc-64/hardware.cfg
yocto-kernel-cache/features/rfkill/non-hardware.cfg
yocto-kernel-cache/ktypes/base/non-hardware.cfg
yocto-kernel-cache/features/aufs/non-hardware.kcf
yocto-kernel-cache/features/ocf/non-hardware.kcf
yocto-kernel-cache/ktypes/base/non-hardware.kcf
yocto-kernel-cache/ktypes/base/hardware.kcf
yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf
Here are explanations for the various files:
- ``hardware.kcf``: Specifies a list of kernel Kconfig files that
contain hardware options only.
- ``non-hardware.kcf``: Specifies a list of kernel Kconfig files that
contain non-hardware options only.
- ``hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options that
are hardware, regardless of whether or not they are within a Kconfig
file specified by a hardware or non-hardware Kconfig file (i.e.
``hardware.kcf`` or ``non-hardware.kcf``).
- ``non-hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options
that are not hardware, regardless of whether or not they are within a
Kconfig file specified by a hardware or non-hardware Kconfig file
(i.e. ``hardware.kcf`` or ``non-hardware.kcf``).
Here is a specific example using the
``kernel-cache/bsp/mti-malta32/hardware.cfg``::
CONFIG_SERIAL_8250
CONFIG_SERIAL_8250_CONSOLE
CONFIG_SERIAL_8250_NR_UARTS
CONFIG_SERIAL_8250_PCI
CONFIG_SERIAL_CORE
CONFIG_SERIAL_CORE_CONSOLE
CONFIG_VGA_ARB
The kernel configuration audit automatically detects
these files (hence the names must be exactly the ones discussed here),
and uses them as inputs when generating warnings about the final
``.config`` file.
A user-specified kernel Metadata repository, or recipe space feature,
can use these same files to classify options that are found within its
``.cfg`` files as hardware or non-hardware, to prevent the OpenEmbedded
build system from producing an error or warning when an option is not in
the final ``.config`` file.

76
sources/poky/documentation/kernel-dev/faq.rst Normal file
View File

@@ -0,0 +1,76 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
**********************
Kernel Development FAQ
**********************
Common Questions and Solutions
==============================
Here are some solutions for common questions.
How do I use my own Linux kernel ``.config`` file?
--------------------------------------------------
Refer to the
":ref:`kernel-dev/common:changing the configuration`"
section for information.
How do I create configuration fragments?
----------------------------------------
A: Refer to the
":ref:`kernel-dev/common:creating configuration fragments`"
section for information.
How do I use my own Linux kernel sources?
-----------------------------------------
Refer to the
":ref:`kernel-dev/common:working with your own sources`"
section for information.
How do I install/not-install the kernel image on the root filesystem?
---------------------------------------------------------------------
The kernel image (e.g. ``vmlinuz``) is provided by the
``kernel-image`` package. Image recipes depend on ``kernel-base``. To
specify whether or not the kernel image is installed in the generated
root filesystem, override ``RRECOMMENDS:${KERNEL_PACKAGE_NAME}-base`` to include or not
include "kernel-image". See the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section in the
Yocto Project Development Tasks Manual for information on how to use an
append file to override metadata.
How do I install a specific kernel module?
------------------------------------------
Linux kernel modules are packaged individually. To ensure a
specific kernel module is included in an image, include it in the
appropriate machine :term:`RRECOMMENDS` variable.
These other variables are useful for installing specific modules:
- :term:`MACHINE_ESSENTIAL_EXTRA_RDEPENDS`
- :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
- :term:`MACHINE_EXTRA_RDEPENDS`
- :term:`MACHINE_EXTRA_RRECOMMENDS`
For example, set the following in the ``qemux86.conf`` file to include
the ``ab123`` kernel modules with images built for the ``qemux86``
machine::
MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123"
For more information, see the
":ref:`kernel-dev/common:incorporating out-of-tree modules`" section.
How do I change the Linux kernel command line?
----------------------------------------------
The Linux kernel command line is
typically specified in the machine config using the :term:`APPEND` variable.
For example, you can add some helpful debug information doing the
following::
APPEND += "printk.time=y initcall_debug debug"

BIN
sources/poky/documentation/kernel-dev/figures/kernel-architecture-overview.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

BIN
sources/poky/documentation/kernel-dev/figures/kernel-dev-flow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
sources/poky/documentation/kernel-dev/figures/kernel-dev-title.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
sources/poky/documentation/kernel-dev/figures/kernel-overview-2-generic.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

20
sources/poky/documentation/kernel-dev/index.rst Normal file
View File

@@ -0,0 +1,20 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=============================================
Yocto Project Linux Kernel Development Manual
=============================================
|
.. toctree::
:caption: Table of Contents
:numbered:
intro
common
advanced
concepts-appx
maint-appx
faq
.. include:: /boilerplate.rst

178
sources/poky/documentation/kernel-dev/intro.rst Normal file
View File

@@ -0,0 +1,178 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
************
Introduction
************
Overview
========
Regardless of how you intend to make use of the Yocto Project, chances
are you will work with the Linux kernel. This manual describes how to
set up your build host to support kernel development, introduces the
kernel development process, provides background information on the Yocto
Linux kernel :term:`Metadata`, describes
common tasks you can perform using the kernel tools, shows you how to
use the kernel Metadata needed to work with the kernel inside the Yocto
Project, and provides insight into how the Yocto Project team develops
and maintains Yocto Linux kernel Git repositories and Metadata.
Each Yocto Project release has a set of Yocto Linux kernel recipes,
whose Git repositories you can view in the Yocto
:yocto_git:`Source Repositories <>` under the "Yocto Linux Kernel"
heading. New recipes for the release track the latest Linux kernel
upstream developments from https://www.kernel.org and introduce
newly-supported platforms. Previous recipes in the release are refreshed
and supported for at least one additional Yocto Project release. As they
align, these previous releases are updated to include the latest from
the Long Term Support Initiative (LTSI) project. You can learn more
about Yocto Linux kernels and LTSI in the
":ref:`kernel-dev/concepts-appx:yocto project kernel development and maintenance`" section.
Also included is a Yocto Linux kernel development recipe
(``linux-yocto-dev.bb``) should you want to work with the very latest in
upstream Yocto Linux kernel development and kernel Metadata development.
.. note::
For more on Yocto Linux kernels, see the
":ref:`kernel-dev/concepts-appx:yocto project kernel development and maintenance`"
section.
The Yocto Project also provides a powerful set of kernel tools for
managing Yocto Linux kernel sources and configuration data. You can use
these tools to make a single configuration change, apply multiple
patches, or work with your own kernel sources.
In particular, the kernel tools allow you to generate configuration
fragments that specify only what you must, and nothing more.
Configuration fragments only need to contain the highest level visible
``CONFIG`` options as presented by the Yocto Linux kernel ``menuconfig``
system. Contrast this against a complete Yocto Linux kernel ``.config``
file, which includes all the automatically selected ``CONFIG`` options.
This efficiency reduces your maintenance effort and allows you to
further separate your configuration in ways that make sense for your
project. A common split separates policy and hardware. For example, all
your kernels might support the ``proc`` and ``sys`` filesystems, but
only specific boards require sound, USB, or specific drivers. Specifying
these configurations individually allows you to aggregate them together
as needed, but maintains them in only one place. Similar logic applies
to separating source changes.
If you do not maintain your own kernel sources and need to make only
minimal changes to the sources, the released recipes provide a vetted
base upon which to layer your changes. Doing so allows you to benefit
from the continual kernel integration and testing performed during
development of the Yocto Project.
If, instead, you have a very specific Linux kernel source tree and are
unable to align with one of the official Yocto Linux kernel recipes,
you have a way to use the Yocto Project Linux kernel tools with your
own kernel sources.
The remainder of this manual provides instructions for completing
specific Linux kernel development tasks. These instructions assume you
are comfortable working with :oe_wiki:`BitBake </Bitbake>` recipes and basic
open-source development tools. Understanding these concepts will
facilitate the process of working with the kernel recipes. If you find
you need some additional background, please be sure to review and
understand the following documentation:
- :doc:`/brief-yoctoprojectqs/index` document.
- :doc:`/overview-manual/index`.
- :ref:`devtool
workflow <sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow>`
as described in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual.
- The ":ref:`dev-manual/layers:understanding and creating layers`"
section in the Yocto Project Development Tasks Manual.
- The ":ref:`kernel-dev/intro:kernel modification workflow`" section.
Kernel Modification Workflow
============================
Kernel modification involves changing the Yocto Project kernel, which
could involve changing configuration options as well as adding new
kernel recipes. Configuration changes can be added in the form of
configuration fragments, while recipe modification comes through the
kernel's ``recipes-kernel`` area in a kernel layer you create.
This section presents a high-level overview of the Yocto Project kernel
modification workflow. The illustration and accompanying list provide
general information and references for further information.
.. image:: figures/kernel-dev-flow.png
:width: 100%
#. *Set up Your Host Development System to Support Development Using the
Yocto Project*: See the ":doc:`/dev-manual/start`" section in
the Yocto Project Development Tasks Manual for options on how to get
a build host ready to use the Yocto Project.
#. *Set Up Your Host Development System for Kernel Development:* It is
recommended that you use ``devtool`` for kernel
development. Alternatively, you can use traditional kernel
development methods with the Yocto Project. Either way, there are
steps you need to take to get the development environment ready.
Using ``devtool`` requires that you have a clean build
of the image. For
more information, see the
":ref:`kernel-dev/common:getting ready to develop using ``devtool```"
section.
Using traditional kernel development requires that you have the
kernel source available in an isolated local Git repository. For more
information, see the
":ref:`kernel-dev/common:getting ready for traditional kernel development`"
section.
#. *Make Changes to the Kernel Source Code if applicable:* Modifying the
kernel does not always mean directly changing source files. However,
if you have to do this, you make the changes to the files in the
Yocto's :term:`Build Directory` if you are using ``devtool``. For more
information, see the
":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
section.
If you are using traditional kernel development, you edit the source
files in the kernel's local Git repository. For more information, see the
":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
section.
#. *Make Kernel Configuration Changes if Applicable:* If your situation
calls for changing the kernel's configuration, you can use
:ref:`menuconfig <kernel-dev/common:using \`\`menuconfig\`\`>`,
which allows you to
interactively develop and test the configuration changes you are
making to the kernel. Saving changes you make with ``menuconfig``
updates the kernel's ``.config`` file.
.. note::
Try to resist the temptation to directly edit an existing ``.config``
file, which is found in the :term:`Build Directory` among the source code
used for the build. Doing so, can produce unexpected results when
the OpenEmbedded build system regenerates the configuration file.
Once you are satisfied with the configuration changes made using
``menuconfig`` and you have saved them, you can directly compare the
resulting ``.config`` file against an existing original and gather
those changes into a
:ref:`configuration fragment file <kernel-dev/common:creating configuration fragments>` to be
referenced from within the kernel's ``.bbappend`` file.
Additionally, if you are working in a BSP layer and need to modify
the BSP's kernel's configuration, you can use ``menuconfig``.
#. *Rebuild the Kernel Image With Your Changes:* Rebuilding the kernel
image applies your changes. Depending on your target hardware, you
can verify your changes on actual hardware or perhaps QEMU.
The remainder of this developer's guide covers common tasks typically
used during kernel development, advanced Metadata usage, and Yocto Linux
kernel maintenance concepts.

233
sources/poky/documentation/kernel-dev/maint-appx.rst Normal file
View File

@@ -0,0 +1,233 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
******************
Kernel Maintenance
******************
Tree Construction
=================
This section describes construction of the Yocto Project kernel source
repositories as accomplished by the Yocto Project team to create Yocto
Linux kernel repositories. These kernel repositories are found under the
heading "Yocto Linux Kernel" at :yocto_git:`/` and
are shipped as part of a Yocto Project release. The team creates these
repositories by compiling and executing the set of feature descriptions
for every BSP and feature in the product. Those feature descriptions
list all necessary patches, configurations, branches, tags, and feature
divisions found in a Yocto Linux kernel. Thus, the Yocto Project Linux
kernel repository (or tree) and accompanying Metadata in the
``yocto-kernel-cache`` are built.
The existence of these repositories allow you to access and clone a
particular Yocto Project Linux kernel repository and use it to build
images based on their configurations and features.
You can find the files used to describe all the valid features and BSPs
in the Yocto Project Linux kernel in any clone of the Yocto Project
Linux kernel source repository and ``yocto-kernel-cache`` Git trees. For
example, the following commands clone the Yocto Project baseline Linux
kernel that branches off ``linux.org`` version 4.12 and the
``yocto-kernel-cache``, which contains stores of kernel Metadata::
$ git clone git://git.yoctoproject.org/linux-yocto-4.12
$ git clone git://git.yoctoproject.org/linux-kernel-cache
For more information on
how to set up a local Git repository of the Yocto Project Linux kernel
files, see the
":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section.
Once you have cloned the kernel Git repository and the cache of Metadata
on your local machine, you can discover the branches that are available
in the repository using the following Git command::
$ git branch -a
Checking out a branch allows you to work with a particular Yocto Linux
kernel. For example, the following commands check out the
"standard/beagleboard" branch of the Yocto Linux kernel repository and
the "yocto-4.12" branch of the ``yocto-kernel-cache`` repository::
$ cd ~/linux-yocto-4.12
$ git checkout -b my-kernel-4.12 remotes/origin/standard/beagleboard
$ cd ~/linux-kernel-cache
$ git checkout -b my-4.12-metadata remotes/origin/yocto-4.12
.. note::
Branches in the ``yocto-kernel-cache`` repository correspond to Yocto Linux
kernel versions (e.g. "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
Once you have checked out and switched to appropriate branches, you can
see a snapshot of all the kernel source files used to build that
particular Yocto Linux kernel for a particular board.
To see the features and configurations for a particular Yocto Linux
kernel, you need to examine the ``yocto-kernel-cache`` Git repository.
As mentioned, branches in the ``yocto-kernel-cache`` repository
correspond to Yocto Linux kernel versions (e.g. ``yocto-4.12``).
Branches contain descriptions in the form of ``.scc`` and ``.cfg``
files.
You should realize, however, that browsing your local
``yocto-kernel-cache`` repository for feature descriptions and patches
is not an effective way to determine what is in a particular kernel
branch. Instead, you should use Git directly to discover the changes in
a branch. Using Git is an efficient and flexible way to inspect changes
to the kernel.
.. note::
Ground up reconstruction of the complete kernel tree is an action
only taken by the Yocto Project team during an active development
cycle. When you create a clone of the kernel Git repository, you are
simply making it efficiently available for building and development.
The following steps describe what happens when the Yocto Project Team
constructs the Yocto Project kernel source Git repository (or tree)
found at :yocto_git:`/` given the introduction of a new
top-level kernel feature or BSP. The following actions effectively
provide the Metadata and create the tree that includes the new feature,
patch, or BSP:
#. *Pass Feature to the OpenEmbedded Build System:* A top-level kernel
feature is passed to the kernel build subsystem. Normally, this
feature is a BSP for a particular kernel type.
#. *Locate Feature:* The file that describes the top-level feature is
located by searching these system directories:
- The in-tree kernel-cache directories, which are located in the
:yocto_git:`yocto-kernel-cache </yocto-kernel-cache/tree/bsp>`
repository organized under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
- Areas pointed to by :term:`SRC_URI` statements found in kernel recipes.
For a typical build, the target of the search is a feature
description in an ``.scc`` file whose name follows this format (e.g.
``beaglebone-standard.scc`` and ``beaglebone-preempt-rt.scc``)::
bsp_root_name-kernel_type.scc
#. *Expand Feature:* Once located, the feature description is either
expanded into a simple script of actions, or into an existing
equivalent script that is already part of the shipped kernel.
#. *Append Extra Features:* Extra features are appended to the top-level
feature description. These features can come from the
:term:`KERNEL_FEATURES`
variable in recipes.
#. *Locate, Expand, and Append Each Feature:* Each extra feature is
located, expanded and appended to the script as described in step
three.
#. *Execute the Script:* The script is executed to produce files
``.scc`` and ``.cfg`` files in appropriate directories of the
``yocto-kernel-cache`` repository. These files are descriptions of
all the branches, tags, patches and configurations that need to be
applied to the base Git repository to completely create the source
(build) branch for the new BSP or feature.
#. *Clone Base Repository:* The base repository is cloned, and the
actions listed in the ``yocto-kernel-cache`` directories are applied
to the tree.
#. *Perform Cleanup:* The Git repositories are left with the desired
branches checked out and any required branching, patching and tagging
has been performed.
The kernel tree and cache are ready for developer consumption to be
locally cloned, configured, and built into a Yocto Project kernel
specific to some target hardware.
.. note::
- The generated ``yocto-kernel-cache`` repository adds to the kernel
as shipped with the Yocto Project release. Any add-ons and
configuration data are applied to the end of an existing branch.
The full repository generation that is found in the official Yocto
Project kernel repositories at :yocto_git:`/` is the
combination of all supported boards and configurations.
- The technique the Yocto Project team uses is flexible and allows
for seamless blending of an immutable history with additional
patches specific to a deployment. Any additions to the kernel
become an integrated part of the branches.
- The full kernel tree that you see on :yocto_git:`/` is
generated through repeating the above steps for all valid BSPs.
The end result is a branched, clean history tree that makes up the
kernel for a given release. You can see the script (``kgit-scc``)
responsible for this in the
:yocto_git:`yocto-kernel-tools </yocto-kernel-tools/tree/tools>`
repository.
- The steps used to construct the full kernel tree are the same
steps that BitBake uses when it builds a kernel image.
Build Strategy
==============
Once you have cloned a Yocto Linux kernel repository and the cache
repository (``yocto-kernel-cache``) onto your development system, you
can consider the compilation phase of kernel development, which is
building a kernel image. Some prerequisites are validated by
the build process before compilation starts:
- The :term:`SRC_URI` points to the
kernel Git repository.
- A BSP build branch with Metadata exists in the ``yocto-kernel-cache``
repository. The branch is based on the Yocto Linux kernel version and
has configurations and features grouped under the
``yocto-kernel-cache/bsp`` directory. For example, features and
configurations for the BeagleBone Board assuming a
``linux-yocto_4.12`` kernel reside in the following area of the
``yocto-kernel-cache`` repository: yocto-kernel-cache/bsp/beaglebone
.. note::
In the previous example, the "yocto-4.12" branch is checked out in
the ``yocto-kernel-cache`` repository.
The OpenEmbedded build system makes sure these conditions are satisfied before
attempting compilation. Other means, however, do exist, such as
bootstrapping a BSP.
Before building a kernel, the build process verifies the tree and
configures the kernel by processing all of the configuration "fragments"
specified by feature descriptions in the ``.scc`` files. As the features
are compiled, associated kernel configuration fragments are noted and
recorded in the series of directories in their compilation order. The
fragments are migrated, pre-processed and passed to the Linux Kernel
Configuration subsystem (``lkc``) as raw input in the form of a
``.config`` file. The ``lkc`` uses its own internal dependency
constraints to do the final processing of that information and generates
the final ``.config`` file that is used during compilation.
Using the board's architecture and other relevant values from the
board's template, kernel compilation is started and a kernel image is
produced.
The other thing that you notice once you configure a kernel is that the
build process generates a build tree that is separate from your kernel's
local Git source repository tree. This build tree has a name that uses
the following form, where ``${MACHINE}`` is the metadata name of the
machine (BSP) and "kernel_type" is one of the Yocto Project supported
kernel types (e.g. "standard")::
linux-${MACHINE}-kernel_type-build
The existing support in the ``kernel.org`` tree achieves this default
functionality.
This behavior means that all the generated files for a particular
machine or BSP are now in the build tree directory. The files include
the final ``.config`` file, all the ``.o`` files, the ``.a`` files, and
so forth. Since each machine or BSP has its own separate
:term:`Build Directory` in its own separate branch of the Git repository,
you can easily switch between different builds.

40
sources/poky/documentation/migration-guides/index.rst Normal file
View File

@@ -0,0 +1,40 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
====================
Release Information
====================
|
Each document in this chapter provides release notes and information about how
to move to one release of the Yocto Project from the previous one.
.. toctree::
migration-general
release-5.0
release-4.3
release-4.2
release-4.1
release-4.0
release-3.4
migration-3.3
migration-3.2
migration-3.1
migration-3.0
migration-2.7
migration-2.6
migration-2.5
migration-2.4
migration-2.3
migration-2.2
migration-2.1
migration-2.0
migration-1.8
migration-1.7
migration-1.6
migration-1.5
migration-1.4
migration-1.3
.. include:: /boilerplate.rst

193
sources/poky/documentation/migration-guides/migration-1.3.rst Normal file
View File

@@ -0,0 +1,193 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.3 (danny)
===================
This section provides migration information for moving to the Yocto
Project 1.3 Release (codename "danny") from the prior release.
.. _1.3-local-configuration:
Local Configuration
-------------------
Differences include changes for
:term:`SSTATE_MIRRORS` and ``bblayers.conf``.
.. _migration-1.3-sstate-mirrors:
SSTATE_MIRRORS
~~~~~~~~~~~~~~
The shared state cache (sstate-cache), as pointed to by
:term:`SSTATE_DIR`, by default now has two-character
subdirectories to prevent issues arising from too many files in the same
directory. Also, native sstate-cache packages, which are built to run on
the host system, will go into a subdirectory named using the distro ID
string. If you copy the newly structured sstate-cache to a mirror
location (either local or remote) and then point to it in
:term:`SSTATE_MIRRORS`, you need to append "PATH"
to the end of the mirror URL so that the path used by BitBake before the
mirror substitution is appended to the path used to access the mirror.
Here is an example::
SSTATE_MIRRORS = "file://.* http://someserver.tld/share/sstate/PATH"
.. _migration-1.3-bblayers-conf:
bblayers.conf
~~~~~~~~~~~~~
The ``meta-yocto`` layer consists of two parts that correspond to the
Poky reference distribution and the reference hardware Board Support
Packages (BSPs), respectively: ``meta-yocto`` and ``meta-yocto-bsp``.
When running BitBake for the first time after upgrading, your
``conf/bblayers.conf`` file will be updated to handle this change and
you will be asked to re-run or restart for the changes to take effect.
.. _1.3-recipes:
Recipes
-------
Differences include changes for the following:
.. _migration-1.3-python-function-whitespace:
Python Function Whitespace
~~~~~~~~~~~~~~~~~~~~~~~~~~
All Python functions must now use four spaces for indentation.
Previously, an inconsistent mix of spaces and tabs existed, which made
extending these functions using ``_append`` or ``_prepend`` complicated
given that Python treats whitespace as syntactically significant. If you
are defining or extending any Python functions (e.g.
``populate_packages``, :ref:`ref-tasks-unpack`, :ref:`ref-tasks-patch` and so forth) in
custom recipes or classes, you need to ensure you are using consistent
four-space indentation.
.. _migration-1.3-proto=-in-src-uri:
proto= in SRC_URI
~~~~~~~~~~~~~~~~~
Any use of ``proto=`` in :term:`SRC_URI` needs to be
changed to ``protocol=``. In particular, this applies to the following
URIs:
- ``svn://``
- ``bzr://``
- ``hg://``
- ``osc://``
Other URIs were already using ``protocol=``. This change improves
consistency.
.. _migration-1.3-nativesdk:
nativesdk
~~~~~~~~~
The suffix ``nativesdk`` is now implemented as a prefix, which simplifies a lot
of the packaging code for :ref:`ref-classes-nativesdk` recipes. All custom
:ref:`ref-classes-nativesdk` recipes, which are relocatable packages that are
native to :term:`SDK_ARCH`, and any references need to be updated to use
``nativesdk-*`` instead of ``*-nativesdk``.
.. _migration-1.3-task-recipes:
Task Recipes
~~~~~~~~~~~~
"Task" recipes are now known as "Package groups" and have been renamed
from ``task-*.bb`` to ``packagegroup-*.bb``. Existing references to the
previous ``task-*`` names should work in most cases as there is an
automatic upgrade path for most packages. However, you should update
references in your own recipes and configurations as they could be
removed in future releases. You should also rename any custom ``task-*``
recipes to ``packagegroup-*``, and change them to inherit
:ref:`ref-classes-packagegroup` instead of ``task``, as well
as taking the opportunity to remove anything now handled by
:ref:`ref-classes-packagegroup`, such as providing ``-dev`` and ``-dbg``
packages, setting :term:`LIC_FILES_CHKSUM`, and so forth. See the
:ref:`ref-classes-packagegroup` section for further details.
.. _migration-1.3-image-features:
IMAGE_FEATURES
~~~~~~~~~~~~~~
Image recipes that previously included ``apps-console-core`` in
:term:`IMAGE_FEATURES` should now include ``splash``
instead to enable the boot-up splash screen. Retaining
``apps-console-core`` will still include the splash screen but generates a
warning. The ``apps-x11-core`` and ``apps-x11-games`` :term:`IMAGE_FEATURES`
features have been removed.
.. _migration-1.3-removed-recipes:
Removed Recipes
~~~~~~~~~~~~~~~
The following recipes have been removed. For most of them, it is
unlikely that you would have any references to them in your own
:term:`Metadata`. However, you should check your metadata
against this list to be sure:
- ``libx11-trim``: Replaced by ``libx11``, which has a negligible
size difference with modern Xorg.
- ``xserver-xorg-lite``: Use ``xserver-xorg``, which has a negligible
size difference when DRI and GLX modules are not installed.
- ``xserver-kdrive``: Effectively unmaintained for many years.
- ``mesa-xlib``: No longer serves any purpose.
- ``galago``: Replaced by telepathy.
- ``gail``: Functionality was integrated into GTK+ 2.13.
- ``eggdbus``: No longer needed.
- ``gcc-*-intermediate``: The build has been restructured to avoid
the need for this step.
- ``libgsmd``: Unmaintained for many years. Functionality now
provided by ``ofono`` instead.
- *contacts, dates, tasks, eds-tools*: Largely unmaintained PIM
application suite. It has been moved to ``meta-gnome`` in
``meta-openembedded``.
In addition to the previously listed changes, the ``meta-demoapps``
directory has also been removed because the recipes in it were not being
maintained and many had become obsolete or broken. Additionally, these
recipes were not parsed in the default configuration. Many of these
recipes are already provided in an updated and maintained form within
the OpenEmbedded community layers such as ``meta-oe`` and
``meta-gnome``. For the remainder, you can now find them in the
``meta-extras`` repository, which is in the
:yocto_git:`Source Repositories <>` at
:yocto_git:`/meta-extras/`.
.. _1.3-linux-kernel-naming:
Linux Kernel Naming
-------------------
The naming scheme for kernel output binaries has been changed to now
include :term:`PE` as part of the filename::
KERNEL_IMAGE_BASE_NAME ?= "${KERNEL_IMAGETYPE}-${PE}-${PV}-${PR}-${MACHINE}-${DATETIME}"
Because the :term:`PE` variable is not set by default, these binary files
could result with names that include two dash characters. Here is an
example::
bzImage--3.10.9+git0+cd502a8814_7144bcc4b8-r0-qemux86-64-20130830085431.bin

237
sources/poky/documentation/migration-guides/migration-1.4.rst Normal file
View File

@@ -0,0 +1,237 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.4 (dylan)
===================
This section provides migration information for moving to the Yocto
Project 1.4 Release (codename "dylan") from the prior release.
.. _migration-1.4-bitbake:
BitBake
-------
Differences include the following:
- *Comment Continuation:* If a comment ends with a line continuation
(\\) character, then the next line must also be a comment. Any
instance where this is not the case, now triggers a warning. You must
either remove the continuation character, or be sure the next line is
a comment.
- *Package Name Overrides:* The runtime package specific variables
:term:`RDEPENDS`,
:term:`RRECOMMENDS`,
:term:`RSUGGESTS`,
:term:`RPROVIDES`,
:term:`RCONFLICTS`,
:term:`RREPLACES`, :term:`FILES`,
:term:`ALLOW_EMPTY`, and the pre, post, install,
and uninstall script functions ``pkg_preinst``, ``pkg_postinst``,
``pkg_prerm``, and ``pkg_postrm`` should always have a package name
override. For example, use ``RDEPENDS_${PN}`` for the main package
instead of :term:`RDEPENDS`. BitBake uses more strict checks when it
parses recipes.
.. _migration-1.4-build-behavior:
Build Behavior
--------------
Differences include the following:
- *Shared State Code:* The shared state code has been optimized to
avoid running unnecessary tasks. For example, the following no longer
populates the target sysroot since that is not necessary::
$ bitbake -c rootfs some-image
Instead, the system just needs to extract the
output package contents, re-create the packages, and construct the
root filesystem. This change is unlikely to cause any problems unless
you have missing declared dependencies.
- *Scanning Directory Names:* When scanning for files in
:term:`SRC_URI`, the build system now uses
:term:`FILESOVERRIDES` instead of
:term:`OVERRIDES` for the directory names. In
general, the values previously in :term:`OVERRIDES` are now in
:term:`FILESOVERRIDES` as well. However, if you relied upon an additional
value you previously added to :term:`OVERRIDES`, you might now need to
add it to :term:`FILESOVERRIDES` unless you are already adding it through
the :term:`MACHINEOVERRIDES` or
:term:`DISTROOVERRIDES` variables, as
appropriate. For more related changes, see the
":ref:`migration-guides/migration-1.4:variables`" section.
.. _migration-1.4-proxies-and-fetching-source:
Proxies and Fetching Source
---------------------------
A new ``oe-git-proxy`` script has been added to replace previous methods
of handling proxies and fetching source from Git. See the
``meta-yocto/conf/site.conf.sample`` file for information on how to use
this script.
.. _migration-1.4-custom-interfaces-file-netbase-change:
Custom Interfaces File (netbase change)
---------------------------------------
If you have created your own custom ``etc/network/interfaces`` file by
creating an append file for the ``netbase`` recipe, you now need to
create an append file for the ``init-ifupdown`` recipe instead, which
you can find in the :term:`Source Directory` at
``meta/recipes-core/init-ifupdown``. For information on how to use
append files, see the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.4-remote-debugging:
Remote Debugging
----------------
Support for remote debugging with the Eclipse IDE is now separated into
an image feature (``eclipse-debug``) that corresponds to the
``packagegroup-core-eclipse-debug`` package group. Previously, the
debugging feature was included through the ``tools-debug`` image
feature, which corresponds to the ``packagegroup-core-tools-debug``
package group.
.. _migration-1.4-variables:
Variables
---------
The following variables have changed:
- :term:`SANITY_TESTED_DISTROS`: This variable now uses a distribution
ID, which is composed of the host distributor ID followed by the
release. Previously,
:term:`SANITY_TESTED_DISTROS` was
composed of the description field. For example, "Ubuntu 12.10"
becomes "Ubuntu-12.10". You do not need to worry about this change if
you are not specifically setting this variable, or if you are
specifically setting it to "".
- :term:`SRC_URI`: The ``${``\ :term:`PN`\ ``}``,
``${``\ :term:`PF`\ ``}``,
``${``\ :term:`P`\ ``}``, and ``FILE_DIRNAME`` directories
have been dropped from the default value of the
:term:`FILESPATH` variable, which is used as the
search path for finding files referred to in
:term:`SRC_URI`. If you have a recipe that relied upon
these directories, which would be unusual, then you will need to add
the appropriate paths within the recipe or, alternatively, rearrange
the files. The most common locations are still covered by ``${``\ :term:`BP`\ ``}``,
``${``\ :term:`BPN`\ ``}``, and "files", which all remain in the default value of
:term:`FILESPATH`.
.. _migration-target-package-management-with-rpm:
Target Package Management with RPM
----------------------------------
If runtime package management is enabled and the RPM backend is
selected, Smart is now installed for package download, dependency
resolution, and upgrades instead of Zypper. For more information on how
to use Smart, run the following command on the target::
smart --help
.. _migration-1.4-recipes-moved:
Recipes Moved
-------------
The following recipes were moved from their previous locations because
they are no longer used by anything in the OpenEmbedded-Core:
- ``clutter-box2d``: Now resides in the ``meta-oe`` layer.
- ``evolution-data-server``: Now resides in the ``meta-gnome`` layer.
- ``gthumb``: Now resides in the ``meta-gnome`` layer.
- ``gtkhtml2``: Now resides in the ``meta-oe`` layer.
- ``gupnp``: Now resides in the ``meta-multimedia`` layer.
- ``gypsy``: Now resides in the ``meta-oe`` layer.
- ``libcanberra``: Now resides in the ``meta-gnome`` layer.
- ``libgdata``: Now resides in the ``meta-gnome`` layer.
- ``libmusicbrainz``: Now resides in the ``meta-multimedia`` layer.
- ``metacity``: Now resides in the ``meta-gnome`` layer.
- ``polkit``: Now resides in the ``meta-oe`` layer.
- ``zeroconf``: Now resides in the ``meta-networking`` layer.
.. _migration-1.4-removals-and-renames:
Removals and Renames
--------------------
The following list shows what has been removed or renamed:
- ``evieext``: Removed because it has been removed from ``xserver``
since 2008.
- *Gtk+ DirectFB:* Removed support because upstream Gtk+ no longer
supports it as of version 2.18.
- ``libxfontcache / xfontcacheproto``: Removed because they were
removed from the Xorg server in 2008.
- ``libxp / libxprintapputil / libxprintutil / printproto``: Removed
because the XPrint server was removed from Xorg in 2008.
- ``libxtrap / xtrapproto``: Removed because their functionality was
broken upstream.
- *linux-yocto 3.0 kernel:* Removed with linux-yocto 3.8 kernel being
added. The linux-yocto 3.2 and linux-yocto 3.4 kernels remain as part
of the release.
- ``lsbsetup``: Removed with functionality now provided by
``lsbtest``.
- ``matchbox-stroke``: Removed because it was never more than a
proof-of-concept.
- ``matchbox-wm-2 / matchbox-theme-sato-2``: Removed because they are
not maintained. However, ``matchbox-wm`` and ``matchbox-theme-sato``
are still provided.
- ``mesa-dri``: Renamed to ``mesa``.
- ``mesa-xlib``: Removed because it was no longer useful.
- ``mutter``: Removed because nothing ever uses it and the recipe is
very old.
- ``orinoco-conf``: Removed because it has become obsolete.
- ``update-modules``: Removed because it is no longer used. The
kernel module ``postinstall`` and ``postrm`` scripts can now do the
same task without the use of this script.
- ``web``: Removed because it is not maintained. Superseded by
``web-webkit``.
- ``xf86bigfontproto``: Removed because upstream it has been disabled
by default since 2007. Nothing uses ``xf86bigfontproto``.
- ``xf86rushproto``: Removed because its dependency in ``xserver``
was spurious and it was removed in 2005.
- ``zypper / libzypp / sat-solver``: Removed and been functionally
replaced with Smart (``python-smartpm``) when RPM packaging is used
and package management is enabled on the target.

355
sources/poky/documentation/migration-guides/migration-1.5.rst Normal file
View File

@@ -0,0 +1,355 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.5 (dora)
==================
This section provides migration information for moving to the Yocto
Project 1.5 Release (codename "dora") from the prior release.
.. _migration-1.5-host-dependency-changes:
Host Dependency Changes
-----------------------
The OpenEmbedded build system now has some additional requirements on
the host system:
- Python 2.7.3+
- Tar 1.24+
- Git 1.7.8+
- Patched version of Make if you are using 3.82. Most distributions
that provide Make 3.82 use the patched version.
If the Linux distribution you are using on your build host does not
provide packages for these, you can install and use the Buildtools
tarball, which provides an SDK-like environment containing them.
For more information on this requirement, see the
":ref:`system-requirements-buildtools`" section.
.. _migration-1.5-atom-pc-bsp:
``atom-pc`` Board Support Package (BSP)
---------------------------------------
The ``atom-pc`` hardware reference BSP has been replaced by a
``genericx86`` BSP. This BSP is not necessarily guaranteed to work on
all x86 hardware, but it will run on a wider range of systems than the
``atom-pc`` did.
.. note::
Additionally, a ``genericx86-64`` BSP has been added for 64-bit Atom
systems.
.. _migration-1.5-bitbake:
BitBake
-------
The following changes have been made that relate to BitBake:
- BitBake now supports a ``_remove`` operator. The addition of this
operator means you will have to rename any items in recipe space
(functions, variables) whose names currently contain ``_remove_`` or
end with ``_remove`` to avoid unexpected behavior.
- BitBake's global method pool has been removed. This method is not
particularly useful and led to clashes between recipes containing
functions that had the same name.
- The "none" server backend has been removed. The "process" server
backend has been serving well as the default for a long time now.
- The ``bitbake-runtask`` script has been removed.
- ``${``\ :term:`P`\ ``}`` and
``${``\ :term:`PF`\ ``}`` are no longer added to
:term:`PROVIDES` by default in ``bitbake.conf``.
These version-specific :term:`PROVIDES` items were seldom used.
Attempting to use them could result in two versions being built
simultaneously rather than just one version due to the way BitBake
resolves dependencies.
.. _migration-1.5-qa-warnings:
QA Warnings
-----------
The following changes have been made to the package QA checks:
- If you have customized :term:`ERROR_QA` or
:term:`WARN_QA` values in your configuration, check
that they contain all of the issues that you wish to be reported.
Previous Yocto Project versions contained a bug that meant that any
item not mentioned in :term:`ERROR_QA` or :term:`WARN_QA` would be treated as
a warning. Consequently, several important items were not already in
the default value of :term:`WARN_QA`. All of the possible QA checks are
now documented in the ":ref:`ref-classes-insane`" section.
- An additional QA check has been added to check if
``/usr/share/info/dir`` is being installed. Your recipe should delete
this file within :ref:`ref-tasks-install` if "make
install" is installing it.
- If you are using the :ref:`ref-classes-buildhistory` class, the check for the
package version going backwards is now controlled using a standard QA check.
Thus, if you have customized your :term:`ERROR_QA` or :term:`WARN_QA` values
and still wish to have this check performed, you should add
"version-going-backwards" to your value for one or the other
variables depending on how you wish it to be handled. See the
documented QA checks in the ":ref:`ref-classes-insane`" section.
.. _migration-1.5-directory-layout-changes:
Directory Layout Changes
------------------------
The following directory changes exist:
- Output SDK installer files are now named to include the image name
and tuning architecture through the :term:`SDK_NAME`
variable.
- Images and related files are now installed into a directory that is
specific to the machine, instead of a parent directory containing
output files for multiple machines. The
:term:`DEPLOY_DIR_IMAGE` variable continues
to point to the directory containing images for the current
:term:`MACHINE` and should be used anywhere there is a
need to refer to this directory. The ``runqemu`` script now uses this
variable to find images and kernel binaries and will use BitBake to
determine the directory. Alternatively, you can set the
:term:`DEPLOY_DIR_IMAGE` variable in the external environment.
- When buildhistory is enabled, its output is now written under the
:term:`Build Directory` rather than :term:`TMPDIR`. Doing so makes
it easier to delete :term:`TMPDIR` and preserve the build history.
Additionally, data for produced SDKs is now split by :term:`IMAGE_NAME`.
- When :ref:`ref-classes-buildhistory` is enabled, its output
is now written under the :term:`Build Directory` rather than :term:`TMPDIR`.
Doing so makes it easier to delete :term:`TMPDIR` and preserve the build
history. Additionally, data for produced SDKs is now split by :term:`IMAGE_NAME`.
- The ``pkgdata`` directory produced as part of the packaging process
has been collapsed into a single machine-specific directory. This
directory is located under ``sysroots`` and uses a machine-specific
name (i.e. ``tmp/sysroots/machine/pkgdata``).
.. _migration-1.5-shortened-git-srcrev-values:
Shortened Git ``SRCREV`` Values
-------------------------------
BitBake will now shorten revisions from Git repositories from the normal
40 characters down to 10 characters within :term:`SRCPV`
for improved usability in path and filenames. This change should be
safe within contexts where these revisions are used because the chances
of spatially close collisions is very low. Distant collisions are not a
major issue in the way the values are used.
.. _migration-1.5-image-features:
``IMAGE_FEATURES``
------------------
The following changes have been made that relate to
:term:`IMAGE_FEATURES`:
- The value of :term:`IMAGE_FEATURES` is now validated to ensure invalid
feature items are not added. Some users mistakenly add package names
to this variable instead of using
:term:`IMAGE_INSTALL` in order to have the
package added to the image, which does not work. This change is
intended to catch those kinds of situations. Valid :term:`IMAGE_FEATURES`
are drawn from ``PACKAGE_GROUP`` definitions,
:term:`COMPLEMENTARY_GLOB` and a new
"validitems" varflag on :term:`IMAGE_FEATURES`. The "validitems" varflag
change allows additional features to be added if they are not
provided using the previous two mechanisms.
- The previously deprecated "apps-console-core" :term:`IMAGE_FEATURES` item
is no longer supported. Add "splash" to :term:`IMAGE_FEATURES` if you
wish to have the splash screen enabled, since this is all that
apps-console-core was doing.
.. _migration-1.5-run:
``/run``
--------
The ``/run`` directory from the Filesystem Hierarchy Standard 3.0 has
been introduced. You can find some of the implications for this change
:oe_git:`here </openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873>`.
The change also means that recipes that install files to ``/var/run``
must be changed. You can find a guide on how to make these changes
`here <https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg31649.html>`__.
.. _migration-1.5-removal-of-package-manager-database-within-image-recipes:
Removal of Package Manager Database Within Image Recipes
--------------------------------------------------------
The image ``core-image-minimal`` no longer adds
``remove_packaging_data_files`` to
:term:`ROOTFS_POSTPROCESS_COMMAND`.
This addition is now handled automatically when "package-management" is
not in :term:`IMAGE_FEATURES`. If you have custom
image recipes that make this addition, you should remove the lines, as
they are not needed and might interfere with correct operation of
postinstall scripts.
.. _migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time:
Images Now Rebuild Only on Changes Instead of Every Time
--------------------------------------------------------
The :ref:`ref-tasks-rootfs` and other related image
construction tasks are no longer marked as "nostamp". Consequently, they
will only be re-executed when their inputs have changed. Previous
versions of the OpenEmbedded build system always rebuilt the image when
requested rather when necessary.
.. _migration-1.5-task-recipes:
Task Recipes
------------
The previously deprecated ``task.bbclass`` has now been dropped. For
recipes that previously inherited from this class, you should rename
them from ``task-*`` to ``packagegroup-*`` and inherit
:ref:`ref-classes-packagegroup` instead.
For more information, see the ":ref:`ref-classes-packagegroup`" section.
.. _migration-1.5-busybox:
BusyBox
-------
By default, we now split BusyBox into two binaries: one that is suid
root for those components that need it, and another for the rest of the
components. Splitting BusyBox allows for optimization that eliminates
the ``tinylogin`` recipe as recommended by upstream. You can disable
this split by setting
:term:`BUSYBOX_SPLIT_SUID` to "0".
.. _migration-1.5-automated-image-testing:
Automated Image Testing
-----------------------
A new automated image testing framework has been added through the
:ref:`ref-classes-testimage` classes. This
framework replaces the older ``imagetest-qemu`` framework.
You can learn more about performing automated image tests in the
":ref:`test-manual/runtime-testing:performing automated runtime testing`"
section in the Yocto Project Test Environment Manual.
.. _migration-1.5-build-history:
Build History
-------------
The changes to Build History are:
- Installed package sizes: ``installed-package-sizes.txt`` for an image
now records the size of the files installed by each package instead
of the size of each compressed package archive file.
- The dependency graphs (``depends*.dot``) now use the actual package
names instead of replacing dashes, dots and plus signs with
underscores.
- The ``buildhistory-diff`` and ``buildhistory-collect-srcrevs``
utilities have improved command-line handling. Use the ``--help``
option for each utility for more information on the new syntax.
For more information on Build History, see the
":ref:`dev-manual/build-quality:maintaining build output quality`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.5-udev:
``udev``
--------
The changes to ``udev`` are:
- ``udev`` no longer brings in ``udev-extraconf`` automatically through
:term:`RRECOMMENDS`, since this was originally
intended to be optional. If you need the extra rules, then add
``udev-extraconf`` to your image.
- ``udev`` no longer brings in ``pciutils-ids`` or ``usbutils-ids``
through :term:`RRECOMMENDS`. These are not needed by ``udev`` itself and
removing them saves around 350KB.
.. _migration-1.5-removed-renamed-recipes:
Removed and Renamed Recipes
---------------------------
- The ``linux-yocto`` 3.2 kernel has been removed.
- ``libtool-nativesdk`` has been renamed to ``nativesdk-libtool``.
- ``tinylogin`` has been removed. It has been replaced by a suid
portion of Busybox. See the ":ref:`migration-1.5-busybox`"
section for more information.
- ``external-python-tarball`` has been renamed to
``buildtools-tarball``.
- ``web-webkit`` has been removed. It has been functionally replaced by
``midori``.
- ``imake`` has been removed. It is no longer needed by any other
recipe.
- ``transfig-native`` has been removed. It is no longer needed by any
other recipe.
- ``anjuta-remote-run`` has been removed. Anjuta IDE integration has
not been officially supported for several releases.
.. _migration-1.5-other-changes:
Other Changes
-------------
Here is a list of short entries describing other changes:
- ``run-postinsts``: Make this generic.
- ``base-files``: Remove the unnecessary ``media/``\ xxx directories.
- ``alsa-state``: Provide an empty ``asound.conf`` by default.
- ``classes/image``: Ensure
:term:`BAD_RECOMMENDATIONS` supports
pre-renamed package names.
- ``classes/rootfs_rpm``: Implement :term:`BAD_RECOMMENDATIONS` for RPM.
- ``systemd``: Remove ``systemd_unitdir`` if ``systemd`` is not in
:term:`DISTRO_FEATURES`.
- ``systemd``: Remove ``init.d`` dir if ``systemd`` unit file is
present and ``sysvinit`` is not a distro feature.
- ``libpam``: Deny all services for the ``OTHER`` entries.
- :ref:`ref-classes-image`: Move ``runtime_mapping_rename`` to avoid conflict
with ``multilib``. See :yocto_bugs:`YOCTO #4993 </show_bug.cgi?id=4993>`
in Bugzilla for more information.
- ``linux-dtb``: Use kernel build system to generate the ``dtb`` files.
- ``kern-tools``: Switch from guilt to new ``kgit-s2q`` tool.

414
sources/poky/documentation/migration-guides/migration-1.6.rst Normal file
View File

@@ -0,0 +1,414 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.6 (daisy)
===================
This section provides migration information for moving to the Yocto
Project 1.6 Release (codename "daisy") from the prior release.
.. _migration-1.6-archiver-class:
``archiver`` Class
------------------
The :ref:`ref-classes-archiver` class has been rewritten and its configuration
has been simplified. For more details on the source archiver, see the
":ref:`dev-manual/licenses:maintaining open source license compliance during your product's lifecycle`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.6-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have been made:
- The ``binutils`` recipe no longer produces a ``binutils-symlinks``
package. ``update-alternatives`` is now used to handle the preferred
``binutils`` variant on the target instead.
- The tc (traffic control) utilities have been split out of the main
``iproute2`` package and put into the ``iproute2-tc`` package.
- The ``gtk-engines`` schemas have been moved to a dedicated
``gtk-engines-schemas`` package.
- The ``armv7a`` with thumb package architecture suffix has changed.
The suffix for these packages with the thumb optimization enabled is
"t2" as it should be. Use of this suffix was not the case in the 1.5
release. Architecture names will change within package feeds as a
result.
.. _migration-1.6-bitbake:
BitBake
-------
The following changes have been made to :term:`BitBake`.
.. _migration-1.6-matching-branch-requirement-for-git-fetching:
Matching Branch Requirement for Git Fetching
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When fetching source from a Git repository using
:term:`SRC_URI`, BitBake will now validate the
:term:`SRCREV` value against the branch. You can specify
the branch using the following form::
SRC_URI = "git://server.name/repository;branch=branchname"
If you do not specify a branch, BitBake looks in the default "master" branch.
Alternatively, if you need to bypass this check (e.g. if you are
fetching a revision corresponding to a tag that is not on any branch),
you can add ";nobranch=1" to the end of the URL within :term:`SRC_URI`.
.. _migration-1.6-bitbake-deps:
Python Definition substitutions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake had some previously deprecated Python definitions within its
``bb`` module removed. You should use their sub-module counterparts
instead:
- ``bb.MalformedUrl``: Use ``bb.fetch.MalformedUrl``.
- ``bb.encodeurl``: Use ``bb.fetch.encodeurl``.
- ``bb.decodeurl``: Use ``bb.fetch.decodeurl``
- ``bb.mkdirhier``: Use ``bb.utils.mkdirhier``.
- ``bb.movefile``: Use ``bb.utils.movefile``.
- ``bb.copyfile``: Use ``bb.utils.copyfile``.
- ``bb.which``: Use ``bb.utils.which``.
- ``bb.vercmp_string``: Use ``bb.utils.vercmp_string``.
- ``bb.vercmp``: Use ``bb.utils.vercmp``.
.. _migration-1.6-bitbake-fetcher:
SVK Fetcher
~~~~~~~~~~~
The SVK fetcher has been removed from BitBake.
.. _migration-1.6-bitbake-console-output:
Console Output Error Redirection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The BitBake console UI will now output errors to ``stderr`` instead of
``stdout``. Consequently, if you are piping or redirecting the output of
``bitbake`` to somewhere else, and you wish to retain the errors, you
will need to add ``2>&1`` (or something similar) to the end of your
``bitbake`` command line.
.. _migration-1.6-task-taskname-overrides:
``task-``\ taskname Overrides
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``task-``\ taskname overrides have been adjusted so that tasks whose
names contain underscores have the underscores replaced by hyphens for
the override so that they now function properly. For example, the task
override for :ref:`ref-tasks-populate_sdk` is
``task-populate-sdk``.
.. _migration-1.6-variable-changes:
Changes to Variables
--------------------
The following variables have changed. For information on the
OpenEmbedded build system variables, see the ":doc:`/ref-manual/variables`" Chapter.
.. _migration-1.6-variable-changes-TMPDIR:
``TMPDIR``
~~~~~~~~~~
:term:`TMPDIR` can no longer be on an NFS mount. NFS does
not offer full POSIX locking and inode consistency and can cause
unexpected issues if used to store :term:`TMPDIR`.
The check for this occurs on startup. If :term:`TMPDIR` is detected on an
NFS mount, an error occurs.
.. _migration-1.6-variable-changes-PRINC:
``PRINC``
~~~~~~~~~
The ``PRINC`` variable has been deprecated and triggers a warning if
detected during a build. For :term:`PR` increments on changes,
use the PR service instead. You can find out more about this service in
the ":ref:`dev-manual/packages:working with a pr service`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.6-variable-changes-IMAGE_TYPES:
``IMAGE_TYPES``
~~~~~~~~~~~~~~~
The "sum.jffs2" option for :term:`IMAGE_TYPES` has
been replaced by the "jffs2.sum" option, which fits the processing
order.
.. _migration-1.6-variable-changes-COPY_LIC_MANIFEST:
``COPY_LIC_MANIFEST``
~~~~~~~~~~~~~~~~~~~~~
The :term:`COPY_LIC_MANIFEST` variable must now
be set to "1" rather than any value in order to enable it.
.. _migration-1.6-variable-changes-COPY_LIC_DIRS:
``COPY_LIC_DIRS``
~~~~~~~~~~~~~~~~~
The :term:`COPY_LIC_DIRS` variable must now be set
to "1" rather than any value in order to enable it.
.. _migration-1.6-variable-changes-PACKAGE_GROUP:
``PACKAGE_GROUP``
~~~~~~~~~~~~~~~~~
The ``PACKAGE_GROUP`` variable has been renamed to
:term:`FEATURE_PACKAGES` to more accurately
reflect its purpose. You can still use ``PACKAGE_GROUP`` but the
OpenEmbedded build system produces a warning message when it encounters
the variable.
.. _migration-1.6-variable-changes-variable-entry-behavior:
Preprocess and Post Process Command Variable Behavior
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following variables now expect a semicolon separated list of
functions to call and not arbitrary shell commands:
- :term:`ROOTFS_PREPROCESS_COMMAND`
- :term:`ROOTFS_POSTPROCESS_COMMAND`
- :term:`SDK_POSTPROCESS_COMMAND`
- :term:`POPULATE_SDK_POST_TARGET_COMMAND`
- :term:`POPULATE_SDK_POST_HOST_COMMAND`
- :term:`IMAGE_POSTPROCESS_COMMAND`
- :term:`IMAGE_PREPROCESS_COMMAND`
- :term:`ROOTFS_POSTUNINSTALL_COMMAND`
- :term:`ROOTFS_POSTINSTALL_COMMAND`
For
migration purposes, you can simply wrap shell commands in a shell
function and then call the function. Here is an example::
my_postprocess_function() {
echo "hello" > ${IMAGE_ROOTFS}/hello.txt
}
ROOTFS_POSTPROCESS_COMMAND += "my_postprocess_function; "
.. _migration-1.6-package-test-ptest:
Package Test (ptest)
--------------------
Package Tests (ptest) are built but not installed by default. For
information on using Package Tests, see the
":ref:`test-manual/ptest:testing packages with ptest`" section in the
Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`"
section.
.. _migration-1.6-build-changes:
Build Changes
-------------
Separate build and source directories have been enabled by default for
selected recipes where it is known to work and for all
recipes that inherit the :ref:`ref-classes-cmake` class. In
future releases the :ref:`ref-classes-autotools` class
will enable a separate :term:`Build Directory` by default as well. Recipes
building Autotools-based software that fails to build with a separate
:term:`Build Directory` should be changed to inherit from the
:ref:`autotools-brokensep <ref-classes-autotools>` class instead of
the :ref:`ref-classes-autotools` or ``autotools_stage`` classes.
.. _migration-1.6-building-qemu-native:
``qemu-native``
---------------
``qemu-native`` now builds without SDL-based graphical output support by
default. The following additional lines are needed in your
``local.conf`` to enable it::
PACKAGECONFIG_pn-qemu-native = "sdl"
ASSUME_PROVIDED += "libsdl-native"
.. note::
The default ``local.conf`` contains these statements. Consequently, if you
are building a headless system and using a default ``local.conf``
file, you will need comment these two lines out.
.. _migration-1.6-core-image-basic:
``core-image-basic``
--------------------
``core-image-basic`` has been renamed to ``core-image-full-cmdline``.
In addition to ``core-image-basic`` being renamed,
``packagegroup-core-basic`` has been renamed to
``packagegroup-core-full-cmdline`` to match.
.. _migration-1.6-licensing:
Licensing
---------
The top-level :term:`LICENSE` file has been changed to better describe the
license of the various components of :term:`OpenEmbedded-Core (OE-Core)`. However,
the licensing itself remains unchanged.
Normally, this change would not cause any side-effects. However, some
recipes point to this file within
:term:`LIC_FILES_CHKSUM` (as
``${COREBASE}/LICENSE``) and thus the accompanying checksum must be
changed from 3f40d7994397109285ec7b81fdeb3b58 to
4d92cd373abda3937c2bc47fbc49d690. A better alternative is to have
:term:`LIC_FILES_CHKSUM` point to a file describing the license that is
distributed with the source that the recipe is building, if possible,
rather than pointing to ``${COREBASE}/LICENSE``.
.. _migration-1.6-cflags-options:
``CFLAGS`` Options
------------------
The "-fpermissive" option has been removed from the default
:term:`CFLAGS` value. You need to take action on
individual recipes that fail when building with this option. You need to
either patch the recipes to fix the issues reported by the compiler, or
you need to add "-fpermissive" to :term:`CFLAGS` in the recipes.
.. _migration-1.6-custom-images:
Custom Image Output Types
-------------------------
Custom image output types, as selected using
:term:`IMAGE_FSTYPES`, must declare their
dependencies on other image types (if any) using a new
:term:`IMAGE_TYPEDEP` variable.
.. _migration-1.6-do-package-write-task:
Tasks
-----
The ``do_package_write`` task has been removed. The task is no longer
needed.
.. _migration-1.6-update-alternatives-provider:
``update-alternative`` Provider
-------------------------------
The default ``update-alternatives`` provider has been changed from
``opkg`` to ``opkg-utils``. This change resolves some troublesome
circular dependencies. The runtime package has also been renamed from
``update-alternatives-cworth`` to ``update-alternatives-opkg``.
.. _migration-1.6-virtclass-overrides:
``virtclass`` Overrides
-----------------------
The ``virtclass`` overrides are now deprecated. Use the equivalent class
overrides instead (e.g. ``virtclass-native`` becomes ``class-native``.)
.. _migration-1.6-removed-renamed-recipes:
Removed and Renamed Recipes
---------------------------
The following recipes have been removed:
- ``packagegroup-toolset-native`` --- this recipe is largely unused.
- ``linux-yocto-3.8`` --- support for the Linux yocto 3.8 kernel has been
dropped. Support for the 3.10 and 3.14 kernels have been added with
the ``linux-yocto-3.10`` and ``linux-yocto-3.14`` recipes.
- ``ocf-linux`` --- this recipe has been functionally replaced using
``cryptodev-linux``.
- ``genext2fs`` --- ``genext2fs`` is no longer used by the build system
and is unmaintained upstream.
- ``js`` --- this provided an ancient version of Mozilla's javascript
engine that is no longer needed.
- ``zaurusd`` --- the recipe has been moved to the ``meta-handheld``
layer.
- ``eglibc 2.17`` --- replaced by the ``eglibc 2.19`` recipe.
- ``gcc 4.7.2`` --- replaced by the now stable ``gcc 4.8.2``.
- ``external-sourcery-toolchain`` --- this recipe is now maintained in
the ``meta-sourcery`` layer.
- ``linux-libc-headers-yocto 3.4+git`` --- now using version 3.10 of the
``linux-libc-headers`` by default.
- ``meta-toolchain-gmae`` --- this recipe is obsolete.
- ``packagegroup-core-sdk-gmae`` --- this recipe is obsolete.
- ``packagegroup-core-standalone-gmae-sdk-target`` --- this recipe is
obsolete.
.. _migration-1.6-removed-classes:
Removed Classes
---------------
The following classes have become obsolete and have been removed:
- ``module_strip``
- ``pkg_metainfo``
- ``pkg_distribute``
- ``image-empty``
.. _migration-1.6-reference-bsps:
Reference Board Support Packages (BSPs)
---------------------------------------
The following reference BSPs changes occurred:
- The BeagleBoard (``beagleboard``) ARM reference hardware has been
replaced by the BeagleBone (``beaglebone``) hardware.
- The RouterStation Pro (``routerstationpro``) MIPS reference hardware
has been replaced by the EdgeRouter Lite (``edgerouter``) hardware.
The previous reference BSPs for the ``beagleboard`` and
``routerstationpro`` machines are still available in a new
``meta-yocto-bsp-old`` layer in the
:yocto_git:`Source Repositories <>` at
:yocto_git:`/meta-yocto-bsp-old/`.

222
sources/poky/documentation/migration-guides/migration-1.7.rst Normal file
View File

@@ -0,0 +1,222 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.7 (dizzy)
===================
This section provides migration information for moving to the Yocto
Project 1.7 Release (codename "dizzy") from the prior release.
.. _migration-1.7-changes-to-setting-qemu-packageconfig-options:
Changes to Setting QEMU ``PACKAGECONFIG`` Options in ``local.conf``
-------------------------------------------------------------------
The QEMU recipe now uses a number of
:term:`PACKAGECONFIG` options to enable various
optional features. The method used to set defaults for these options
means that existing ``local.conf`` files will need to be modified to
append to :term:`PACKAGECONFIG` for ``qemu-native`` and ``nativesdk-qemu``
instead of setting it. In other words, to enable graphical output for
QEMU, you should now have these lines in ``local.conf``::
PACKAGECONFIG_append_pn-qemu-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
.. _migration-1.7-minimum-git-version:
Minimum Git version
-------------------
The minimum :ref:`overview-manual/development-environment:git`
version required on the
build host is now 1.7.8 because the ``--list`` option is now required by
BitBake's Git fetcher. As always, if your host distribution does not
provide a version of Git that meets this requirement, you can use the
:term:`buildtools` tarball that does. See the
":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
section for more information.
.. _migration-1.7-autotools-class-changes:
Autotools Class Changes
-----------------------
The following :ref:`ref-classes-autotools` class changes occurred:
- *A separate :term:`Build Directory` is now used by default:* The
:ref:`ref-classes-autotools` class has been changed to use a directory for
building (:term:`B`), which is separate from the source directory
(:term:`S`). This is commonly referred to as ``B != S``, or
an out-of-tree build.
If the software being built is already capable of building in a
directory separate from the source, you do not need to do anything.
However, if the software is not capable of being built in this
manner, you will need to either patch the software so that it can
build separately, or you will need to change the recipe to inherit
the :ref:`autotools-brokensep <ref-classes-autotools>` class instead
of the :ref:`ref-classes-autotools` or ``autotools_stage`` classes.
- The ``--foreign`` option is no longer passed to ``automake`` when
running ``autoconf``: This option tells ``automake`` that a
particular software package does not follow the GNU standards and
therefore should not be expected to distribute certain files such as
``ChangeLog``, ``AUTHORS``, and so forth. Because the majority of
upstream software packages already tell ``automake`` to enable
foreign mode themselves, the option is mostly superfluous. However,
some recipes will need patches for this change. You can easily make
the change by patching ``configure.ac`` so that it passes "foreign"
to ``AM_INIT_AUTOMAKE()``. See :oe_git:`this
commit </openembedded-core/commit/?id=01943188f85ce6411717fb5bf702d609f55813f2>`
for an example showing how to make the patch.
.. _migration-1.7-binary-configuration-scripts-disabled:
Binary Configuration Scripts Disabled
-------------------------------------
Some of the core recipes that package binary configuration scripts now
disable the scripts due to the scripts previously requiring error-prone
path substitution. Software that links against these libraries using
these scripts should use the much more robust ``pkg-config`` instead.
The list of recipes changed in this version (and their configuration
scripts) is as follows::
directfb (directfb-config)
freetype (freetype-config)
gpgme (gpgme-config)
libassuan (libassuan-config)
libcroco (croco-6.0-config)
libgcrypt (libgcrypt-config)
libgpg-error (gpg-error-config)
libksba (ksba-config)
libpcap (pcap-config)
libpcre (pcre-config)
libpng (libpng-config, libpng16-config)
libsdl (sdl-config)
libusb-compat (libusb-config)
libxml2 (xml2-config)
libxslt (xslt-config)
ncurses (ncurses-config)
neon (neon-config)
npth (npth-config)
pth (pth-config)
taglib (taglib-config)
Additionally, support for ``pkg-config`` has been added to some recipes in the
previous list in the rare cases where the upstream software package does
not already provide it.
.. _migration-1.7-glibc-replaces-eglibc:
``eglibc 2.19`` Replaced with ``glibc 2.20``
--------------------------------------------
Because ``eglibc`` and ``glibc`` were already fairly close, this
replacement should not require any significant changes to other software
that links to ``eglibc``. However, there were a number of minor changes
in ``glibc 2.20`` upstream that could require patching some software
(e.g. the removal of the ``_BSD_SOURCE`` feature test macro).
``glibc 2.20`` requires version 2.6.32 or greater of the Linux kernel.
Thus, older kernels will no longer be usable in conjunction with it.
For full details on the changes in ``glibc 2.20``, see the upstream
release notes
`here <https://sourceware.org/ml/libc-alpha/2014-09/msg00088.html>`__.
.. _migration-1.7-kernel-module-autoloading:
Kernel Module Autoloading
-------------------------
The :term:`module_autoload_* <module_autoload>` variable is now
deprecated and a new
:term:`KERNEL_MODULE_AUTOLOAD` variable
should be used instead. Also, :term:`module_conf_* <module_conf>`
must now be used in conjunction with a new
:term:`KERNEL_MODULE_PROBECONF` variable.
The new variables no longer require you to specify the module name as
part of the variable name. This change not only simplifies usage but
also allows the values of these variables to be appropriately
incorporated into task signatures and thus trigger the appropriate tasks
to re-execute when changed. You should replace any references to
``module_autoload_*`` with :term:`KERNEL_MODULE_AUTOLOAD`, and add any
modules for which ``module_conf_*`` is specified to
:term:`KERNEL_MODULE_PROBECONF`.
.. _migration-1.7-qa-check-changes:
QA Check Changes
----------------
The following changes have occurred to the QA check process:
- Additional QA checks ``file-rdeps`` and ``build-deps`` have been
added in order to verify that file dependencies are satisfied (e.g.
package contains a script requiring ``/bin/bash``) and build-time
dependencies are declared, respectively. For more information, please
see the ":doc:`/ref-manual/qa-checks`" chapter.
- Package QA checks are now performed during a new
:ref:`ref-tasks-package_qa` task rather than being
part of the :ref:`ref-tasks-package` task. This allows
more parallel execution. This change is unlikely to be an issue
except for highly customized recipes that disable packaging tasks
themselves by marking them as ``noexec``. For those packages, you
will need to disable the :ref:`ref-tasks-package_qa` task as well.
- Files being overwritten during the
:ref:`ref-tasks-populate_sysroot` task now
trigger an error instead of a warning. Recipes should not be
overwriting files written to the sysroot by other recipes. If you
have these types of recipes, you need to alter them so that they do
not overwrite these files.
You might now receive this error after changes in configuration or
metadata resulting in orphaned files being left in the sysroot. If
you do receive this error, the way to resolve the issue is to delete
your :term:`TMPDIR` or to move it out of the way and
then re-start the build. Anything that has been fully built up to
that point and does not need rebuilding will be restored from the
shared state cache and the rest of the build will be able to proceed
as normal.
.. _migration-1.7-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``x-load``: This recipe has been superseded by U-Boot SPL for all
Cortex-based TI SoCs. For legacy boards, the ``meta-ti`` layer, which
contains a maintained recipe, should be used instead.
- ``ubootchart``: This recipe is obsolete. A ``bootchart2`` recipe has
been added to functionally replace it.
- ``linux-yocto 3.4``: Support for the linux-yocto 3.4 kernel has been
dropped. Support for the 3.10 and 3.14 kernels remains, while support
for version 3.17 has been added.
- ``eglibc`` has been removed in favor of ``glibc``. See the
":ref:`migration-1.7-glibc-replaces-eglibc`" section for more information.
.. _migration-1.7-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous change occurred:
- The build history feature now writes ``build-id.txt`` instead of
``build-id``. Additionally, ``build-id.txt`` now contains the full
build header as printed by BitBake upon starting the build. You
should manually remove old "build-id" files from your existing build
history repositories to avoid confusion. For information on the build
history feature, see the
":ref:`dev-manual/build-quality:maintaining build output quality`"
section in the Yocto Project Development Tasks Manual.

183
sources/poky/documentation/migration-guides/migration-1.8.rst Normal file
View File

@@ -0,0 +1,183 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.8 (fido)
==================
This section provides migration information for moving to the Yocto
Project 1.8 Release (codename "fido") from the prior release.
.. _migration-1.8-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``owl-video``: Functionality replaced by ``gst-player``.
- ``gaku``: Functionality replaced by ``gst-player``.
- ``gnome-desktop``: This recipe is now available in ``meta-gnome`` and
is no longer needed.
- ``gsettings-desktop-schemas``: This recipe is now available in
``meta-gnome`` and is no longer needed.
- ``python-argparse``: The ``argparse`` module is already provided in
the default Python distribution in a package named
``python-argparse``. Consequently, the separate ``python-argparse``
recipe is no longer needed.
- ``telepathy-python, libtelepathy, telepathy-glib, telepathy-idle, telepathy-mission-control``:
All these recipes have moved to ``meta-oe`` and are consequently no
longer needed by any recipes in OpenEmbedded-Core.
- ``linux-yocto_3.10`` and ``linux-yocto_3.17``: Support for the
linux-yocto 3.10 and 3.17 kernels has been dropped. Support for the
3.14 kernel remains, while support for 3.19 kernel has been added.
- ``poky-feed-config-opkg``: This recipe has become obsolete and is no
longer needed. Use ``distro-feed-config`` from ``meta-oe`` instead.
- ``libav 0.8.x``: ``libav 9.x`` is now used.
- ``sed-native``: No longer needed. A working version of ``sed`` is
expected to be provided by the host distribution.
.. _migration-1.8-bluez:
BlueZ 4.x / 5.x Selection
-------------------------
Proper built-in support for selecting BlueZ 5.x in preference to the
default of 4.x now exists. To use BlueZ 5.x, simply add "bluez5" to your
:term:`DISTRO_FEATURES` value. If you had
previously added append files (``*.bbappend``) to make this selection,
you can now remove them.
Additionally, a ``bluetooth`` class has been added to make selection of
the appropriate bluetooth support within a recipe a little easier. If
you wish to make use of this class in a recipe, add something such as
the following::
inherit bluetooth
PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', '${BLUEZ}', '', d)}"
PACKAGECONFIG[bluez4] = "--enable-bluetooth,--disable-bluetooth,bluez4"
PACKAGECONFIG[bluez5] = "--enable-bluez5,--disable-bluez5,bluez5"
.. _migration-1.8-kernel-build-changes:
Kernel Build Changes
--------------------
The kernel build process was changed to place the source in a common shared work
area and to place build artifacts separately in the source code tree. In theory,
migration paths have been provided for most common usages in kernel recipes but
this might not work in all cases. In particular, users need to ensure that
``${S}`` (source files) and ``${B}`` (build artifacts) are used correctly in
functions such as :ref:`ref-tasks-configure` and :ref:`ref-tasks-install`. For
kernel recipes that do not inherit from :ref:`ref-classes-kernel-yocto` or
include ``linux-yocto.inc``, you might wish to refer to the ``linux.inc`` file
in the ``meta-oe`` layer for the kinds of changes you need to make. For reference,
here is the
:oe_git:`commit </meta-openembedded/commit/meta-oe/recipes-kernel/linux/linux.inc?id=fc7132ede27ac67669448d3d2845ce7d46c6a1ee>`
where the ``linux.inc`` file in ``meta-oe`` was updated.
Recipes that rely on the kernel source code and do not inherit the
:ref:`module <ref-classes-module>` classes might need to add explicit
dependencies on the :ref:`ref-tasks-shared_workdir` kernel task, for example::
do_configure[depends] += "virtual/kernel:do_shared_workdir"
.. _migration-1.8-ssl:
SSL 3.0 is Now Disabled in OpenSSL
----------------------------------
SSL 3.0 is now disabled when building OpenSSL. Disabling SSL 3.0 avoids
any lingering instances of the POODLE vulnerability. If you feel you
must re-enable SSL 3.0, then you can add an append file (``*.bbappend``)
for the ``openssl`` recipe to remove "-no-ssl3" from
:term:`EXTRA_OECONF`.
.. _migration-1.8-default-sysroot-poisoning:
Default Sysroot Poisoning
-------------------------
``gcc's`` default sysroot and include directories are now "poisoned". In
other words, the sysroot and include directories are being redirected to
a non-existent location in order to catch when host directories are
being used due to the correct options not being passed. This poisoning
applies both to the cross-compiler used within the build and to the
cross-compiler produced in the SDK.
If this change causes something in the build to fail, it almost
certainly means the various compiler flags and commands are not being
passed correctly to the underlying piece of software. In such cases, you
need to take corrective steps.
.. _migration-1.8-rebuild-improvements:
Rebuild Improvements
--------------------
Changes have been made to the :ref:`ref-classes-base`,
:ref:`ref-classes-autotools`, and :ref:`ref-classes-cmake` classes to clean out
generated files when the :ref:`ref-tasks-configure` task needs to be
re-executed.
One of the improvements is to attempt to run "make clean" during the
:ref:`ref-tasks-configure` task if a ``Makefile`` exists. Some software packages
do not provide a working clean target within their make files. If you
have such recipes, you need to set
:term:`CLEANBROKEN` to "1" within the recipe, for example::
CLEANBROKEN = "1"
.. _migration-1.8-qa-check-and-validation-changes:
QA Check and Validation Changes
-------------------------------
The following QA Check and Validation Changes have occurred:
- Usage of ``PRINC`` previously triggered a warning. It now triggers an
error. You should remove any remaining usage of ``PRINC`` in any
recipe or append file.
- An additional QA check has been added to detect usage of ``${D}`` in
:term:`FILES` values where :term:`D` values
should not be used at all. The same check ensures that ``$D`` is used
in ``pkg_preinst/pkg_postinst/pkg_prerm/pkg_postrm`` functions
instead of ``${D}``.
- :term:`S` now needs to be set to a valid value within a
recipe. If :term:`S` is not set in the recipe, the directory is not
automatically created. If :term:`S` does not point to a directory that
exists at the time the :ref:`ref-tasks-unpack` task
finishes, a warning will be shown.
- :term:`LICENSE` is now validated for correct
formatting of multiple licenses. If the format is invalid (e.g.
multiple licenses are specified with no operators to specify how the
multiple licenses interact), then a warning will be shown.
.. _migration-1.8-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- The ``send-error-report`` script now expects a "-s" option to be
specified before the server address. This assumes a server address is
being specified.
- The ``oe-pkgdata-util`` script now expects a "-p" option to be
specified before the ``pkgdata`` directory, which is now optional. If
the ``pkgdata`` directory is not specified, the script will run
BitBake to query :term:`PKGDATA_DIR` from the
build environment.

280
sources/poky/documentation/migration-guides/migration-2.0.rst Normal file
View File

@@ -0,0 +1,280 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.0 (jethro)
====================
This section provides migration information for moving to the Yocto
Project 2.0 Release (codename "jethro") from the prior release.
.. _migration-2.0-gcc-5:
GCC 5
-----
The default compiler is now GCC 5.2. This change has required fixes for
compilation errors in a number of other recipes.
One important example is a fix for when the Linux kernel freezes at boot
time on ARM when built with GCC 5. If you are using your own kernel
recipe or source tree and building for ARM, you will likely need to
apply this
`patch <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a077224fd35b2f7fbc93f14cf67074fc792fbac2>`__.
The standard ``linux-yocto`` kernel source tree already has a workaround
for the same issue.
For further details, see https://gcc.gnu.org/gcc-5/changes.html
and the porting guide at
https://gcc.gnu.org/gcc-5/porting_to.html.
Alternatively, you can switch back to GCC 4.9 or 4.8 by setting
:term:`GCCVERSION` in your configuration, as follows::
GCCVERSION = "4.9%"
.. _migration-2.0-Gstreamer-0.10-removed:
Gstreamer 0.10 Removed
----------------------
Gstreamer 0.10 has been removed in favor of Gstreamer 1.x. As part of
the change, recipes for Gstreamer 0.10 and related software are now
located in ``meta-multimedia``. This change results in Qt4 having Phonon
and Gstreamer support in QtWebkit disabled by default.
.. _migration-2.0-removed-recipes:
Removed Recipes
---------------
The following recipes have been moved or removed:
- ``bluez4``: The recipe is obsolete and has been moved due to
``bluez5`` becoming fully integrated. The ``bluez4`` recipe now
resides in ``meta-oe``.
- ``gamin``: The recipe is obsolete and has been removed.
- ``gnome-icon-theme``: The recipe's functionally has been replaced by
``adwaita-icon-theme``.
- Gstreamer 0.10 Recipes: Recipes for Gstreamer 0.10 have been removed
in favor of the recipes for Gstreamer 1.x.
- ``insserv``: The recipe is obsolete and has been removed.
- ``libunique``: The recipe is no longer used and has been moved to
``meta-oe``.
- ``midori``: The recipe's functionally has been replaced by
``epiphany``.
- ``python-gst``: The recipe is obsolete and has been removed since it
only contains bindings for Gstreamer 0.10.
- ``qt-mobility``: The recipe is obsolete and has been removed since it
requires ``Gstreamer 0.10``, which has been replaced.
- ``subversion``: All 1.6.x versions of this recipe have been removed.
- ``webkit-gtk``: The older 1.8.3 version of this recipe has been
removed in favor of ``webkitgtk``.
.. _migration-2.0-bitbake-datastore-improvements:
BitBake datastore improvements
------------------------------
The method by which BitBake's datastore handles overrides has changed.
Overrides are now applied dynamically and ``bb.data.update_data()`` is
now a no-op. Thus, ``bb.data.update_data()`` is no longer required in
order to apply the correct overrides. In practice, this change is
unlikely to require any changes to Metadata. However, these minor
changes in behavior exist:
- All potential overrides are now visible in the variable history as
seen when you run the following::
$ bitbake -e
- ``d.delVar('VARNAME')`` and
``d.setVar('VARNAME', None)`` result in the variable and all
of its overrides being cleared out. Before the change, only the
non-overridden values were cleared.
.. _migration-2.0-shell-message-function-changes:
Shell Message Function Changes
------------------------------
The shell versions of the BitBake message functions (i.e. ``bbdebug``,
``bbnote``, ``bbwarn``, ``bbplain``, ``bberror``, and ``bbfatal``) are
now connected through to their BitBake equivalents ``bb.debug()``,
``bb.note()``, ``bb.warn()``, ``bb.plain()``, ``bb.error()``, and
``bb.fatal()``, respectively. Thus, those message functions that you
would expect to be printed by the BitBake UI are now actually printed.
In practice, this change means two things:
- If you now see messages on the console that you did not previously
see as a result of this change, you might need to clean up the calls
to ``bbwarn``, ``bberror``, and so forth. Or, you might want to
simply remove the calls.
- The ``bbfatal`` message function now suppresses the full error log in
the UI, which means any calls to ``bbfatal`` where you still wish to
see the full error log should be replaced by ``die`` or
``bbfatal_log``.
.. _migration-2.0-extra-development-debug-package-cleanup:
Extra Development/Debug Package Cleanup
---------------------------------------
The following recipes have had extra ``dev/dbg`` packages removed:
- ``acl``
- ``apmd``
- ``aspell``
- ``attr``
- ``augeas``
- ``bzip2``
- ``cogl``
- ``curl``
- ``elfutils``
- ``gcc-target``
- ``libgcc``
- ``libtool``
- ``libxmu``
- ``opkg``
- ``pciutils``
- ``rpm``
- ``sysfsutils``
- ``tiff``
- ``xz``
All of the above recipes now conform to the standard packaging scheme
where a single ``-dev``, ``-dbg``, and ``-staticdev`` package exists per
recipe.
.. _migration-2.0-recipe-maintenance-tracking-data-moved-to-oe-core:
Recipe Maintenance Tracking Data Moved to OE-Core
-------------------------------------------------
Maintenance tracking data for recipes that was previously part of
``meta-yocto`` has been moved to :term:`OpenEmbedded-Core (OE-Core)`. The change
includes ``package_regex.inc`` and ``distro_alias.inc``, which are
typically enabled when using the ``distrodata`` class. Additionally, the
contents of ``upstream_tracking.inc`` has now been split out to the
relevant recipes.
.. _migration-2.0-automatic-stale-sysroot-file-cleanup:
Automatic Stale Sysroot File Cleanup
------------------------------------
Stale files from recipes that no longer exist in the current
configuration are now automatically removed from sysroot as well as
removed from any other place managed by shared state. This automatic
cleanup means that the build system now properly handles situations such
as renaming the build system side of recipes, removal of layers from
``bblayers.conf``, and :term:`DISTRO_FEATURES`
changes.
Additionally, work directories for old versions of recipes are now
pruned. If you wish to disable pruning old work directories, you can set
the following variable in your configuration::
SSTATE_PRUNE_OBSOLETEWORKDIR = "0"
.. _migration-2.0-linux-yocto-kernel-metadata-repository-now-split-from-source:
``linux-yocto`` Kernel Metadata Repository Now Split from Source
----------------------------------------------------------------
The ``linux-yocto`` tree has up to now been a combined set of kernel
changes and configuration (meta) data carried in a single tree. While
this format is effective at keeping kernel configuration and source
modifications synchronized, it is not always obvious to developers how
to manipulate the Metadata as compared to the source.
Metadata processing has now been removed from the
:ref:`ref-classes-kernel-yocto` class and the external
Metadata repository ``yocto-kernel-cache``, which has always been used
to seed the ``linux-yocto`` "meta" branch. This separate ``linux-yocto``
cache repository is now the primary location for this data. Due to this
change, ``linux-yocto`` is no longer able to process combined trees.
Thus, if you need to have your own combined kernel repository, you must
do the split there as well and update your recipes accordingly. See the
``meta/recipes-kernel/linux/linux-yocto_4.1.bb`` recipe for an example.
.. _migration-2.0-additional-qa-checks:
Additional QA checks
--------------------
The following QA checks have been added:
- Added a "host-user-contaminated" check for ownership issues for
packaged files outside of ``/home``. The check looks for files that
are incorrectly owned by the user that ran BitBake instead of owned
by a valid user in the target system.
- Added an "invalid-chars" check for invalid (non-UTF8) characters in
recipe metadata variable values (i.e.
:term:`DESCRIPTION`,
:term:`SUMMARY`, :term:`LICENSE`, and
:term:`SECTION`). Some package managers do not support
these characters.
- Added an "invalid-packageconfig" check for any options specified in
:term:`PACKAGECONFIG` that do not match any
:term:`PACKAGECONFIG` option defined for the recipe.
.. _migration-2.0-miscellaneous:
Miscellaneous Changes
---------------------
These additional changes exist:
- ``gtk-update-icon-cache`` has been renamed to ``gtk-icon-utils``.
- The ``tools-profile`` :term:`IMAGE_FEATURES`
item as well as its corresponding packagegroup and
``packagegroup-core-tools-profile`` no longer bring in ``oprofile``.
Bringing in ``oprofile`` was originally added to aid compilation on
resource-constrained targets. However, this aid has not been widely
used and is not likely to be used going forward due to the more
powerful target platforms and the existence of better
cross-compilation tools.
- The :term:`IMAGE_FSTYPES` variable's default
value now specifies ``ext4`` instead of ``ext3``.
- All support for the ``PRINC`` variable has been removed.
- The ``packagegroup-core-full-cmdline`` packagegroup no longer brings
in ``lighttpd`` due to the fact that bringing in ``lighttpd`` is not
really in line with the packagegroup's purpose, which is to add full
versions of command-line tools that by default are provided by
``busybox``.

432
sources/poky/documentation/migration-guides/migration-2.1.rst Normal file
View File

@@ -0,0 +1,432 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.1 (krogoth)
=====================
This section provides migration information for moving to the Yocto
Project 2.1 Release (codename "krogoth") from the prior release.
.. _migration-2.1-variable-expansion-in-python-functions:
Variable Expansion in Python Functions
--------------------------------------
Variable expressions, such as ``${VARNAME}`` no longer expand
automatically within Python functions. Suppressing expansion was done to
allow Python functions to construct shell scripts or other code for
situations in which you do not want such expressions expanded. For any
existing code that relies on these expansions, you need to change the
expansions to expand the value of individual variables through
``d.getVar()``. To alternatively expand more complex expressions, use
``d.expand()``.
.. _migration-2.1-overrides-must-now-be-lower-case:
Overrides Must Now be Lower-Case
--------------------------------
The convention for overrides has always been for them to be lower-case
characters. This practice is now a requirement as BitBake's datastore
now assumes lower-case characters in order to give a slight performance
boost during parsing. In practical terms, this requirement means that
anything that ends up in :term:`OVERRIDES` must now
appear in lower-case characters (e.g. values for :term:`MACHINE`,
:term:`TARGET_ARCH`, :term:`DISTRO`, and also recipe names if
``_pn-``\ recipename overrides are to be effective).
.. _migration-2.1-expand-parameter-to-getvar-and-getvarflag-now-mandatory:
Expand Parameter to ``getVar()`` and ``getVarFlag()`` is Now Mandatory
----------------------------------------------------------------------
The expand parameter to ``getVar()`` and ``getVarFlag()`` previously
defaulted to False if not specified. Now, however, no default exists so
one must be specified. You must change any ``getVar()`` calls that do
not specify the final expand parameter to calls that do specify the
parameter. You can run the following ``sed`` command at the base of a
layer to make this change::
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
sed -e 's:\(\.getVarFlag([^,()]*,[^,()]*\)):\1, False):g' -i `grep -ril getVarFlag *`
.. note::
The reason for this change is that it prepares the way for changing
the default to True in a future Yocto Project release. This future
change is a much more sensible default than False. However, the
change needs to be made gradually as a sudden change of the default
would potentially cause side-effects that would be difficult to
detect.
.. _migration-2.1-makefile-environment-changes:
Makefile Environment Changes
----------------------------
:term:`EXTRA_OEMAKE` now defaults to "" instead of
"-e MAKEFLAGS=". Setting :term:`EXTRA_OEMAKE` to "-e MAKEFLAGS=" by default
was a historical accident that has required many classes (e.g.
:ref:`ref-classes-autotools`, ``module``) and recipes to override this default in order
to work with sensible build systems. When upgrading to the release, you
must edit any recipe that relies upon this old default by either setting
:term:`EXTRA_OEMAKE` back to "-e MAKEFLAGS=" or by explicitly setting any
required variable value overrides using :term:`EXTRA_OEMAKE`, which is
typically only needed when a Makefile sets a default value for a
variable that is inappropriate for cross-compilation using the "="
operator rather than the "?=" operator.
.. _migration-2.1-libexecdir-reverted-to-prefix-libexec:
``libexecdir`` Reverted to ``${prefix}/libexec``
------------------------------------------------
The use of ``${libdir}/${BPN}`` as ``libexecdir`` is different as
compared to all other mainstream distributions, which either uses
``${prefix}/libexec`` or ``${libdir}``. The use is also contrary to the
GNU Coding Standards (i.e.
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html)
that suggest ``${prefix}/libexec`` and also notes that any
package-specific nesting should be done by the package itself. Finally,
having ``libexecdir`` change between recipes makes it very difficult for
different recipes to invoke binaries that have been installed into
``libexecdir``. The Filesystem Hierarchy Standard (i.e.
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html) now
recognizes the use of ``${prefix}/libexec/``, giving distributions the
choice between ``${prefix}/lib`` or ``${prefix}/libexec`` without
breaking FHS.
.. _migration-2.1-ac-cv-sizeof-off-t-no-longer-cached-in-site-files:
``ac_cv_sizeof_off_t`` is No Longer Cached in Site Files
--------------------------------------------------------
For recipes inheriting the :ref:`ref-classes-autotools`
class, ``ac_cv_sizeof_off_t`` is no longer cached in the site files for
``autoconf``. The reason for this change is because the
``ac_cv_sizeof_off_t`` value is not necessarily static per architecture
as was previously assumed. Rather, the value changes based on whether
large file support is enabled. For most software that uses ``autoconf``,
this change should not be a problem. However, if you have a recipe that
bypasses the standard :ref:`ref-tasks-configure` task
from the :ref:`ref-classes-autotools` class and the software the recipe is building
uses a very old version of ``autoconf``, the recipe might be incapable
of determining the correct size of ``off_t`` during :ref:`ref-tasks-configure`.
The best course of action is to patch the software as necessary to allow
the default implementation from the :ref:`ref-classes-autotools` class to work such
that ``autoreconf`` succeeds and produces a working configure script,
and to remove the overridden :ref:`ref-tasks-configure` task such that the default
implementation does get used.
.. _migration-2.1-image-generation-split-out-from-filesystem-generation:
Image Generation is Now Split Out from Filesystem Generation
------------------------------------------------------------
Previously, for image recipes the :ref:`ref-tasks-rootfs`
task assembled the filesystem and then from that filesystem generated
images. With this Yocto Project release, image generation is split into
separate :ref:`ref-tasks-image` tasks for clarity both in
operation and in the code.
For most cases, this change does not present any problems. However, if
you have made customizations that directly modify the :ref:`ref-tasks-rootfs` task
or that mention :ref:`ref-tasks-rootfs`, you might need to update those changes.
In particular, if you had added any tasks after :ref:`ref-tasks-rootfs`, you
should make edits so that those tasks are after the
:ref:`ref-tasks-image-complete` task rather than
after :ref:`ref-tasks-rootfs` so that your added tasks run at the correct
time.
A minor part of this restructuring is that the post-processing definitions and
functions have been moved from the :ref:`ref-classes-image` class to the
:ref:`rootfs-postcommands <ref-classes-rootfs*>` class. Functionally,
however, they remain unchanged.
.. _migration-2.1-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed in the 2.1 release:
- ``gcc`` version 4.8: Versions 4.9 and 5.3 remain.
- ``qt4``: All support for Qt 4.x has been moved out to a separate
``meta-qt4`` layer because Qt 4 is no longer supported upstream.
- ``x11vnc``: Moved to the ``meta-oe`` layer.
- ``linux-yocto-3.14``: No longer supported.
- ``linux-yocto-3.19``: No longer supported.
- ``libjpeg``: Replaced by the ``libjpeg-turbo`` recipe.
- ``pth``: Became obsolete.
- ``liboil``: Recipe is no longer needed and has been moved to the
``meta-multimedia`` layer.
- ``gtk-theme-torturer``: Recipe is no longer needed and has been moved
to the ``meta-gnome`` layer.
- ``gnome-mime-data``: Recipe is no longer needed and has been moved to
the ``meta-gnome`` layer.
- ``udev``: Replaced by the ``eudev`` recipe for compatibility when
using ``sysvinit`` with newer kernels.
- ``python-pygtk``: Recipe became obsolete.
- ``adt-installer``: Recipe became obsolete. See the
":ref:`migration-guides/migration-2.1:adt removed`" section for more information.
.. _migration-2.1-class-changes:
Class Changes
-------------
The following classes have changed:
- ``autotools_stage``: Removed because the
:ref:`ref-classes-autotools` class now provides its
functionality. Recipes that inherited from ``autotools_stage`` should
now inherit from :ref:`ref-classes-autotools` instead.
- ``boot-directdisk``: Merged into the ``image-vm`` class. The
``boot-directdisk`` class was rarely directly used. Consequently,
this change should not cause any issues.
- ``bootimg``: Merged into the :ref:`ref-classes-image-live` class. The
``bootimg`` class was rarely directly used. Consequently, this change should
not cause any issues.
- ``packageinfo``: Removed due to its limited use by the Hob UI, which
has itself been removed.
.. _migration-2.1-build-system-ui-changes:
Build System User Interface Changes
-----------------------------------
The following changes have been made to the build system user interface:
- *Hob GTK+-based UI*: Removed because it is unmaintained and based on
the outdated GTK+ 2 library. The Toaster web-based UI is much more
capable and is actively maintained. See the
":ref:`toaster-manual/setup-and-use:using the toaster web interface`"
section in the Toaster User Manual for more information on this
interface.
- *"puccho" BitBake UI*: Removed because is unmaintained and no longer
useful.
.. _migration-2.1-adt-removed:
ADT Removed
-----------
The Application Development Toolkit (ADT) has been removed because its
functionality almost completely overlapped with the :ref:`standard
SDK <sdk-manual/using:using the standard sdk>` and the
:ref:`extensible SDK <sdk-manual/extensible:using the extensible sdk>`. For
information on these SDKs and how to build and use them, see the
:doc:`/sdk-manual/index` manual.
.. note::
The Yocto Project Eclipse IDE Plug-in is still supported and is not
affected by this change.
.. _migration-2.1-poky-reference-distribution-changes:
Poky Reference Distribution Changes
-----------------------------------
The following changes have been made for the Poky distribution:
- The ``meta-yocto`` layer has been renamed to ``meta-poky`` to better
match its purpose, which is to provide the Poky reference
distribution. The ``meta-yocto-bsp`` layer retains its original name
since it provides reference machines for the Yocto Project and it is
otherwise unrelated to Poky. References to ``meta-yocto`` in your
``conf/bblayers.conf`` should automatically be updated, so you should
not need to change anything unless you are relying on this naming
elsewhere.
- The :ref:`ref-classes-uninative` class is now enabled
by default in Poky. This class attempts to isolate the build system
from the host distribution's C library and makes re-use of native
shared state artifacts across different host distributions practical.
With this class enabled, a tarball containing a pre-built C library
is downloaded at the start of the build.
The :ref:`ref-classes-uninative` class is enabled through the
``meta/conf/distro/include/yocto-uninative.inc`` file, which for
those not using the Poky distribution, can include to easily enable
the same functionality.
Alternatively, if you wish to build your own ``uninative`` tarball,
you can do so by building the ``uninative-tarball`` recipe, making it
available to your build machines (e.g. over HTTP/HTTPS) and setting a
similar configuration as the one set by ``yocto-uninative.inc``.
- Static library generation, for most cases, is now disabled by default
in the Poky distribution. Disabling this generation saves some build
time as well as the size used for build output artifacts.
Disabling this library generation is accomplished through a
``meta/conf/distro/include/no-static-libs.inc``, which for those not
using the Poky distribution can easily include to enable the same
functionality.
Any recipe that needs to opt-out of having the ``--disable-static``
option specified on the configure command line either because it is
not a supported option for the configure script or because static
libraries are needed should set the following variable::
DISABLE_STATIC = ""
- The separate ``poky-tiny`` distribution now uses the musl C library
instead of a heavily pared down ``glibc``. Using musl results in a
smaller distribution and facilitates much greater maintainability
because musl is designed to have a small footprint.
If you have used ``poky-tiny`` and have customized the ``glibc``
configuration you will need to redo those customizations with musl
when upgrading to the new release.
.. _migration-2.1-packaging-changes:
Packaging Changes
-----------------
The following changes have been made to packaging:
- The ``runuser`` and ``mountpoint`` binaries, which were previously in
the main ``util-linux`` package, have been split out into the
``util-linux-runuser`` and ``util-linux-mountpoint`` packages,
respectively.
- The ``python-elementtree`` package has been merged into the
``python-xml`` package.
.. _migration-2.1-tuning-file-changes:
Tuning File Changes
-------------------
The following changes have been made to the tuning files:
- The "no-thumb-interwork" tuning feature has been dropped from the ARM
tune include files. Because interworking is required for ARM EABI,
attempting to disable it through a tuning feature no longer makes
sense.
.. note::
Support for ARM OABI was deprecated in gcc 4.7.
- The ``tune-cortexm*.inc`` and ``tune-cortexr4.inc`` files have been
removed because they are poorly tested. Until the OpenEmbedded build
system officially gains support for CPUs without an MMU, these tuning
files would probably be better maintained in a separate layer if
needed.
.. _migration-2.1-supporting-gobject-introspection:
Supporting GObject Introspection
--------------------------------
This release supports generation of GLib Introspective Repository (GIR)
files through GObject introspection, which is the standard mechanism for
accessing GObject-based software from runtime environments. You can
enable, disable, and test the generation of this data. See the
":ref:`dev-manual/gobject-introspection:enabling gobject introspection support`"
section in the Yocto Project Development Tasks Manual for more
information.
.. _migration-2.1-miscellaneous-changes:
Miscellaneous Changes
---------------------
These additional changes exist:
- The minimum Git version has been increased to 1.8.3.1. If your host
distribution does not provide a sufficiently recent version, you can
install the :term:`buildtools`, which will provide it. See the
:ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`
section for more information on the :term:`buildtools` tarball.
- The buggy and incomplete support for the RPM version 4 package
manager has been removed. The well-tested and maintained support for
RPM version 5 remains.
- Previously, the following list of packages were removed if
package-management was not in
:term:`IMAGE_FEATURES`, regardless of any
dependencies::
update-rc.d
base-passwd
shadow
update-alternatives
run-postinsts
With the Yocto Project 2.1 release, these packages are
only removed if "read-only-rootfs" is in :term:`IMAGE_FEATURES`, since
they might still be needed for a read-write image even in the absence
of a package manager (e.g. if users need to be added, modified, or
removed at runtime).
- The
:ref:`devtool modify <sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component>`
command now defaults to extracting the source since that is most
commonly expected. The ``-x`` or ``--extract`` options are now no-ops. If
you wish to provide your own existing source tree, you will now need
to specify either the ``-n`` or ``--no-extract`` options when running
``devtool modify``.
- If the formfactor for a machine is either not supplied or does not
specify whether a keyboard is attached, then the default is to assume
a keyboard is attached rather than assume no keyboard. This change
primarily affects the Sato UI.
- The ``.debug`` directory packaging is now automatic. If your recipe
builds software that installs binaries into directories other than
the standard ones, you no longer need to take care of setting
``FILES_${PN}-dbg`` to pick up the resulting ``.debug`` directories
as these directories are automatically found and added.
- Inaccurate disk and CPU percentage data has been dropped from
:ref:`ref-classes-buildstats` output. This data has been replaced with
``getrusage()`` data and corrected IO statistics. You will probably
need to update any custom code that reads the :ref:`ref-classes-buildstats` data.
- The ``meta/conf/distro/include/package_regex.inc`` is now deprecated.
The contents of this file have been moved to individual recipes.
.. note::
Because this file will likely be removed in a future Yocto Project
release, it is suggested that you remove any references to the
file that might be in your configuration.
- The ``v86d/uvesafb`` has been removed from the ``genericx86`` and
``genericx86-64`` reference machines, which are provided by the
``meta-yocto-bsp`` layer. Most modern x86 boards do not rely on this
file and it only adds kernel error messages during startup. If you do
still need to support ``uvesafb``, you can simply add ``v86d`` to
your image.
- Build sysroot paths are now removed from debug symbol files. Removing
these paths means that remote GDB using an unstripped build system
sysroot will no longer work (although this was never documented to
work). The supported method to accomplish something similar is to set
``IMAGE_GEN_DEBUGFS`` to "1", which will generate a companion debug
image containing unstripped binaries and associated debug sources
alongside the image.

456
sources/poky/documentation/migration-guides/migration-2.2.rst Normal file
View File

@@ -0,0 +1,456 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.2 (morty)
===================
This section provides migration information for moving to the Yocto
Project 2.2 Release (codename "morty") from the prior release.
.. _migration-2.2-minimum-kernel-version:
Minimum Kernel Version
----------------------
The minimum kernel version for the target system and for SDK is now
3.2.0, due to the upgrade to ``glibc 2.24``. Specifically, for
AArch64-based targets the version is 3.14. For Nios II-based targets,
the minimum kernel version is 3.19.
.. note::
For x86 and x86_64, you can reset :term:`OLDEST_KERNEL`
to anything down to 2.6.32 if desired.
.. _migration-2.2-staging-directories-in-sysroot-simplified:
Staging Directories in Sysroot Has Been Simplified
--------------------------------------------------
The way directories are staged in sysroot has been simplified and
introduces the new :term:`SYSROOT_DIRS`,
:term:`SYSROOT_DIRS_NATIVE`, and ``SYSROOT_DIRS_BLACKLIST``
(replaced by :term:`SYSROOT_DIRS_IGNORE` in version 4.0). See the
:oe_lists:`v2 patch series on the OE-Core Mailing List
</pipermail/openembedded-core/2016-May/121365.html>`
for additional information.
.. _migration-2.2-removal-of-old-images-from-tmp-deploy-now-enabled:
Removal of Old Images and Other Files in ``tmp/deploy`` Now Enabled
-------------------------------------------------------------------
Removal of old images and other files in ``tmp/deploy/`` is now enabled
by default due to a new staging method used for those files. As a result
of this change, the ``RM_OLD_IMAGE`` variable is now redundant.
.. _migration-2.2-python-changes:
Python Changes
--------------
The following changes for Python occurred:
.. _migration-2.2-bitbake-now-requires-python-3.4:
BitBake Now Requires Python 3.4+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake requires Python 3.4 or greater.
.. _migration-2.2-utf-8-locale-required-on-build-host:
UTF-8 Locale Required on Build Host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A UTF-8 locale is required on the build host due to Python 3. Since
C.UTF-8 is not a standard, the default is en_US.UTF-8.
.. _migration-2.2-metadata-now-must-use-python-3-syntax:
Metadata Must Now Use Python 3 Syntax
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The metadata is now required to use Python 3 syntax. For help preparing
metadata, see any of the many Python 3 porting guides available.
Alternatively, you can reference the conversion commits for BitBake and
you can use :term:`OpenEmbedded-Core (OE-Core)` as a guide for changes.
Particular areas of interest are:
- subprocess command-line pipes needing locale decoding
- the syntax for octal values changed
- the ``iter*()`` functions changed name
- iterators now return views, not lists
- changed names for Python modules
.. _migration-2.2-target-python-recipes-switched-to-python-3:
Target Python Recipes Switched to Python 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most target Python recipes have now been switched to Python 3.
Unfortunately, systems using RPM as a package manager and providing
online package-manager support through SMART still require Python 2.
.. note::
Python 2 and recipes that use it can still be built for the target as
with previous versions.
.. _migration-2.2-buildtools-tarball-includes-python-3:
``buildtools-tarball`` Includes Python 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The :term:`buildtools` tarball now includes Python 3.
.. _migration-2.2-uclibc-replaced-by-musl:
uClibc Replaced by musl
-----------------------
uClibc has been removed in favor of musl. Musl has matured, is better
maintained, and is compatible with a wider range of applications as
compared to uClibc.
.. _migration-2.2-B-no-longer-default-working-directory-for-tasks:
``${B}`` No Longer Default Working Directory for Tasks
------------------------------------------------------
``${``\ :term:`B`\ ``}`` is no longer the default working directory for tasks.
Consequently, any custom tasks you define now need to either have the
``[``\ :ref:`dirs <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]``
flag set, or the task needs to change into the appropriate working directory
manually (e.g using ``cd`` for a shell task).
.. note::
The preferred method is to use the
[dirs]
flag.
.. _migration-2.2-runqemu-ported-to-python:
``runqemu`` Ported to Python
----------------------------
``runqemu`` has been ported to Python and has changed behavior in some
cases. Previous usage patterns continue to be supported.
The new ``runqemu`` is a Python script. Machine knowledge is no longer
hardcoded into ``runqemu``. You can choose to use the ``qemuboot``
configuration file to define the BSP's own arguments and to make it
bootable with ``runqemu``. If you use a configuration file, use the
following form::
image-name-machine.qemuboot.conf
The configuration file
enables fine-grained tuning of options passed to QEMU without the
``runqemu`` script hard-coding any knowledge about different machines.
Using a configuration file is particularly convenient when trying to use
QEMU with machines other than the ``qemu*`` machines in
:term:`OpenEmbedded-Core (OE-Core)`. The ``qemuboot.conf`` file is generated by the
``qemuboot`` class when the root filesystem is being built (i.e. build
rootfs). QEMU boot arguments can be set in BSP's configuration file and
the ``qemuboot`` class will save them to ``qemuboot.conf``.
If you want to use ``runqemu`` without a configuration file, use the
following command form::
$ runqemu machine rootfs kernel [options]
Supported machines are as follows:
- qemuarm
- qemuarm64
- qemux86
- qemux86-64
- qemuppc
- qemumips
- qemumips64
- qemumipsel
- qemumips64el
Consider the
following example, which uses the ``qemux86-64`` machine, provides a
root filesystem, provides an image, and uses the ``nographic`` option::
$ runqemu qemux86-64 tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4 tmp/deploy/images/qemux86-64/bzImage nographic
Here is a list of variables that can be set in configuration files
such as ``bsp.conf`` to enable the BSP to be booted by ``runqemu``::
QB_SYSTEM_NAME: QEMU name (e.g. "qemu-system-i386")
QB_OPT_APPEND: Options to append to QEMU (e.g. "-show-cursor")
QB_DEFAULT_KERNEL: Default kernel to boot (e.g. "bzImage")
QB_DEFAULT_FSTYPE: Default FSTYPE to boot (e.g. "ext4")
QB_MEM: Memory (e.g. "-m 512")
QB_MACHINE: QEMU machine (e.g. "-machine virt")
QB_CPU: QEMU cpu (e.g. "-cpu qemu32")
QB_CPU_KVM: Similar to QB_CPU except used for kvm support (e.g. "-cpu kvm64")
QB_KERNEL_CMDLINE_APPEND: Options to append to the kernel's -append
option (e.g. "console=ttyS0 console=tty")
QB_DTB: QEMU dtb name
QB_AUDIO_DRV: QEMU audio driver (e.g. "alsa", set it when support audio)
QB_AUDIO_OPT: QEMU audio option (e.g. "-soundhw ac97,es1370"), which is used
when QB_AUDIO_DRV is set.
QB_KERNEL_ROOT: Kernel's root (e.g. /dev/vda)
QB_TAP_OPT: Network option for 'tap' mode (e.g.
"-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device virtio-net-device,netdev=net0").
runqemu will replace "@TAP@" with the one that is used, such as tap0, tap1 ...
QB_SLIRP_OPT: Network option for SLIRP mode (e.g. "-netdev user,id=net0 -device virtio-net-device,netdev=net0")
QB_ROOTFS_OPT: Used as rootfs (e.g.
"-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device virtio-blk-device,drive=disk0").
runqemu will replace "@ROOTFS@" with the one which is used, such as
core-image-minimal-qemuarm64.ext4.
QB_SERIAL_OPT: Serial port (e.g. "-serial mon:stdio")
QB_TCPSERIAL_OPT: tcp serial port option (e.g.
" -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device virtconsole,chardev=virtcon"
runqemu will replace "@PORT@" with the port number which is used.
To use ``runqemu``, set :term:`IMAGE_CLASSES` as
follows and run ``runqemu``:
.. note::
"QB" means "QEMU Boot".
.. note::
For command-line syntax, use ``runqemu help``.
::
IMAGE_CLASSES += "qemuboot"
.. _migration-2.2-default-linker-hash-style-changed:
Default Linker Hash Style Changed
---------------------------------
The default linker hash style for ``gcc-cross`` is now "sysv" in order
to catch recipes that are building software without using the
OpenEmbedded :term:`LDFLAGS`. This change could result in
seeing some "No GNU_HASH in the elf binary" QA issues when building such
recipes. You need to fix these recipes so that they use the expected
:term:`LDFLAGS`. Depending on how the software is built, the build system
used by the software (e.g. a Makefile) might need to be patched.
However, sometimes making this fix is as simple as adding the following
to the recipe::
TARGET_CC_ARCH += "${LDFLAGS}"
.. _migration-2.2-kernel-image-base-name-no-longer-uses-kernel-imagetype:
``KERNEL_IMAGE_BASE_NAME`` no Longer Uses ``KERNEL_IMAGETYPE``
--------------------------------------------------------------
The ``KERNEL_IMAGE_BASE_NAME`` variable no longer uses the
:term:`KERNEL_IMAGETYPE` variable to create the
image's base name. Because the OpenEmbedded build system can now build
multiple kernel image types, this part of the kernel image base name as
been removed leaving only the following::
KERNEL_IMAGE_BASE_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}-${DATETIME}"
If you have recipes or
classes that use ``KERNEL_IMAGE_BASE_NAME`` directly, you might need to
update the references to ensure they continue to work.
.. _migration-2.2-imgdeploydir-replaces-deploy-dir-image-for-most-use-cases:
``IMGDEPLOYDIR`` Replaces ``DEPLOY_DIR_IMAGE`` for Most Use Cases
-----------------------------------------------------------------
The :term:`IMGDEPLOYDIR` variable was introduced to allow sstate caching of
image creation results. Image recipes defining custom :term:`IMAGE_CMD` or
doing postprocessing on the generated images need to be adapted to use
:term:`IMGDEPLOYDIR` instead of :term:`DEPLOY_DIR_IMAGE`. :term:`IMAGE_MANIFEST`
creation and symlinking of the most recent image file will fail otherwise.
.. _migration-2.2-bitbake-changes:
BitBake Changes
---------------
The following changes took place for BitBake:
- The "goggle" UI and standalone image-writer tool have been removed as
they both require GTK+ 2.0 and were not being maintained.
- The Perforce fetcher now supports :term:`SRCREV` for
specifying the source revision to use, be it
``${``\ :term:`AUTOREV`\ ``}``, changelist number,
p4date, or label, in preference to separate
:term:`SRC_URI` parameters to specify these. This
change is more in-line with how the other fetchers work for source
control systems. Recipes that fetch from Perforce will need to be
updated to use :term:`SRCREV` in place of specifying the source revision
within :term:`SRC_URI`.
- Some of BitBake's internal code structures for accessing the recipe
cache needed to be changed to support the new multi-configuration
functionality. These changes will affect external tools that use
BitBake's tinfoil module. For information on these changes, see the
changes made to the scripts supplied with OpenEmbedded-Core:
:yocto_git:`1 </poky/commit/?id=189371f8393971d00bca0fceffd67cc07784f6ee>`
and
:yocto_git:`2 </poky/commit/?id=4a5aa7ea4d07c2c90a1654b174873abb018acc67>`.
- The task management code has been rewritten to avoid using ID
indirection in order to improve performance. This change is unlikely
to cause any problems for most users. However, the setscene
verification function as pointed to by
``BB_SETSCENE_VERIFY_FUNCTION`` needed to change signature.
Consequently, a new variable named ``BB_SETSCENE_VERIFY_FUNCTION2``
has been added allowing multiple versions of BitBake to work with
suitably written metadata, which includes OpenEmbedded-Core and Poky.
Anyone with custom BitBake task scheduler code might also need to
update the code to handle the new structure.
.. _migration-2.2-swabber-has-been-removed:
Swabber has Been Removed
------------------------
Swabber, a tool that was intended to detect host contamination in the
build process, has been removed, as it has been unmaintained and unused
for some time and was never particularly effective. The OpenEmbedded
build system has since incorporated a number of mechanisms including
enhanced QA checks that mean that there is less of a need for such a
tool.
.. _migration-2.2-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``augeas``: No longer needed and has been moved to ``meta-oe``.
- ``directfb``: Unmaintained and has been moved to ``meta-oe``.
- ``gcc``: Removed 4.9 version. Versions 5.4 and 6.2 are still present.
- ``gnome-doc-utils``: No longer needed.
- ``gtk-doc-stub``: Replaced by ``gtk-doc``.
- ``gtk-engines``: No longer needed and has been moved to
``meta-gnome``.
- ``gtk-sato-engine``: Became obsolete.
- ``libglade``: No longer needed and has been moved to ``meta-oe``.
- ``libmad``: Unmaintained and functionally replaced by ``libmpg123``.
``libmad`` has been moved to ``meta-oe``.
- ``libowl``: Became obsolete.
- ``libxsettings-client``: No longer needed.
- ``oh-puzzles``: Functionally replaced by ``puzzles``.
- ``oprofileui``: Became obsolete. OProfile has been largely supplanted
by perf.
- ``packagegroup-core-directfb.bb``: Removed.
- ``core-image-directfb.bb``: Removed.
- ``pointercal``: No longer needed and has been moved to ``meta-oe``.
- ``python-imaging``: No longer needed and moved to ``meta-python``
- ``python-pyrex``: No longer needed and moved to ``meta-python``.
- ``sato-icon-theme``: Became obsolete.
- ``swabber-native``: Swabber has been removed. See the :ref:`entry on
Swabber <migration-guides/migration-2.2:swabber has been removed>`.
- ``tslib``: No longer needed and has been moved to ``meta-oe``.
- ``uclibc``: Removed in favor of musl.
- ``xtscal``: No longer needed and moved to ``meta-oe``
.. _migration-2.2-removed-classes:
Removed Classes
---------------
The following classes have been removed:
- ``distutils-native-base``: No longer needed.
- ``distutils3-native-base``: No longer needed.
- ``sdl``: Only set :term:`DEPENDS` and
:term:`SECTION`, which are better set within the
recipe instead.
- ``sip``: Mostly unused.
- ``swabber``: See the :ref:`entry on
Swabber <migration-guides/migration-2.2:swabber has been removed>`.
.. _migration-2.2-minor-packaging-changes:
Minor Packaging Changes
-----------------------
The following minor packaging changes have occurred:
- ``grub``: Split ``grub-editenv`` into its own package.
- ``systemd``: Split container and vm related units into a new package,
systemd-container.
- ``util-linux``: Moved ``prlimit`` to a separate
``util-linux-prlimit`` package.
.. _migration-2.2-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- ``package_regex.inc``: Removed because the definitions
``package_regex.inc`` previously contained have been moved to their
respective recipes.
- Both ``devtool add`` and ``recipetool create`` now use a fixed
:term:`SRCREV` by default when fetching from a Git
repository. You can override this in either case to use
``${``\ :term:`AUTOREV`\ ``}`` instead by using the
``-a`` or ``--autorev`` command-line option
- ``distcc``: GTK+ UI is now disabled by default.
- ``packagegroup-core-tools-testapps``: Removed Piglit.
- :ref:`ref-classes-image`: Renamed COMPRESS(ION) to CONVERSION. This change
means that ``COMPRESSIONTYPES``, ``COMPRESS_DEPENDS`` and
``COMPRESS_CMD`` are deprecated in favor of ``CONVERSIONTYPES``,
``CONVERSION_DEPENDS`` and :term:`CONVERSION_CMD`. The ``COMPRESS*``
variable names will still work in the 2.2 release but metadata that
does not need to be backwards-compatible should be changed to use the
new names as the ``COMPRESS*`` ones will be removed in a future
release.
- ``gtk-doc``: A full version of ``gtk-doc`` is now made available.
However, some old software might not be capable of using the current
version of ``gtk-doc`` to build documentation. You need to change
recipes that build such software so that they explicitly disable
building documentation with ``gtk-doc``.

515
sources/poky/documentation/migration-guides/migration-2.3.rst Normal file
View File

@@ -0,0 +1,515 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.3 (pyro)
==================
This section provides migration information for moving to the Yocto
Project 2.3 Release (codename "pyro") from the prior release.
.. _migration-2.3-recipe-specific-sysroots:
Recipe-specific Sysroots
------------------------
The OpenEmbedded build system now uses one sysroot per recipe to resolve
long-standing issues with configuration script auto-detection of
undeclared dependencies. Consequently, you might find that some of your
previously written custom recipes are missing declared dependencies,
particularly those dependencies that are incidentally built earlier in a
typical build process and thus are already likely to be present in the
shared sysroot in previous releases.
Consider the following:
- *Declare Build-Time Dependencies:* Because of this new feature, you
must explicitly declare all build-time dependencies for your recipe.
If you do not declare these dependencies, they are not populated into
the sysroot for the recipe.
- *Specify Pre-Installation and Post-Installation Native Tool
Dependencies:* You must specifically specify any special native tool
dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
the :term:`PACKAGE_WRITE_DEPS` variable.
Specifying these dependencies ensures that these tools are available
if these scripts need to be run on the build host during the
:ref:`ref-tasks-rootfs` task.
As an example, see the ``dbus`` recipe. You will see that this recipe
has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
:term:`DISTRO_FEATURES`. In the example,
``systemd-systemctl-native`` is added to :term:`PACKAGE_WRITE_DEPS`,
which is also conditional on "systemd" being in :term:`DISTRO_FEATURES`.
- Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``: You need to
examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
steps to take.
Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
were in previous Yocto Project releases. However, since a separate
sysroot is now being populated for every recipe and if existing
functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
relocation, then you will need to change these to use a
post-installation script that is installed by a function added to
:term:`SYSROOT_PREPROCESS_FUNCS`.
For an example, see the :ref:`ref-classes-pixbufcache` class in ``meta/classes/`` in
the :ref:`overview-manual/development-environment:yocto project source repositories`.
.. note::
The
SSTATEPOSTINSTFUNCS
variable itself is now deprecated in favor of the
do_populate_sysroot[postfuncs]
task. Consequently, if you do still have any function or functions
that need to be called after the sysroot component is created for
a recipe, then you would be well advised to take steps to use a
post installation script as described previously. Taking these
steps prepares your code for when
SSTATEPOSTINSTFUNCS
is removed in a future Yocto Project release.
- *Specify the Sysroot when Using Certain External Scripts:* Because
the shared sysroot is now gone, the scripts
``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
such that you need to specify which recipe's
:term:`STAGING_DIR_NATIVE` is used.
.. note::
You can find more information on how recipe-specific sysroots work in
the ":ref:`ref-classes-staging`" section.
.. _migration-2.3-path-variable:
``PATH`` Variable
-----------------
Within the environment used to run build tasks, the environment variable
``PATH`` is now sanitized such that the normal native binary paths
(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
directory containing symbolic links linking only to the binaries from
the host mentioned in the :term:`HOSTTOOLS` and
:term:`HOSTTOOLS_NONFATAL` variables is added
to ``PATH``.
Consequently, any native binaries provided by the host that you need to
call needs to be in one of these two variables at the configuration
level.
Alternatively, you can add a native recipe (i.e. ``-native``) that
provides the binary to the recipe's :term:`DEPENDS`
value.
.. note::
PATH
is not sanitized in the same way within ``devshell``.
If it were, you would have difficulty running host tools for
development and debugging within the shell.
.. _migration-2.3-scripts:
Changes to Scripts
------------------
The following changes to scripts took place:
- ``oe-find-native-sysroot``: The usage for the
``oe-find-native-sysroot`` script has changed to the following::
$ . oe-find-native-sysroot recipe
You must now supply a recipe for recipe
as part of the command. Prior to the Yocto Project 2.3 release, it
was not necessary to provide the script with the command.
- ``oe-run-native``: The usage for the ``oe-run-native`` script has
changed to the following::
$ oe-run-native native_recipe tool
You must
supply the name of the native recipe and the tool you want to run as
part of the command. Prior to the Yocto Project 2.3 release, it
was not necessary to provide the native recipe with the command.
- ``cleanup-workdir``: The ``cleanup-workdir`` script has been
removed because the script was found to be deleting files it should
not have, which lead to broken build trees. Rather than trying to
delete portions of :term:`TMPDIR` and getting it wrong,
it is recommended that you delete :term:`TMPDIR` and have it restored
from shared state (sstate) on subsequent builds.
- ``wipe-sysroot``: The ``wipe-sysroot`` script has been removed as
it is no longer needed with recipe-specific sysroots.
.. _migration-2.3-functions:
Changes to Functions
--------------------
The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
and related functions have been removed in favor of ``d.getVar()``,
``d.setVar()``, and so forth.
You need to fix any references to these old functions.
.. _migration-2.3-bitbake-changes:
BitBake Changes
---------------
The following changes took place for BitBake:
- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
graphical dependency explorer UI ``depexp`` was replaced by
``taskexp`` ("Task Explorer"), which provides a graphical way of
exploring the ``task-depends.dot`` file. The data presented by Task
Explorer is much more accurate than the data that was presented by
``depexp``. Being able to visualize the data is an often requested
feature as standard ``*.dot`` file viewers cannot usual cope with the
size of the ``task-depends.dot`` file.
- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
``pn-depends.dot`` files as previously generated using the
``bitbake -g`` command have been removed. A ``recipe-depends.dot``
file is now generated as a collapsed version of ``task-depends.dot``
instead.
The reason for this change is because ``package-depends.dot`` and
``pn-depends.dot`` largely date back to a time before task-based
execution and do not take into account task-level dependencies
between recipes, which could be misleading.
- *Mirror Variable Splitting Changes:* Mirror variables including
:term:`MIRRORS`, :term:`PREMIRRORS`,
and :term:`SSTATE_MIRRORS` can now separate
values entirely with spaces. Consequently, you no longer need "\\n".
BitBake looks for pairs of values, which simplifies usage. There
should be no change required to existing mirror variable values
themselves.
- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
"rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
instead of an "rsh" parameter. This new optional parameter is used
when the "protocol" parameter is set to "svn+ssh". You can only use
the new parameter to specify the ``ssh`` program used by SVN. The SVN
fetcher passes the new parameter through the ``SVN_SSH`` environment
variable during the :ref:`ref-tasks-fetch` task.
See the
":ref:`bitbake-user-manual/bitbake-user-manual-fetching:subversion (svn) fetcher (\`\`svn://\`\`)`"
section in the BitBake User Manual for additional information.
- ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
Removed: Because the mechanism they were part of is no longer
necessary with recipe-specific sysroots, the
``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
variables have been removed.
.. _migration-2.3-absolute-symlinks:
Absolute Symbolic Links
-----------------------
Absolute symbolic links (symlinks) within staged files are no longer
permitted and now trigger an error. Any explicit creation of symlinks
can use the ``lnr`` script, which is a replacement for ``ln -r``.
If the build scripts in the software that the recipe is building are
creating a number of absolute symlinks that need to be corrected, you
can inherit ``relative_symlinks`` within the recipe to turn those
absolute symlinks into relative symlinks.
.. _migration-2.3-gplv2-and-gplv3-moves:
GPLv2 Versions of GPLv3 Recipes Moved
-------------------------------------
Older GPLv2 versions of GPLv3 recipes have moved to a separate
``meta-gplv2`` layer.
If you use :term:`INCOMPATIBLE_LICENSE` to
exclude GPLv3 or set :term:`PREFERRED_VERSION`
to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
``meta-gplv2`` layer to your configuration.
.. note::
You can ``find meta-gplv2`` layer in the OpenEmbedded layer index at
:oe_layer:`/meta-gplv2`.
These relocated GPLv2 recipes do not receive the same level of
maintenance as other core recipes. The recipes do not get security fixes
and upstream no longer maintains them. In fact, the upstream community
is actively hostile towards people that use the old versions of the
recipes. Moving these recipes into a separate layer both makes the
different needs of the recipes clearer and clearly identifies the number
of these recipes.
.. note::
The long-term solution might be to move to BSD-licensed replacements
of the GPLv3 components for those that need to exclude GPLv3-licensed
components from the target system. This solution will be investigated
for future Yocto Project releases.
.. _migration-2.3-package-management-changes:
Package Management Changes
--------------------------
The following package management changes took place:
- Smart package manager is replaced by DNF package manager. Smart has
become unmaintained upstream, is not ported to Python 3.x.
Consequently, Smart needed to be replaced. DNF is the only feasible
candidate.
The change in functionality is that the on-target runtime package
management from remote package feeds is now done with a different
tool that has a different set of command-line options. If you have
scripts that call the tool directly, or use its API, they need to be
fixed.
For more information, see the `DNF
Documentation <https://dnf.readthedocs.io/en/latest/>`__.
- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
- DNF is API-incompatible with Rpm 5.x and porting it and
maintaining the port is non-trivial.
- Rpm 5.x itself has limited maintenance upstream, and the Yocto
Project is one of the very few remaining users.
- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
- Version 6.x of Berkeley DB has largely been rejected by the open
source community due to its AGPLv3 license. As a result, most
mainstream open source projects that require DB are still
developed and tested with DB 5.x.
- In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
Thus, no reason exists to continue carrying DB 6.x in OE-core.
- ``createrepo`` is replaced with ``createrepo_c``.
``createrepo_c`` is the current incarnation of the tool that
generates remote repository metadata. It is written in C as compared
to ``createrepo``, which is written in Python. ``createrepo_c`` is
faster and is maintained.
- Architecture-independent RPM packages are "noarch" instead of "all".
This change was made because too many places in DNF/RPM4 stack
already make that assumption. Only the filenames and the architecture
tag has changed. Nothing else has changed in OE-core system,
particularly in the :ref:`ref-classes-allarch` class.
- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
currently supported. This issue will be fully addressed in a future
Yocto Project release. See :yocto_bugs:`defect 11209 </show_bug.cgi?id=11209>`
for more information on a solution to package feed signing with RPM
in the Yocto Project 2.3 release.
- OPKG now uses the libsolv backend for resolving package dependencies
by default. This is vastly superior to OPKG's internal ad-hoc solver
that was previously used. This change does have a small impact on
disk (around 500 KB) and memory footprint.
.. note::
For further details on this change, see the
:yocto_git:`commit message </poky/commit/?id=f4d4f99cfbc2396e49c1613a7d237b9e57f06f81>`.
.. _migration-2.3-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``linux-yocto 4.8``: Version 4.8 has been removed. Versions 4.1
(LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
- ``python-smartpm``: Functionally replaced by ``dnf``.
- ``createrepo``: Replaced by the ``createrepo-c`` recipe.
- ``rpmresolve``: No longer needed with the move to RPM 4 as RPM
itself is used instead.
- ``gstreamer``: Removed the GStreamer Git version recipes as they
have been stale. ``1.10.``\ x recipes are still present.
- ``alsa-conf-base``: Merged into ``alsa-conf`` since ``libasound``
depended on both. Essentially, no way existed to install only one of
these.
- ``tremor``: Moved to ``meta-multimedia``. Fixed-integer Vorbis
decoding is not needed by current hardware. Thus, GStreamer's ivorbis
plugin has been disabled by default eliminating the need for the
``tremor`` recipe in :term:`OpenEmbedded-Core (OE-Core)`.
- ``gummiboot``: Replaced by ``systemd-boot``.
.. _migration-2.3-wic-changes:
Wic Changes
-----------
The following changes have been made to Wic:
.. note::
For more information on Wic, see the
":ref:`dev-manual/wic:creating partitioned images using wic`"
section in the Yocto Project Development Tasks Manual.
- *Default Output Directory Changed:* Wic's default output directory is
now the current directory by default instead of the unusual
``/var/tmp/wic``.
The ``-o`` and ``--outdir`` options remain unchanged and are used to
specify your preferred output directory if you do not want to use the
default directory.
- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
it duplicates functionality of the rawcopy plugin.
.. _migration-2.3-qa-changes:
QA Changes
----------
The following QA checks have changed:
- ``unsafe-references-in-binaries``: The
``unsafe-references-in-binaries`` QA check, which was disabled by
default, has now been removed. This check was intended to detect
binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
the case where the user has ``/usr`` on a separate filesystem to
``/``.
The removed QA check was buggy. Additionally, ``/usr`` residing on a
separate partition from ``/`` is now a rare configuration.
Consequently, ``unsafe-references-in-binaries`` was removed.
- ``file-rdeps``: The ``file-rdeps`` QA check is now an error by
default instead of a warning. Because it is an error instead of a
warning, you need to address missing runtime dependencies.
For additional information, see the
:ref:`ref-classes-insane` class and the
":ref:`ref-manual/qa-checks:errors and warnings`" section.
.. _migration-2.3-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- In this release, a number of recipes have been changed to ignore the
``largefile`` :term:`DISTRO_FEATURES` item,
enabling large file support unconditionally. This feature has always
been enabled by default. Disabling the feature has not been widely
tested.
.. note::
Future releases of the Yocto Project will remove entirely the
ability to disable the
largefile
feature, which would make it unconditionally enabled everywhere.
- If the :term:`DISTRO_VERSION` value contains
the value of the :term:`DATE` variable, which is the
default between Poky releases, the :term:`DATE` value is explicitly
excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
displayed at the login prompt, in order to avoid conflicts with
Multilib enabled. Regardless, the :term:`DATE` value is inaccurate if the
``base-files`` recipe is restored from shared state (sstate) rather
than rebuilt.
If you need the build date recorded in ``/etc/issue*`` or anywhere
else in your image, a better method is to define a post-processing
function to do it and have the function called from
:term:`ROOTFS_POSTPROCESS_COMMAND`.
Doing so ensures the value is always up-to-date with the created
image.
- Dropbear's ``init`` script now disables DSA host keys by default.
This change is in line with the systemd service file, which supports
RSA keys only, and with recent versions of OpenSSH, which deprecates
DSA host keys.
- The :ref:`ref-classes-buildhistory` class now
correctly uses tabs as separators between all columns in
``installed-package-sizes.txt`` in order to aid import into other
tools.
- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
:term:`DISTRO_FEATURES` feature. Distributions that previously set::
USE_LDCONFIG = "0"
should now instead use the following::
DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
- The default value of
:term:`COPYLEFT_LICENSE_INCLUDE` now
includes all versions of AGPL licenses in addition to GPL and LGPL.
.. note::
The default list is not intended to be guaranteed as a complete
safe list. You should seek legal advice based on what you are
distributing if you are unsure.
- Kernel module packages are now suffixed with the kernel version in
order to allow module packages from multiple kernel versions to
co-exist on a target system. If you wish to return to the previous
naming scheme that does not include the version suffix, use the
following::
KERNEL_MODULE_PACKAGE_SUFFIX = ""
- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
``*.la`` files are not actually needed on Linux and relocating them
is an unnecessary burden.
If you need to preserve these ``.la`` files (e.g. in a custom
distribution), you must change :term:`INHERIT_DISTRO` such that
":ref:`ref-classes-remove-libtool`" is not included
in the value.
- Extensible SDKs built for GCC 5+ now refuse to install on a
distribution where the host GCC version is 4.8 or 4.9. This change
resulted from the fact that the installation is known to fail due to
the way the ``uninative`` shared state (sstate) package is built. See
the :ref:`ref-classes-uninative` class for additional information.
- All :ref:`ref-classes-native` and :ref:`ref-classes-nativesdk` recipes now
use a separate :term:`DISTRO_FEATURES` value instead of sharing the value
used by recipes for the target, in order to avoid unnecessary rebuilds.
The :term:`DISTRO_FEATURES` for :ref:`ref-classes-native` recipes
is :term:`DISTRO_FEATURES_NATIVE` added to an intersection of
:term:`DISTRO_FEATURES` and :term:`DISTRO_FEATURES_FILTER_NATIVE`.
For :ref:`ref-classes-nativesdk` recipes, the corresponding
variables are :term:`DISTRO_FEATURES_NATIVESDK` and
:term:`DISTRO_FEATURES_FILTER_NATIVESDK`.
- The ``FILESDIR`` variable, which was previously deprecated and rarely
used, has now been removed. You should change any recipes that set
``FILESDIR`` to set :term:`FILESPATH` instead.
- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
longer needed with recipe-specific sysroots.

319
sources/poky/documentation/migration-guides/migration-2.4.rst Normal file
View File

@@ -0,0 +1,319 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.4 (rocko)
===================
This section provides migration information for moving to the Yocto
Project 2.4 Release (codename "rocko") from the prior release.
.. _migration-2.4-memory-resident-mode:
Memory Resident Mode
--------------------
A persistent mode is now available in BitBake's default operation,
replacing its previous "memory resident mode" (i.e.
``oe-init-build-env-memres``). Now you only need to set
:term:`BB_SERVER_TIMEOUT` to a timeout (in
seconds) and BitBake's server stays resident for that amount of time
between invocations. The ``oe-init-build-env-memres`` script has been
removed since a separate environment setup script is no longer needed.
.. _migration-2.4-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes that have
occurred:
- ``python3`` Changes:
- The main "python3" package now brings in all of the standard
Python 3 distribution rather than a subset. This behavior matches
what is expected based on traditional Linux distributions. If you
wish to install a subset of Python 3, specify ``python-core`` plus
one or more of the individual packages that are still produced.
- ``python3``: The ``bz2.py``, ``lzma.py``, and
``_compression.py`` scripts have been moved from the
``python3-misc`` package to the ``python3-compression`` package.
- ``binutils``: The ``libbfd`` library is now packaged in a separate
"libbfd" package. This packaging saves space when certain tools (e.g.
``perf``) are installed. In such cases, the tools only need
``libbfd`` rather than all the packages in ``binutils``.
- ``util-linux`` Changes:
- The ``su`` program is now packaged in a separate "util-linux-su"
package, which is only built when "pam" is listed in the
:term:`DISTRO_FEATURES` variable.
``util-linux`` should not be installed unless it is needed because
``su`` is normally provided through the shadow file format. The
main ``util-linux`` package has runtime dependencies (i.e.
:term:`RDEPENDS`) on the ``util-linux-su`` package
when "pam" is in :term:`DISTRO_FEATURES`.
- The ``switch_root`` program is now packaged in a separate
"util-linux-switch-root" package for small :term:`Initramfs` images that
do not need the whole ``util-linux`` package or the busybox
binary, which are both much larger than ``switch_root``. The main
``util-linux`` package has a recommended runtime dependency (i.e.
:term:`RRECOMMENDS`) on the
``util-linux-switch-root`` package.
- The ``ionice`` program is now packaged in a separate
"util-linux-ionice" package. The main ``util-linux`` package has a
recommended runtime dependency (i.e. :term:`RRECOMMENDS`) on the
``util-linux-ionice`` package.
- ``initscripts``: The ``sushell`` program is now packaged in a
separate "initscripts-sushell" package. This packaging change allows
systems to pull ``sushell`` in when ``selinux`` is enabled. The
change also eliminates needing to pull in the entire ``initscripts``
package. The main ``initscripts`` package has a runtime dependency
(i.e. :term:`RDEPENDS`) on the ``sushell`` package when "selinux" is in
:term:`DISTRO_FEATURES`.
- ``glib-2.0``: The ``glib-2.0`` package now has a recommended
runtime dependency (i.e. :term:`RRECOMMENDS`) on the ``shared-mime-info``
package, since large portions of GIO are not useful without the MIME
database. You can remove the dependency by using the
:term:`BAD_RECOMMENDATIONS` variable if
``shared-mime-info`` is too large and is not required.
- *Go Standard Runtime:* The Go standard runtime has been split out
from the main ``go`` recipe into a separate ``go-runtime`` recipe.
.. _migration-2.4-removed-recipes:
Removed Recipes
---------------
- ``acpitests``: This recipe is not maintained.
- ``autogen-native``: No longer required by Grub, oe-core, or
meta-oe.
- ``bdwgc``: Nothing in OpenEmbedded-Core requires this recipe. It
has moved to meta-oe.
- ``byacc``: This recipe was only needed by rpm 5.x and has moved to
meta-oe.
- ``gcc (5.4)``: The 5.4 series dropped the recipe in favor of 6.3 /
7.2.
- ``gnome-common``: Deprecated upstream and no longer needed.
- ``go-bootstrap-native``: Go 1.9 does its own bootstrapping so this
recipe has been removed.
- ``guile``: This recipe was only needed by ``autogen-native`` and
``remake``. The recipe is no longer needed by either of these
programs.
- ``libclass-isa-perl``: This recipe was previously needed for LSB 4,
no longer needed.
- ``libdumpvalue-perl``: This recipe was previously needed for LSB 4,
no longer needed.
- ``libenv-perl``: This recipe was previously needed for LSB 4, no
longer needed.
- ``libfile-checktree-perl``: This recipe was previously needed for
LSB 4, no longer needed.
- ``libi18n-collate-perl``: This recipe was previously needed for LSB
4, no longer needed.
- ``libiconv``: This recipe was only needed for ``uclibc``, which was
removed in the previous release. ``glibc`` and ``musl`` have their
own implementations. ``meta-mingw`` still needs ``libiconv``, so it
has been moved to ``meta-mingw``.
- ``libpng12``: This recipe was previously needed for LSB. The
current ``libpng`` is 1.6.x.
- ``libpod-plainer-perl``: This recipe was previously needed for LSB
4, no longer needed.
- ``linux-yocto (4.1)``: This recipe was removed in favor of 4.4,
4.9, 4.10 and 4.12.
- ``mailx``: This recipe was previously only needed for LSB
compatibility, and upstream is defunct.
- ``mesa (git version only)``: The git version recipe was stale with
respect to the release version.
- ``ofono (git version only)``: The git version recipe was stale with
respect to the release version.
- ``portmap``: This recipe is obsolete and is superseded by
``rpcbind``.
- ``python3-pygpgme``: This recipe is old and unmaintained. It was
previously required by ``dnf``, which has switched to official
``gpgme`` Python bindings.
- ``python-async``: This recipe has been removed in favor of the
Python 3 version.
- ``python-gitdb``: This recipe has been removed in favor of the
Python 3 version.
- ``python-git``: This recipe was removed in favor of the Python 3
version.
- ``python-mako``: This recipe was removed in favor of the Python 3
version.
- ``python-pexpect``: This recipe was removed in favor of the Python
3 version.
- ``python-ptyprocess``: This recipe was removed in favor of Python
the 3 version.
- ``python-pycurl``: Nothing is using this recipe in
OpenEmbedded-Core (i.e. ``meta-oe``).
- ``python-six``: This recipe was removed in favor of the Python 3
version.
- ``python-smmap``: This recipe was removed in favor of the Python 3
version.
- ``remake``: Using ``remake`` as the provider of ``virtual/make`` is
broken. Consequently, this recipe is not needed in OpenEmbedded-Core.
.. _migration-2.4-kernel-device-tree-move:
Kernel Device Tree Move
-----------------------
Kernel Device Tree support is now easier to enable in a kernel recipe.
The Device Tree code has moved to a :ref:`ref-classes-kernel-devicetree` class.
Functionality is automatically enabled for any recipe that inherits the
:ref:`kernel <ref-classes-kernel>` class and sets the :term:`KERNEL_DEVICETREE`
variable. The previous mechanism for doing this,
``meta/recipes-kernel/linux/linux-dtb.inc``, is still available to avoid
breakage, but triggers a deprecation warning. Future releases of the
Yocto Project will remove ``meta/recipes-kernel/linux/linux-dtb.inc``.
It is advisable to remove any ``require`` statements that request
``meta/recipes-kernel/linux/linux-dtb.inc`` from any custom kernel
recipes you might have. This will avoid breakage in post 2.4 releases.
.. _migration-2.4-package-qa-changes:
Package QA Changes
------------------
- The "unsafe-references-in-scripts" QA check has been removed.
- If you refer to ``${COREBASE}/LICENSE`` within
:term:`LIC_FILES_CHKSUM` you receive a
warning because this file is a description of the license for
OE-Core. Use ``${COMMON_LICENSE_DIR}/MIT`` if your recipe is
MIT-licensed and you cannot use the preferred method of referring to
a file within the source tree.
.. _migration-2.4-readme-changes:
``README`` File Changes
-----------------------
- The main Poky ``README`` file has been moved to the ``meta-poky``
layer and has been renamed ``README.poky``. A symlink has been
created so that references to the old location work.
- The ``README.hardware`` file has been moved to ``meta-yocto-bsp``. A
symlink has been created so that references to the old location work.
- A ``README.qemu`` file has been created with coverage of the
``qemu*`` machines.
.. _migration-2.4-miscellaneous-changes:
Miscellaneous Changes
---------------------
- The ``ROOTFS_PKGMANAGE_BOOTSTRAP`` variable and any references to it
have been removed. You should remove this variable from any custom
recipes.
- The ``meta-yocto`` directory has been removed.
.. note::
In the Yocto Project 2.1 release
meta-yocto
was renamed to
meta-poky
and the
meta-yocto
subdirectory remained to avoid breaking existing configurations.
- The ``maintainers.inc`` file, which tracks maintainers by listing a
primary person responsible for each recipe in OE-Core, has been moved
from ``meta-poky`` to OE-Core (i.e. from
``meta-poky/conf/distro/include`` to ``meta/conf/distro/include``).
- The :ref:`ref-classes-buildhistory` class now makes
a single commit per build rather than one commit per subdirectory in
the repository. This behavior assumes the commits are enabled with
:term:`BUILDHISTORY_COMMIT` = "1", which
is typical. Previously, the :ref:`ref-classes-buildhistory` class made one commit
per subdirectory in the repository in order to make it easier to see
the changes for a particular subdirectory. To view a particular
change, specify that subdirectory as the last parameter on the
``git show`` or ``git diff`` commands.
- The ``x86-base.inc`` file, which is included by all x86-based machine
configurations, now sets :term:`IMAGE_FSTYPES`
using ``?=`` to "live" rather than appending with ``+=``. This change
makes the default easier to override.
- BitBake fires multiple "BuildStarted" events when multiconfig is
enabled (one per configuration). For more information, see the
":ref:`bitbake-user-manual/bitbake-user-manual-metadata:events`"
section in the BitBake User Manual.
- By default, the ``security_flags.inc`` file sets a
:term:`GCCPIE` variable with an option to enable
Position Independent Executables (PIE) within ``gcc``. Enabling PIE
in the GNU C Compiler (GCC), makes Return Oriented Programming (ROP)
attacks much more difficult to execute.
- OE-Core now provides a ``bitbake-layers`` plugin that implements a
"create-layer" subcommand. The implementation of this subcommand has
resulted in the ``yocto-layer`` script being deprecated and will
likely be removed in the next Yocto Project release.
- The ``vmdk``, ``vdi``, and ``qcow2`` image file types are now used in
conjunction with the "wic" image type through :term:`CONVERSION_CMD`.
Consequently, the equivalent image types are now ``wic.vmdk``,
``wic.vdi``, and ``wic.qcow2``, respectively.
- ``do_image_<type>[depends]`` has replaced ``IMAGE_DEPENDS_<type>``.
If you have your own classes that implement custom image types, then
you need to update them.
- OpenSSL 1.1 has been introduced. However, the default is still 1.0.x
through the :term:`PREFERRED_VERSION`
variable. This preference is set is due to the remaining
compatibility issues with other software. The
:term:`PROVIDES` variable in the openssl 1.0 recipe
now includes "openssl10" as a marker that can be used in
:term:`DEPENDS` within recipes that build software
that still depend on OpenSSL 1.0.
- To ensure consistent behavior, BitBake's "-r" and "-R" options (i.e.
prefile and postfile), which are used to read or post-read additional
configuration files from the command line, now only affect the
current BitBake command. Before these BitBake changes, these options
would "stick" for future executions.

304
sources/poky/documentation/migration-guides/migration-2.5.rst Normal file
View File

@@ -0,0 +1,304 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.5 (sumo)
==================
This section provides migration information for moving to the Yocto
Project 2.5 Release (codename "sumo") from the prior release.
.. _migration-2.5-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes that have
occurred:
- ``bind-libs``: The libraries packaged by the bind recipe are in a
separate ``bind-libs`` package.
- ``libfm-gtk``: The ``libfm`` GTK+ bindings are split into a
separate ``libfm-gtk`` package.
- ``flex-libfl``: The flex recipe splits out libfl into a separate
``flex-libfl`` package to avoid too many dependencies being pulled in
where only the library is needed.
- ``grub-efi``: The ``grub-efi`` configuration is split into a
separate ``grub-bootconf`` recipe. However, the dependency
relationship from ``grub-efi`` is through a virtual/grub-bootconf
provider making it possible to have your own recipe provide the
dependency. Alternatively, you can use a BitBake append file to bring
the configuration back into the ``grub-efi`` recipe.
- *armv7a Legacy Package Feed Support:* Legacy support is removed for
transitioning from ``armv7a`` to ``armv7a-vfp-neon`` in package
feeds, which was previously enabled by setting
``PKGARCHCOMPAT_ARMV7A``. This transition occurred in 2011 and active
package feeds should by now be updated to the new naming.
.. _migration-2.5-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``gcc``: The version 6.4 recipes are replaced by 7.x.
- ``gst-player``: Renamed to ``gst-examples`` as per upstream.
- ``hostap-utils``: This software package is obsolete.
- ``latencytop``: This recipe is no longer maintained upstream. The
last release was in 2009.
- ``libpfm4``: The only file that requires this recipe is
``oprofile``, which has been removed.
- ``linux-yocto``: The version 4.4, 4.9, and 4.10 recipes have been
removed. Versions 4.12, 4.14, and 4.15 remain.
- ``man``: This recipe has been replaced by modern ``man-db``
- ``mkelfimage``: This tool has been removed in the upstream coreboot
project, and is no longer needed with the removal of the ELF image
type.
- ``nativesdk-postinst-intercept``: This recipe is not maintained.
- ``neon``: This software package is no longer maintained upstream
and is no longer needed by anything in OpenEmbedded-Core.
- ``oprofile``: The functionality of this recipe is replaced by
``perf`` and keeping compatibility on an ongoing basis with ``musl``
is difficult.
- ``pax``: This software package is obsolete.
- ``stat``: This software package is not maintained upstream.
``coreutils`` provides a modern stat binary.
- ``zisofs-tools-native``: This recipe is no longer needed because
the compressed ISO image feature has been removed.
.. _migration-2.5-scripts-and-tools-changes:
Scripts and Tools Changes
-------------------------
- ``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer``: The
``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer`` scripts
previously shipped with poky but not in OpenEmbedded-Core have been
removed. These scripts are not maintained and are outdated. In many
cases, they are also limited in scope. The
``bitbake-layers create-layer`` command is a direct replacement for
``yocto-layer``. See the documentation to create a BSP or kernel
recipe in the ":ref:`bsp-guide/bsp:bsp kernel recipe example`" section.
- ``devtool finish``: ``devtool finish`` now exits with an error if
there are uncommitted changes or a rebase/am in progress in the
recipe's source repository. If this error occurs, there might be
uncommitted changes that will not be included in updates to the
patches applied by the recipe. A -f/--force option is provided for
situations that the uncommitted changes are inconsequential and you
want to proceed regardless.
- ``scripts/oe-setup-rpmrepo`` script: The functionality of
``scripts/oe-setup-rpmrepo`` is replaced by
``bitbake package-index``.
- ``scripts/test-dependencies.sh`` script: The script is largely made
obsolete by the recipe-specific sysroots functionality introduced in
the previous release.
.. _migration-2.5-bitbake-changes:
BitBake Changes
---------------
- The ``--runall`` option has changed. There are two different
behaviors people might want:
- *Behavior A:* For a given target (or set of targets) look through
the task graph and run task X only if it is present and will be
built.
- *Behavior B:* For a given target (or set of targets) look through
the task graph and run task X if any recipe in the taskgraph has
such a target, even if it is not in the original task graph.
The ``--runall`` option now performs "Behavior B". Previously
``--runall`` behaved like "Behavior A". A ``--runonly`` option has
been added to retain the ability to perform "Behavior A".
- Several explicit "run this task for all recipes in the dependency
tree" tasks have been removed (e.g. ``fetchall``, ``checkuriall``,
and the ``*all`` tasks provided by the ``distrodata`` and
:ref:`ref-classes-archiver` classes). There is a BitBake option to complete this for
any arbitrary task. For example::
bitbake <target> -c fetchall
should now be replaced with::
bitbake <target> --runall=fetch
.. _migration-2.5-python-and-python3-changes:
Python and Python 3 Changes
---------------------------
Here are auto-packaging changes to Python and Python 3:
The script-managed ``python-*-manifest.inc`` files that were previously
used to generate Python and Python 3 packages have been replaced with a
JSON-based file that is easier to read and maintain. A new task is
available for maintainers of the Python recipes to update the JSON file
when upgrading to new Python versions. You can now edit the file
directly instead of having to edit a script and run it to update the
file.
One particular change to note is that the Python recipes no longer have
build-time provides for their packages. This assumes ``python-foo`` is
one of the packages provided by the Python recipe. You can no longer run
``bitbake python-foo`` or have a
:term:`DEPENDS` on ``python-foo``,
but doing either of the following causes the package to work as
expected::
IMAGE_INSTALL_append = " python-foo"
or ::
RDEPENDS_${PN} = "python-foo"
The earlier build-time provides behavior was a quirk of the
way the Python manifest file was created. For more information on this
change please see :yocto_git:`this commit
</poky/commit/?id=8d94b9db221d1def42f091b991903faa2d1651ce>`.
.. _migration-2.5-miscellaneous-changes:
Miscellaneous Changes
---------------------
- The :ref:`ref-classes-kernel` class supports building packages for multiple kernels.
If your kernel recipe or ``.bbappend`` file mentions packaging at
all, you should replace references to the kernel in package names
with ``${KERNEL_PACKAGE_NAME}``. For example, if you disable
automatic installation of the kernel image using
``RDEPENDS_kernel-base = ""`` you can avoid warnings using
``RDEPENDS_${KERNEL_PACKAGE_NAME}-base = ""`` instead.
- The :ref:`ref-classes-buildhistory` class commits changes to the repository by
default so you no longer need to set ``BUILDHISTORY_COMMIT = "1"``.
If you want to disable commits you need to set
``BUILDHISTORY_COMMIT = "0"`` in your configuration.
- The ``beaglebone`` reference machine has been renamed to
``beaglebone-yocto``. The ``beaglebone-yocto`` BSP is a reference
implementation using only mainline components available in
OpenEmbedded-Core and ``meta-yocto-bsp``, whereas Texas Instruments
maintains a full-featured BSP in the ``meta-ti`` layer. This rename
avoids the previous name clash that existed between the two BSPs.
- The :ref:`ref-classes-update-alternatives` class no longer works with SysV ``init``
scripts because this usage has been problematic. Also, the
``sysklogd`` recipe no longer uses ``update-alternatives`` because it
is incompatible with other implementations.
- By default, the :ref:`ref-classes-cmake` class uses
``ninja`` instead of ``make`` for building. This improves build
performance. If a recipe is broken with ``ninja``, then the recipe
can set ``OECMAKE_GENERATOR = "Unix Makefiles"`` to change back to
``make``.
- The previously deprecated ``base_*`` functions have been removed in
favor of their replacements in ``meta/lib/oe`` and
``bitbake/lib/bb``. These are typically used from recipes and
classes. Any references to the old functions must be updated. The
following table shows the removed functions and their replacements:
+------------------------------+----------------------------------------------------------+
| *Removed* | *Replacement* |
+==============================+==========================================================+
| base_path_join() | oe.path.join() |
+------------------------------+----------------------------------------------------------+
| base_path_relative() | oe.path.relative() |
+------------------------------+----------------------------------------------------------+
| base_path_out() | oe.path.format_display() |
+------------------------------+----------------------------------------------------------+
| base_read_file() | oe.utils.read_file() |
+------------------------------+----------------------------------------------------------+
| base_ifelse() | oe.utils.ifelse() |
+------------------------------+----------------------------------------------------------+
| base_conditional() | oe.utils.conditional() |
+------------------------------+----------------------------------------------------------+
| base_less_or_equal() | oe.utils.less_or_equal() |
+------------------------------+----------------------------------------------------------+
| base_version_less_or_equal() | oe.utils.version_less_or_equal() |
+------------------------------+----------------------------------------------------------+
| base_contains() | bb.utils.contains() |
+------------------------------+----------------------------------------------------------+
| base_both_contain() | oe.utils.both_contain() |
+------------------------------+----------------------------------------------------------+
| base_prune_suffix() | oe.utils.prune_suffix() |
+------------------------------+----------------------------------------------------------+
| oe_filter() | oe.utils.str_filter() |
+------------------------------+----------------------------------------------------------+
| oe_filter_out() | oe.utils.str_filter_out() (or use the \_remove operator) |
+------------------------------+----------------------------------------------------------+
- Using ``exit 1`` to explicitly defer a postinstall script until first
boot is now deprecated since it is not an obvious mechanism and can
mask actual errors. If you want to explicitly defer a postinstall to
first boot on the target rather than at ``rootfs`` creation time, use
``pkg_postinst_ontarget()`` or call
``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
Any failure of a ``pkg_postinst()`` script (including ``exit 1``)
will trigger a warning during :ref:`ref-tasks-rootfs`.
For more information, see the
":ref:`dev-manual/new-recipe:post-installation scripts`"
section in the Yocto Project Development Tasks Manual.
- The ``elf`` image type has been removed. This image type was removed
because the ``mkelfimage`` tool that was required to create it is no
longer provided by coreboot upstream and required updating every time
``binutils`` updated.
- Support for .iso image compression (previously enabled through
``COMPRESSISO = "1"``) has been removed. The userspace tools
(``zisofs-tools``) are unmaintained and ``squashfs`` provides better
performance and compression. In order to build a live image with
squashfs+lz4 compression enabled you should now set
``LIVE_ROOTFS_TYPE = "squashfs-lz4"`` and ensure that ``live`` is in
:term:`IMAGE_FSTYPES`.
- Recipes with an unconditional dependency on ``libpam`` are only
buildable with ``pam`` in :term:`DISTRO_FEATURES`. If the dependency is
truly optional then it is recommended that the dependency be
conditional upon ``pam`` being in :term:`DISTRO_FEATURES`.
- For EFI-based machines, the bootloader (``grub-efi`` by default) is
installed into the image at /boot. Wic can be used to split the
bootloader into separate boot and root filesystem partitions if necessary.
- Patches whose context does not match exactly (i.e. where patch
reports "fuzz" when applying) will generate a warning. For an example
of this see :yocto_git:`this commit
</poky/commit/?id=cc97bc08125b63821ce3f616771830f77c456f57>`.
- Layers are expected to set ``LAYERSERIES_COMPAT_layername`` to match
the version(s) of OpenEmbedded-Core they are compatible with. This is
specified as codenames using spaces to separate multiple values (e.g.
"rocko sumo"). If a layer does not set
``LAYERSERIES_COMPAT_layername``, a warning will is shown. If a layer
sets a value that does not include the current version ("sumo" for
the 2.5 release), then an error will be produced.
- The ``TZ`` environment variable is set to "UTC" within the build
environment in order to fix reproducibility problems in some recipes.

447
sources/poky/documentation/migration-guides/migration-2.6.rst Normal file
View File

@@ -0,0 +1,447 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.6 (thud)
==================
This section provides migration information for moving to the Yocto
Project 2.6 Release (codename "thud") from the prior release.
.. _migration-2.6-gcc-changes:
GCC 8.2 is Now Used by Default
------------------------------
The GNU Compiler Collection version 8.2 is now used by default for
compilation. For more information on what has changed in the GCC 8.x
release, see https://gcc.gnu.org/gcc-8/changes.html.
If you still need to compile with version 7.x, GCC 7.3 is also provided.
You can select this version by setting the and can be selected by
setting the :term:`GCCVERSION` variable to "7.%" in
your configuration.
.. _migration-2.6-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- *beecrypt*: No longer needed since moving to RPM 4.
- *bigreqsproto*: Replaced by ``xorgproto``.
- *calibrateproto*: Removed in favor of ``xinput``.
- *compositeproto*: Replaced by ``xorgproto``.
- *damageproto*: Replaced by ``xorgproto``.
- *dmxproto*: Replaced by ``xorgproto``.
- *dri2proto*: Replaced by ``xorgproto``.
- *dri3proto*: Replaced by ``xorgproto``.
- *eee-acpi-scripts*: Became obsolete.
- *fixesproto*: Replaced by ``xorgproto``.
- *fontsproto*: Replaced by ``xorgproto``.
- *fstests*: Became obsolete.
- *gccmakedep*: No longer used.
- *glproto*: Replaced by ``xorgproto``.
- *gnome-desktop3*: No longer needed. This recipe has moved to ``meta-oe``.
- *icon-naming-utils*: No longer used since the Sato theme was removed in 2016.
- *inputproto*: Replaced by ``xorgproto``.
- *kbproto*: Replaced by ``xorgproto``.
- *libusb-compat*: Became obsolete.
- *libuser*: Became obsolete.
- *libnfsidmap*: No longer an external requirement since ``nfs-utils`` 2.2.1. ``libnfsidmap`` is now integrated.
- *libxcalibrate*: No longer needed with ``xinput``
- *mktemp*: Became obsolete. The ``mktemp`` command is provided by both ``busybox`` and ``coreutils``.
- *ossp-uuid*: Is not being maintained and has mostly been replaced by ``uuid.h`` in ``util-linux``.
- *pax-utils*: No longer needed. Previous QA tests that did use this recipe are now done at build time.
- *pcmciautils*: Became obsolete.
- *pixz*: No longer needed. ``xz`` now supports multi-threaded compression.
- *presentproto*: Replaced by ``xorgproto``.
- *randrproto*: Replaced by ``xorgproto``.
- *recordproto*: Replaced by ``xorgproto``.
- *renderproto*: Replaced by ``xorgproto``.
- *resourceproto*: Replaced by ``xorgproto``.
- *scrnsaverproto*: Replaced by ``xorgproto``.
- *trace-cmd*: Became obsolete. ``perf`` replaced this recipe's functionally.
- *videoproto*: Replaced by ``xorgproto``.
- *wireless-tools*: Became obsolete. Superseded by ``iw``.
- *xcmiscproto*: Replaced by ``xorgproto``.
- *xextproto*: Replaced by ``xorgproto``.
- *xf86dgaproto*: Replaced by ``xorgproto``.
- *xf86driproto*: Replaced by ``xorgproto``.
- *xf86miscproto*: Replaced by ``xorgproto``.
- *xf86-video-omapfb*: Became obsolete. Use kernel modesetting driver instead.
- *xf86-video-omap*: Became obsolete. Use kernel modesetting driver instead.
- *xf86vidmodeproto*: Replaced by ``xorgproto``.
- *xineramaproto*: Replaced by ``xorgproto``.
- *xproto*: Replaced by ``xorgproto``.
- *yasm*: No longer needed since previous usages are now satisfied by ``nasm``.
.. _migration-2.6-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have been made:
- *cmake*: ``cmake.m4`` and ``toolchain`` files have been moved to
the main package.
- *iptables*: The ``iptables`` modules have been split into
separate packages.
- *alsa-lib*: ``libasound`` is now in the main ``alsa-lib`` package
instead of ``libasound``.
- *glibc*: ``libnss-db`` is now in its own package along with a
``/var/db/makedbs.sh`` script to update databases.
- *python and python3*: The main package has been removed from
the recipe. You must install specific packages or ``python-modules``
/ ``python3-modules`` for everything.
- *systemtap*: Moved ``systemtap-exporter`` into its own package.
.. _migration-2.6-xorg-protocol-dependencies:
XOrg Protocol dependencies
--------------------------
The ``*proto`` upstream repositories have been combined into one
"xorgproto" repository. Thus, the corresponding recipes have also been
combined into a single ``xorgproto`` recipe. Any recipes that depend
upon the older ``*proto`` recipes need to be changed to depend on the
newer ``xorgproto`` recipe instead.
For names of recipes removed because of this repository change, see the
:ref:`migration-guides/migration-2.6:removed recipes` section.
.. _migration-2.6-distutils-distutils3-fetching-dependencies:
``distutils`` and ``distutils3`` Now Prevent Fetching Dependencies During the ``do_configure`` Task
---------------------------------------------------------------------------------------------------
Previously, it was possible for Python recipes that inherited the
``distutils`` and ``distutils3`` classes to fetch code
during the :ref:`ref-tasks-configure` task to satisfy
dependencies mentioned in ``setup.py`` if those dependencies were not
provided in the sysroot (i.e. recipes providing the dependencies were
missing from :term:`DEPENDS`).
.. note::
This change affects classes beyond just the two mentioned (i.e. ``distutils``
and ``distutils3``). Any recipe that inherits ``distutils*`` classes are
affected. For example, the ``setuptools`` and :ref:`ref-classes-setuptools3`
recipes are affected since they inherit the ``distutils*`` classes.
Fetching these types of dependencies that are not provided in the
sysroot negatively affects the ability to reproduce builds. This type of
fetching is now explicitly disabled. Consequently, any missing
dependencies in Python recipes that use these classes now result in an
error during the :ref:`ref-tasks-configure` task.
.. _migration-2.6-linux-yocto-configuration-audit-issues-now-correctly-reported:
``linux-yocto`` Configuration Audit Issues Now Correctly Reported
-----------------------------------------------------------------
Due to a bug, the kernel configuration audit functionality was not
writing out any resulting warnings during the build. This issue is now
corrected. You might notice these warnings now if you have a custom
kernel configuration with a ``linux-yocto`` style kernel recipe.
.. _migration-2.6-image-kernel-artifact-naming-changes:
Image/Kernel Artifact Naming Changes
------------------------------------
The following changes have been made:
- Name variables (e.g. :term:`IMAGE_NAME`) use a new
:term:`IMAGE_VERSION_SUFFIX` variable instead of
:term:`DATETIME`. Using :term:`IMAGE_VERSION_SUFFIX`
allows easier and more direct changes.
The :term:`IMAGE_VERSION_SUFFIX` variable is set in the ``bitbake.conf``
configuration file as follows::
IMAGE_VERSION_SUFFIX = "-${DATETIME}"
- Several variables have changed names for consistency::
Old Variable Name New Variable Name
========================================================
KERNEL_IMAGE_BASE_NAME KERNEL_IMAGE_NAME
KERNEL_IMAGE_SYMLINK_NAME KERNEL_IMAGE_LINK_NAME
MODULE_TARBALL_BASE_NAME MODULE_TARBALL_NAME
MODULE_TARBALL_SYMLINK_NAME MODULE_TARBALL_LINK_NAME
INITRAMFS_BASE_NAME INITRAMFS_NAME
- The ``MODULE_IMAGE_BASE_NAME`` variable has been removed. The module
tarball name is now controlled directly with the
:term:`MODULE_TARBALL_NAME` variable.
- The :term:`KERNEL_DTB_NAME` and
:term:`KERNEL_DTB_LINK_NAME` variables
have been introduced to control kernel Device Tree Binary (DTB)
artifact names instead of mangling ``KERNEL_IMAGE_*`` variables.
- The :term:`KERNEL_FIT_NAME` and
:term:`KERNEL_FIT_LINK_NAME` variables
have been introduced to specify the name of flattened image tree
(FIT) kernel images similar to other deployed artifacts.
- The :term:`MODULE_TARBALL_NAME` and
:term:`MODULE_TARBALL_LINK_NAME`
variable values no longer include the "module-" prefix or ".tgz"
suffix. These parts are now hardcoded so that the values are
consistent with other artifact naming variables.
- Added the :term:`INITRAMFS_LINK_NAME`
variable so that the symlink can be controlled similarly to other
artifact types.
- :term:`INITRAMFS_NAME` now uses
"${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" instead
of "${PV}-${PR}-${MACHINE}-${DATETIME}", which makes it consistent
with other variables.
.. _migration-2.6-serial-console-deprecated:
``SERIAL_CONSOLE`` Deprecated
-----------------------------
The ``SERIAL_CONSOLE`` variable has been functionally replaced by the
:term:`SERIAL_CONSOLES` variable for some time. With the Yocto Project 2.6
release, ``SERIAL_CONSOLE`` has been officially deprecated.
``SERIAL_CONSOLE`` will continue to work as before for the 2.6 release.
However, for the sake of future compatibility, it is recommended that
you replace all instances of ``SERIAL_CONSOLE`` with :term:`SERIAL_CONSOLES`.
.. note::
The only difference in usage is that :term:`SERIAL_CONSOLES`
expects entries to be separated using semicolons as compared to
``SERIAL_CONSOLE``, which expects spaces.
.. _migration-2.6-poky-sets-unknown-configure-option-to-qa-error:
Configure Script Reports Unknown Options as Errors
--------------------------------------------------
If the configure script reports an unknown option, this now triggers a
QA error instead of a warning. Any recipes that previously got away with
specifying such unknown options now need to be fixed.
.. _migration-2.6-override-changes:
Override Changes
----------------
The following changes have occurred:
- The ``virtclass-native`` and ``virtclass-nativesdk`` Overrides Have
Been Removed: The ``virtclass-native`` and ``virtclass-nativesdk``
overrides have been deprecated since 2012 in favor of
``class-native`` and ``class-nativesdk``, respectively. Both
``virtclass-native`` and ``virtclass-nativesdk`` are now dropped.
.. note::
The ``virtclass-multilib-`` overrides for multilib are still valid.
- The ``forcevariable`` Override Now Has a Higher Priority Than
``libc`` Overrides: The ``forcevariable`` override is documented to
be the highest priority override. However, due to a long-standing
quirk of how :term:`OVERRIDES` is set, the ``libc``
overrides (e.g. ``libc-glibc``, ``libc-musl``, and so forth)
erroneously had a higher priority. This issue is now corrected.
It is likely this change will not cause any problems. However, it is
possible with some unusual configurations that you might see a change
in behavior if you were relying on the previous behavior. Be sure to
check how you use ``forcevariable`` and ``libc-*`` overrides in your
custom layers and configuration files to ensure they make sense.
- The ``build-${BUILD_OS}`` Override Has Been Removed: The
``build-${BUILD_OS}``, which is typically ``build-linux``, override
has been removed because building on a host operating system other
than a recent version of Linux is neither supported nor recommended.
Dropping the override avoids giving the impression that other host
operating systems might be supported.
- The "_remove" operator now preserves whitespace. Consequently, when
specifying list items to remove, be aware that leading and trailing
whitespace resulting from the removal is retained.
See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:removal (override style syntax)`"
section in the BitBake User Manual for a detailed example.
.. _migration-2.6-systemd-configuration-now-split-out-to-systemd-conf:
``systemd`` Configuration is Now Split Into ``systemd-conf``
------------------------------------------------------------
The configuration for the ``systemd`` recipe has been moved into a
``systemd-conf`` recipe. Moving this configuration to a separate recipe
avoids the ``systemd`` recipe from becoming machine-specific for cases
where machine-specific configurations need to be applied (e.g. for
``qemu*`` machines).
Currently, the new recipe packages the following files::
${sysconfdir}/machine-id
${sysconfdir}/systemd/coredump.conf
${sysconfdir}/systemd/journald.conf
${sysconfdir}/systemd/logind.conf
${sysconfdir}/systemd/system.conf
${sysconfdir}/systemd/user.conf
If you previously used bbappend files to append the ``systemd`` recipe to
change any of the listed files, you must do so for the ``systemd-conf``
recipe instead.
.. _migration-2.6-automatic-testing-changes:
Automatic Testing Changes
-------------------------
This section provides information about automatic testing changes:
- ``TEST_IMAGE`` Variable Removed: Prior to this release, you set the
``TEST_IMAGE`` variable to "1" to enable automatic testing for
successfully built images. The ``TEST_IMAGE`` variable no longer
exists and has been replaced by the
:term:`TESTIMAGE_AUTO` variable.
- Inheriting the :ref:`ref-classes-testimage` and :ref:`ref-classes-testsdk`
classes: best practices now dictate that you use the :term:`IMAGE_CLASSES`
variable rather than the :term:`INHERIT` variable when you inherit the
:ref:`ref-classes-testimage` and :ref:`ref-classes-testsdk` classes used
for automatic testing.
.. _migration-2.6-openssl-changes:
OpenSSL Changes
---------------
`OpenSSL <https://www.openssl.org/>`__ has been upgraded from 1.0 to
1.1. By default, this upgrade could cause problems for recipes that have
both versions in their dependency chains. The problem is that both
versions cannot be installed together at build time.
.. note::
It is possible to have both versions of the library at runtime.
.. _migration-2.6-bitbake-changes:
BitBake Changes
---------------
The server logfile ``bitbake-cookerdaemon.log`` is now always placed in
the :term:`Build Directory` instead of the current directory.
.. _migration-2.6-security-changes:
Security Changes
----------------
The Poky distribution now uses security compiler flags by default.
Inclusion of these flags could cause new failures due to stricter
checking for various potential security issues in code.
.. _migration-2.6-post-installation-changes:
Post Installation Changes
-------------------------
You must explicitly mark post installs to defer to the target. If you
want to explicitly defer a postinstall to first boot on the target
rather than at root filesystem creation time, use ``pkg_postinst_ontarget()`` or
call ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
Any failure of a ``pkg_postinst()`` script (including exit 1) triggers
an error during the :ref:`ref-tasks-rootfs` task.
For more information on post-installation behavior, see the
":ref:`dev-manual/new-recipe:post-installation scripts`"
section in the Yocto Project Development Tasks Manual.
.. _migration-2.6-python-3-profile-guided-optimizations:
Python 3 Profile-Guided Optimization
------------------------------------
The ``python3`` recipe now enables profile-guided optimization. Using
this optimization requires a little extra build time in exchange for
improved performance on the target at runtime. Additionally, the
optimization is only enabled if the current
:term:`MACHINE` has support for user-mode emulation in
QEMU (i.e. "qemu-usermode" is in
:term:`MACHINE_FEATURES`, which it is by
default).
If you wish to disable Python profile-guided optimization regardless of
the value of :term:`MACHINE_FEATURES`, then ensure that
:term:`PACKAGECONFIG` for the ``python3`` recipe
does not contain "pgo". You could accomplish the latter using the
following at the configuration level::
PACKAGECONFIG_remove_pn-python3 = "pgo"
Alternatively, you can set :term:`PACKAGECONFIG` using an append file
for the ``python3`` recipe.
.. _migration-2.6-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes occurred:
- Default to using the Thumb-2 instruction set for armv7a and above. If
you have any custom recipes that build software that needs to be
built with the ARM instruction set, change the recipe to set the
instruction set as follows::
ARM_INSTRUCTION_SET = "arm"
- ``run-postinsts`` no longer uses ``/etc/*-postinsts`` for
``dpkg/opkg`` in favor of built-in postinst support. RPM behavior
remains unchanged.
- The ``NOISO`` and ``NOHDD`` variables are no longer used. You now
control building ``*.iso`` and ``*.hddimg`` image types directly by
using the :term:`IMAGE_FSTYPES` variable.
- The ``scripts/contrib/mkefidisk.sh`` has been removed in favor of
Wic.
- ``kernel-modules`` has been removed from
:term:`RRECOMMENDS` for ``qemumips`` and
``qemumips64`` machines. Removal also impacts the ``x86-base.inc``
file.
.. note::
``genericx86`` and ``genericx86-64`` retain ``kernel-modules`` as part of
the :term:`RRECOMMENDS` variable setting.
- The ``LGPLv2_WHITELIST_GPL-3.0`` variable has been removed. If you
are setting this variable in your configuration, set or append it to
the ``WHITELIST_GPL-3.0`` variable instead.
- ``${ASNEEDED}`` is now included in the
:term:`TARGET_LDFLAGS` variable directly. The
remaining definitions from ``meta/conf/distro/include/as-needed.inc``
have been moved to corresponding recipes.
- Support for DSA host keys has been dropped from the OpenSSH recipes.
If you are still using DSA keys, you must switch over to a more
secure algorithm as recommended by OpenSSH upstream.
- The ``dhcp`` recipe now uses the ``dhcpd6.conf`` configuration file
in ``dhcpd6.service`` for IPv6 DHCP rather than re-using
``dhcpd.conf``, which is now reserved for IPv4.

181
sources/poky/documentation/migration-guides/migration-2.7.rst Normal file
View File

@@ -0,0 +1,181 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.7 (warrior)
=====================
This section provides migration information for moving to the Yocto
Project 2.7 Release (codename "warrior") from the prior release.
.. _migration-2.7-bitbake-changes:
BitBake Changes
---------------
The following changes have been made to BitBake:
- BitBake now checks anonymous Python functions and pure Python
functions (e.g. ``def funcname:``) in the metadata for tab
indentation. If found, BitBake produces a warning.
- BitBake now checks
:term:`BBFILE_COLLECTIONS` for duplicate
entries and triggers an error if any are found.
.. _migration-2.7-eclipse-support-dropped:
Eclipse Support Removed
-----------------------
Support for the Eclipse IDE has been removed. Support continues for
those releases prior to 2.7 that did include support. The 2.7 release
does not include the Eclipse Yocto plugin.
.. _migration-2.7-qemu-native-splits-system-and-user-mode-parts:
``qemu-native`` Splits the System and User-Mode Parts
-----------------------------------------------------
The system and user-mode parts of ``qemu-native`` are now split.
``qemu-native`` provides the user-mode components and
``qemu-system-native`` provides the system components. If you have
recipes that depend on QEMU's system emulation functionality at build
time, they should now depend upon ``qemu-system-native`` instead of
``qemu-native``.
.. _migration-2.7-upstream-tracking.inc-removed:
The ``upstream-tracking.inc`` File Has Been Removed
---------------------------------------------------
The previously deprecated ``upstream-tracking.inc`` file is now removed.
Any ``UPSTREAM_TRACKING*`` variables are now set in the corresponding
recipes instead.
Remove any references you have to the ``upstream-tracking.inc`` file in
your configuration.
.. _migration-2.7-distro-features-libc-removed:
The ``DISTRO_FEATURES_LIBC`` Variable Has Been Removed
------------------------------------------------------
The ``DISTRO_FEATURES_LIBC`` variable is no longer used. The ability to
configure glibc using kconfig has been removed for quite some time
making the ``libc-*`` features set no longer effective.
Remove any references you have to ``DISTRO_FEATURES_LIBC`` in your own
layers.
.. _migration-2.7-license-values:
License Value Corrections
-------------------------
The following corrections have been made to the
:term:`LICENSE` values set by recipes:
- *socat*: Corrected :term:`LICENSE` to be "GPLv2" rather than "GPLv2+".
- *libgfortran*: Set license to "GPL-3.0-with-GCC-exception".
- *elfutils*: Removed "Elfutils-Exception" and set to "GPLv2" for shared libraries
.. _migration-2.7-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes.
- ``bind``: The ``nsupdate`` binary has been moved to the
``bind-utils`` package.
- Debug split: The default debug split has been changed to create
separate source packages (i.e. ``package_name-dbg`` and
``package_name-src``). If you are currently using ``dbg-pkgs`` in
:term:`IMAGE_FEATURES` to bring in debug
symbols and you still need the sources, you must now also add
``src-pkgs`` to :term:`IMAGE_FEATURES`. Source packages remain in the
target portion of the SDK by default, unless you have set your own
value for :term:`SDKIMAGE_FEATURES` that
does not include ``src-pkgs``.
- Mount all using ``util-linux``: ``/etc/default/mountall`` has moved
into the -mount sub-package.
- Splitting binaries using ``util-linux``: ``util-linux`` now splits
each binary into its own package for fine-grained control. The main
``util-linux`` package pulls in the individual binary packages using
the :term:`RRECOMMENDS` and
:term:`RDEPENDS` variables. As a result, existing
images should not see any changes assuming
:term:`NO_RECOMMENDATIONS` is not set.
- ``netbase/base-files``: ``/etc/hosts`` has moved from ``netbase`` to
``base-files``.
- ``tzdata``: The main package has been converted to an empty meta
package that pulls in all ``tzdata`` packages by default.
- ``lrzsz``: This package has been removed from
``packagegroup-self-hosted`` and
``packagegroup-core-tools-testapps``. The X/Y/ZModem support is less
likely to be needed on modern systems. If you are relying on these
packagegroups to include the ``lrzsz`` package in your image, you now
need to explicitly add the package.
.. _migration-2.7-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- *gcc*: Drop version 7.3 recipes. Version 8.3 now remains.
- *linux-yocto*: Drop versions 4.14 and 4.18 recipes. Versions 4.19 and 5.0 remain.
- *go*: Drop version 1.9 recipes. Versions 1.11 and 1.12 remain.
- *xvideo-tests*: Became obsolete.
- *libart-lgpl*: Became obsolete.
- *gtk-icon-utils-native*: These tools are now provided by gtk+3-native
- *gcc-cross-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
- *gcc-crosssdk-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
- *glibc-initial*: Removed because the benefits of having it for site_config are currently outweighed by the cost of building the recipe.
.. _migration-2.7-removed-classes:
Removed Classes
---------------
The following classes have been removed:
- *distutils-tools*: This class was never used.
- *bugzilla.bbclass*: Became obsolete.
- *distrodata*: This functionally has been replaced by a more modern tinfoil-based implementation.
.. _migration-2.7-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes occurred:
- The ``distro`` subdirectory of the Poky repository has been removed
from the top-level ``scripts`` directory.
- Perl now builds for the target using
`perl-cross <https://arsv.github.io/perl-cross/>`_ for better
maintainability and improved build performance. This change should
not present any problems unless you have heavily customized your Perl
recipe.
- ``arm-tunes``: Removed the "-march" option if mcpu is already added.
- ``update-alternatives``: Convert file renames to
:term:`PACKAGE_PREPROCESS_FUNCS`
- ``base/pixbufcache``: Obsolete ``sstatecompletions`` code has been
removed.
- :ref:`ref-classes-native` class: :term:`RDEPENDS` handling has been enabled.
- ``inetutils``: This recipe has rsh disabled.

323
sources/poky/documentation/migration-guides/migration-3.0.rst Normal file
View File

@@ -0,0 +1,323 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.0 (zeus)
==================
This section provides migration information for moving to the Yocto
Project 3.0 Release (codename "zeus") from the prior release.
.. _migration-3.0-init-system-selection:
Init System Selection
---------------------
Changing the init system manager previously required setting a number of
different variables. You can now change the manager by setting the
``INIT_MANAGER`` variable and the corresponding include files (i.e.
``conf/distro/include/init-manager-*.conf``). Include files are provided
for four values: "none", "sysvinit", "systemd", and "mdev-busybox". The
default value, "none", for ``INIT_MANAGER`` should allow your current
settings to continue working. However, it is advisable to explicitly set
``INIT_MANAGER``.
.. _migration-3.0-lsb-support-removed:
LSB Support Removed
-------------------
Linux Standard Base (LSB) as a standard is not current, and is not well
suited for embedded applications. Support can be continued in a separate
layer if needed. However, presently LSB support has been removed from
the core.
As a result of this change, the ``poky-lsb`` derivative distribution
configuration that was also used for testing alternative configurations
has been replaced with a ``poky-altcfg`` distribution that has LSB parts
removed.
.. _migration-3.0-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed.
- ``core-image-lsb-dev``: Part of removed LSB support.
- ``core-image-lsb``: Part of removed LSB support.
- ``core-image-lsb-sdk``: Part of removed LSB support.
- ``cve-check-tool``: Functionally replaced by the ``cve-update-db``
recipe and :ref:`ref-classes-cve-check` class.
- ``eglinfo``: No longer maintained. ``eglinfo`` from ``mesa-demos`` is
an adequate and maintained alternative.
- ``gcc-8.3``: Version 8.3 removed. Replaced by 9.2.
- ``gnome-themes-standard``: Only needed by gtk+ 2.x, which has been
removed.
- ``gtk+``: GTK+ 2 is obsolete and has been replaced by gtk+3.
- ``irda-utils``: Has become obsolete. IrDA support has been removed
from the Linux kernel in version 4.17 and later.
- ``libnewt-python``: ``libnewt`` Python support merged into main
``libnewt`` recipe.
- ``libsdl``: Replaced by newer ``libsdl2``.
- ``libx11-diet``: Became obsolete.
- ``libxx86dga``: Removed obsolete client library.
- ``libxx86misc``: Removed. Library is redundant.
- ``linux-yocto``: Version 5.0 removed, which is now redundant (5.2 /
4.19 present).
- ``lsbinitscripts``: Part of removed LSB support.
- ``lsb``: Part of removed LSB support.
- ``lsbtest``: Part of removed LSB support.
- ``openssl10``: Replaced by newer ``openssl`` version 1.1.
- ``packagegroup-core-lsb``: Part of removed LSB support.
- ``python-nose``: Removed the Python 2.x version of the recipe.
- ``python-numpy``: Removed the Python 2.x version of the recipe.
- ``python-scons``: Removed the Python 2.x version of the recipe.
- ``source-highlight``: No longer needed.
- ``stress``: Replaced by ``stress-ng``.
- ``vulkan``: Split into ``vulkan-loader``, ``vulkan-headers``, and
``vulkan-tools``.
- ``weston-conf``: Functionality moved to ``weston-init``.
.. _migration-3.0-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have occurred.
- The :wikipedia:`Epiphany <GNOME_Web>` browser
has been dropped from ``packagegroup-self-hosted`` as it has not been
needed inside ``build-appliance-image`` for quite some time and was
causing resource problems.
- ``libcap-ng`` Python support has been moved to a separate
``libcap-ng-python`` recipe to streamline the build process when the
Python bindings are not needed.
- ``libdrm`` now packages the file ``amdgpu.ids`` into a separate
``libdrm-amdgpu`` package.
- ``python3``: The ``runpy`` module is now in the ``python3-core``
package as it is required to support the common "python3 -m" command
usage.
- ``distcc`` now provides separate ``distcc-client`` and
``distcc-server`` packages as typically one or the other are needed,
rather than both.
- ``python*-setuptools`` recipes now separately package the
``pkg_resources`` module in a ``python-pkg-resources`` /
``python3-pkg-resources`` package as the module is useful independent
of the rest of the setuptools package. The main ``python-setuptools``
/ ``python3-setuptools`` package depends on this new package so you
should only need to update dependencies unless you want to take
advantage of the increased granularity.
.. _migration-3.0-cve-checking:
CVE Checking
------------
``cve-check-tool`` has been functionally replaced by a new
``cve-update-db`` recipe and functionality built into the :ref:`ref-classes-cve-check`
class. The result uses NVD JSON data feeds rather than the deprecated
XML feeds that ``cve-check-tool`` was using, supports CVSSv3 scoring,
and makes other improvements.
Additionally, the ``CVE_CHECK_CVE_WHITELIST`` variable has been replaced
by ``CVE_CHECK_WHITELIST`` (replaced by :term:`CVE_CHECK_IGNORE` in version 4.0).
.. _migration-3.0-bitbake-changes:
BitBake Changes
---------------
The following BitBake changes have occurred.
- ``addtask`` statements now properly validate dependent tasks.
Previously, an invalid task was silently ignored. With this change,
the invalid task generates a warning.
- Other invalid ``addtask`` and ``deltask`` usages now trigger these
warnings: "multiple target tasks arguments with addtask / deltask",
and "multiple before/after clauses".
- The "multiconfig" prefix is now shortened to "mc". "multiconfig" will
continue to work, however it may be removed in a future release.
- The ``bitbake -g`` command no longer generates a
``recipe-depends.dot`` file as the contents (i.e. a reprocessed
version of ``task-depends.dot``) were confusing.
- The ``bb.build.FuncFailed`` exception, previously raised by
``bb.build.exec_func()`` when certain other exceptions have occurred,
has been removed. The real underlying exceptions will be raised
instead. If you have calls to ``bb.build.exec_func()`` in custom
classes or ``tinfoil-using`` scripts, any references to
``bb.build.FuncFailed`` should be cleaned up.
- Additionally, the ``bb.build.exec_func()`` no longer accepts the
"pythonexception" parameter. The function now always raises
exceptions. Remove this argument in any calls to
``bb.build.exec_func()`` in custom classes or scripts.
- The ``BB_SETSCENE_VERIFY_FUNCTION2`` variable is no longer used. In
the unlikely event that you have any references to it, they should be
removed.
- The ``RunQueueExecuteScenequeue`` and ``RunQueueExecuteTasks`` events
have been removed since setscene tasks are now executed as part of
the normal runqueue. Any event handling code in custom classes or
scripts that handles these two events need to be updated.
- The arguments passed to functions used with
:term:`BB_HASHCHECK_FUNCTION`
have changed. If you are using your own custom hash check function,
see :yocto_git:`/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725`
for details.
- Task specifications in ``BB_TASKDEPDATA`` and class implementations
used in signature generator classes now use "<fn>:<task>" everywhere
rather than the "." delimiter that was being used in some places.
This change makes it consistent with all areas in the code. Custom
signature generator classes and code that reads ``BB_TASKDEPDATA``
need to be updated to use ':' as a separator rather than '.'.
.. _migration-3.0-sanity-checks:
Sanity Checks
-------------
The following sanity check changes occurred.
- :term:`SRC_URI` is now checked for usage of two
problematic items:
- "${PN}" prefix/suffix use --- warnings always appear if ${PN} is
used. You must fix the issue regardless of whether multiconfig or
anything else that would cause prefixing/suffixing to happen.
- Github archive tarballs --- these are not guaranteed to be stable.
Consequently, it is likely that the tarballs will be refreshed and
thus the :term:`SRC_URI` checksums will fail to apply. It is recommended
that you fetch either an official release tarball or a specific
revision from the actual Git repository instead.
Either one of these items now trigger a warning by default. If you
wish to disable this check, remove ``src-uri-bad`` from
:term:`WARN_QA`.
- The ``file-rdeps`` runtime dependency check no longer expands
:term:`RDEPENDS` recursively as there is no mechanism
to ensure they can be fully computed, and thus races sometimes result
in errors either showing up or not. Thus, you might now see errors
for missing runtime dependencies that were previously satisfied
recursively. Here is an example: package A contains a shell script
starting with ``#!/bin/bash`` but has no dependency on bash. However,
package A depends on package B, which does depend on bash. You need
to add the missing dependency or dependencies to resolve the warning.
- Setting ``DEPENDS_${PN}`` anywhere (i.e. typically in a recipe) now
triggers an error. The error is triggered because
:term:`DEPENDS` is not a package-specific variable
unlike RDEPENDS. You should set :term:`DEPENDS` instead.
- systemd currently does not work well with the musl C library because
only upstream officially supports linking the library with glibc.
Thus, a warning is shown when building systemd in conjunction with
musl.
.. _migration-3.0-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred.
- The ``gnome`` class has been removed because it now does very little.
You should update recipes that previously inherited this class to do
the following::
inherit gnomebase gtk-icon-cache gconf mime
- The ``meta/recipes-kernel/linux/linux-dtb.inc`` file has been
removed. This file was previously deprecated in favor of setting
:term:`KERNEL_DEVICETREE` in any kernel
recipe and only produced a warning. Remove any ``include`` or
``require`` statements pointing to this file.
- :term:`TARGET_CFLAGS`,
:term:`TARGET_CPPFLAGS`,
:term:`TARGET_CXXFLAGS`, and
:term:`TARGET_LDFLAGS` are no longer exported
to the external environment. This change did not require any changes
to core recipes, which is a good indicator that no changes will be
required. However, if for some reason the software being built by one
of your recipes is expecting these variables to be set, then building
the recipe will fail. In such cases, you must either export the
variable or variables in the recipe or change the scripts so that
exporting is not necessary.
- You must change the host distro identifier used in
:term:`NATIVELSBSTRING` to use all lowercase
characters even if it does not contain a version number. This change
is necessary only if you are not using
:ref:`ref-classes-uninative` and :term:`SANITY_TESTED_DISTROS`.
- In the ``base-files`` recipe, writing the hostname into
``/etc/hosts`` and ``/etc/hostname`` is now done within the main
:ref:`ref-tasks-install` function rather than in the
``do_install_basefilesissue`` function. The reason for the change is
because ``do_install_basefilesissue`` is more easily overridden
without having to duplicate the hostname functionality. If you have
done the latter (e.g. in a ``base-files`` bbappend), then you should
remove it from your customized ``do_install_basefilesissue``
function.
- The ``wic --expand`` command now uses commas to separate "key:value"
pairs rather than hyphens.
.. note::
The wic command-line help is not updated.
You must update any scripts or commands where you use
``wic --expand`` with multiple "key:value" pairs.
- UEFI image variable settings have been moved from various places to a
central ``conf/image-uefi.conf``. This change should not influence
any existing configuration as the ``meta/conf/image-uefi.conf`` in
the core metadata sets defaults that can be overridden in the same
manner as before.
- ``conf/distro/include/world-broken.inc`` has been removed. For cases
where certain recipes need to be disabled when using the musl C
library, these recipes now have ``COMPATIBLE_HOST_libc-musl`` set
with a comment that explains why.

277
sources/poky/documentation/migration-guides/migration-3.1.rst Normal file
View File

@@ -0,0 +1,277 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.1 (dunfell)
=====================
This section provides migration information for moving to the Yocto
Project 3.1 Release (codename "dunfell") from the prior release.
.. _migration-3.1-minimum-system-requirements:
Minimum system requirements
---------------------------
The following versions / requirements of build host components have been
updated:
- gcc 5.0
- python 3.5
- tar 1.28
- ``rpcgen`` is now required on the host (part of the ``libc-dev-bin``
package on Ubuntu, Debian and related distributions, and the
``glibc`` package on RPM-based distributions).
Additionally, the ``makeinfo`` and ``pod2man`` tools are *no longer*
required on the host.
.. _migration-3.1-mpc8315e-rdb-removed:
mpc8315e-rdb machine removed
----------------------------
The MPC8315E-RDB machine is old/obsolete and unobtainable, thus given
the maintenance burden the ``mpc8315e-rdb`` machine configuration that
supported it has been removed in this release. The removal does leave a
gap in official PowerPC reference hardware support; this may change in
future if a suitable machine with accompanying support resources is
found.
.. _migration-3.1-python-2-removed:
Python 2 removed
----------------
Due to the expiration of upstream support in January 2020, support for
Python 2 has now been removed; it is recommended that you use Python 3
instead. If absolutely needed there is a meta-python2 community layer
containing Python 2, related classes and various Python 2-based modules,
however it should not be considered as supported.
.. _migration-3.1-reproducible-builds:
Reproducible builds now enabled by default
------------------------------------------
In order to avoid unnecessary differences in output files (aiding binary
reproducibility), the Poky distribution configuration
(``DISTRO = "poky"``) now inherits the ``reproducible_build`` class by
default.
.. _migration-3.1-ptest-feature-impact:
Impact of ptest feature is now more significant
-----------------------------------------------
The Poky distribution configuration (``DISTRO = "poky"``) enables ptests
by default to enable runtime testing of various components. In this
release, a dependency needed to be added that has resulted in a
significant increase in the number of components that will be built just
when building a simple image such as core-image-minimal. If you do not
need runtime tests enabled for core components, then it is recommended
that you remove "ptest" from
:term:`DISTRO_FEATURES` to save a significant
amount of build time e.g. by adding the following in your configuration::
DISTRO_FEATURES_remove = "ptest"
.. _migration-3.1-removed-recipes:
Removed recipes
---------------
The following recipes have been removed:
- ``chkconfig``: obsolete
- ``console-tools``: obsolete
- ``enchant``: replaced by ``enchant2``
- ``foomatic-filters``: obsolete
- ``libidn``: no longer needed, moved to meta-oe
- ``libmodulemd``: replaced by ``libmodulemd-v1``
- ``linux-yocto``: drop 4.19, 5.2 version recipes (5.4 now provided)
- ``nspr``: no longer needed, moved to meta-oe
- ``nss``: no longer needed, moved to meta-oe
- ``python``: Python 2 removed (Python 3 preferred)
- ``python-setuptools``: Python 2 version removed (python3-setuptools
preferred)
- ``sysprof``: no longer needed, moved to meta-oe
- ``texi2html``: obsolete
- ``u-boot-fw-utils``: functionally replaced by ``libubootenv``
.. _migration-3.1-features-check:
features_check class replaces distro_features_check
---------------------------------------------------
The ``distro_features_check`` class has had its functionality expanded,
now supporting ``ANY_OF_MACHINE_FEATURES``,
``REQUIRED_MACHINE_FEATURES``, ``CONFLICT_MACHINE_FEATURES``,
``ANY_OF_COMBINED_FEATURES``, ``REQUIRED_COMBINED_FEATURES``,
``CONFLICT_COMBINED_FEATURES``. As a result the class has now been
renamed to ``features_check``; the ``distro_features_check`` class still
exists but generates a warning and redirects to the new class. In
preparation for a future removal of the old class it is recommended that
you update recipes currently inheriting ``distro_features_check`` to
inherit :ref:`ref-classes-features_check` instead.
.. _migration-3.1-removed-classes:
Removed classes
---------------
The following classes have been removed:
- ``distutils-base``: moved to meta-python2
- ``distutils``: moved to meta-python2
- ``libc-common``: merged into the glibc recipe as nothing else used
it.
- ``python-dir``: moved to meta-python2
- ``pythonnative``: moved to meta-python2
- ``setuptools``: moved to meta-python2
- ``tinderclient``: dropped as it was obsolete.
.. _migration-3.1-src-uri-checksums:
SRC_URI checksum behaviour
--------------------------
Previously, recipes by tradition included both SHA256 and MD5 checksums
for remotely fetched files in :term:`SRC_URI`, even
though only one is actually mandated. However, the MD5 checksum does not
add much given its inherent weakness; thus when a checksum fails only
the SHA256 sum will now be printed. The md5sum will still be verified if
it is specified.
.. _migration-3.1-npm:
npm fetcher changes
-------------------
The npm fetcher has been completely reworked in this release. The npm
fetcher now only fetches the package source itself and no longer the
dependencies; there is now also an npmsw fetcher which explicitly
fetches the shrinkwrap file and the dependencies. This removes the
slightly awkward ``NPM_LOCKDOWN`` and ``NPM_SHRINKWRAP`` variables which
pointed to local files; the lockdown file is no longer needed at all.
Additionally, the package name in ``npm://`` entries in
:term:`SRC_URI` is now specified using a ``package``
parameter instead of the earlier ``name`` which overlapped with the
generic ``name`` parameter. All recipes using the npm fetcher will need
to be changed as a result.
An example of the new scheme::
SRC_URI = "npm://registry.npmjs.org;package=array-flatten;version=1.1.1 \
npmsw://${THISDIR}/npm-shrinkwrap.json"
Another example where the sources are fetched from git rather than an npm repository::
SRC_URI = "git://github.com/foo/bar.git;protocol=https \
npmsw://${THISDIR}/npm-shrinkwrap.json"
devtool and recipetool have also been updated to match with the npm
fetcher changes. Other than producing working and more complete recipes
for npm sources, there is also a minor change to the command line for
devtool: the ``--fetch-dev`` option has been renamed to ``--npm-dev`` as
it is npm-specific.
.. _migration-3.1-packaging-changes:
Packaging changes
-----------------
- ``intltool`` has been removed from ``packagegroup-core-sdk`` as it is
rarely needed to build modern software --- gettext can do most of the
things it used to be needed for. ``intltool`` has also been removed
from ``packagegroup-core-self-hosted`` as it is not needed to for
standard builds.
- git: ``git-am``, ``git-difftool``, ``git-submodule``, and
``git-request-pull`` are no longer perl-based, so are now installed
with the main ``git`` package instead of within ``git-perltools``.
- The ``ldconfig`` binary built as part of glibc has now been moved to
its own ``ldconfig`` package (note no ``glibc-`` prefix). This
package is in the :term:`RRECOMMENDS` of the main
``glibc`` package if ``ldconfig`` is present in
:term:`DISTRO_FEATURES`.
- ``libevent`` now splits each shared library into its own package (as
Debian does). Since these are shared libraries and will be pulled in
through the normal shared library dependency handling, there should
be no impact to existing configurations other than less unnecessary
libraries being installed in some cases.
- linux-firmware now has a new package for ``bcm4366c`` and includes
available NVRAM config files into the ``bcm43340``, ``bcm43362``,
``bcm43430`` and ``bcm4356-pcie`` packages.
- ``harfbuzz`` now splits the new ``libharfbuzz-subset.so`` library
into its own package to reduce the main package size in cases where
``libharfbuzz-subset.so`` is not needed.
.. _migration-3.1-package-qa-warnings:
Additional warnings
-------------------
Warnings will now be shown at :ref:`ref-tasks-package_qa` time in the following
circumstances:
- A recipe installs ``.desktop`` files containing ``MimeType`` keys but
does not inherit the new :ref:`ref-classes-mime-xdg` class
- A recipe installs ``.xml`` files into ``${datadir}/mime/packages``
but does not inherit the :ref:`ref-classes-mime` class
.. _migration-3.1-x86-live-wic:
``wic`` image type now used instead of ``live`` by default for x86
------------------------------------------------------------------
``conf/machine/include/x86-base.inc`` (inherited by most x86 machine
configurations) now specifies ``wic`` instead of ``live`` by default in
:term:`IMAGE_FSTYPES`. The ``live`` image type will
likely be removed in a future release so it is recommended that you use
``wic`` instead.
.. _migration-3.1-misc:
Miscellaneous changes
---------------------
- The undocumented ``SRC_DISTRIBUTE_LICENSES`` variable has now been
removed in favour of a new ``AVAILABLE_LICENSES`` variable which is
dynamically set based upon license files found in
``${COMMON_LICENSE_DIR}`` and ``${LICENSE_PATH}``.
- The tune definition for big-endian microblaze machines is now
``microblaze`` instead of ``microblazeeb``.
- ``newlib`` no longer has built-in syscalls. ``libgloss`` should then
provide the syscalls, ``crt0.o`` and other functions that are no
longer part of ``newlib`` itself. If you are using
``TCLIBC = "newlib"`` this now means that you must link applications
with both ``newlib`` and ``libgloss``, whereas before ``newlib``
would run in many configurations by itself.

Some files were not shown because too many files have changed in this diff Show More