tqma6-yocto-mirror/docs/02-mirror-creation/step-by-step-mirror-creation.md

Step-by-Step Yocto Mirror Creation

Document ID: PROC-MIRROR-001
Date: 2026-03-01
System: Ubuntu 22.04 LTS
Environment: Outside corporate network (Internet access required)
Target: TQMa6UL Yocto Scarthgap mirror for offline deployment

---

Prerequisites

1.1 Host System Requirements

Resource	Minimum	Recommended	Notes
CPU	4 cores	8+ cores	Parallel builds
RAM	8 GB	16+ GB	BitBake is memory-intensive
Disk Space	100 GB	200+ GB	Sources + build artifacts
OS	Ubuntu 22.04	Ubuntu 22.04 LTS	Verified platform
Internet	Required	Broadband	For initial download

1.2 Required Packages

Install on Ubuntu 22.04:

sudo apt update
sudo apt install -y \
    gawk wget git diffstat unzip texinfo gcc build-essential \
    chrpath socat cpio python3 python3-pip python3-pexpect \
    xz-utils debianutils iputils-ping python3-git \
    python3-jinja2 libegl1-mesa libsdl1.2-dev xterm \
    locales lz4 zstd rpcsvc-proto \
    file bsdmainutils curl

# Ensure Python 3 is default
sudo apt install -y python3-distutils

1.3 Locale Configuration

sudo locale-gen en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

---

Step 1: Create Working Directory

# Create base directory for mirror creation
mkdir -p ~/tqma6-yocto-mirror
export MIRROR_BASE=~/tqma6-yocto-mirror
cd $MIRROR_BASE

# Create subdirectories
mkdir -p sources layers downloads sstate-cache

---

Step 2: Clone Yocto Poky (Reference Distribution)

cd $MIRROR_BASE/sources

# Clone Poky (Yocto reference) - Scarthgap branch
git clone -b scarthgap --depth 1 \
    https://git.yoctoproject.org/git/poky.git \
    poky-scarthgap

cd poky-scarthgap
# Full clone for complete history (optional but recommended for approval)
git fetch --unshallow

cd $MIRROR_BASE/sources

Verification:

cd poky-scarthgap
git log --oneline -1
# Should show: scarthgap branch, recent commit
git describe --tags
# Should show: yocto-5.0.x or similar

---

Step 3: Clone OpenEmbedded Meta-Layer

cd $MIRROR_BASE/sources

# Clone meta-openembedded (contains essential recipes)
git clone -b scarthgap --depth 1 \
    https://git.openembedded.org/meta-openembedded \
    meta-openembedded

cd meta-openembedded
git fetch --unshallow

cd $MIRROR_BASE/sources

Included sub-layers:

• meta-oe - Base additional recipes
• meta-python - Python packages
• meta-networking - Network tools
• meta-filesystems - Filesystem tools
• meta-perl - Perl modules

---

Step 4: Clone TQ BSP Layer

cd $MIRROR_BASE/sources

# Clone TQ BSP layer - specific version for TQMa6UL
git clone -b scarthgap.TQ.ARM.BSP.0006 --depth 1 \
    https://github.com/tq-systems/meta-tq.git \
    meta-tq

cd meta-tq
git fetch --unshallow

# Verify the branch
git branch -a | grep scarthgap

cd $MIRROR_BASE/sources

Critical: Ensure you use the exact tag scarthgap.TQ.ARM.BSP.0006 for reproducibility.

---

Step 5: Initialize Build Environment

cd $MIRROR_BASE/sources/poky-scarthgap

# Source the build environment
source oe-init-build-env ../build

# This creates the build directory and sets up environment
# You should now be in: ~/tqma6-yocto-mirror/build

---

Step 6: Configure Build for TQMa6UL

6.1 Edit conf/local.conf

# Edit the local.conf file
vim conf/local.conf

Add/modify these settings:

# Machine selection for TQMa6UL on MBa6x
MACHINE ??= "tqma6ulx-mba6ulx"

# Parallelism (adjust to your CPU)
BB_NUMBER_THREADS ?= "8"
PARALLEL_MAKE ?= "-j8"

# Download directory (for mirroring)
DL_DIR ?= "${MIRROR_BASE}/downloads"

# Shared state cache
SSTATE_DIR ?= "${MIRROR_BASE}/sstate-cache"

# Keep source code for license compliance
RM_WORK_EXCLUDE += "*"
INHERIT += "archiver"
ARCHIVER_MODE[src] = "original"
ARCHIVER_MODE[diff] = "1"

# Generate license information
COPY_LIC_MANIFEST = "1"
COPY_LIC_DIRS = "1"

# Create source mirror
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL ?= "file://${MIRROR_BASE}/mirror/"
BB_GENERATE_MIRROR_TARBALLS = "1"

6.2 Add Layers to conf/bblayers.conf

vim conf/bblayers.conf

Content:

BBLAYERS ?= " \
  ${MIRROR_BASE}/sources/poky-scarthgap/meta \
  ${MIRROR_BASE}/sources/poky-scarthgap/meta-poky \
  ${MIRROR_BASE}/sources/poky-scarthgap/meta-yocto-bsp \
  ${MIRROR_BASE}/sources/meta-openembedded/meta-oe \
  ${MIRROR_BASE}/sources/meta-openembedded/meta-python \
  ${MIRROR_BASE}/sources/meta-openembedded/meta-networking \
  ${MIRROR_BASE}/sources/meta-openembedded/meta-filesystems \
  ${MIRROR_BASE}/sources/meta-tq/meta-tq \
  "

---

Step 7: First Build (Downloads All Sources)

# Still in build directory
# This will download all required source packages

bitbake core-image-minimal

Expected duration: 2-6 hours (first build, depending on hardware)

What happens:

1. BitBake parses all recipes
2. Downloads source code for every package
3. Builds toolchain, kernel, rootfs
4. Creates deployment images

---

Step 8: Create Source Mirror

8.1 Generate Mirror Tarballs

# Create mirror directory
mkdir -p ${MIRROR_BASE}/mirror

# Copy downloaded sources
cp -r ${MIRROR_BASE}/downloads/* ${MIRROR_BASE}/mirror/ 2>/dev/null || true

# Create source archive with archiver
bitbake core-image-minimal -c do_populate_lic

8.2 Collect License Information

# Generate license manifest
bitbake core-image-minimal -c do_populate_lic

# Copy license files
mkdir -p ${MIRROR_BASE}/licenses
cp -r ${MIRROR_BASE}/build/tmp/deploy/licenses/* ${MIRROR_BASE}/licenses/

# Create license CSV (see separate script)

---

Step 9: Package the Mirror

9.1 Create Archive Structure

cd ${MIRROR_BASE}

# Create organized structure for approval
mkdir -p mirror-package/{sources,licenses,build-instructions}

# Copy source layers (git repositories)
cp -r sources/* mirror-package/sources/

# Copy license information
cp -r licenses/* mirror-package/licenses/ 2>/dev/null || true

# Copy downloads (source tarballs)
mkdir -p mirror-package/downloads
cp -r downloads/* mirror-package/downloads/ 2>/dev/null || true

# Document the configuration
cp build/conf/local.conf mirror-package/build-instructions/
cp build/conf/bblayers.conf mirror-package/build-instructions/

9.2 Create Archive

cd ${MIRROR_BASE}

# Create timestamped archive
export TIMESTAMP=$(date +%Y%m%d-%H%M%S)
export ARCHIVE_NAME="tqma6-yocto-mirror-scarthgap-${TIMESTAMP}.tar.gz"

tar czf ${ARCHIVE_NAME} \
    --exclude='.git/objects/pack/*' \
    mirror-package/

# Calculate checksum
sha256sum ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.sha256

echo "Archive created: ${ARCHIVE_NAME}"
echo "Size: $(du -h ${ARCHIVE_NAME} | cut -f1)"

---

Step 10: Generate License Table

See: License Table Generation Script

The license table must include:

• Package name
• Version
• License (SPDX identifier)
• License file location
• Source location
• Military use approval (Yes/No/Conditional)

---

Verification Checklist

Before delivering mirror:

• All git repositories cloned (poky, meta-openembedded, meta-tq)
• Complete source downloads in downloads/ directory
• Build successful: core-image-minimal completed
• License information generated in licenses/
• Archive created with timestamp
• SHA256 checksum calculated
• License compliance table generated

---

Troubleshooting

Problem: Fetch failures during build

Solution: Check internet connectivity, proxy settings. Some corporate networks block git:// protocol:

git config --global url."https://".insteadOf git://

Problem: Disk space exhausted

Solution: Ensure at least 100GB free. Clean build if needed:

bitbake -c cleanall core-image-minimal

Problem: Permission denied

Solution: Ensure proper user permissions, avoid building as root.

---

Next Steps

1. Transfer archive to corporate network (USB, secure transfer)
2. Submit for software approval with license table
3. Deploy in corporate environment following Deployment Guide

---

Document Version: 1.0
Author: Siggi ⚙️
Last Updated: 2026-03-01