diff --git a/docs/02-mirror-creation/step-by-step-mirror-creation.md b/docs/02-mirror-creation/step-by-step-mirror-creation.md index 2985d239..feb7b7ec 100644 --- a/docs/02-mirror-creation/step-by-step-mirror-creation.md +++ b/docs/02-mirror-creation/step-by-step-mirror-creation.md @@ -125,6 +125,13 @@ export MACHINE=tqma6ul-multi-mba6ulx > The `ACCEPT_FSL_EULA=1` environment variable auto-accepts it. > Without this variable, the setup script blocks waiting for input. +**CRITICAL:** Some recipes (firmware-imx) check local.conf, not just the +environment variable. Add this to your local.conf: + +```bash +echo 'ACCEPT_FSL_EULA = "1"' >> conf/local.conf +``` + **Verify:** ```bash bitbake -e | grep -E "^MACHINE=|^DISTRO=|^DL_DIR=" @@ -256,6 +263,9 @@ EOF # Build cd sources/ci-meta-tq export ACCEPT_FSL_EULA=1 + +# CRITICAL: Also add to local.conf +echo 'ACCEPT_FSL_EULA = "1"' >> conf/local.conf export MACHINE=tqma6ul-multi-mba6ulx . ./setup-environment build_ut mainline diff --git a/docs/02-mirror-creation/step-by-step-mirror-creation.md.bak b/docs/02-mirror-creation/step-by-step-mirror-creation.md.bak new file mode 100644 index 00000000..2985d239 --- /dev/null +++ b/docs/02-mirror-creation/step-by-step-mirror-creation.md.bak @@ -0,0 +1,284 @@ +# Step-by-Step Yocto Mirror Creation with ci-meta-tq + +**Document ID:** PROC-MIRROR-001-REV3 +**Date:** 2026-03-01 +**Verified:** 2026-03-01 (Debian 13 + Ubuntu 22.04 Docker, incl. air-gapped deployment) +**System:** Ubuntu 22.04 LTS (recommended), Debian 12/13 (works with warning) +**Target:** TQMa6UL Yocto Scarthgap mirror (UT build) + +--- + +## Hardware Target + +| Field | Value | +|-------|-------| +| **SoM** | TQMa6UL (sticker: `TQMA6U-AB`) | +| **SoC** | NXP i.MX6 UltraLite (Cortex-A7, single core, 32-bit) | +| **Carrier** | MBa6ULx | +| **Machine** | `tqma6ul-multi-mba6ulx` | +| **BSP Config** | `mainline` (NOT `imx`!) | + +> **Warning:** Do NOT use `tqma6qdl-multi-mba6x` — that is for the Quad/Dual +> variant on a different carrier board. The UltraLite is ONLY in `mainline`. + +--- + +## Prerequisites + +### 1.1 Host System + +| Resource | Minimum | Recommended | +|----------|---------|-------------| +| CPU | 4 cores | 8+ cores | +| RAM | 8 GB | 16+ GB | +| Disk | 100 GB | 200+ GB | +| OS | Ubuntu 22.04 LTS | Ubuntu 22.04 LTS | + +> **Note:** Debian 12/13 also works. BitBake will show a warning +> ("Host distribution has not been validated") but builds succeed. + +### 1.2 Required Packages (Ubuntu 22.04) + +```bash +sudo apt update +sudo apt install -y \ + git python3 jq bash grep gawk wget diffstat \ + chrpath cpio texinfo gcc g++ make file tar \ + bzip2 gzip xz-utils zstd lz4 patch perl \ + python3-pexpect socat unzip rsync bc \ + libsdl1.2-dev xterm +``` + +### 1.3 Locale + +```bash +sudo locale-gen en_US.UTF-8 +export LC_ALL=en_US.UTF-8 +export LANG=en_US.UTF-8 +``` + +--- + +## Step 1: Clone ci-meta-tq + +```bash +export UT_BASE=~/UT +mkdir -p ${UT_BASE} && cd ${UT_BASE} + +# Clone at the Scarthgap release tag +git clone --branch scarthgap.TQ.ARM.BSP.0006 \ + --recurse-submodules \ + https://github.com/tq-systems/ci-meta-tq.git + +cd ci-meta-tq +git submodule sync && git submodule update --init +``` + +**Verify:** +```bash +./ci/ls-configs --file +# Expected: imx, ls, mainline, ti + +./ci/ls-machines --file --config=mainline | grep tqma6ul +# Expected: tqma6ul-multi-mba6ulx +``` + +--- + +## Step 2: Configure Downloads Directory + +```bash +# Create shared download directory (outside build tree) +sudo mkdir -p /srv/yocto/downloads /srv/yocto/sstate-cache +sudo chown -R "$USER:$USER" /srv/yocto + +# Create site.conf (global Yocto config) +mkdir -p ~/.yocto +cat > ~/.yocto/site.conf << 'EOF' +# UT Project - Download Directories +DL_DIR ?= "/srv/yocto/downloads" +SSTATE_DIR ?= "/srv/yocto/sstate-cache" +EOF +``` + +> **Important:** Do NOT add premirror/own-mirrors config yet! +> That is only for the air-gapped target machine (Step 6). + +--- + +## Step 3: Initialize Build Environment + +```bash +cd ${UT_BASE}/ci-meta-tq + +# CRITICAL: Accept the NXP/FSL EULA (required for firmware-imx) +export ACCEPT_FSL_EULA=1 + +# Set machine +export MACHINE=tqma6ul-multi-mba6ulx + +# Initialize (creates build_ut/ directory) +. ./setup-environment build_ut mainline +``` + +> **Note:** You will see a Qt6 GPL Exception EULA prompt. +> The `ACCEPT_FSL_EULA=1` environment variable auto-accepts it. +> Without this variable, the setup script blocks waiting for input. + +**Verify:** +```bash +bitbake -e | grep -E "^MACHINE=|^DISTRO=|^DL_DIR=" +# Expected: +# MACHINE="tqma6ul-multi-mba6ulx" +# DISTRO="spaetzle" +# DL_DIR="/srv/yocto/downloads" +``` + +--- + +## Step 4: Download All Sources (Fetch Only) + +For mirror creation, you do NOT need a full build. Fetch-only is faster: + +```bash +# Fetch all sources for the target image (no compilation) +bitbake tq-image-small-debug --runall=fetch +``` + +**Expected:** ~510 fetch tasks, takes 30-60 minutes depending on bandwidth. + +**Verify:** +```bash +du -sh /srv/yocto/downloads/ +# Expected: ~5 GB + +ls /srv/yocto/downloads/*.done | wc -l +# Expected: ~138 completed downloads +``` + +> **Alternative:** For a full build (also populates sstate-cache): +> ```bash +> bitbake tq-image-small-debug +> ``` +> This takes 4-8 hours but also verifies the build actually succeeds. + +--- + +## Step 5: Generate License Table + +The license table is generated from BitBake metadata, NOT from build output: + +```bash +cd ${UT_BASE}/ci-meta-tq +export ACCEPT_FSL_EULA=1 +export MACHINE=tqma6ul-multi-mba6ulx +. ./setup-environment build_ut mainline + +# Generate the build list (all 264 recipes) +bitbake tq-image-small-debug -g +# Creates: pn-buildlist, task-depends.dot + +# Extract license + SRC_URI for every recipe +echo "Package,Version,License,SRC_URI,Recipe_File" > license-table.csv + +while IFS= read -r recipe; do + output=$(bitbake -e "$recipe" 2>/dev/null) + pv=$(echo "$output" | grep '^PV=' | head -1 | sed 's/^PV="//' | sed 's/"$//') + license=$(echo "$output" | grep '^LICENSE=' | head -1 | sed 's/^LICENSE="//' | sed 's/"$//') + src_uri=$(echo "$output" | grep '^SRC_URI=' | head -1 | sed 's/^SRC_URI="//' | sed 's/"$//') + recipe_file=$(echo "$output" | grep '^FILE=' | head -1 | sed 's/^FILE="//' | sed 's/"$//') + first_url=$(echo "$src_uri" | grep -oP '(https?|ftp|git)://[^\s]+' | head -1 | sed 's/;.*//') + recipe_file=$(echo "$recipe_file" | sed "s|.*/sources/||") + license=$(echo "$license" | tr ',' ';') + echo "\"$recipe\",\"$pv\",\"$license\",\"$first_url\",\"$recipe_file\"" >> license-table.csv +done < pn-buildlist + +echo "Done: $(wc -l license-table.csv) entries" +``` + +> **Note:** This takes ~30 minutes (one `bitbake -e` call per recipe). +> All Yocto variables (`${GNU_MIRROR}`, `${PV}`, etc.) are fully resolved. + +--- + +## Step 6: Package the Mirror + +```bash +cd ${UT_BASE} + +TIMESTAMP=$(date +%Y%m%d) +ARCHIVE="UT-tqma6ul-yocto-scarthgap-${TIMESTAMP}" + +mkdir -p ${ARCHIVE}/{sources,downloads,configs,docs} + +# Copy repository (with submodules) +cp -r ci-meta-tq ${ARCHIVE}/sources/ + +# Copy all downloaded sources +cp -r /srv/yocto/downloads/* ${ARCHIVE}/downloads/ + +# Copy configs +cp ~/.yocto/site.conf ${ARCHIVE}/configs/ +cp ci-meta-tq/build_ut/conf/local.conf ${ARCHIVE}/configs/ 2>/dev/null +cp ci-meta-tq/build_ut/conf/bblayers.conf ${ARCHIVE}/configs/ 2>/dev/null + +# Copy license table +cp ci-meta-tq/build_ut/license-table.csv ${ARCHIVE}/docs/ + +# Create archive +tar czf ${ARCHIVE}.tar.gz ${ARCHIVE}/ +sha256sum ${ARCHIVE}.tar.gz > ${ARCHIVE}.tar.gz.sha256 + +echo "Archive: ${ARCHIVE}.tar.gz ($(du -h ${ARCHIVE}.tar.gz | cut -f1))" +``` + +--- + +## Step 7: Air-Gapped Deployment + +On the target machine (no internet): + +```bash +# Extract +tar xzf UT-tqma6ul-yocto-scarthgap-*.tar.gz +cd UT-tqma6ul-yocto-scarthgap-*/ + +# Configure premirror (point to local downloads) +mkdir -p ~/.yocto +cat > ~/.yocto/site.conf << EOF +DL_DIR = "$(pwd)/downloads" +SSTATE_DIR = "/tmp/sstate-cache" +SOURCE_MIRROR_URL = "file://$(pwd)/downloads/" +INHERIT += "own-mirrors" +BB_NO_NETWORK = "1" +EOF + +# Build +cd sources/ci-meta-tq +export ACCEPT_FSL_EULA=1 +export MACHINE=tqma6ul-multi-mba6ulx +. ./setup-environment build_ut mainline + +bitbake tq-image-small-debug +``` + +--- + +## Verified Results (2026-03-01) + +| Step | Status | Notes | +|------|--------|-------| +| Clone ci-meta-tq | ✅ | Tag scarthgap.TQ.ARM.BSP.0006 | +| setup-environment | ✅ | Requires ACCEPT_FSL_EULA=1 | +| Fetch all sources | ✅ | 510/510 tasks, ~5 GB | +| License extraction | ✅ | 264/264 recipes, 224 URLs | +| ci/ls-machines | ✅ | tqma6ul-multi-mba6ulx found | +| Debian 13 host | ✅ | Warning but functional | +| Ubuntu 22.04 host | ✅ | Tested in Docker, all steps pass | +| Air-gapped fetch (BB_NO_NETWORK=1) | ✅ | 510/510 from local mirror | +| Full build | ⬜ | Not tested (sandbox RAM limit, needs 8GB+) | + +--- + +**Author:** Siggi ⚙️ +**Revision:** 3.1 (Ubuntu 22.04 + air-gap verified)