tqma6-yocto-mirror/docs/02-mirror-creation/step-by-step-mirror-creation.md

# Step-by-Step Yocto Mirror Creation with ci-meta-tq

**Document ID:** PROC-MIRROR-001-REV2  
**Date:** 2026-03-01  
**System:** Ubuntu 22.04 LTS  
**Environment:** Outside corporate network (Internet required)  
**Target:** TQMa6UL Yocto Scarthgap mirror (UT build)  
**Build Name:** UT (Universität/Projekt)

---

## Hardware Target Confirmation

Based on sticker analysis and code review:

**Sticker:** TQMA6U-AB RK REV.0405  
**Board:** TQMa6UL (NXP i.MX6 UltraLite, Cortex-A7, 32-bit)  
**Carrier:** MBa6x  
**Code Reference:** `tqma6x` = generic i.MX6 family identifier in legacy codebase

**Machine Configuration:** `tqma6ulx-mba6ulx`  
*Alternative (ci-meta-tq multi-config):* `tqma6qdl-multi-mba6x`

---

## Prerequisites

### 1.1 Host System Requirements

| Resource | Minimum | Recommended | Notes |
|----------|---------|-------------|-------|
| CPU | 4 cores | 8+ cores | Parallel builds |
| RAM | 8 GB | 16+ GB | BitBake memory-intensive |
| Disk Space | 100 GB | 200+ GB | Sources + build artifacts |
| OS | Ubuntu 22.04 | Ubuntu 22.04 LTS | Verified platform |
| Internet | Required | Broadband | For initial download only |

### 1.2 Required Packages

Install on Ubuntu 22.04:

```bash
sudo apt update
sudo apt install -y \
    git python3 jq bash grep gawk wget diffstat \
    chrpath cpio texinfo gcc g++ make file tar \
    bzip2 gzip xz-utils zstd lz4 patch perl \
    python3-pexpect socat unzip rsync bc \
    libsdl1.2-dev xterm
```

### 1.3 Locale Configuration

```bash
sudo locale-gen en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
```

---

## Step 1: Create Working Directory

```bash
# Create base directory for mirror creation
export UT_BASE=~/UT-yocto-mirror
mkdir -p ${UT_BASE}
cd ${UT_BASE}

# Create subdirectories
mkdir -p mirror-bundle build-scripts
```

---

## Step 2: Clone ci-meta-tq Repository

The official TQ CI repository includes all required layers as submodules:

```bash
cd ${UT_BASE}

# Clone ci-meta-tq with all submodules (Scarthgap branch)
git clone --branch scarthgap.TQ.ARM.BSP.0006 \
    --recurse-submodules \
    https://github.com/tq-systems/ci-meta-tq.git \
    ci-meta-tq

cd ci-meta-tq

# Verify submodules are initialized
git submodule sync
git submodule update --init

# Verify structure
ls -la sources/
# Should show: meta-tq, poky, meta-openembedded, etc.
```

**Verification:**
```bash
cd ${UT_BASE}/ci-meta-tq
./ci/ls-configs --file
./ci/ls-machines --file --config=imx
# Should list tqma6ulx-mba6ulx and other machines
```

---

## Step 3: Configure Build Environment for UT

### 3.1 Create Site Configuration

Create central configuration file `~/.yocto/site.conf`:

```bash
mkdir -p ~/.yocto

cat > ~/.yocto/site.conf <<'EOF'
# UT Project Configuration
# Build: UT (Universität/Projekt)
# Target: TQMa6UL on MBa6x
# Date: 2026-03-01

# Mirror directories (outside home for space)
DL_DIR ?= "/srv/yocto/downloads"
SSTATE_DIR ?= "/srv/yocto/sstate-cache"

# Create directories
EOF

# Create system directories
sudo mkdir -p /srv/yocto/downloads /srv/yocto/sstate-cache
sudo chown -R "$USER:$USER" /srv/yocto
```

### 3.2 Initialize Build Environment

```bash
cd ${UT_BASE}/ci-meta-tq

# Set machine for TQMa6UL
export MACHINE=tqma6ulx-mba6ulx

# Alternative if using multi-config:
# export MACHINE=tqma6qdl-multi-mba6x

# Initialize build environment (creates build_ut directory)
. ./setup-environment build_ut imx

# Verify configuration
bitbake -e | grep -E "^MACHINE=|^DL_DIR=|^SSTATE_DIR="
```

---

## Step 4: First Build (Downloads All Sources)

This step downloads all required source packages for offline use:

```bash
cd ${UT_BASE}/ci-meta-tq

# Ensure environment is set
export MACHINE=tqma6ulx-mba6ulx
. ./setup-environment build_ut imx

# Build the image (downloads all sources)
bitbake tq-image-small-debug

# Alternative images:
# bitbake core-image-minimal
# bitbake core-image-full-cmdline
```

**Expected duration:** 4-8 hours (first build, depends on hardware)

**What happens:**
1. BitBake parses all recipes from all layers
2. Downloads source code for every package to `/srv/yocto/downloads`
3. Populates shared state cache in `/srv/yocto/sstate-cache`
4. Builds toolchain, kernel, bootloader, rootfs
5. Creates deployment images in `build_ut/tmp/deploy/images/`

---

## Step 5: Create Source Mirror for Air-Gap

### 5.1 Configure Premirror

Add premirror configuration to site.conf:

```bash
cat >> ~/.yocto/site.conf <<'EOF'

# --- Offline / Premirror Setup ---
# These settings enable offline builds after mirror creation

SOURCE_MIRROR_URL ?= "file:///srv/yocto/premirror/"
INHERIT += "own-mirrors"

PREMIRRORS:prepend = " \
    git://.*/.* file:///srv/yocto/premirror/ \n \
    ftp://.*/.* file:///srv/yocto/premirror/ \n \
    http://.*/.* file:///srv/yocto/premirror/ \n \
    https://.*/.* file:///srv/yocto/premirror/ \n \
"
EOF

# Create premirror directory
sudo mkdir -p /srv/yocto/premirror
sudo chown -R "$USER:$USER" /srv/yocto
```

### 5.2 Fill Mirror with ci-meta-tq Script

Use TQ's provided script to populate the mirror:

```bash
cd ${UT_BASE}/ci-meta-tq

# Set machine and config
export MACHINE=tqma6ulx-mba6ulx

# Fill mirror (downloads all git repos and tarballs)
ci/fill_mirror build_ut imx

# Verify mirror contents
ls -la /srv/yocto/premirror/
du -sh /srv/yocto/premirror/
du -sh /srv/yocto/downloads/
du -sh /srv/yocto/sstate-cache/
```

### 5.3 Generate License Information

```bash
cd ${UT_BASE}/ci-meta-tq
. ./setup-environment build_ut imx

# Generate license manifest
bitbake tq-image-small-debug -c do_populate_lic

# Copy license files
mkdir -p ${UT_BASE}/mirror-bundle/licenses
cp -r build_ut/tmp/deploy/licenses/* ${UT_BASE}/mirror-bundle/licenses/

# Create license table (see separate script)
```

---

## Step 6: Package the Mirror

### 6.1 Create Archive Structure

```bash
cd ${UT_BASE}

# Create organized structure for approval
mkdir -p UT-mirror-package/{sources,premirror,downloads,sstate-cache,licenses,configs}

# Copy ci-meta-tq repository (with submodules)
cp -r ci-meta-tq UT-mirror-package/sources/

# Copy mirror contents
cp -r /srv/yocto/premirror/* UT-mirror-package/premirror/ 2>/dev/null || true
cp -r /srv/yocto/downloads/* UT-mirror-package/downloads/ 2>/dev/null || true

# Copy licenses
cp -r mirror-bundle/licenses/* UT-mirror-package/licenses/ 2>/dev/null || true

# Copy configuration
cp ~/.yocto/site.conf UT-mirror-package/configs/
cp ci-meta-tq/build_ut/conf/local.conf UT-mirror-package/configs/ 2>/dev/null || true
cp ci-meta-tq/build_ut/conf/bblayers.conf UT-mirror-package/configs/ 2>/dev/null || true

# Create build script
cat > UT-mirror-package/build-instructions/README.txt <<'EOF'
UT Yocto Mirror Package
=======================

Target: TQMa6UL on MBa6x (i.MX6 UltraLite)
Yocto: Scarthgap 5.0 LTS
BSP: TQ scarthgap.TQ.ARM.BSP.0006
Build: UT (Universität/Projekt)
2038 Status: Compliant (64-bit time_t on 32-bit ARM)

Directory Structure:
- sources/       : ci-meta-tq repository with all submodules
- premirror/     : Git repositories and source tarballs
- downloads/     : Downloaded source files
- sstate-cache/  : Build cache (optional)
- licenses/      : License manifests for all packages
- configs/       : Site configuration and build configs

See build-instructions/ for detailed deployment guide.
EOF

# Create detailed deployment guide
cp docs/04-deployment/corporate-network-deployment.md UT-mirror-package/build-instructions/
```

### 6.2 Create Compressed Archive

```bash
cd ${UT_BASE}

# Create timestamped archive
export TIMESTAMP=$(date +%Y%m%d-%H%M%S)
export ARCHIVE_NAME="UT-tqma6ul-yocto-scarthgap-${TIMESTAMP}.tar.gz"

tar czf ${ARCHIVE_NAME} \
    --exclude='.git/objects/pack/*' \
    --exclude='*.tmp' \
    --exclude='*.log' \
    UT-mirror-package/

# Calculate checksums
sha256sum ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.sha256
md5sum ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.md5

echo "Archive created: ${ARCHIVE_NAME}"
echo "Size: $(du -h ${ARCHIVE_NAME} | cut -f1)"
echo "SHA256: $(cat ${ARCHIVE_NAME}.sha256 | cut -d' ' -f1)"
```

---

## Step 7: Generate License Table

### 7.1 Create License CSV

```bash
cd ${UT_BASE}

# Generate from BitBake license manifest
cat > generate-license-table.sh <<'SCRIPT'
#!/bin/bash
# Generate license table for software approval

OUTPUT="UT-license-table.csv"
echo "Package,Version,Recipe,License,MilitaryUse,SourceLocation,Notes" > $OUTPUT

# Process license manifest
find build_ut/tmp/deploy/licenses -name "*.csv" -o -name "*.manifest" | while read f; do
    # Parse and format for approval
    # This is a template - actual parsing depends on manifest format
    echo "Processing: $f"
done

echo "License table generated: $OUTPUT"
SCRIPT

chmod +x generate-license-table.sh
```

### 7.2 Manual License Review

Key packages to verify for military use:

| Package | License | Military Use | Verification |
|---------|---------|--------------|--------------|
| linux-yocto | GPL-2.0-only | ✅ Yes | Kernel source included |
| u-boot-tq | GPL-2.0+ | ✅ Yes | Source included |
| glibc | LGPL-2.1+ | ✅ Yes | Source included |
| busybox | GPL-2.0-only | ✅ Yes | Source included |
| imx-firmware | Proprietary | ⚠️ Check | Review NXP license |

---

## Verification Checklist

Before delivering mirror:

- [ ] ci-meta-tq cloned with all submodules
- [ ] MACHINE=tqma6ulx-mba6ulx configured
- [ ] Build "UT" completed successfully
- [ ] /srv/yocto/downloads/ populated
- [ ] /srv/yocto/premirror/ populated by ci/fill_mirror
- [ ] License manifests generated
- [ ] Archive created with timestamp
- [ ] SHA256 checksum calculated
- [ ] License table (CSV) generated

---

## Deliverables for Software Approval

1. **UT-mirror-package.tar.gz** - Complete source mirror
2. **UT-license-table.csv** - License compliance table
3. **SHA256 checksums** - Integrity verification
4. **Build instructions** - Corporate deployment guide
5. **2038-compliance-doc** - Analysis document

---

## Next Steps

1. **Transfer archive** to corporate network (secure USB transfer)
2. **Submit for software approval** with license table
3. **Deploy in air-gapped environment** per deployment guide
4. **Build verification** - confirm offline build succeeds

---

**Document Version:** 2.0  
**Author:** Siggi ⚙️  
**Build Name:** UT  
**Target:** TQMa6UL  
**Last Updated:** 2026-03-01