Initial documentation for TQMa6UL Yocto mirror project
Add comprehensive documentation for 2038-compliant BSP migration: - README.md: Project overview, 2038 compliance verification - 2038-problem-analysis.md: Detailed technical analysis confirming Scarthgap (5.0) provides 64-bit time_t on 32-bit ARM - step-by-step-mirror-creation.md: Complete procedure for creating offline mirror on Ubuntu 22.04 outside corporate network - license-compliance.md: License categories, approval criteria, and table templates for military use approval - corporate-network-deployment.md: Installation and build setup for air-gapped corporate environment Target: TQMa6UL-AB (i.MX6 UltraLite) on MBa6x BSP: TQ scarthgap.TQ.ARM.BSP.0006 2038 Status: Verified compliant (kernel 6.6 + glibc 2.38+) Repo: https://code.gegen.autos/openclaw/tqma6-yocto-mirror
This commit is contained in:
OpenClaw (Siggi)
375
docs/02-mirror-creation/step-by-step-mirror-creation.md
Normal file
375
docs/02-mirror-creation/step-by-step-mirror-creation.md
Normal file
@@ -0,0 +1,375 @@
|
||||
# Step-by-Step Yocto Mirror Creation
|
||||
|
||||
**Document ID:** PROC-MIRROR-001
|
||||
**Date:** 2026-03-01
|
||||
**System:** Ubuntu 22.04 LTS
|
||||
**Environment:** Outside corporate network (Internet access required)
|
||||
**Target:** TQMa6UL Yocto Scarthgap mirror for offline deployment
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### 1.1 Host System Requirements
|
||||
|
||||
| Resource | Minimum | Recommended | Notes |
|
||||
|----------|---------|-------------|-------|
|
||||
| CPU | 4 cores | 8+ cores | Parallel builds |
|
||||
| RAM | 8 GB | 16+ GB | BitBake is memory-intensive |
|
||||
| Disk Space | 100 GB | 200+ GB | Sources + build artifacts |
|
||||
| OS | Ubuntu 22.04 | Ubuntu 22.04 LTS | Verified platform |
|
||||
| Internet | Required | Broadband | For initial download |
|
||||
|
||||
### 1.2 Required Packages
|
||||
|
||||
Install on Ubuntu 22.04:
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y \
|
||||
gawk wget git diffstat unzip texinfo gcc build-essential \
|
||||
chrpath socat cpio python3 python3-pip python3-pexpect \
|
||||
xz-utils debianutils iputils-ping python3-git \
|
||||
python3-jinja2 libegl1-mesa libsdl1.2-dev xterm \
|
||||
locales lz4 zstd rpcsvc-proto \
|
||||
file bsdmainutils curl
|
||||
|
||||
# Ensure Python 3 is default
|
||||
sudo apt install -y python3-distutils
|
||||
```
|
||||
|
||||
### 1.3 Locale Configuration
|
||||
|
||||
```bash
|
||||
sudo locale-gen en_US.UTF-8
|
||||
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
|
||||
export LC_ALL=en_US.UTF-8
|
||||
export LANG=en_US.UTF-8
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Create Working Directory
|
||||
|
||||
```bash
|
||||
# Create base directory for mirror creation
|
||||
mkdir -p ~/tqma6-yocto-mirror
|
||||
export MIRROR_BASE=~/tqma6-yocto-mirror
|
||||
cd $MIRROR_BASE
|
||||
|
||||
# Create subdirectories
|
||||
mkdir -p sources layers downloads sstate-cache
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Clone Yocto Poky (Reference Distribution)
|
||||
|
||||
```bash
|
||||
cd $MIRROR_BASE/sources
|
||||
|
||||
# Clone Poky (Yocto reference) - Scarthgap branch
|
||||
git clone -b scarthgap --depth 1 \
|
||||
https://git.yoctoproject.org/git/poky.git \
|
||||
poky-scarthgap
|
||||
|
||||
cd poky-scarthgap
|
||||
# Full clone for complete history (optional but recommended for approval)
|
||||
git fetch --unshallow
|
||||
|
||||
cd $MIRROR_BASE/sources
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
cd poky-scarthgap
|
||||
git log --oneline -1
|
||||
# Should show: scarthgap branch, recent commit
|
||||
git describe --tags
|
||||
# Should show: yocto-5.0.x or similar
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Clone OpenEmbedded Meta-Layer
|
||||
|
||||
```bash
|
||||
cd $MIRROR_BASE/sources
|
||||
|
||||
# Clone meta-openembedded (contains essential recipes)
|
||||
git clone -b scarthgap --depth 1 \
|
||||
https://git.openembedded.org/meta-openembedded \
|
||||
meta-openembedded
|
||||
|
||||
cd meta-openembedded
|
||||
git fetch --unshallow
|
||||
|
||||
cd $MIRROR_BASE/sources
|
||||
```
|
||||
|
||||
**Included sub-layers:**
|
||||
- `meta-oe` - Base additional recipes
|
||||
- `meta-python` - Python packages
|
||||
- `meta-networking` - Network tools
|
||||
- `meta-filesystems` - Filesystem tools
|
||||
- `meta-perl` - Perl modules
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Clone TQ BSP Layer
|
||||
|
||||
```bash
|
||||
cd $MIRROR_BASE/sources
|
||||
|
||||
# Clone TQ BSP layer - specific version for TQMa6UL
|
||||
git clone -b scarthgap.TQ.ARM.BSP.0006 --depth 1 \
|
||||
https://github.com/tq-systems/meta-tq.git \
|
||||
meta-tq
|
||||
|
||||
cd meta-tq
|
||||
git fetch --unshallow
|
||||
|
||||
# Verify the branch
|
||||
git branch -a | grep scarthgap
|
||||
|
||||
cd $MIRROR_BASE/sources
|
||||
```
|
||||
|
||||
**Critical:** Ensure you use the exact tag `scarthgap.TQ.ARM.BSP.0006` for reproducibility.
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Initialize Build Environment
|
||||
|
||||
```bash
|
||||
cd $MIRROR_BASE/sources/poky-scarthgap
|
||||
|
||||
# Source the build environment
|
||||
source oe-init-build-env ../build
|
||||
|
||||
# This creates the build directory and sets up environment
|
||||
# You should now be in: ~/tqma6-yocto-mirror/build
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Configure Build for TQMa6UL
|
||||
|
||||
### 6.1 Edit `conf/local.conf`
|
||||
|
||||
```bash
|
||||
# Edit the local.conf file
|
||||
vim conf/local.conf
|
||||
```
|
||||
|
||||
**Add/modify these settings:**
|
||||
|
||||
```conf
|
||||
# Machine selection for TQMa6UL on MBa6x
|
||||
MACHINE ??= "tqma6ulx-mba6ulx"
|
||||
|
||||
# Parallelism (adjust to your CPU)
|
||||
BB_NUMBER_THREADS ?= "8"
|
||||
PARALLEL_MAKE ?= "-j8"
|
||||
|
||||
# Download directory (for mirroring)
|
||||
DL_DIR ?= "${MIRROR_BASE}/downloads"
|
||||
|
||||
# Shared state cache
|
||||
SSTATE_DIR ?= "${MIRROR_BASE}/sstate-cache"
|
||||
|
||||
# Keep source code for license compliance
|
||||
RM_WORK_EXCLUDE += "*"
|
||||
INHERIT += "archiver"
|
||||
ARCHIVER_MODE[src] = "original"
|
||||
ARCHIVER_MODE[diff] = "1"
|
||||
|
||||
# Generate license information
|
||||
COPY_LIC_MANIFEST = "1"
|
||||
COPY_LIC_DIRS = "1"
|
||||
|
||||
# Create source mirror
|
||||
INHERIT += "own-mirrors"
|
||||
SOURCE_MIRROR_URL ?= "file://${MIRROR_BASE}/mirror/"
|
||||
BB_GENERATE_MIRROR_TARBALLS = "1"
|
||||
```
|
||||
|
||||
### 6.2 Add Layers to `conf/bblayers.conf`
|
||||
|
||||
```bash
|
||||
vim conf/bblayers.conf
|
||||
```
|
||||
|
||||
**Content:**
|
||||
```conf
|
||||
BBLAYERS ?= " \
|
||||
${MIRROR_BASE}/sources/poky-scarthgap/meta \
|
||||
${MIRROR_BASE}/sources/poky-scarthgap/meta-poky \
|
||||
${MIRROR_BASE}/sources/poky-scarthgap/meta-yocto-bsp \
|
||||
${MIRROR_BASE}/sources/meta-openembedded/meta-oe \
|
||||
${MIRROR_BASE}/sources/meta-openembedded/meta-python \
|
||||
${MIRROR_BASE}/sources/meta-openembedded/meta-networking \
|
||||
${MIRROR_BASE}/sources/meta-openembedded/meta-filesystems \
|
||||
${MIRROR_BASE}/sources/meta-tq/meta-tq \
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 7: First Build (Downloads All Sources)
|
||||
|
||||
```bash
|
||||
# Still in build directory
|
||||
# This will download all required source packages
|
||||
|
||||
bitbake core-image-minimal
|
||||
```
|
||||
|
||||
**Expected duration:** 2-6 hours (first build, depending on hardware)
|
||||
|
||||
**What happens:**
|
||||
1. BitBake parses all recipes
|
||||
2. Downloads source code for every package
|
||||
3. Builds toolchain, kernel, rootfs
|
||||
4. Creates deployment images
|
||||
|
||||
---
|
||||
|
||||
## Step 8: Create Source Mirror
|
||||
|
||||
### 8.1 Generate Mirror Tarballs
|
||||
|
||||
```bash
|
||||
# Create mirror directory
|
||||
mkdir -p ${MIRROR_BASE}/mirror
|
||||
|
||||
# Copy downloaded sources
|
||||
cp -r ${MIRROR_BASE}/downloads/* ${MIRROR_BASE}/mirror/ 2>/dev/null || true
|
||||
|
||||
# Create source archive with archiver
|
||||
bitbake core-image-minimal -c do_populate_lic
|
||||
```
|
||||
|
||||
### 8.2 Collect License Information
|
||||
|
||||
```bash
|
||||
# Generate license manifest
|
||||
bitbake core-image-minimal -c do_populate_lic
|
||||
|
||||
# Copy license files
|
||||
mkdir -p ${MIRROR_BASE}/licenses
|
||||
cp -r ${MIRROR_BASE}/build/tmp/deploy/licenses/* ${MIRROR_BASE}/licenses/
|
||||
|
||||
# Create license CSV (see separate script)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 9: Package the Mirror
|
||||
|
||||
### 9.1 Create Archive Structure
|
||||
|
||||
```bash
|
||||
cd ${MIRROR_BASE}
|
||||
|
||||
# Create organized structure for approval
|
||||
mkdir -p mirror-package/{sources,licenses,build-instructions}
|
||||
|
||||
# Copy source layers (git repositories)
|
||||
cp -r sources/* mirror-package/sources/
|
||||
|
||||
# Copy license information
|
||||
cp -r licenses/* mirror-package/licenses/ 2>/dev/null || true
|
||||
|
||||
# Copy downloads (source tarballs)
|
||||
mkdir -p mirror-package/downloads
|
||||
cp -r downloads/* mirror-package/downloads/ 2>/dev/null || true
|
||||
|
||||
# Document the configuration
|
||||
cp build/conf/local.conf mirror-package/build-instructions/
|
||||
cp build/conf/bblayers.conf mirror-package/build-instructions/
|
||||
```
|
||||
|
||||
### 9.2 Create Archive
|
||||
|
||||
```bash
|
||||
cd ${MIRROR_BASE}
|
||||
|
||||
# Create timestamped archive
|
||||
export TIMESTAMP=$(date +%Y%m%d-%H%M%S)
|
||||
export ARCHIVE_NAME="tqma6-yocto-mirror-scarthgap-${TIMESTAMP}.tar.gz"
|
||||
|
||||
tar czf ${ARCHIVE_NAME} \
|
||||
--exclude='.git/objects/pack/*' \
|
||||
mirror-package/
|
||||
|
||||
# Calculate checksum
|
||||
sha256sum ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.sha256
|
||||
|
||||
echo "Archive created: ${ARCHIVE_NAME}"
|
||||
echo "Size: $(du -h ${ARCHIVE_NAME} | cut -f1)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 10: Generate License Table
|
||||
|
||||
**See:** [License Table Generation Script](../03-license-analysis/generate-license-table.sh)
|
||||
|
||||
The license table must include:
|
||||
- Package name
|
||||
- Version
|
||||
- License (SPDX identifier)
|
||||
- License file location
|
||||
- Source location
|
||||
- Military use approval (Yes/No/Conditional)
|
||||
|
||||
---
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Before delivering mirror:
|
||||
|
||||
- [ ] All git repositories cloned (poky, meta-openembedded, meta-tq)
|
||||
- [ ] Complete source downloads in `downloads/` directory
|
||||
- [ ] Build successful: `core-image-minimal` completed
|
||||
- [ ] License information generated in `licenses/`
|
||||
- [ ] Archive created with timestamp
|
||||
- [ ] SHA256 checksum calculated
|
||||
- [ ] License compliance table generated
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Problem: Fetch failures during build
|
||||
|
||||
**Solution:** Check internet connectivity, proxy settings. Some corporate networks block git:// protocol:
|
||||
```bash
|
||||
git config --global url."https://".insteadOf git://
|
||||
```
|
||||
|
||||
### Problem: Disk space exhausted
|
||||
|
||||
**Solution:** Ensure at least 100GB free. Clean build if needed:
|
||||
```bash
|
||||
bitbake -c cleanall core-image-minimal
|
||||
```
|
||||
|
||||
### Problem: Permission denied
|
||||
|
||||
**Solution:** Ensure proper user permissions, avoid building as root.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Transfer archive** to corporate network (USB, secure transfer)
|
||||
2. **Submit for software approval** with license table
|
||||
3. **Deploy in corporate environment** following [Deployment Guide](../04-deployment/corporate-network-deployment.md)
|
||||
|
||||
---
|
||||
|
||||
**Document Version:** 1.0
|
||||
**Author:** Siggi ⚙️
|
||||
**Last Updated:** 2026-03-01
|
||||
Reference in New Issue
Block a user