JetPatch Unified Linux Repository

Overview

This article describes how to create and manage local mirror repositories for various Linux-based operating systems using Docker containers. By deploying OS package repositories in Docker containers, you only need a single Docker host server to run multiple repository containers. This solution minimizes hardware requirements and reduces operational overhead.

Note: For access to the latest tarball and further assistance, please contact JetPatch Support.

1. Video Demonstration

For a step-by-step visual walkthrough of the installation and configuration procedures, watch the following video presentation:

2. Benefits

• Reduced Hardware Requirements: Leverage Docker containers to minimize physical resources.

• Lower Operational Overhead: Simplify management by consolidating repository services on a single Docker host.

• Modular Architecture: Each OS flavor/version runs independently and can be maintained separately.

3. Solution Architecture

• Docker Host: Manages the installation of Docker and deployment of repository containers.

• Repository Containers: Each OS flavor/version runs in its own container with an Nginx web server to serve the repository.

• Nginx Proxy Container: Provides reverse proxy functionality and HTTPS termination, routing web requests to the appropriate OS repository container.

4. Prerequisites

Host Machine Requirements

• Supported OS for Docker Host: RHEL8, OL8

• Access: SSH access with root or sudo privileges

• Network: High-speed internet (≥ 100 Mbps) and proper firewall configuration

• System Updates: Run yum update (or equivalent) to ensure the host is current

• Disk Space: Sufficient storage as per the sizing guide

Guest OS Requirements

• Supported Operating Systems:
  Alma Linux 8/9, RHEL 7/8/9, Oracle Linux 7/8/9, Ubuntu 18/20/22

• Additional Credentials:

  • Oracle Linux: Subscription and CSI details

  • Red Hat Enterprise Linux: Customer Portal subscription credentials

• Special Note for RHEL9: Deactivate the EVC mode on the host VM to avoid the “Fatal glibc error: CPU does not support x86-64-v2.”

NOTE: The repository data is stored by default in /var/lib/docker/. You can modify the default location by editing the repos_install.conf file if necessary.

5. Supported Container Operating Systems

• Alma Linux 8/9

• RHEL 7/8/9

• Oracle Linux 7/8/9

• Ubuntu 18/20/22/24

• Rocky Linux 8/9

6. Components Overview

Installation Script

• Path: /repos_install.sh

• Functionality:

  • Reads configuration from /etc/repos_install.conf

  • Removes previous containers and images for each OS flavor/version

  • Pulls the Docker image for each configured OS and the Nginx proxy image from the JetPatch Docker Hub repository

  • Creates Docker volumes for each OS flavor/version (old volumes remain intact)

  • Creates a Docker network named repos_network for container communication

Management Script

• Path: /bin/jetpatch_repos.sh

• Functionality:
  Manages repository containers with start, stop, and restart options, reading from /etc/repos.conf

Repository Containers

• Location: Repositories are stored at /var/www/html/

• Volume Name: Each container mounts a Docker volume named <os>_repos_vol

• Web Server: Each container runs Nginx serving the repository content

• Behavior: Repositories are created at container start and updated every 24 hours. Cache is preserved across restarts.

Nginx Proxy Container

• Function:

  • Implements reverse proxy and HTTPS termination

  • Listens on HTTP/HTTPS and routes requests to appropriate OS repository containers based on URI endpoints

• Example URL:
  To access the RHEL 7 repository, use http://<HOST-IP>/rhel7/ or https://<HOST-IP>/rhel7/ (include trailing slash to avoid 404 errors).

7. Installation & Management Steps

Preparation

1. Login and Create Workspace:

2. Obtain the Software Package:
   Copy jetpatch_repos.tar.gz into the work_repos folder or download it using:  

   wget --no-check-certificate https://<ASK-SUPPORT-FOR-LATEST-VERSION>

3. Extract the Tarball:

   tar -xf jetpatch_repos.tar.gz

Configuration

4. Edit Installation Config File:
   Update /etc/repos_install.conf to specify the OS flavors/versions. For example:

   OS_REPOS="centos7 rhel8"

5. Update Docker Hub Details:
   Ensure the Docker Hub account details are set correctly:

   DOCKER_HUB_ACCOUNT="itjetpatch/repos"

6. Execute Installation Script:

   sh repos_install.sh

   Note: On RHEL 8.7, if Docker CE installation fails, install it manually:

   dnf install --allowerasing docker-ce

7. Verify Installation:
   Check Docker images:

   docker images

Management

8. Configure Repository Settings:
   Copy the sample configuration to /etc/repos.conf and edit as needed:

   cp extras/repos.conf /etc/repos.conf

   • Configure subscription credentials and additional repository settings for RHEL, OL, etc.

9. Start Repository Containers:

   jetpatch_repos.sh start

10. Verify Running Containers:

    docker ps

11. Test Repository Access:
    Open a web browser and navigate to:

    http://<HOST-IP>/rhel8/

    IMPORTANT: Always include the trailing slash or you might encounter error 404.

8. Configuring Client Machines

For each client server, perform the following:

1. Disable Current Repositories:
   Use the provided disable-all-repos.sh script.

2. Create a New .repo File:
   In the /etc/yum.repos.d/  directory, create a file (e.g., local_repos.repo) with content similar to:

   CentOS 7 Example:

   [centos7-base]
   name=CentOS 7 Base
   baseurl=http://<HOST-IP>/centos7/base/
   enabled=1
   
   [centos7-updates]
   name=CentOS 7 Updates
   baseurl=http://<HOST-IP>/centos7/updates/
   enabled=1

3. Disable Default .repo Files:
   Move or rename any previous repo files to prevent conflicts.

4. Refresh Repository Metadata:

   For DNF:

   sudo dnf clean all
   sudo dnf makecache
   sudo dnf repolist

   For YUM (RHEL7):

   sudo yum clean all
   sudo yum makecache
   sudo yum repolist

Repeat similar steps for RHEL, Ubuntu, Oracle Linux, AlmaLinux, etc., using the repository URLs provided in the Appendix.

9. Advanced Configuration & Troubleshooting

Enabling HTTPS Support

On an Existing Deployment

1. Edit repos_install.conf:
   Clear OS repository configuration:

   OS_REPOS=""

2. Stop Services:

   jetpatch_repos.sh stop

3. Update the Nginx Proxy Image: Run the installation script:

   sh repos_install.sh

4. Configure HTTPS in /etc/repos.conf:

   ENABLE_SSL="1"
   NGINX_PROXY_SSL_DIR="/root/nginx_proxy_certs"

5. Restart Services:

   jetpatch_repos.sh start

On a Fresh Deployment

• Prior to starting the service (jetpatch_repos.sh start), update /etc/repos.conf with:

  ENABLE_SSL="1"
  NGINX_PROXY_SSL_DIR="/root/nginx_proxy_certs"

• Then, start the service as described above.

RHEL SCA – Simple Content Access

• Background:
  Red Hat systems now enable SCA by default (since November 2024).

• Configuration:
  To disable SCA and revert to the previous registration method, add the following in /etc/repos.conf:

  RHEL7_SCA_DISABLED="1"
  RHEL8_SCA_DISABLED="1"
  RHEL9_SCA_DISABLED="1"

• Note: A pre-check is implemented to avoid duplicate registration attempts when the system is already registered.

Mirroring Additional RHEL Repositories

• Default Repositories (e.g., RHEL 8):

  • rhel-8-for-x86_64-baseos-rpms

  • rhel-8-for-x86_64-appstream-rpms

• To Add More:
  Edit /etc/repos.conf and update the variable (example for RHEL8):

  RHEL_8_REPO_LIST="rhel-8-for-x86_64-baseos-rpms rhel-8-for-x86_64-appstream-rpms rhel-8-for-x86_64-appstream-source-rpms rhel-8-for-x86_64-baseos-e4s-debug-rpms"

  Important: When adding new entries, always include the default repositories.

Adding Ubuntu Pro Subscription with ESM Repositories

• Pre-requisite: JetPatch Version 4.2.7+

• Configuration in /etc/repos.conf:

  For Ubuntu 20.04:

  UBUNTU20_PRO_TOKEN="<ubuntu pro token>"

  For Ubuntu 22.04:

  UBUNTU22_PRO_TOKEN="<ubuntu pro token>"

  For Ubuntu 24.04:

  UBUNTU24_PRO_TOKEN="<ubuntu pro token>"

•  

• On the Ubuntu Client Machine:
  Add the following entries to /etc/apt/sources.list:

  Ubuntu 20.04 Example:

  deb [trusted=yes] http://<Repo-IP>/ubuntu20/mirror/esm.ubuntu.com/apps/ubuntu/ focal-apps-updates main
  deb [trusted=yes] http://<Repo-IP>/ubuntu20/mirror/esm.ubuntu.com/apps/ubuntu/ focal-apps-security main
  deb [trusted=yes] http://<Repo-IP>/ubuntu20/mirror/esm.ubuntu.com/infra/ubuntu/ focal-infra-updates main
  deb [trusted=yes] http://<Repo-IP>/ubuntu20/mirror/esm.ubuntu.com/infra/ubuntu/ focal-infra-security main

  Ubuntu 22.04: Similar entries using the /ubuntu22/ endpoint.

  Ubuntu 24.04: Add ESM entries at /etc/apt/sources.list.d/ubuntu.sources to be seen like so:

• Types: deb
  URIs: http://<HOST-IP>/ubuntu24/mirror/esm.ubuntu.com/apps/ubuntu/
  Suites: noble-apps-updates noble-apps-security
  Components: main
  Signed-By: /etc/apt/keyrings/esm-apps.gpg
  
  Types: deb
  URIs: http://<HOST-IP>/ubuntu24/mirror/esm.ubuntu.com/infra/ubuntu/
  Suites: noble-infra-updates noble-infra-security
  Components: main
  Signed-By: /etc/apt/keyrings/esm-infra.gpg

Install the ESM keys using below commands

• sudo install -m 0755 -d /etc/apt/keyrings
  
  # esm-apps key
  tmpdir="$(mktemp -d)"; GNUPGHOME="$tmpdir" gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys AB01A101DB53907B
  GNUPGHOME="$tmpdir" gpg --export AB01A101DB53907B | sudo gpg --dearmor -o /etc/apt/keyrings/esm-apps.gpg
  rm -rf "$tmpdir"
  
  # esm-infra key
  tmpdir="$(mktemp -d)"; GNUPGHOME="$tmpdir" gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys 4067E40313CB4B13
  GNUPGHOME="$tmpdir" gpg --export 4067E40313CB4B13 | sudo gpg --dearmor -o /etc/apt/keyrings/esm-infra.gpg
  rm -rf "$tmpdir"
  
  sudo chmod 644 /etc/apt/keyrings/esm-*.gpg

Modifying Cron Job Schedule

• Default Behavior:
  The repository sync task is triggered by /etc/cron.daily/uln-yum-mirror.

• To Modify the Schedule:

  1. Login to the Docker Container:

     docker exec -it <container_name> /bin/bash

  2. Copy the Cron File:

     cp /etc/cron.daily/uln-yum-mirror /etc/cron.hourly/

  3. Edit the /etc/crontab File:
     Adjust the timing to run the script hourly, weekly, or at a specified time.

General Troubleshooting

• Network Issues: Verify firewall settings between the host and repository.

• View Docker Logs:
  For example, for an Oracle Linux container:

  docker logs -n 100 -f oraclelinux7_repos_con

• Common Error – RHEL9 Glibc Issue:
  Deactivate EVC mode on the host VM if encountering:

  Fatal glibc error: CPU does not support x86-64-v2

• Overlay Storage Troubleshooting:

  • Check and set DOCKER_STORAGE_DRIVER and DEFAULT_DOCKER_STORAGE_DRIVER in repos_install.conf

  • Ensure DOCKER_ROOT_DIR is correctly set and that NFS remote storage is not used.

• Ubuntu Repository 404 Errors:
  Regenerate package index files by:

  1. Logging into the container:

     docker exec -it ubuntu22_repos_con /bin/bash

  2. Deleting old index files:

     sudo rm -rf /var/lib/apt/lists/*

  3. Updating package information:

     sudo apt-get update

10. Upgrading the Repository Version

1. Stop the Current Service:

   jetpatch_repos.sh stop

2. Download the New Version:
   Place the new version in a separate directory and extract it.

3. Compare Configuration Files:
   Use a diff tool to compare the old and new versions of repos_install.conf.

4. Merge Changes:
   Manually merge any new configuration parameters into the old file.

5. Run the Installation Script:

   sh repos_install.sh

6. Update /etc/repos.conf:
   Compare and merge changes from extras/repos.conf as necessary.

7. Restart the Containers:

   jetpatch_repos.sh start

8. Verify Operation:

   docker ps

9. Test the Repository:
   Browse to:

   http://<HOST-IP>/rhel8/

11. Appendix: Sample Repo Configuration Files

Below is an example of a repo configuration file used on RHEL 8 repo client machines.

Oracle Linux 8 Example:

local_ol8_baseos_latest]
name=Oracle Linux 
baseurl=http://<Server-IP>/repo/OracleLinux/ol8_baseos_latest/$basearch/
gpgkey=http://<Server-IP>/RPM-GPG-KEY-oracle-ol8
gpgcheck=1
enabled=1

[local_ol8_appstream]
name=Oracle Linux AppStream 
baseurl=http://<Server-IP>/repo/OracleLinux/ol8_appstream/$basearch/
gpgkey=http://<Server-IP>/RPM-GPG-KEY-oracle-ol8
gpgcheck=1
enabled=1

[local_ol8_UEKR6]
name=Latest Unbreakable Enterprise Kernel for Oracle Linux
baseurl=http://<Server-IP>/repo/OracleLinux/ol8_UEKR6/$basearch/
gpgkey=http://<Server-IP>/RPM-GPG-KEY-oracle-ol8
gpgcheck=1
enabled=1

Sample configuration files for all supported operating systems are available for download below. Please select the file that corresponds to your repo client's Guest OS.