JetPatch Unified Linux Repository

In this article, we will describe the process of creating a Docker image and corresponding containers for local mirror repositories of various Linux-based Operating Systems, flavors, and versions. 

By creating OS package repositories in Docker containers, the deployment architecture would need only the Docker host server on which repository Docker containers for various OS flavors can be deployed. 

Please contact JetPatch support for access.

What are the benefits of creating Repositories in Docker Containers?

• Reduces hardware resource requirements
• Reduces operational overhead 

Solution Architecture

• The docker package is installed on Host machine for managing the docker images and containers.
• Each OS flavor and version has an independent docker image and container for providing the local repository.
• The repository container has the OS repository and also exposes the repository through HTTP/HTTPS using Nginx web server.
• There is an additional docker image and container which provides the reverse
  proxy functionality using Nginx. The Proxy container listens on HTTP/HTTPS for web
  requests and forwards the requests to appropriate OS repository containers based on
  the request URI endpoint.

Prerequisites

The solution would require the following components:

1. Host Machine Requirements
   1. Use a supported Operating Systems (the Docker container): RHEL8, OL8, or CentOS7 
   2. Have SSH access (root or sudo permissions) to the host machine
   3. Have a high-speed internet connection (Fast Ethernet 100 Mbps and above)
   4. Update your host OS using the appropriate command, such as "yum update" or a similar command, depending on your OS.
   5. Be able to communicate with relevant internet repos (no firewall issues)
   6. Adequate disk space for the configured repositories. (see sizing guide) 
2. Guest OS Requirements
   1. Scan your environment to identify the list of operating systems you want to support from the following options: AlmaLinux 8/9, RHEL 7/8/9, Oracle Linux 7/8, CentOS 7, Ubuntu 18/20/22.
   2. For Oracle Linux guest OS: Subscription and CSI details
   3. For RedHat guest OS: Customer Portal subscription credentials
   4. To run RHEL9 as a guest OS, it's necessary to deactivate the EVC mode for the host VM. (error Fatal glibc error: CPU does not support x86-64-v2)

Note: The repository data is typically stored in the /var/lib/docker/ directory unless specified otherwise. To modify the storage location before installing Docker, you can make changes to the repos_install.conf file.

Supported Container Operating Systems

• AlmaLinux 8/9
• RHEL 7/8/9
• Oracle Linux 7/8
• CentOS 7
• Ubuntu 18/20/22

Components

Installation Script

1. Script Path: /repos_install.sh
2. Reads configuration from /etc/repos_install.conf
3. Removes the previous containers and images for each of the configured OS flavors and versions.
4. Pull the Docker image for each of the configured OS flavor and version, and Nginx Proxy from the configured JetPatch Docker Hub repository.
5. Create Docker volumes for each of the configured OS flavors and versions, if not available. (Note: Old Docker volumes are not deleted)
6. Creates a Docker Network named ‘repos_network’. All the containers are part of this network.

The Management Script

1. Script path - /bin/jetpatch_repos.sh
2. Reads config from /etc/repos.conf
3. Start/Stop/Restart the repository containers

Repository Containers

Each of the repository containers has the following features:

1. The repositories are created at /var/www/html/.
2. Docker volume is named as _repos_vol mounted at /var/www/html
3. Nginx web server with Root Directory at /var/www/html/
4. Repository created at start of container.
5. Repository updated every 24 hours.
6. The repository containers retain the cache in the corresponding Docker volumes. Restart of a container will use the previously downloaded repository cache.

Nginx Proxy Container

Nginx Proxy container runs Nginx web server. The web server implements a reverse proxy and https termination for the repository containers. The OS repositories can be accessed using Host IP and OS name as endpoint. The reverse proxy supports both HTTP and HTTPS requests.

Ex: To access RHEL 7 repository, URL to use would be http:///rhel7/ or https:///rhel7/

Installation & Management Steps:

1. Login to the Centos 7 Host server as 'root' user, create a work folder, and cd to the
same.

# mkdir work_repos
# cd work_repos 

2. Copy the given tarball (jetpatch_repos.tar.gz) in this new folder.  You can also use the following 'wget' command. Please contact support to get the latest version.

# wget --no-check-certificate https://<ASK-SUPPORT-FOR-LATEST-VERSION>

3. "Untar" the tarball using the following command

# tar -xf jetpatch_repos.tar

4. Edit the installation config file ‘repos_install.conf’ and configure the OS flavors and version for which Docker image/containers of local repositories have to be installed
Ex: Enabling local Centos 7 and RHEL 8

OS_REPOS=”centos7 rhel8” 

5. Edit the Docker Hub account and repository details from which the Docker images have to be pulled. Currently, images are at the following location

DOCKER_HUB_ACCOUNT="itjetpatch/repos"

6. Execute the installation script repos_install.sh

#sh repos_install.sh

Note: The docker-ce installation may fail on RHEL 8.7 Please install it manually (dnf install --allowerasing docker-ce) and then rerun repos_install.sh 

7. Verify the Docker images have been installed

# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
itjetpatch/repos rhel8_repos_img eeb77a198132 6 days ago
324MB
itjetpatch/repos centos7_repos_img d145b88e9904 6 days ago
683MB

8. Copy the sample local repository configuration file from extras/repos.conf to
/etc/repos.conf

# cp extras/repos.conf /etc/repos.conf

9. Edit the /etc/repos.conf file to enable the required repository containers and
other configurations.

Examples:

• Edit your RedHat 7 and 8 Subscription details (user, password) 
• Additional RHEL repositories are now supported on all RHEL distributions. If configured, must also include default repositories (see example below for RHEL7)
  Note: Redhat "Simple Content Access" is not currently supported. Disable it before installation the first run (jetpatch_repos.sh start). After the docker registration it can be enabled again.   

# For RHEL8 repo, RedHat Customer Portal subscription credentials 
RHEL_8_USERNAME="" 
RHEL_8_PASSWORD="" 

# For RHEL7 repo, RedHat Customer Portal subscription credentials 
RHEL_7_USERNAME="" 
RHEL_7_PASSWORD=""
RHEL_7_REPO_LIST="rhel-7-server-optional-rpms rhel-7-server-rpms rhel-server-rhscl-7-rpms rhel-7-server-extras-rpms rhel-7-server-supplementary-rpms"

• Edit your OL 7 and 8 (Subscription and CSI details).

ORACLELINUX_7_USERNAME="" 
ORACLELINUX_7_PASSWORD="" 
ORACLELINUX_7_CSI="" 

ORACLELINUX_8_USERNAME="" 
ORACLELINUX_8_PASSWORD="" 
ORACLELINUX_8_CSI="" 

10. Start the Docker containers using the following command

# jetpatch_repos.sh start

11. Verify that all the configured containers are started

# docker ps

12. Test the repo by browsing to the web page: 

EX: http://<HOST-IP>/rhel8/

IMPORTANT: If the trailing slash is omitted, you might encounter a 404 error.

Endpoint URL of Repositories to be Configured on Client Machines

Notes:

1. The following steps should be performed on all client servers that are configured to sync with the local repositories (as per the relevant OS). We highly recommend using our run-task capabilities, to easily distribute the configurations
2. The "HOST-IP" parameter value can be either the IP of the host or the hostname. 

For every OS type: (i.e RHEL7, OEL8)

1. Disable all current repositories in the Local Repository server by running disable-all-repos.sh attachment on the client-server. 

2. Create a .repo file that will be added to the /etc/yum.repos.d directory on every server using the repositories. That file should look similar to the following file.

vi /etc/yum.repos.d/name_of_repo_file.repo

3. Insert the relevant configuration from the below list and save file. 

Centos 7
http://<HOST-IP>/centos7/base/
http://<HOST-IP>/centos7/centosplus/
http://<HOST-IP>/centos7/extras/
http://<HOST-IP>/centos7/updates/

RHEL 7
http://<HOST-IP>/rhel7/main/
http://<HOST-IP>/rhel7/server/
http://<HOST-IP>/rhel7/rhscl/

RHEL 8
http://<HOST-IP>/rhel8/baseOS/
http://<HOST-IP>/rhel8/AppStream/

Ubuntu 18.04
deb http://<HOST-IP>/ubuntu18/mirror/archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse
deb http://<HOST-IP>/ubuntu18/mirror/archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://<HOST-IP>/ubuntu18/mirror/archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
deb http://<HOST-IP>/ubuntu18/mirror/archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://<HOST-IP>/ubuntu18/mirror/archive.ubuntu.com/ubuntu/ bionic-proposed main restricted universe multiverse

Ubuntu 20.04
deb http://<HOST-IP>/ubuntu20/mirror/archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb http://<HOST-IP>/ubuntu20/mirror/archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://<HOST-IP>/ubuntu20/mirror/archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb http://<HOST-IP>/ubuntu20/mirror/archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse

Oracle Linux 7
http://<HOST-IP>/oraclelinux7/OracleLinux/OL7/latest/$basearch/
http:/<HOST-IP>/oraclelinux7/OracleLinux/OL7/addons/$basearch/

Oracle Linux 8
http://<HOST-IP>/oraclelinux8/ol8_x86_64_baseos_latest/
http://<HOST-IP>/oraclelinux8/ol8_x86_64_appstream/
http://<HOST-IP>/oraclelinux8/l8_x86_64_addons/

Ubuntu 22.04
deb http://<HOST-IP>/ubuntu22/mirror/archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse
deb http://<HOST-IP>/ubuntu22/mirror/archive.ubuntu.com/ubuntu/ jammy-updates main restricted universe multiverse
deb http://<HOST-IP>/ubuntu22/mirror/archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
deb http://<HOST-IP>/ubuntu22/mirror/archive.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse

AlmaLinux 8
http://<HOST-IP>/almalinux8/baseos/
http://<HOST-IP>/almalinux8/appstream/
http://<HOST-IP>/almalinux8/extras/

AlmaLinux 9
http://<HOST-IP>/almalinux9/baseos/
http://<HOST-IP>/almalinux9/appstream/
http://<HOST-IP>/almalinux9/extras/

4. Disable all “default” and previous .repo files  by relocating them to a different directory:

5. Clean the Repository Metadata:

sudo dnf clean all
sudo dnf makecache

6. List Enabled Repositories:

sudo dnf repolist

Note:

1. For RHEL7 'yum' package manager is used.

Clean the Repository Metadata:

sudo yum clean all
sudo yum makecache

List Enabled Repositories:

sudo yum repolist

2. For certain operating systems, the link might vary. You can verify this by navigating to http://<HOST-IP>/<OS-NAME>/ and then adding all the repositories listed there to the configuration file.

Troubleshooting and Advanced Configuration

HTTPS support for docker OS repo

Pre-Requisites:

• SSL certificate file for use (self-signed, from domain ca or from public/commercial ca) 
• The SSL certificate to be used must be trusted by all the repo clients. (steps for this are dependent on client os but are normally published by the os vendor in their documentation/community portal).

Enabling HTTPS on the existing docker repository host

1. Edit the file 'Repo/repos_install.conf' with the following change to only update the nginx proxy docker image(without affecting currently enabled OS repos)

OS_REPOS=""

       2. Stop the service - #jetpatch_repos.sh stop

       3. Execute #repos_install.sh' script to update the nginx proxy docker image(the latest image will be pulled from the docker hub)

       4. Edit the file /etc/repos.conf with the following changes to enable HTTPS support:

ENABLE_SSL="1"

NGINX_PROXY_SSL_DIR="/root/nginx_proxy_certs" - Path of the folder where the SSL certificates are located(Certificate file names should be 'cert.crt' and 'cert.key')

        5. Start the service - #jetpatch_repos.sh start

Enabling HTTPS on a fresh deployment of the docker repository host

1. Before starting the service ' #jetpatch_repos.sh start' do the following changes to /etc/repos.conf' to enable HTTPS support

ENABLE_SSL="1"

NGINX_PROXY_SSL_DIR="/root/nginx_proxy_certs" - Path of the folder where the SSL certificates are located(Certificate file names should be 'cert.crt' and 'cert.key')

      2. Start the service - #jetpatch_repos.sh start

Mirror additional RHEL repositories

By default, the RHEL7/8/9 container mirrors default (usually two) repositories: 

1. rhel-8-for-x86_64-baseos-rpms 

2. rhel-8-for-x86_64-appstream-rpms

For additional repositories, please edit /etc/repos.conf file using the following variable: “RHEL_X_REPO_LIST”. See the usage example, below:

RHEL_8_REPO_LIST="rhel-8-for-x86_64-baseos-rpms rhel-8-for-x86_64-appstream-rpms rhel-8-for-x86_64-appstream-source-rpms rhel-8-for-x86_64-baseos-e4s-debug-rpms"

The path on the webserver for each of the repositories is based on the repository's Repo-ID.

EX: http://<HOST-IP>/rhel8/rhel-8-for-x86_64-baseos-rpms/

IMPORTANT: When configuring new repos(RHEL_X_REPO_LIST), a list of all repos needs to be given and it MUST include the default ones as well. 

Adding Ubuntu Pro Subscription Registration Option with ESM Repositories

Pre-req: JetPatch Version 4.2.7+

JetPatch Unified Linux Repository allows mirroring of ESM repositories by configuring the Ubuntu Pro Subscription token by adding the below parameter in /etc/repos.conf:

Note: esm-apps and esm-infra are the default ESM repositories while other ones are optional and/or extra repositories.

Configuration on JetPatch Unified Linux Repository

For Ubuntu 20.04:

#UBUNTU20_PRO_TOKEN="<ubuntu pro token>"

For Ubuntu 22.04:

#UBUNTU22_PRO_TOKEN="<ubuntu pro token>"

Note: The Ubuntu Pro token can be obtained from the dashboard of your Ubuntu Pro account.

Note: Once using a Secondary ULR same settings needs to be done there also.

Configuration on Ubuntu Client Machine

On the Ubuntu client machine, add the following entries to /etc/apt/sources.list to enable installation and updates from the ESM repositories:

Ubuntu 20.04

Ubuntu 22.04

Modify Cron's Job Schedule

docker exec -it oraclelinux7_repos_con    /bin/bash

# jetpatch_repos.sh stop

#sh repos_install.sh cleanupall

#sh repos_install.sh clean <os>
Example: 
#sh repos_install.sh clean rhel7

#sh repos_install.sh clean
Ex:
#sh repos_install.sh clean

# jetpatch_repos.sh stop

# sh repos_install.sh

# jetpatch_repos.sh start

# docker ps

http://<HOST-IP>/rhel8/

docker logs -n 100 -f oraclelinux7_repos_con

Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

docker exec -it ubuntu22_repos_con /bin/bash

sudo rm -rf /var/lib/apt/lists/*

 sudo apt-get update

[ol8_baseos_latest]
name=Oracle Linux $releasever BaseOS Latest ($basearch)
baseurl=http://30.30.55.155/oraclelinux8/ol8_x86_64_baseos_latest/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol8_appstream_latest]
name=Oracle Linux $releasever AppStream Latest ($basearch)
baseurl=http://30.30.55.155/oraclelinux8/ol8_x86_64_appstream/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol8_addons_latest]
name=Oracle Linux $releasever Addons Latest ($basearch)
baseurl=http://30.30.55.155/oraclelinux8/ol8_x86_64_addons/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol8_UEKR6_latest]
name=Oracle Linux $releasever UEKR6 Latest ($basearch)
baseurl=http://30.30.55.155/oraclelinux8/ol8_x86_64_UEKR6/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1