Build a CentOS7 server for: pxe boot, kickstart, reposync, repotrack, nfs, https (STEP 7)

STEP 7 – CREATE /etc/yum.repos.d/c7x64.repo for the CentOS 7 64-bit REPOSYNC CLIENTS:


note: client machine has to have nss-mdns (avahi-daemon) working, or file will need to be edited with server’s IP ADDRESS.

provide a copy of the file at “http://c7pxe.local/repos/client-files/c7x64.repo” from
folder “/var/www/html/repos/client-files/”

subsequent kickstart scripts will get this file during system installation.


# File:/etc/yum.repos.d/c7x64.repo
[c7x64-base]
name=CentOS Base
baseurl=http://c7pxe.local/repos/c7x64/base/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled=1

[c7x64-epel]
name=CentOS EPEL
baseurl=http://c7pxe.local/repos/c7x64/epel/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
enabled=1

[c7x64-extras]
name=CentOS Extras
baseurl=http://c7pxe.local/repos/c7x64/extras/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled=1

[c7x64-updates]
name=CentOS Updates
baseurl=http://c7pxe.local/repos/c7x64/updates/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled=1


 

Build a CentOS7 server for: pxe boot, kickstart, reposync, repotrack, nfs, https (STEP 6)

STEP 6 – build REPOSYNC and REPOTRACK commands for SCRIPTs (for getting filtered packages from BASE, UPDATES, and EXTRAS.
The same pattern works fine for EPEL, but I switched back to using REPOTRACK for EPEL, because I rarely utilize more than a few hundred MB of the EPEL packages and finally got around to parsing out a list.
  • “rs32-reposync-update.sh”    # for getting base, updates, and extras.
  • “rs64-reposync-update.sh”    # for getting base, extras

And, a REPOTRACK script for getting selected packages from EPEL:

  • “rt64-c7x64-update.sh”

An easy way to begin using REPOTRACK is to query existing systems for what packages are installed from a given repo.  To gather my EPEL package list, I ran these two commands on all of my application servers:

  • repoquery -a –installed –qf “%{ui_from_repo} %{name}” | grep ‘^@epel’
  • repoquery -a –installed –qf “%-20{ui_from_repo} %-30{name} %-7{arch} %{epoch}:%-12{version} %-4{release}” | grep ‘^@epel’

Parse the results however you want, and you’ve got the basis for your repotrack script.  Repotrack doesn’t behave exactly like reposync, but it DOES get dependencies for packages, so it isn’t necessary to identify everything in advance.  It also works with wildcards.


Synchronize the directories for the repositories.
The –newest-only option puts only the latest version of each package in the repos.
note: “repoid” must match a repo name in “/etc/yum{$}.repos.d/{$}.repo”

For the “rs-c7x64-update.sh” script:

  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=base –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/
  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=extras –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/
  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=updates –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/

list the package URLs, but don’t actually download anything.


  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=base –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/ –urls

  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=extras –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/ –urls
  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=updates –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/ –urls
  • reposync –config=/etc/yumrsc7x64.conf –gpgcheck –plugins –repoid=epel –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x64/ –urls

Create (or update) new repodata for the local repositories. Note the option “-g comps.xml” to update the package group information.

  • createrepo /var/www/html/repos/c7x64/base/ -g comps.xml
  • createrepo /var/www/html/repos/c7x64/extras/
  • createrepo /var/www/html/repos/c7x64/updates/
  • createrepo /var/www/html/repos/c7x64/epel/

Do this so that clients will be able to read from the httpd repos:
chcon -R -t httpd_sys_content_t /var/www/html/repos/
Something about “SE Linux” according to a doc at RHEL.
Might throw errors, not sure if it has to be done *after* the first time RepoSync runs and populates the local repos, or if it doesn’t apply when the security settings on the vm are “permissive”.

For the “rs-c7x32-update.sh” script:

  • reposync –config=/etc/yumrsc7x32.conf –gpgcheck –plugins –repoid=base –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x32/
  • reposync –config=/etc/yumrsc7x32.conf –gpgcheck –plugins –repoid=extras –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x32/
  • reposync –config=/etc/yumrsc7x32.conf –gpgcheck –plugins –repoid=updates –newest-only –delete –downloadcomps –download-metadata –download_path=/var/www/html/repos/c7x32/
Create (or update) new repodata for the local repositories. Note the option “-g comps.xml” to update the package group information.
  • createrepo /var/www/html/repos/c7x32/base/ -g comps.xml
  • createrepo /var/www/html/repos/c7x32/extras/
  • createrepo /var/www/html/repos/c7x32/updates/


For the “rt64-c7x64-update.sh” script:


The repotrack options are very similar to reposync:

  • –config=CONFIG      # defaults to /etc/yum.conf
  • –config=/etc/yumrsc7x64.conf
  • –arch=ARCH # defaults to “current arch” (whatever the systems is currently running at).
  • –repoid=REPOID # default is all enabled.
  • –download_path=DESTDIR # Path to download packages to.
  • –urls # only show download urls, don’t download files.
  • –newest # defaults to newest-only.

For my script, I’ve sorted commands/packages into common groups.  The script is literally the following commands and comments:

# common packages:
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest etckeeper htop nss-mdns epel-release

# security packages:
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest haveged tripwire fail2ban

# only for a Gui VM:
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest imlib2 openbox openbox-libs python2-pyxdg tweak yumex

# for python pypi … note: using “python36-pip” gets “python36, python36-libs, and python36-setuptools”.
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest python36-pip 

# for a DNS server:
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest dhcping 

# for a docker server… note: using “docker*” gets python dependencies; no need to type them all out.
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest docker*

# some git servers want/need this… note: using “github2fedmsg” gets dependencies; no need to type them all out.
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest github2fedmsg

# for a kitchen sink PYTHON36*…  using python36* gets a lot of stuff… including… numpy, pyvomi, pytests, requests, tkinter, virtualenv
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest python36*

# for more GIT* packages… includes items like gitolite, gitstats, git-extras, git-tools, 
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest git*

# for MYSQL* … fairly small list of items, some connectors, utilities, and some perl.
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest mysql*

# for SQLITE* … small list of items
repotrack –config=/etc/yumrsc7x64.conf –repoid=epel –download_path=/var/www/html/repos/c7x64/epel/Packages/ –newest sqlite*

# BUILD THE LOCAL REPOS:
createrepo /var/www/html/repos/c7x64/epel/


Build a CentOS7 server for: pxe boot, kickstart, reposync, repotrack, nfs, https (STEP 5)

STEP 5 – configure an EXCLUDE LINE for the YUM CONFIG files:

nano /etc/yum.conf
  • exclude list, filters out stuff I don’t need to download:
  • this is the c7pxe list I’m using as of 2019-03-22.
  • NOTE: when we’re done with this setup, we’ll have a least three yum config files on this server:
    • 1) the regular /etc/yum.conf the OS uses.
    • 2) –config=/etc/yumrsc7x64.conf
    • 3) –config=/etc/yumrsc7x32.conf
in yum config files pulling packages for 64-bit instances:

exclude=*.i386,*.i686,*-debug-*,*-demo-*,*-src-*,alsa*,ant-*,*android*,*amarok*,b43-openfwwf,bfa-firmware,bpg-*,cockpit*,culmus-*,cinnamon*,*-doc-*,*-docs-*,elementary*,ekiga*,fluid-soundfont*,firefox*,gimp-*,gutenprint-*,graphviz-*,gnuradio*,glassfish*,i3-*,ivtv*,iwl*firmware,kde*,kubernetes*,java-1.6*,java-1.7*,josm-*,jetty*,jboss*,jakarta*,libreoffice*,libertas*,maven*,mecurial*,moodle*,mythes-*,NetworkManager-team,libteam,nextcloud*,openblas*,openarena-*,qemu*,ql2100-firmware,ql2200-firmware,ql23xx-firmware,ql2400-firmware,ql2500-firmware,quantum-*,rt61pci-firmware,rt73usb-firmware,rust*,shogun*,subversion*,seamonkey*,texlive*,thunderbird*,uhd*,vtk*,wine*,wannier*,xemacs*,xqilla*,youtube*,zd1211-firmware,kacst-*,lohit-*,thai-scalable-*,*-chinese-*,*-de-*,*-es-*,*-fr-*,*-ga-*,*-hu-*,*-ja-*,*-lb-*,*-lv-*,*-ko-*,*-ru-*,*-zh-*,0ad-*,mingw*,horai*

# same exclude list with added line breaks for easier reading:
exclude=*.i386,*.i686,*-debug-*,*-demo-*,*-src-*,
alsa*,ant-*,*android*,*amarok*,
b43-openfwwf,bfa-firmware,bpg-*,
cockpit*,culmus-*,cinnamon*,*-doc-*,*-docs-*,elementary*,ekiga*,
fluid-soundfont*,firefox*,gimp-*,
gutenprint-*,graphviz-*,gnuradio*,glassfish*,
i3-*,ivtv*,iwl*firmware,kde*,kubernetes*,
java-1.6*,java-1.7*,josm-*,jetty*,jboss*,jakarta*,
libreoffice*,libertas*,
maven*,mecurial*,moodle*,mythes-*,NetworkManager-team,libteam,
nextcloud*,openblas*,openarena-*,
qemu*,ql2100-firmware,ql2200-firmware,ql23xx-firmware,ql2400-firmware,
ql2500-firmware,quantum-*,
rt61pci-firmware,rt73usb-firmware,rust*,
shogun*,subversion*,seamonkey*,texlive*,thunderbird*,
uhd*,vtk*,wine*,wannier*,xemacs*,xqilla*,youtube*,zd1211-firmware,
kacst-*,lohit-*,thai-scalable-*,*-chinese-*,
*-de-*,*-es-*,*-fr-*,*-ga-*,*-hu-*,*-ja-*,*-lb-*,*-lv-*,*-ko-*,*-ru-*,*-zh-*,0ad-*,mingw*,horai


 

in yum config files pulling packages for 32-bit instances:

exclude=*-debug-*,*-demo-*,*-src-*,alsa*,ant-*,*android*,*amarok*,b43-openfwwf,bfa-firmware,bpg-*,cockpit*,culmus-*,cinnamon*,*-doc-*,*-docs-*,elementary*,ekiga*,fluid-soundfont*,firefox*,gimp-*,gnome*,google-noto-*,gstreamer*,gutenprint-*,graphviz-*,gnuradio*,glassfish*,i3-*,ImageMagick*,ivtv*,iwl*firmware,kde*,kubernetes*,java-1.6*,java-1.7*,josm-*,jetty*,jboss*,jakarta*,libreoffice*,libertas*,maven*,mecurial*,moodle*,mythes-*,NetworkManager-team,libteam,nextcloud*,openblas*,openarena-*,qemu*,ql2100-firmware,ql2200-firmware,ql23xx-firmware,ql2400-firmware,ql2500-firmware,quantum-*,rt61pci-firmware,rt73usb-firmware,rust*,shogun*,subversion*,seamonkey*,texlive*,thunderbird*,uhd*,vtk*,wine*,wannier*,xemacs*,xqilla*,youtube*,zd1211-firmware,kacst-*,lohit-*,thai-scalable-*,*-chinese-*,*-de-*,*-es-*,*-fr-*,*-ga-*,*-hu-*,*-ja-*,*-lb-*,*-lv-*,*-ko-*,*-ru-*,*-zh-*,0ad-*,mingw*,horai*

 

 

Build a CentOS7 server for: pxe boot, kickstart, reposync, repotrack, nfs, https (STEP 4)

STEP 4 – CONFIG RepoSync/RepoTrack to support multiple OS Distros, Releases, and Architectures:

As of 2019-03-22, on c7pxe, the c7x64 and c7x32 repos take about this much space…
  • 4.0G /var/www/html/repos/c7x64/base/        # filter allows for GNOME desktop.
  • 206M /var/www/html/repos/c7x64/extras/
  • 942M /var/www/html/repos/c7x64/updates/
  • 187M /var/www/html/repos/c7x64/epel/
  • 3.4G /var/www/html/repos/c7x32/base/          # filter also excludes GUI desktops.

REPOSYNC OPTIONS (not all of them):
-c CONFIG, –config=CONFIG
             Config file to use (defaults to /etc/yum.conf).
  • CREATE extra YUM config files, one for each {Distro-Release-Arch} being sync’d:
    •  /etc/yumrsc7x64.conf # configure an exclude= line.
    •  /etc/yumrsc7x32.conf # configure an exclude= line.
  • CREATE extra YUM repo directory, one for each {Distro-Release-Arch} being sync’d:
    •  /etc/yumrsc7x64.repos.d
    •  /etc/yumrsc7x32.repos.d
  • CREATE matching repo files under each directory:
    • /etc/yumrsc7x64.repos.d/c7x64.repo # {base,extras,updates,epel}
    • /etc/yumrsc7x32.repos.d/c7x32.repo # {base,extras,updates}
  • EDIT each YUM config files to specify where to find it’s “*.repo” files:
    •  IN “/etc/yumrsc7x64.conf”, add the line “reposdir=/etc/yumrsc7x64.repos.d
    •  IN “/etc/yumrsc7x32.conf”, add the line “reposdir=/etc/yumrsc7x32.repos.d
  • define reposync CONFIG OPTION for each {Distro-Release-Arch} being sync’d (for the reposync script(s):
    •  –config=/etc/yumrsc7x64.conf
    •  –config=/etc/yumrsc7x32.conf
-a ARCH, –arch=ARCH
              Act as if running the specified arch (default: current arch,
              note: does not override $releasever. x86_64 is a superset for
              i*86.).
  •  –arch= # can skip this option/flag for the x64 version, default already works.
  •  –arch= # for the x32… I’m not sure whether it is supposed to be “i386”, “i686”, or something else.
    • note: 2019-03-22, seems to working without actually using this flag.
-r REPOID, –repoid=REPOID
              Specify repo ids to query, can be specified multiple times
             (default is all enabled).
  • This flag refers to the [repo name] in the configured “*.repo” files.
  • “REPOID” must match a name in the targeted /etc/yum{$}.repos.d/{$}.repo file(s).
  • So, if you keep the names simple there, then this flag remains much simpler.
  • the options used by my repo configs and reposync/repotrack scripts:
    •  –repoid=base
    •  –repoid=extras
    •  –repoid=updates
    •  –repoid=epel
-d, –delete
              Delete local packages no longer present in repository.
 -p DESTDIR, –download_path=DESTDIR
              Path to download packages to: defaults to current directory.
  • My local repos are being stored/served in these locations (don’t put this syntax in the script):
    • /var/www/html/repos/c7x64/{base,extras,updates,epel}
    • /var/www/html/repos/c7x32/{base,extras,updates}
  • note: the reposync utility will make (or use existing) subfolder(s) matching the repo name, so, my reposync scripts only need:
    • –download_path=/var/www/html/repos/c7x64/
    • –download_path=/var/www/html/repos/c7x32/
-u, –urls
              Just list urls of what would be downloaded, don’t download.
-n, –newest-only
              Download only newest packages per-repo.

Build a CentOS7 server for: pxe boot, kickstart, reposync, repotrack, nfs, https (summary of steps)

  • INTRODUCTION:  Overview of the starting point for this install, and reasons why to do it.
  • STEP  1 – clone an existing “minimal” VM (or build one).
  • STEP  2 – prepare to install/config PXE/RepoSync/RepoTrack (load software packages).
  • STEP  3 – add/config a 2nd virtual hard disk for the repo files.
  • STEP  4 – CONFIG RepoSync/RepoTrack to support multiple OS Distros, Releases, and Architectures.
  • STEP  5 – configure an EXCLUDE LINE for YUM CONFIG files
  • STEP  6 – build REPOSYNC commands for SCRIPT “rs-c7x64-update.sh”
  • STEP  7 – CREATE /etc/yum.repos.d/c7x64.repo for the CentOS 7 64-bit REPOSYNC CLIENTS
  • STEP  8 – CREATE /etc/yum.repos.d/c7x32.repo for the CentOS 7 32-bit REPOSYNC CLIENTS
  • STEP  9 – ENABLE and CONFIGURE PXE (uses vmware dhcp; does not require CentOS NTP/DHCP/DNS/vsftd/xinetd)
  • STEP 10 – Create a PXE BOOT MENU
  • STEP 11 – create the kickstart files referenced by the PXE Boot menu:
  • STEP 12 – Put the required PXE client boot files in place.
  • STEP 13 – Provide PXE boot server info to DHCP clients, via VMware Fusion vnet config (not a CentOS DHCP server).
  • STEP 14 – Test PXE Boot and Kickstart installation.
  • SIDEBAR 1 – Alternate ways to provide PXE BOOT IMAGES to clients (a brief summary)
  • SIDEBAR 2 – Optional NFS SHARE: convenient for exploring repo contents from a gui desktop VM.
  • SIDEBAR 3 – PXE client note re memory:  the boot image uses a ramdisk.