0

I continually find that certain things "just don't work" when run as preseed/late_command in an Ubuntu Bionic preseed file. (The same applies to autoinstall for Ubuntu Focal.) In this particular case, I'm attempting to run an Ansible Role (specifically, the Canonical Ubuntu 18.04 LTS for Ansible STIG) against the target as a late_command:

d-i    preseed/late_command    string \
    in-target /usr/bin/curl -fsSL -o /opt/stig.zip {{ di_preseed.stig_role_url }}; \
    in-target /usr/bin/unzip -qq /opt/stig.zip -d /opt; \
    in-target /usr/bin/unzip -qq /opt/ubuntu1804STIG-ansible.zip -d /opt; \
    in-target /bin/sh -c -- 'cd /opt && ANSIBLE_LOG_PATH=/var/log/ansible.log /bin/sh enforce.sh'

Here enforce.sh is just a wrapper around ansible-playbook.

This install from ISO on Virtualbox fails with:

Failed to run preseeded command ... <command> finished with exit code 2

I am still able to log in to the box as ubuntu and become root. I see that Ansible was successfully installed (it is originally specified in pkgsel/include).

I then open up /var/log/installer/syslog on the target and find that ansible-playbook failed when it could not find rsyslog.service:

enter image description here

This is confusing as rsyslog.service is most certainly enabled and active after the install, which I can confirm with systemctl (status|is-active) rsyslog.

So what I'm seeking to understand here is:

  • Why is Ansible unable to find rsyslog.service during install even though it appears to be enabled?
  • What factors are different about the installation that result in things seeming to commonly break or not be available?
  • Would I be better of running this as an onboot script under init.rc, then last line it removes itself after done?

Related: Certain commands (e.g. modprobe or usermod) fail as late-commands in Ubuntu autoinstall

Brad Solomon
  • 103
  • 1
  • 15

1 Answers1

1

The key thing to remember is that the in-target commands are executed in a chroot environment. They are not executed in a fully booted system where core processes like systemd are running and available.

Testing

I setup a playbook based on the STIG tasks in the screenshot and ran it from the preseed. I saw the same results as you.

  • Playbook
---
- hosts: localhost
  gather_facts: no
  connection: local

  tasks:
    - name: check if rsyslog.service is installed
      shell: ! systemctl list-unit-files | grep "^rsyslog.service[ \t]\+"
      changed_when: False
      check_mode: no
      register: result
      failed_when: result.rc > 1

    - name: stigrule_219160_rsyslog_enable
      service:
        name: rsyslog.service
        enabled: "yes"
  • Partial preseed file
d-i pkgsel/include string ansible
d-i preseed/late_command string \
    wget -P /target/ http://REDACTED/my_playbook.yml ; \
    in-target ansible-playbook -i /dev/null -b -v /my_playbook.yml

Bionic packages ansible 2.5.1 and that version of the service module appears to execute systemctl show rsyslog.service. This does not work in a chroot environment. To demonstrate, if I open a terminal in the installer environment and run in-target systemctl show rsyslog.service then the log file will show the output

in-target: Running in chroot, ignoring request: show

Potential Fix

I did find a patch in Ansible 2.3 that addresses the problem that systemctl will ignore commands when running in a chroot environment. This was only applied to the systemd module though and not the service module. I updated my playbook and this does run successfully.

---
- hosts: localhost
  gather_facts: no
  connection: local

  tasks:
    - name: check if rsyslog.service is installed
      shell: ! systemctl list-unit-files | grep "^rsyslog.service[ \t]\+"
      changed_when: False
      check_mode: no
      register: result
      failed_when: result.rc > 1

    - name: stigrule_219160_rsyslog_enable fixed
      systemd:
        name: rsyslog.service
        enabled: "yes"

Therefore, you may be able to progress further by modifying the STIG task(s) from using the service module to using the systemd module.

Andrew Lowther
  • 5,811
  • 1
  • 15
  • 23
  • Thanks for this helpful answer. I had a breakthrough when i discovered today that `ctrl+z` will get you into a bash shell as root in the installer environment with /cdrom and /target sitting on the filesystem (the former being the mounted iso). That opened up a whole lot of easier debugging including discovering, as you mention here, that most of systemctl isn't available in a chroot environment. So that's not an ansible problem, per se, and the fact that curtin in-target is a chroot /target under the hood seems to be the underlying cause of most behavioral issues I've encountered. – Brad Solomon Aug 04 '21 at 17:33
  • FWIW, we'll likely choose to go the route of using ansible from a separate control node via ssh (as it's really supposed to be done, I would argue) after the installation process has completed for a group of nodes and they are accessible via ssh, where chroot is out of the picture. – Brad Solomon Aug 04 '21 at 17:35
  • Thanks to being able to get into the installer shell I've now been able to solve this one: https://askubuntu.com/a/1355980/919528. You might find that interesting – Brad Solomon Aug 04 '21 at 18:14
  • 1
    @BradSolomon I do think running ansible from a control node will be less problematic. There should be an option to open a shell from the "Help" drop-down in the installer UI. I usually just use `Alt-F2`. See https://askubuntu.com/a/1257186/376778 . You can even SSH into the installer environment if you set (or capture) the `installer` user password (YMMV depending on subiquity version): https://askubuntu.com/a/1322129/376778 – Andrew Lowther Aug 04 '21 at 18:48