4

I want to upgrade my Ubuntu 16.04 server to Ubuntu 18.04 and I'm running the following commands to do so;

apt update -y
apt upgrade -y
do-release-upgrade

apt update command runs fine with the following output;

# apt update -y
Hit:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security InRelease
Hit:2 https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
5 packages can be upgraded. Run 'apt list --upgradable' to see them.

However when running the apt upgrade command, it returns the following errors;

# apt upgrade -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done

*The following packages could receive security updates with UA Infra: ESM service enabled:
  libkrb5-3 libgssapi-krb5-2 libk5crypto3 libkrb5support0 libzstd1
Learn more about UA Infra: ESM service for Ubuntu 16.04 at https://ubuntu.com/16-04

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

The following packages have been kept back:
  libk5crypto3 libkrb5support0
The following packages will be upgraded:
  libzstd1
1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
1 esm-infra security update
Need to get 189 kB of archives.
After this operation, 132 kB of additional disk space will be used.
Err:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1+esm1
  401  Unauthorized
E: Failed to fetch https://esm.ubuntu.com/infra/ubuntu/pool/main/libz/libzstd/libzstd1_1.3.3+dfsg-2ubuntu1+esm1_amd64.deb  401  Unauthorized

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I'm not sure what to do at this point. How can I solve this?

Marry Jane
  • 225
  • 2
  • 7
  • How long ago did you try? 401 can mean that a repo is down for whatever reason, it seems up now, alternatively you could try just removing `esm.ubuntu.com` form your sources list, since you are upgrading, you don't need extended support. – Mark Kirby Oct 24 '21 at 13:21
  • Well I tried a few minutes ago but I have multiple servers and I'm doing them all at the same time and while some of them works fine, some of them gives this error so I don't think the repo is down, I think it's related to my server. – Marry Jane Oct 24 '21 at 13:23
  • @MarkKirby also I don't have `esm.ubuntu.com` in my sources list at all. – Marry Jane Oct 24 '21 at 13:42
  • 1
    You say you're running 16.04 ESM, but those look like 18.04 ESM sources (they say `bionic` instead of `xenial`). 18.04 does not need to be ESM, since it's still in Community Support. This suggests that somebody already tried a `do-release-upgrade` that failed. Perhaps the upgrader left the altered sources in place (it happens). You can change the sources back to Xenial. – user535733 Oct 24 '21 at 13:50
  • @user535733 Yes, I tried that myself but it failed. How can I revert this back? – Marry Jane Oct 24 '21 at 14:04
  • The fact that it failed suggests that you might need to be prepared for a backup-and-reinstall. Depends upon the reason (check the logs!) Revert the sources change by simply substituting release names in /etc/apt/sources.list and /etc/apt/sources.list.d/* – user535733 Oct 24 '21 at 14:39

3 Answers3

6

Two ways exist depending on currently running Ubuntu version.

(a) Ubuntu 16.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted
deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted
EOF

  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted"
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted"

and then resume upgrade to 18.04 LTS:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list
sudo do-release-upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list.

(b) Ubuntu 18.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted
deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted
EOF

  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted"
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted"

and then install all necessary upgrades by:

sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list

sudo apt-get update
sudo apt-get upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list while replacing xenial with bionic.

N0rbert
  • 97,162
  • 34
  • 239
  • 423
  • 2
    Thank you for your answer. It is very detailed. For my case, only running the command `sudo apt-get purge ubuntu-advantage-tools --autoremove` alone solved my issue. I didn't have to do the other changes. – Marry Jane Oct 25 '21 at 12:58
  • On an 18.04 system, the source of the problem was the `/etc/apt/sources.list.d/ubuntu-esm-infra.list` file. Commenting out the sources in there did the trick. I'm sure deleting it would have worked, too. – Lambart Jun 07 '23 at 19:37
3

On an 18.04 system (at least), look for a /etc/apt/sources.list.d/ubuntu-esm-infra.list file. That file erroneously lists sources for ESM updates that you would need to pay for, and apt is dutifully trying to download them.

I disabled those sources by putting a # before each of the two deb lines. The file now looks like this:

# Written by ubuntu-advantage-tools
#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-security main
# deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-security main

#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates main
# deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-updates main

I think deleting the file would have worked, too.

Now, when I update the system again, using my usual update command:

$ sudo apt update && sudo apt autoremove && sudo apt upgrade

...the apt upgrade portion doesn't try to download those unauthorized sources, and the upgrade is able to finish!

Lambart
  • 2,660
  • 22
  • 27
1

When you register a system with Ubuntu Pro a /etc/apt/auth.conf.d/90ubuntu-advantage file is created. This file contains credentials tied to your Ubuntu account which gives that system permission to retrieve ESM updates.

I registered a system and initially it had no issues, but after some weeks attempts to retrieve packages from https://esm.ubuntu.com/ gave a HTTP 401 "Unauthorized" error.

I resolved the issue by detaching and reattaching the machine using the current Token value from https://ubuntu.com/pro/dashboard.

This regenerated the /etc/apt/auth.conf.d/90ubuntu-advantage file with current credentials.

  1. sudo pro detach
  2. sudo pro attach TOKEN_VALUE_HERE
  3. sudo apt-get update

At this point I could apply updates (e.g., via apt, apt-get, aptitude) as before.

deoren
  • 353
  • 2
  • 9