8

snapcraft cleanbuild is supposed to let me build snaps in clean-room environments. Accordingly it starts from a minimal LXD image and installs necessary packages. However, it seems to use the base archive.ubuntu.com mirror, which is excruciatingly slow. How do I cache the packages it download?

I could set up apt-cacher-ng or another caching proxy, but there's still the problem of getting the snap/LXD combination to use it. It would be best if it had an internal cache like pbuilder does, but if not, then how do I convince it use a different mirror or a proxy for the mirror?

muru
  • 193,181
  • 53
  • 473
  • 722
  • Could you try Proxy Configuration from this ref https://insights.ubuntu.com/2016/03/16/lxd-2-0-installing-and-configuring-lxd-212/ – user.dz Apr 20 '17 at 14:38
  • 1
    @user.dz Unless I'm mistaken that's configuration for the LXD daemon, not for the LXD containers. At any rate, I installed apt-cacher-ng and set it as the proxy according to those instances, but a new run of `snapcraft cleanbuild` isn't using it (apt-cacher-ng has no changes in logs or in the cache dir) – muru Apr 20 '17 at 15:13
  • 1
    (The configuration did take effect: a-c-ng doesn't like HTTPS CONNECT requests and I had to unset the HTTPS proxy setting before LXD started working again.) – muru Apr 20 '17 at 15:15
  • Do you think you could accomplish what you want by modifying the hosts file (at least temporarily) to redirect connections destined for archive.ubuntu.com to a server with your cached packages? – b_laoshi Apr 25 '17 at 02:26
  • @b_laoshi that would need to be done in the container, not the host server, and if I could do that, I would be able to edit the `sources.list` or `apt.conf` to set a different mirror or a proxy. But snapcraft sets up a container and gets started on building without giving me a chance to do anything, and it seems like a racy way to try and do it manually after snapcraft has started the container. – muru Apr 25 '17 at 02:30
  • 1
    [Here's](https://tribaal.io/making-lxd-fly-on-ubuntu-as-well.html) a nice article which details how you can use a package proxy on the host computer, and configure lxd to use that. – popey May 15 '17 at 11:01
  • 1
    @popey thanks! Would you write an answer summarizing that? I used `environment.http_proxy` instead of `user.user-data/apt`, and a more general squid/apt-cacher-ng proxy instead of squid-deb-proxy, but the core idea is there – muru May 18 '17 at 19:06
  • @muru if popey doesn't write an answer would you consider answering your question yourself please? That would shorten the "unanswered list" for those of us browsing through it. Thanks! – WinEunuuchs2Unix Jul 02 '17 at 21:23

1 Answers1

0

The basic principles as outlined in popey's suggestion work:

  1. Set up a caching proxy on the host (or elsewhere, depending on your preferences, I'll assume host): apt-cacher-ng, squid-deb-proxy or squid itself.

  2. Get the host IP address (as seen by the container):

    export LXD_ADDRESS=$(ip -4 -o address show dev lxdbr0 | awk -F'[ /]*' '{print $4}')

  3. Use it to set the default profile proxy settings:

    printf '%s\n' '#cloud-config' 'apt:' " proxy: http://$LXD_ADDRESS:8000" |
  lxc profile set default user.user-data -
  4. Success!

However, there are a couple of caveats.

snapcraft cleanbuild downloads a lot of packages by some other mechanism than the usual apt-get, which isn't affected by this configuration. This insanity is seen in this part of the output:

Preparing to pull glue
Pulling glue
Preparing to pull cassandra
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
...
Get:46 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [3004 B]
Fetched 26.0 MB in 6s (4237 kB/s)
Get:1 ca-certificates-java_20160321_all.deb [12.9 kB]
Fetched 12.9 kB in 0s (0 B/s)
Get:1 libxdmcp6_1.1.2-1.1_amd64.deb [11.0 kB]
Fetched 11.0 kB in 0s (0 B/s)
Get:1 init-system-helpers_1.29ubuntu4_all.deb [32.3 kB]
Fetched 32.3 kB in 0s (0 B/s)
Get:1 default-jre-headless_1.8-56ubuntu2_amd64.deb [4380 B]
Fetched 4380 B in 0s (0 B/s)
Get:1 default-jdk_1.8-56ubuntu2_amd64.deb [968 B]
Fetched 968 B in 0s (0 B/s)
Get:1 libkrb5-3_1.13.2+dfsg-5ubuntu2_amd64.deb [273 kB]

There are quite a few of these, and the individual requests make them slow as hell anyway, and that's not counting that that the apt proxy settings set earlier don't affect them.

So, I set the http_proxy environment variable instead:

lxc profile set default environment.http_proxy "http://$LXD_ADDRESS:3128"

And similarly for HTTPS, since some other downloads in the container used HTTPS. I usually prefer apt-cacher-ng, but it doesn't support HTTPS CONNECT, so I switched to squid.

muru
  • 193,181
  • 53
  • 473
  • 722