Installing Prophet (pystan build) on Docker container (buster)

Question

I'd like to install fbprophet in a Docker container. My Dockerfile looks like this:

FROM python:3.7

RUN pip install --upgrade pip
RUN pip install fbprophet

Building the image errors on the "pip install fbprophet" step with this stack trace:

    Running setup.py install for fbprophet ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7ei5jssc/fbprophet_9a3a667ec353402389a02258feccfe51/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7ei5jssc/fbprophet_9a3a667ec353402389a02258feccfe51/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-741oj2zp/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7m/fbprophet
         cwd: /tmp/pip-install-7ei5jssc/fbprophet_9a3a667ec353402389a02258feccfe51/
    Complete output (10 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/fbprophet
    creating build/lib/fbprophet/stan_model
    Importing plotly failed. Interactive plots will not work.
    INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec NOW.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7ei5jssc/fbprophet_9a3a667ec353402389a02258feccfe51/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7ei5jssc/fbprophet_9a3a667ec353402389a02258feccfe51/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-741oj2zp/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7m/fbprophet Check the logs for full command output.

There are a number of suggestions I've found online that I've tried unsuccessfully (at this point I built the container without fbprophet and ran it, so I could try commands live):

1. Here the problem was that gcc/g++ weren't installed.

• I tried apt install g++ gcc, still fails to build (they were already installed)

2. pnmartinez's comment here suggests some missing build tools.

• I did apt-get install build-essential libncursesw5-dev libreadline-gplv2-dev libssl-dev libgdbm-dev libc6-dev libsqlite3-dev libbz2-dev libffi-dev zlib1g-dev (after running apt-get upgrade, apt-get update, apt-get dist-upgrade), and retried with same result.

3. This GH issue suggests downgrading to pystan==2.18.

• Before this commit introduced in v0.6, Prophet restricted itself to PyStan <=2.18.1, so I tried pip install prophet==0.5, which still failed. (I went all the way down to 0.2, still failed.) I then tried pip3 uninstall pystan && pip3 install pystan==2.18 && pip3 install fbprophet==0.5 with the same result.

4. Install from conda-forge instead

• I looked into this, but installing a working conda env in a docker container seems pretty complicated

5. I also started trying to install on an Alpine-based container instead of Debian.

• I started running into issues there before the Prophet stage, and in general I would assume building a complicated C++ project won't be easier on a non-standard distro.

It appears that the problem occurs when gcc tries to build a PyStan model. My guess is that it is simply running out of memory (the cc1plus proc on top uses 2GB, about 90% of RAM, someone claims here it can use up to 8GB). However, this is pretty common software, and I succeeded in installing fbprophet with the same Dockerfile a few months back, so I'm not sure why this has become a problem. (I've tried building it both on macOS (Big Sur) and Linux (Fedora 33) with the same result.)

Answer 1

Figured it out - as it was a memory issue, I just needed to allocate more memory to the Docker VM. This answer explains how to do it with the GUI. A lot of time wasted...

Answer 2

Well, that's what I found - I gave 6 CPUs to Docker VM, so when it tries to build a code, GCC spawns 6 threads, each of them consuming too much RAM. I can say - even 16 Gb might be not enough. So, limiting the number of CPUs for the Docker VM by the, let's say, 4 CPU might be a good choice.

Answer 3

For anyone else having this issue, I was building with base image python:3.7-slim Changing to python:3.7 fixed the issue

Answer 4

For anyone wanting to go the Conda route (OP's #4 idea and what worked best for me), here's how to do it:

FROM python:3-slim

RUN apt-get update && \
    apt-get install -y wget && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Install Miniconda.
RUN wget --quiet 
https://repo.anaconda.com/miniconda/Miniconda3-latest- 
Linux-x86_64.sh -O /tmp/install_miniconda.sh && \
 /bin/bash /tmp/install_miniconda.sh -b -p /opt/conda
ENV PATH=/opt/conda/bin:$PATH

# Install FB Prophet.
RUN conda install -c conda-forge Prophet

Answer 5

With Mac OSX Big Sur 11.6

My Dockerfile includes

FROM python:3.7-slim

RUN pip install --upgrade pip
RUN pip install pystan==2.19
RUN pip install fbprophet