This is an archival dump of old wiki content --- see scipy.org for current material

Red Hat 3.4.6, Python 2.5, GCC 3.4, FFTW, ATLAS, LAPACK

Here is the complete set of instructions I used to build everything using the GNU compilers. I am assuming that you are using the bash shell. I made the following files listed below and placed them in a common directory:

I execute these with the following script which runs them in term while recording a log of everything. 

export BASE=/data/gcc_version
. go.bash

go.bash

export BASE=${BASE:-$1}

mkdir -p logs/

function exec_and_log () {
    bash -v $1 2>&1 | tee logs/$1.log
}

setup_files=`ls ??_*.bash | sort -n`

for file in $setup_files
do
  exec_and_log $file
done

This gets the list of setup files from the current directory (using the ls command), sorts them numerically (hence the numeric prefix), and executes them one at a time placing the output into a log file in the logs sub-directory.

01_setup.bash

mkdir -p ${BASE}/zips
mkdir -p ${BASE}/src
mkdir -p ${BASE}/src/svn
mkdir -p ${BASE}/apps

cd ${BASE}
cat > pythonrc.py <<EOF
# Python startup script: sets up things like the history
# and tab completion features for the interpreter.
import os
import rlcompleter, readline
readline.parse_and_bind('tab: complete')
readline.parse_and_bind('"\M-OA": history-search-backward')
readline.parse_and_bind('"\M-[A": history-search-backward')
readline.parse_and_bind('"\M-\C-OA": history-search-backward')
readline.parse_and_bind('"\M-\C-[A": history-search-backward')
readline.parse_and_bind('"\M-OB": history-search-forward')
readline.parse_and_bind('"\M-[B": history-search-forward')
readline.parse_and_bind('"\M-\C-OB": history-search-forward')
readline.parse_and_bind('"\M-\C-[B": history-search-forward')

# Save and load history from a local file .pyhist
if False:
    import atexit
    histfile = ".pyhist"
    try:
        readline.read_history_file(histfile)
    except IOError:
        pass
    atexit.register(readline.write_history_file, histfile)
    del histfile

# Load user's startup file ~/.pythonrc.py if it exists
user_startup_filename = os.path.expanduser("~/.pythonrc.py")

if os.access(user_startup_filename,os.R_OK):
   del os
   execfile(user_startup_filename)
else:
   del os

del user_startup_filename
EOF

cat > setenv_gcc <<EOF
# Setup path and environment for using gcc compilers and programs
# compiled with gcc.
BASE=${BASE}

export CC=gcc

shopt -s extglob

PATH=":\${PATH%:}:"             # Add leading and trailing colons
function add () { PATH=":\$1\${PATH//:\$1:/:}"; }
add \${BASE}/apps/tcl8.4.14_gcc/bin/
add \${BASE}/apps/Python-2.5_gcc/bin/
add \${BASE}/apps/fftw-3.1.2_gcc/bin/
add \${BASE}/src/ATLAS-3.6.0_gcc/bin/Linux_P4SSE2/
add \${BASE}/apps/freetype-2.3.2_gcc/bin/
add \${BASE}/apps/libpng-1.2.16_gcc/bin/
PATH="\${PATH//+(:)/:}"         # Remove duplicate colons
PATH="\${PATH%:}"               # Remove trailing colon
PATH="\${PATH#:}"               # Remove leading colon
export PATH

CPATH=":\${CPATH%:}:"           # Add leading and trailing colons
function add () { CPATH=":\$1\${CPATH//:\$1:/:}"; }
add \${BASE}/apps/tcl8.4.14_gcc/include/
add \${BASE}/apps/Python-2.5_gcc/include/
add \${BASE}/apps/fftw-3.1.2_gcc/include/
add \${BASE}/src/ATLAS-3.6.0_gcc/include/Linux_P4SSE2/
add \${BASE}/apps/freetype-2.3.2_gcc/include/
add \${BASE}/apps/libpng-1.2.16_gcc/include/
CPATH="\${CPATH//+(:)/:}"       # Remove duplicate colons
CPATH="\${CPATH%:}"             # Remove trailing colon
CPATH="\${CPATH#:}"             # Remove leading colon
export CPATH

LD_LIBRARY_PATH=":\${LD_LIBRARY_PATH%:}:" # Add leading and trailing colons
function add () { LD_LIBRARY_PATH=":\$1\${LD_LIBRARY_PATH//:\$1:/:}"; }
add \${BASE}/apps/tcl8.4.14_gcc/lib/
add \${BASE}/apps/Python-2.5_gcc/lib/
add \${BASE}/apps/fftw-3.1.2_gcc/lib/
add \${BASE}/src/ATLAS-3.6.0_gcc/lib/Linux_P4SSE2/
add \${BASE}/apps/freetype-2.3.2_gcc/lib/
add \${BASE}/apps/libpng-1.2.16_gcc/lib/
LD_LIBRARY_PATH="\${LD_LIBRARY_PATH//+(:)/:}" # Remove duplicate colons
LD_LIBRARY_PATH="\${LD_LIBRARY_PATH%:}"       # Remove trailing colon
LD_LIBRARY_PATH="\${LD_LIBRARY_PATH#:}"       # Remove leading colon
export LD_LIBRARY_PATH

MANPATH=":\${MANPATH%:}:"       # Add leading and trailing colons
function add () { MANPATH=":\$1\${MANPATH//:\$1:/:}"; }
add \${BASE}/apps/tcl8.4.14_gcc/man/
add \${BASE}/apps/Python-2.5_gcc/man/
add \${BASE}/apps/fftw-3.1.2_gcc/share/man/
add \${BASE}/apps/libpng-1.2.16_gcc/share/man/
MANPATH="\${MANPATH//+(:)/:}"   # Remove duplicate colons
MANPATH="\${MANPATH%:}"         # Remove trailing colon
MANPATH="\${MANPATH#:}"         # Remove leading colon
export MANPATH

export PYTHONSTARTUP=\${BASE}/pythonrc.py

EOF

This uses the bash cat <<EOF construct to make the setenv_gcc script and a pythonrc.rc. A bit of bashing makes setenv_gcc removed duplicates from PATH etc. so that they do not become too long. The pythonrc.rc file includes the readline module which provides a much nice interface experience (there is also a section at the end that will save a readline history in a local file .pyhist, but it is not enabled by default). This is just here to show people how to do this.

02_download.bash

# Source file to set environment up
. ${BASE}/setenv_gcc

# Download files and check MD5 sums
cd ${BASE}/zips
wget -nv http://prdownloads.sourceforge.net/tcl/tcl8.4.14-src.tar.gz
wget -nv http://prdownloads.sourceforge.net/tcl/tk8.4.14-src.tar.gz
wget -nv http://www.python.org/ftp/python/2.5/Python-2.5.tar.bz2
wget -nv http://www.netlib.org/lapack/lapack-3.1.1.tgz
wget -nv http://downloads.sourceforge.net/math-atlas/atlas3.6.0.tar.bz2
wget -nv http://www.fftw.org/fftw-3.1.2.tar.gz
wget -nv http://download.savannah.gnu.org/releases/freetype/freetype-2.3.2.tar.bz2
wget -nv http://downloads.sourceforge.net/libpng/libpng-1.2.16.tar.bz2
wget -nv http://www.zlib.net/zlib-1.2.3.tar.gz

# These are from svn sources: if SVN does not work,
# you will have to get these manually.
cd ${BASE}/src/svn
svn co http://svn.scipy.org/svn/numpy/trunk numpy
svn co http://svn.scipy.org/svn/scipy/trunk scipy
svn co http://matplotlib.svn.sourceforge.net/svnroot/matplotlib/trunk/matplotlib matplotlib

cat > md5sums.md5 <<EOF
51c6bf74d3ffdb0bd866ecdac6ff6460  tcl8.4.14-src.tar.gz
d12f591f5689f95c82bfb9c1015407bb  tk8.4.14-src.tar.gz
ddb7401e711354ca83b7842b733825a3  Python-2.5.tar.bz2
00b21551a899bcfbaa7b8443e1faeef9  lapack-3.1.1.tgz
df2ee2eb65d1c08ee93d04370172c262  atlas3.6.0.tar.bz2
08f2e21c9fd02f4be2bd53a62592afa4  fftw-3.1.2.tar.gz
119e1fe126fcfa5a70bc56db55f573d5  freetype-2.3.2.tar.bz2
7a1ca4f49bcffdec60d50f48460642bd  libpng-1.2.16.tar.bz2
debc62758716a169df9f62e6ab2bc634  zlib-1.2.3.tar.gz
EOF

md5sum -cw md5sums.md5

Here we download all of the files, and compute their checksums. The NumPy, SciPy, and matplotlib packages are downloaded from the svn archives. To be consistent later on, we package these, but it would probably be better to simply leave these in the src directory and simply update them before building.

03_tck-tk.bash

# Build and Install Tcl/Tk for graphics support
. ${BASE}/setenv_gcc; env

cd ${BASE}/src
tar -zxvf ${BASE}/zips/tcl8.4.14-src.tar.gz
mv tcl8.4.14 tcl8.4.14_gcc

cd ${BASE}/src/tcl8.4.14_gcc/unix
./configure --prefix=${BASE}/apps/tcl8.4.14_gcc/
make
make install

cd ${BASE}/src
tar -zxvf ${BASE}/zips/tk8.4.14-src.tar.gz
mv tk8.4.14 tk8.4.14_gcc

cd ${BASE}/src/tk8.4.14_gcc/unix
./configure --prefix=${BASE}/apps/tcl8.4.14_gcc/
make
make install

04_python.bash

# Build and Install Python 2.5
. ${BASE}/setenv_gcc; env

cd ${BASE}/src
tar -jxvf ${BASE}/zips/Python-2.5.tar.bz2
mv Python-2.5 Python-2.5_gcc

cd ${BASE}/src/Python-2.5_gcc

./configure --prefix=${BASE}/apps/Python-2.5_gcc/\
            LDFLAGS="-L\${BASE}/apps/tcl8.4.14_gcc/lib/"\
            CPPFLAGS="-I\${BASE}/apps/tcl8.4.14_gcc/include/"

make
make test
make install

Note that part of the Python installation uses the python setup.py file and this does not grok the LD_LIBRARY_FLAGS or the CPATH environmental variables. Instead, it requires the LDFLAGS and CPPFLAGS variables, but apparently, it is not enough to just set these (somehow, they don't get transmitted from the environment to the interpreter). Thus, we must manually specify them in the configuration step so that python knows where to look. (If you install things to the usual place /usr/local/lib and /usr/local/include, then python will find them.)

05_lapack.bash

# Build LAPACK
. ${BASE}/setenv_gcc; env


cd ${BASE}/src
tar -zxvf ${BASE}/zips/lapack-3.1.1.tgz
mv lapack-3.1.1 lapack-3.1.1_gcc

cd ${BASE}/src/lapack-3.1.1_gcc
cp INSTALL/make.inc.LINUX make.inc

make lapacklib

The ATLAS library we will install next provides only a subset of LAPACK, so we need to make the full library first. We will merge it with the fast components of ATLAS after installing ATLAS.

06_atlas.bash

# Build ATLAS
. ${BASE}/setenv_gcc; env


cd ${BASE}/src
tar -jxvf ${BASE}/zips/atlas3.6.0.tar.bz2
mv ATLAS ATLAS-3.6.0_gcc

cd ${BASE}/src/ATLAS-3.6.0_gcc
make config <<EOF
023
y

y
y




y

EOF
make install arch=Linux_P4SSE2
make sanity_test arch=Linux_P4SSE2

# Merge LAPACK
cd ${BASE}/src/ATLAS-3.6.0_gcc/lib/Linux_P4SSE2/
mkdir tmp
cd tmp
ar x ../liblapack.a
cp ${BASE}/src/lapack-3.1.1_gcc/lapack_LINUX.a ../liblapack.a
ar r ../liblapack.a *.o
cd ..
rm -rf tmp

cd ${BASE}/src/ATLAS-3.6.0_gcc
make sanity_test arch=Linux_P4SSE2

The Automatically Tuned Linear Algebra Software (ATLAS) library provides a fast implementation of the BLAS for computing with arrays. This is important for speed gains using NumPy. Get it from https://sourceforge.net/project/showfiles.php?group_id=23725.

The configuration utility asks several questions which you should answer (in the previous script I just simulated the responses.) The result is a configuration for your architecture (Linux_P4SSE2 in this case). Note that ATLAS does not install itself elsewhere, so we just use it in the src directory.

Finally, the LAPACK library included with ATLAS is not complete. However, the LAPACK library we made in the previous step is not fast. The solution is to make a hybrid LAPACK library in the ATLAS directory. (Instructions for doing this are given at http://math-atlas.sourceforge.net/errata.html#completelp.) This is the last part of the script.

07_fftw.bash

# Build and install FFTW
. ${BASE}/setenv_gcc; env

cd ${BASE}/src
tar -zxvf ${BASE}/zips/fftw-3.1.2.tar.gz
mv fftw-3.1.2 fftw-3.1.2_gcc

cd ${BASE}/src/fftw-3.1.2_gcc
./configure --prefix=${BASE}/apps/fftw-3.1.2_gcc/
make
make check
make install

The FFTW is a tuned library for Fast Fourier Transforms used by SciPy.

08_numpy.bash

############# Build python components
. ${BASE}/setenv_gcc; env

# First make sure we use the correct python
which python

# Build and Install NumPy
cd ${BASE}/src/svn
mv numpy numpy_gcc

cd ${BASE}/src/svn/numpy_gcc
cat > site.cfg <<EOF
[atlas]
library_dirs = ${BASE}/src/ATLAS-3.6.0_gcc/lib/Linux_P4SSE2/
#libraries = lapack, f77blas, cblas, atlas
EOF

python setup.py build
python setup.py install

cd ../../
python -c "import numpy;numpy.test(100)"

09_scipy.bash

# Build and Install SciPy
. ${BASE}/setenv_gcc; env

cd ${BASE}/src/svn
mv scipy scipy_gcc

cd ${BASE}/src/svn/scipy_gcc
cat > site.cfg <<EOF
[atlas]
library_dirs = ${BASE}/src/ATLAS-3.6.0_gcc/lib/Linux_P4SSE2/

[fftw3]
library_dirs = ${BASE}/apps/fftw-3.1.2_gcc/lib/
EOF

python setup.py build
python setup.py install

cd ../../
python -c "import scipy;scipy.test(100)"

10_freetype.bash

# Build and install the freetype library
. ${BASE}/setenv_gcc; env

cd ${BASE}/src
tar -jxvf ${BASE}/zips/freetype-2.3.2.tar.bz2
mv freetype-2.3.2 freetype-2.3.2_gcc

cd ${BASE}/src/freetype-2.3.2_gcc
./configure --prefix=${BASE}/apps/freetype-2.3.2_gcc/
make
make install

Depending on your distribution of Linux, you may not need to install the FreeType library. Here is the installation in case you do. (If you do not get graphics output later you may need this.) The sources are available from http://freetype.sourceforge.net/download.html#stable.

11_libpng.bash

# Build and install libpng
. ${BASE}/setenv_gcc; env

cd ${BASE}/src
tar -jxvf ${BASE}/zips/libpng-1.2.16.tar.bz2
mv libpng-1.2.16 libpng-1.2.16_gcc

cd ${BASE}/src/libpng-1.2.16_gcc
./configure --prefix=${BASE}/apps/libpng-1.2.16_gcc/
make
make install

Again, depending on your distribution, you may or may not need the LibPNG libraries. The sources are available from http://www.libpng.org/pub/png/libpng.html.

12_matplotlib.bash

# Build and Install matplotlib
. ${BASE}/setenv_gcc

cd ${BASE}/src/svn
mv matplotlib matplotlib_gcc

cd ${BASE}/src/svn/matplotlib_gcc
python setup.py build
python setup.py install

You may also need to install the zlib library which is available from http://zlib.net/. The installation is canonical.

To check that it all worked, try plotting a simple graph: 

-bash-3.00$ . ${BASE}/setenv_gcc
-bash-3.00$ python
Python 2.5 (r25:51908, Mar 29 2007, 16:48:58)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from pylab import *
>>> x=linspace(-10,10,1000)
>>> y=sin(x)
>>> ion()
>>> plot(x,y)
[<matplotlib.lines.Line2D instance at 0xb79da6ac>]

This should produce a nice plot. If you do not see a plot but see no errors, then your backend is probably not set properly. Check the compilation log file for errors and warnings. One potential problem seems to be compiling matplotlib from a terminal that cannot open a windows. This causes the installation scripts to think you can't display windows and it seems to turn off the display options. Recompiling from a terminal that can open a window should fix this. (There is probably a way of directly forcing this too but I do not know it yet.)

GNU Compilers

A C++ compiler is not presently needed for Python or SciPy but may be useful for Weave code. We need the C compiler and the Fortran compilers. Here is what Red Hat provided: 

$ gcc --version
gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
Copyright (C) 2006 Free Software Foundation, Inc.
...
$ g77 --version
GNU Fortran (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
Copyright (C) 2006 Free Software Foundation, Inc.
...

CategoryInstallation

SciPy: Installing_SciPy/Linux/BuildingFromSource/GCC_1 (last edited 2015-10-24 17:48:25 by anonymous)