Installing AMGX and MiniFE on GPU architecture machine (Buzzard)

Table of Contents 

Download software

 

Installation

  • AMGX
  1. unzip 2014.12.22_amgx_redhat-6.5_cuda-6.5_openmpi-1.7.2.zip
  2. We do not have CUDA 6.5 so we create symbolic links with out current installation

    cd lib
    ln -s /usr/local/cuda/lib64/libcudart.so.8.0 libcudart.so.6.5
    ln -s /usr/local/cuda/lib64/libcublas.so.8.0 libcublas.so.6.5
    ln -s /usr/local/cuda/lib64/libcusparse.so.8.0 libcusparse.so.6.5
    ln -s /usr/mpi/gcc/openmpi-1.10.5a1/lib64/libmpi.so libmpi.so.1
     

  3. Declare the path for the library
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/sw/cs/amgx/lib/
     
  4. Compile
    cd ../examples
    make

 

  • MiniFE
  1. git clone https://github.com/Mantevo/miniFE.git
     
  2. cd miniFE/nvamg/src
     
  3. vim cg_solve.hpp

        Convert line 38 from:
       #include "nvamg_c.h"
       to
       #include "amgx_capi.h"

       Replace all the NVAMG calls to AMGX, execute the following command from the vim editor
       %s/NVAMG/AMGX/g

       close the file

     4. vim Makefile

        Declare the AMGX path in line 12
     
        NVAMG_PATH = /sw/cs/amgx/

        Convert line 25 from:

        LDFLAGS=$(CFLAGS) -L$(NVAMG_PATH)/lib -lnvamgsh
        to
        LDFLAGS=$(CFLAGS) -L$(NVAMG_PATH)/lib -lamgxsh
        close file
  

    5. vim make_targets

       Add option "-ldl" at the end of line 23
       close file

    6. Compile
        make
        The executable miniFE.x is created

        cd ../basic
        vim get_common_files

        Convert line 3 from:
        dir=../../common
        to
        dir=../src
       
        close file

        Execute make twice
 

    7. Example

       cp gold_files/1x1x2_A.mtx.1.0 A.mtx.1.0
       cp gold_files/1x1x2_A.mtx.2.0 A.mtx.2.0
       cp gold_files/1x1x2_A.mtx.2.1 A.mtx.2.1
       cp gold_files/1x1x2_b.vec.1.0 b.vec.1.0
       cp gold_files/1x1x2_b.vec.2.0 b.vec.2.0
       cp gold_files/1x1x2_b.vec.2.1 b.vec.2.1
       cp gold_files/1x1x2_x.vec.1.0 x.vec.1.0
       cp gold_files/1x1x2_x.vec.2.0 x.vec.2.0
       cp gold_files/1x1x2_x.vec.2.1 x.vec.2.1
 

       Execute

      ./run_tests
     
      running miniFE tests...
      running miniFE test for np=1, nx=1 ny=1 nz=2...
      tests passed

      running miniFE test for np=2, nx=1 ny=1 nz=2...
      tests passed