Research:ChaNGaBuildInstructions

From Astronomy Facility Wiki

Jump to: navigation, search

Specific instructions for Research:ChaNGa builds on various parallel architectures are documented below. Prefered Charm++ build options are noted. If the machine corresponds to a NSF Teragrid facility, the particular machine will be noted.

An *autobuild* script is periodically run to verify the builds on all supported platforms. The status of these builds will be documented on a web page to be determined.

Contents

[edit]

i386 Suse/Redhat Beowulf

[edit]

net-linux

In this configuration, Charm uses a UDP protocol for communicating over the network.

It builds out of the box with configure/make.

[edit]

Issues

With gcc version 4.0.X you make get the error ctype_base.h:58: error: '<anonymous enum>' is/uses anonymous type. I believe this is a bug in these versions of the compiler. A work-around is:

  • Copy /usr/include/ctype.h to the changa directory.
  • Edit the enum definition to make it non-anonymous. E.g change line 325 to be: 
enum __XXX_ISbit_enum
{
_ISupper = _ISbit (0), /* UPPERCASE. */
  • Edit the makefile so that CXXFLAGS has -I. in it.
  • make!
[edit]

SGI Altix (Cobalt at NCSA)

[edit]

mpi-linux-ia64-mpt-icc

Builds and runs out of the box.

[edit]

IBM SP4 (Copper at NCSA)

[edit]

mpi-sp

  • May need to configure --host aix if the C/C++ compiler produces an executable that needs an MPI environment to run.
  • Need to use gmake instead of make
  • Builds fine otherwise.
  • Note that charmrun is just a wrapper around poe. It is more robust just to use poe to start a parallel job.
[edit]

lapi

  • For systems with a Federation switch (ARSC iceberg), directly using IBM's communication layer may give better performance.
  • Need to use gmake instead of make
  • Builds fine otherwise.
  • Note that charmrun is just a wrapper around poe. It is more robust just to use poe to start a parallel job.
  • Set the network using # @ network.LAPI = css0,,US in the loadleveler command file.
[edit]

Cray XT3 (Bigben at PSC)

The Cray OS (catamount) does not have the xdr library available. Download it from our distribution site, and compile it with "gcc" before building ChaNGa.

[edit]

mpi-crayxt3-gcc4

  • The following commands need to be executed before Charm and ChaNGa can be built.
    module load gcc/4.0.2
    module remove acml
  • configure needs to be run as ./configure -host linux since the cray front end is actually a cross-compilation environment.
  • charmrun doesn't work on bigben. Use the standard pbsyod to run ChaNGa.
[edit]

mpi-crayxt3

This uses the default pathScale compiler which may give better performance. However building charm is a little tricky.

  • Use build charm++ mpi-crayxt3 to build charm.
  • Change to the tmp directory and edit conv-mach.sh and change the CMK_SEQ_CC and CMK_SEQ_LD definitions to gcc.
  • Type make charm++ to rebuild with these changes.
  • Build ChaNGa with the standard configure and make commands.
  • As above, use pbsyod to run.

If you do not make the changes above, you will get a mysterious alloca() error.

[edit]

Dell Dual Core Linux cluster (lonestar at TACC)

[edit]

mpi-linux-amd64

Charm should be built with MPICC=mpicc MPICXX=mpiCC build charm++ mpi-linux-amd64 -O3 -xT

Configures and builds out of the box.

  • However, an FPE error occurs unless the system memory allocator is used. Edit the Makefile to add -memory os to the LDFLAGS definition (but before the '#', which is a comment character).
[edit]

Sun Opteron Infiniband cluster (midnight at ARSC)

[edit]

mpi-linux-amd64

Charm should be build by first setting your programming environment to use gcc: module load voltairempi-S-1.gcc, and then building charm with: build charm++ mpi-linux-amd64

The pathscale compiler should not be used at the moment. There are several files which cause the compiler to crash(!) at the highest optimization level, and it breaks the Charm++ thread layer. On the other hand, the PGI compiler seems to work fine.

It has been reported that ChaNGa runs faster (per core) on the 16-way nodes than on the 4-way nodes.

[edit]

IBM Bluegene/L

This architecture does not come with an XDR library which ChaNGa uses for machine independent output. For this machine a compiled version of the XDR library is provided on our distribution site. Download the file xdr.tgz from the distribution site and unpack it in the ChaNGa directory. The configure script for ChaNGa will then detect it and link to it appropriately.

Actually, the configure script is not so automatic. First rename the "xdr" directory to "rpc". Second, "../structures/config.h" needs to be told that "xdr_hyper()" is available. Edit it to contain the line #define HAVE_XDR_HYPER = 1. Third the changa Makefile needs to be changed to add "XDR_OBJS" to the LDLIBS macro.

Retrieved from "http://librarian.phys.washington.edu/astro/index.php/Research:ChaNGaBuildInstructions"
Personal tools