Research:Salsa
From Astronomy Facility Wiki
Also See the documentation on the Salsa Page
Salsa is our first working parallel, interactive analysis tool. It's composed of two pieces, a server running in parallel on the back end, and a Java client running on your local machine. Our current implementation uses some script hackery to automagically start and connect to the server when you fire up the client.
Contents |
Obtaining Salsa
The 0.3 release of Salsa can be downloaded from the NASA AISRP Code and Algorithm Library.
The lastest version can be checked out of our CVS repository (CVSROOT=hpcc.astro.washington.edu:/net/CVS/nchilada)
with
cvs checkout salsa.
Building Salsa
Also see the README file in the ResolutionServer directory of the salsa distribution.
Prerequisite software
- Charm++ (CVS version)
- boost (1.31 or greater)
- Java JDK 1.4.X.
- Apache Xerces-C++ XML parser
- Python (2.2 or greater)
Build Resolution Server
- Install Charm++ first.
Note that you will probably need the latest CVS version of Charm++ rather than the released version. If you get a "Charmxi syntax error", during the Resolutionserver build, then you need a later version of Charm++.
- Compile the "liveViz" and "pythonCCS" modules of Charm++
cd tmp/libs/ck-libs/liveViz; make
cd tmp/libs/ck-libs/pythonCCS; make
A symbolic link may be needed in the pythonCSS directory.
E.g. ln -s /usr/include/python2.2 python
or
ln -s /astro/net/python/include/python2.5 python
- Install and build the Apache Xerces C++ XML Parser ([1]) in
the parent directory. Set the XERCESCROOT environment variable to point at this directory.
- Set the environment variable "CHARM_DIR" to point at the directory in which the source of charm is located.
- Type
configurein the ResolutionServer directory.
- Type
makein the ResolutionServer directory.
Build Interactive Client
The java client is in the separate directory InteractiveClient. Make sure the "CHARM_DIR" environment variable points to the charm source directory (there are some java class libraries that are neede there) and type make in the InteractiveClient directory. This should build the Salsa.jar Java archive.
Starting Salsa on a Workstation
Although Salsa is designed as a parallel analysis application, it can be run on a workstation, or a group of workstations on a network. In order to do this, one of the "net" versions of charm++ needs to be used for the build. For example, for a linux workstation, build charm++ with
build charm++ net-linux
in the charm directory. When ResolutionServer is built, a charmrun executable is produced which does the job of starting the server up on multiple (or just one) workstations. An example command line is
charmrun ResolutionServer ++server +p 1 simlist
which would start ResolutionServer on one processor on the machine specified in the nodelist or ~/.nodelist. An example of this file is
group main ++shell /usr/bin/ssh -X host mymachine
The charmrun command will report which port it is listening on. E.g.
ccs: Server IP = 127.0.0.1, Server port = 32991
This information can now be used to start the Java interactive client:
java -jar Salsa.jar localhost 32991 tipsyfile
All this is somewhat automated in the Run_Salsa.sh script in the ResolutionServer directory.
Python execution
The server is actually running a python interpreter into which the client is feeding commands. If you want to access it directly, use the Write and Execute Code menu item. Be aware that for any complicated python, the stack size on the server needs to be increased from its default. The ResolutionServer will need to be started with
charmrun ++server ResolutionServer +p 1 +stacksize 1000000 simlist
Starting Salsa on a Cluster
Starting ResolutionServer
ResolutionServer is a parallel program, and so starting it depends on the nature of your parallel machine. Here is a script to run ResolutionServer as an MPI job on the Student Cluster (Robert). This script is available in ~trq/bin.x86_64/salsa_bsub on robert.
#!/bin/sh
# Usage:
# salsa_bsub nproc
NPROC=$1
echo set up tunnel with \" ssh -v -L16512:XXX:16512 robert\" replacing XXX
echo with IP address given below.
bsub -q short -n $NPROC -I << EOF
#!/bin/sh -x
rm -f nodelist
for i in \$LSB_HOSTS ; do
echo \$i >> nodelist
done
NPROC=\`wc nodelist | awk '{print $1;}'\`
mpirun_ssh -hostfile nodelist -np $NPROC /home/trq/bin.x86_64/ResolutionServer ++server ++server-port 16512 simlist
EOF
The crux of the script is the penultimate line where an MPI job started using all the nodes in nodelist. The ++server ++server-port 16512 options tell it to start listening for commands on the local internet port 16512. simlist can be a file listing available simulations, but if it doesn't exist, it is ignored.
Tunnelling
Unfortunately, the internet port that ResolutionServer is listening to is buried deep within the cluster and is inaccessible to the outside world. Hence one needs to set up a "tunnel" to access it. The concept is that a separate port is setup (say, on your workstation) such that any connections to this port are forwarded to the port inside the cluster via an ssh connection to the cluster front end. The command to set this up would be, e.g.,
ssh -v -L16512:10.127.1.235:16512 robert
If this is run on your workstation, then ssh listens for connections on port 16512 on your workstation, and forwards any connections through robert to port 16512 on the host "10.127.1.235" (an internal node on robert). The host number is reported by ResolutionServer when it starts so that you know which internal node to connect to.
That's the hard part. Any ideas for automating this process would be greatly appreciated.
Interactive Client
With the tunnelling established, the java client can be started on your workstation:
java -jar Salsa.jar localhost 16512 /path-on-robert/simulation_file
This connects to the 16512 port on your local machine, which gets forwarded by the tunnelling to the parallel server. The simulation file (tipsy or NChilada format) needs to be specified with respect to where ResolutionServer is run on robert.
Starting Salsa on a Supercomputer
See Research:SalsaBuildInstructions for details on particular supercomputers.
Using the Interactive Client
The java program will open a window and request an image from the server. If all goes well, and image will be displayed. The image can be rotated and zoomed as in Tipsy. New colorings can be defined in the Coloring item of the Manage drop-down menu. Ranges of individual attributes can be examined in the Attributes entry of the Manage menu. Subsets can be selected using the shift button and dragging, similar to Tipsy.
Using the Mouse in the Graphics Window
As with Tipsy, left mouse clicks will zoom in on the image, and center mouse clicks will zoom out. Unlike tipsy, rotations will not reset the zoom scale. If the potential attribute is defined, the view will rotate around the particle in the image with the lowest potential.
A shift left click and drag will outline a box to be selected. Upon release, the simulation will be rotated by 90 degrees so that the third dimension of the box can be specified, again with a left mouse click and drag.
A CTRL left click and drag will define a sphere. The initial mouse position will define a center of the sphere, and the release point will set the boundary. The third dimension of the center of the sphere will be defined by the center of rotation.
Using Python
Under the Write and execute code menu item a window can be popped up into which python can be entered and executed on the server. The charm module contains methods for operating on the data.
print dir(charm) will give a list of these methods. print charm.__doc__ will give brief documentation. More complete documentation can be found in Research:Salsa Python
A particularly interesting method is charm.runLocalParticleCode(string). This will cause the method localparticle to be executed with each particle as an argument. The string argument should be python code containing a definition of localparticle that will manipulate the particle attributes. E.g.
charm.runLocalParticleCode("def localparticle(p):\n p.mass = 2*p.mass")
will double the mass of every particle.
This method can also be accessed through the Write and execute local code menu. In that case, the def statement is implied, and the particle data can be address through the variable p. E.g., the above can be accomplished by typing p.mass = 2*p.mass in the LocalParticle Code text box.
Taco: a command line client
The CommandClient directory of the Salsa distribution contains taco, a command line "shell" client that allows you to type python at the ResolutionServer. This may make it easier to perform scripting operations with the ResolutionServer. Note that you can connect to the same ResolutionServer that you started with Salsa, so you can visualize and do scripting at the same time. The usage is:
taco hostname port
