This section describes how to unpack, make, and run SPPARKS.
2.1 What's in the SPPARKS distributionWhen you download SPPARKS you will need to unzip and untar the downloaded file with the following commands, after placing the tarball in an appropriate directory.
gunzip spparks*.tar.gz tar xvf spparks*.tar
This will create a spparks directory containing two files and several sub-directories:
| README | text file |
| LICENSE | the GNU General Public License (GPL) |
| doc | documentation |
| examples | test problems |
| python | Python wrapper |
| src | source files |
| tools | auxiliary tools |
This section has the following sub-sections:
Building SPPARKS can be non-trivial. You will likely need to edit a makefile, there are compiler options, additional libraries can be used (MPI, JPEG), etc. Please read this section carefully. If you are not comfortable with makefiles, or building codes on a Unix platform, or running an MPI job on your machine, please find a local expert to help you.
Building a SPPARKS executable:The src directory contains the C++ source and header files for SPPARKS. It also contains a top-level Makefile and a MAKE sub-directory with low-level Makefile.* files for several machines. From within the src directory, type "make" or "gmake". You should see a list of available choices. If one of those is the machine and options you want, say serial or mpi or linux, then type one of the commands:
make serial make mpi gmake linux
Try the "serial" and "mpi" targets first, since they are generic and should typically work on any machine, assuming you have the GNU g++ compiler (for the serial version) and MPI installed (for the mpi version).
Note that on a multi-processor or multi-core platform you can launch a parallel make, by using the "-j" switch with the make command, which will typically build SPPARKS more quickly.
If you get no errors and an executable like spk_serial or spk_mpi is produced, you're done; it's your lucky day.
IMPORTANT NOTE: You need a C++ compiler that is C++11 compliant to build SPPARKS. Almost all current C++ compilers are; you just need to use a -std=c++11 flag when compiling, as in the src/MAKE/Makefile.machine files provided with SPPARKS.
(1) If the make command breaks immediately with errors that indicate it can't find files with a "*" in their names, this can be because your machine's make doesn't support wildcard expansion in a makefile. Try gmake instead of make.
(2) Other errors typically occur because the low-level Makefile isn't setup correctly for your machine. If your platform is named "foo", you need to create a Makefile.foo in the MAKE sub-directory. Use whatever existing file is closest to your platform as a starting point. See the next section for more instructions.
These are the issues you need to address when editing a low-level Makefile for your machine. With a couple exceptions, the only portion of the file you should need to edit is the "System-specific Settings" section.
(1) Change the first line of Makefile.foo to include the word "foo" and whatever other options you set. This is the line you will see if you just type "make".
(2) The "compiler/linker settings" section lists compiler and linker settings for your C++ compiler, including path and optimization flags. You can use g++, the open-source GNU compiler, which is available on all Unix systems. You can also use mpicc which will typically be available if MPI is installed on your system, though you should check which actual compiler it wraps. You can also point to a specific compiler; for example see MAKE/Makefile.spencer.gnu where an environment variable MPI_HOME is used to specify path to mpicxx and mpicc compilers.
Vendor compilers often produce faster code. On boxes with Intel CPUs, we suggest using the commercial Intel icc compiler, which can be downloaded from Intel's compiler site.
If building a C++ code on your machine requires additional libraries, then you should list them as part of the LIB variable.
The DEPFLAGS setting is what triggers the C++ compiler to create a dependency list for a source file. This speeds re-compilation when source (*.cpp) or header (*.h) files are edited. Some compilers do not support dependency file creation, or may use a different switch than -D. GNU g++ works with -D. If your compiler can't create dependency files (a long list of errors involving *.d files), then you'll need to create a Makefile.foo patterned after Makefile.storm, which uses different rules that do not involve dependency files.
(3) The "system-specific settings" section has 3 parts.
(3.a) The SPK_INC variable is used to include options that turn on system-dependent ifdefs within the SPPARKS code. The settings that are currently recogized are:
The read_sites and dump commands will read/write gzipped files if you compile with -DSPPARKS_GZIP. It requires that your Unix support the "popen" command.
If you use -DSPPARKS_JPEG, the dump image command will be able to write out JPEG image files. If not, it will only be able to write out text-based PPM image files. For JPEG files, you must also link SPPARKS with a JPEG library. See section (3.c) below for more details on this.
Use at most one of the -DSPPARKS_SMALLBIG, -DSPPARKS_BIGBIG, -DSPPARKS_SMALLSMALL settings. The default is -DSPPARKS_SMALLBIG. These settings refer to use of 4-byte (small) vs 8-byte (big) integers within SPPARKS, as specified in src/spktype.h. The only reason to use the BIGBIG setting is to enable simulation of systems with more than 2 billion sites. Normally, the only reason to use SMALLSMALL is if your machine does not support 64-bit integers. See the Additional build tips section below for more details on these settings.
(3.b) The 3 MPI variables are used to specify an MPI library to build SPPARKS with.
If you want SPPARKS to run in parallel, you must have an MPI library installed on your platform. If you use an MPI-wrapped compiler, such as "mpicc" to build SPPARKS, you can probably leave these 3 variables blank. If you do not use "mpicc" as your compiler/linker, then you need to specify where the mpi.h file (MPI_INC) and the MPI library (MPI_PATH) is found and its name (MPI_LIB).
If you are installing MPI yourself, we recommend Argonne's MPICH 1.2 or 2.0 or OpenMPI. MPICH can be downloaded from the Argonne MPI site. OpenMPI can be downloaded the OpenMPI site. LAM MPI should also work. If you are running on a big parallel platform, your system people or the vendor should have already installed a version of MPI, which will be faster than MPICH or OpenMPI or LAM, so find out how to build and link with it. If you use MPICH or OpenMPI or LAM, you will have to configure and build it for your platform. The MPI configure script should have compiler options to enable you to use the same compiler you are using for the SPPARKS build, which can avoid problems that can arise when linking SPPARKS to the MPI library.
If you just want SPPARKS to run on a single processor, you can use the STUBS library in place of MPI, since you don't need a true MPI library installed on your system. See the Makefile.serial file for how to specify the 3 MPI variables. You will also need to build the STUBS library for your platform before making SPPARKS itself. From the STUBS dir, type "make" and it will hopefully create a libmpi.a suitable for linking to SPPARKS. If this build fails, you will need to edit the STUBS/Makefile for your platform.
The file STUBS/mpi.cpp has a CPU timer function MPI_Wtime() that calls gettimeofday() . If your system doesn't support gettimeofday() , you'll need to insert code to call another timer. Note that the ANSI-standard function clock() rolls over after an hour or so, and is therefore insufficient for timing long SPPARKS simulations.
(3.c) The 3 JPG variables are used to specify a JPEG library which SPPARKS uses when writing a JPEG file via the dump image command. These can be left blank if you are not using the -DSPPARKS_JPEG switch discussed above in section (3.a).
A standard JPEG library usually goes by the name libjpeg.a and has an associated header file jpeglib.h. Whichever JPEG library you have on your platform, you'll need to set the appropriate JPG_INC, JPG_PATH, and JPG_LIB variables in Makefile.foo so that the compiler and linker can find it.
That's it. Once you have a correct Makefile.foo and you have pre-built any other libraries it will use (e.g. MPI, JPEG), all you need to do from the src directory is type one of these 2 commands:
That's it. Once you have a correct Makefile.foo and you have pre-built the MPI library it uses, all you need to do from the src directory is type one of these 2 commands:
make foo gmake foo
You should get the executable spk_foo when the build is complete.
(1) Building SPPARKS for multiple platforms.
You can make SPPARKS for multiple platforms from the same src directory. Each target creates its own object sub-directory called Obj_name where it stores the system-specific *.o files.
(2) Cleaning up.
Typing "make clean" will delete all *.o object files created when SPPARKS is built.
(3) Changing the SPPARKS size limits via -DSPPARKS_SMALLBIG or -DSPPARKS_BIGBIG or -DSPPARKS_SMALLSMALL
As explained above, any of these 3 settings can be specified on the SPK_INC line in your low-level src/MAKE/Makefile.foo.
The default is -DSPPARKS_SMALLBIG which allows for systems with up to 2^31 sites (about 2 billion). This is because the site IDs are stored in 32-bit integers.
To allow for larger systems, compile with -DSPPARKS_BIGBIG. This stores site IDs in 64-bit integers. This enables systems with up to 2^63 sites (about 9e18).
If your system does not support 8-byte integers, you will need to compile with the -DSPPARKS_SMALLSMALL setting. This will restrict the total number of sites to 2^31 (about 2 billion), as well as store some simulation statistics in 4-byte integers.
Note that in src/lmptype.h there are definitions of all these data types as well as the MPI data types associated with them. The MPI types need to be consistent with the associated C data types, or else SPPARKS will generate a run-time error. As far as we know, the settings defined in src/spktype.h are portable and work on every current system.
In all cases, the size of problem that can be run on a per-processor basis is limited by 4-byte integer storage to 2^31 sites per processor (about 2 billion). This should not normally be a limitation since such a problem would have a huge per-processor memory and would run very slowly in terms of CPU secs per Monte Carlo interation.
Building for a Mac:OS X is BSD Unix, so it already works. See the Makefile.mac file.
SPPARKS is just C++ with MPI calls, so it should be possible to build it for a Windows box, either using a Linux installation such as cygwin (see src/MAKE/Makefile.cygwin), or importing the source files into Visual Studio C++ and building it there. For the latter you are on your own. The SPPARKS developers do not use Windows. But if you figure out how to do it, or create a Visual Studio project that works, please let us know, and we can release the instructions/files for how to do this as part of SPPARKS.
The source code for SPPARKS is structured as a large set of core files which are always used, plus optional packages, which are groups of files that enable a specific set of features. You can see the list of both standard and user-contributed packages by typing "make package".
Currently there is only one optional package: STITCH. It is dicussed more below.
Any or all packages can be included or excluded when SPPARKS is built. You may wish to exclude certain packages if you will never run certain kinds of simulations.
By default, SPPARKS includes no packages.
Packages are included or excluded by typing "make yes-name" or "make no-name", where "name" is the name of the package. You can also type "make yes-all" or "make no-all" to include/exclude all packages. These commands work by simply moving files back and forth between the main src directory and sub-directories with the package name, so that the files are seen or not seen when SPPARKS is built. After you have included or excluded a package, you must re-build SPPARKS.
Additional make options exist to help manage SPPARKS files that exist in both the src directory and in package sub-directories. You do not normally need to use these commands unless you are editing SPPARKS files or have downloaded a patch from the SPPARKS WWW site. Typing "make package-update" will overwrite src files with files from the package directories if the package has been included. It should be used after a patch is installed, since patches only update the master package version of a file. Typing "make package-overwrite" will overwrite files in the package directories with src files. Typing "make package-check" will list differences between src and package versions of the same files.
2.3.1 STITCH package
The STITCH package allows SPPARKS to use the Stitch library for I/O, which is included in the SPPARKS distribution in lib/stitch. At some point the Stitch library will have its own website and will also be downloadable there.
Stitch is an efficient I/O API and database format with a native python interface. Stitch files can read in to start a simulation and/or output during a simulation. A novel aspect of stitch is that it enables out-of-core computations by building a simulation domain analogously to the way an additive manufactured (AM) part is built. It merges outputs written over time to efficiently construct a much larger simulation domain that would otherwise be impossible to model in one simulation. Stitching workflows can be created to perform multiple SPPARKS simulations representing an additive manufacturing process; such simulations can produce huge numbers of lattice sites representing an entire AM build that would otherwise be impossible to simulate due to length scale and computational resource limitations. Stitch is intended and primarily focused on microstructural evolution simulations such as welding and additive manufacturing but other applications may be possible.
Building SPPARKS with the STITCH package enables these commands to use stitch-related options:
See the am_path and stitch sub-directories in the examples directory for models and scripts which use the Stitch library.
You can build SPPARKS with stitch support in one of 3 ways.
(1) From the src directory using make
% cd spparks/src % make lib-stitch args="-b" # build the Stitch library and set links to it % make yes-stitch # install the STITCH package % make mpi # or whichever machine target you wish
(2) From the lib directory using Install.py
% cd spparks/lib % python Install.py -b # build the Stitch library and set links to it % cd spparks/src % make yes-stitch # install the STITCH package % make mpi # or whichever machine target you wish
(3) Manual build of the Stitch library you have downloaded to your system
% cd $STITCHDIR # STITCHDIR = the Stitch library directory % make # build Stitch with default Makefile % make -f Makefile.custom # build Stitch with custom Makefile % cd spparks/lib/stitch % ln -s $STITCHDIR liblink # set two links in SPPARKS lib/stitch % ln -s $STITCHDIR includelink % cd spparks/src % make yes-stitch # install the STITCH package % make mpi # or whichever machine target you wish
To un-install the STITCH package from SPPARKS, do the following:
% cd spparks/src % make no-stitch # un-install the STITCH package files % make mpi # re-build SPPARKS w/out the STITCH package
SPPARKS can be built as either a static or shared library, which can then be called from another application or a scripting language. See this section for more info on coupling SPPARKS to other codes. See this section for more info on wrapping and running SPPARKS from Python.
To build SPPARKS as a static library (*.a file on Linux), type
make makelib make -f Makefile.lib foo
where foo is the machine name. This kind of library is typically used to statically link a driver application to SPPARKS, so that you can insure all dependencies are satisfied at compile time. Note that inclusion or exclusion of any desired optional packages should be done before typing "make makelib". The first "make" command will create a current Makefile.lib with all the file names in your src dir. The second "make" command will use it to build SPPARKS as a static library, using the ARCHIVE and ARFLAGS settings in src/MAKE/Makefile.foo. The build will create the file libspparks_foo.a which another application can link to.
To build SPPARKS as a shared library (*.so file on Linux), which can be dynamically loaded, e.g. from Python, type
make makeshlib make -f Makefile.shlib foo
where foo is the machine name. This kind of library is required when wrapping SPPARKS with Python; see Section_python for details. Again, note that inclusion or exclusion of any desired optional packages should be done before typing "make makelib". The first "make" command will create a current Makefile.shlib with all the file names in your src dir. The second "make" command will use it to build SPPARKS as a shared library, using the SHFLAGS and SHLIBFLAGS settings in src/MAKE/Makefile.foo. The build will create the file libspparks_foo.so which another application can link to dyamically. It will also create a soft link libspparks.so, which the Python wrapper uses by default.
Note that for a shared library to be usable by a calling program, all the auxiliary libraries it depends on must also exist as shared libraries. This will be the case for libraries included with SPPARKS, such as the dummy MPI library in src/STUBS since they are always built as shared libraries with the -fPIC switch. However, if a library like MPI does not exist as a shared library, the second make command will generate an error. This means you will need to install a shared library version of the package. The build instructions for the library should tell you how to do this.
As an example, here is how to build and install the MPICH library, a popular open-source version of MPI, distributed by Argonne National Labs, as a shared library in the default /usr/local/lib location:
./configure --enable-shared make make install
You may need to use "sudo make install" in place of the last line if you do not have write privileges for /usr/local/lib. The end result should be the file /usr/local/lib/libmpich.so.
The operating system finds shared libraries to load at run-time using the environment variable LD_LIBRARY_PATH. So you may wish to copy the file src/libspparks.so or src/libspparks_g++.so (for example) to a place the system can find it by default, such as /usr/local/lib, or you may wish to add the SPPARKS src directory to LD_LIBRARY_PATH, so that the current version of the shared library is always available to programs that use it.
For the csh or tcsh shells, you would add something like this to your ~/.cshrc file:
setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/home/sjplimp/spparks/src
Either flavor of library (static or shared0 allows one or more SPPARKS objects to be instantiated from the calling program.
When used from a C++ program, all of SPPARKS is wrapped in a SPPARKS_NS namespace; you can safely use any of its classes and methods from within the calling code, as needed.
When used from a C or Fortran program or a scripting language like Python, the library has a simple function-style interface, provided in src/library.cpp and src/library.h.
See the sample codes in examples/COUPLE/simple for examples of C++ and C and Fortran codes that invoke SPPARKS thru its library interface. There are other examples as well in the COUPLE directory which are discussed in Section_howto 2 of the manual. See Section_python of the manual for a description of the Python wrapper provided with SPPARKS that operates through the SPPARKS library interface.
The files src/library.cpp and library.h define the C-style API for using SPPARKS as a library. See Section_howto 3 of the manual for a description of the interface and how to extend it for your needs.
By default, SPPARKS runs by reading commands from stdin; e.g. spk_linux < in.file. This means you first create an input script (e.g. in.file) containing the desired commands. This section describes how input scripts are structured and what commands they contain.
You can test SPPARKS on any of the sample inputs provided in the examples directory. Input scripts are named in.* and sample outputs are named log.*.name.P where name is a machine and P is the number of processors it was run on.
Here is how you might run the Potts model tests on a Linux box, using mpirun to launch a parallel job:
cd src make linux cp spk_linux ../examples/lj cd ../examples/potts mpirun -np 4 spk_linux < in.potts
The screen output from SPPARKS is described in a section below. As it runs, SPPARKS also writes a log.spparks file with the same information.
Note that this sequence of commands copies the SPPARKS executable (spk_linux) to the directory with the input files. This may not be necessary, but some versions of MPI reset the working directory to where the executable is, rather than leave it as the directory where you launch mpirun from (if you launch spk_linux on its own and not under mpirun). If that happens, SPPARKS will look for additional input files and write its output files to the executable directory, rather than your working directory, which is probably not what you want.
If SPPARKS encounters errors in the input script or while running a simulation it will print an ERROR message and stop or a WARNING message and continue. See this section for a discussion of the various kinds of errors SPPARKS can or can't detect, a list of all ERROR and WARNING messages, and what to do about them.
SPPARKS can run a problem on any number of processors, including a single processor. SPPARKS can run as large a problem as will fit in the physical memory of one or more processors. If you run out of memory, you must run on more processors or setup a smaller problem.
At run time, SPPARKS recognizes several optional command-line switches which may be used in any order. For example, spk_ibm might be launched as follows:
mpirun -np 16 spk_ibm -var f tmp.out -log my.log -screen none < in.alloy
These are the command-line options:
-echo style
Set the style of command echoing. The style can be none or screen or log or both. Depending on the style, each command read from the input script will be echoed to the screen and/or logfile. This can be useful to figure out which line of your script is causing an input error. The default value is log. The echo style can also be set by using the echo command in the input script itself.
-partition 8x2 4 5 ...
Invoke SPPARKS in multi-partition mode. When SPPARKS is run on P processors and this switch is not used, SPPARKS runs in one partition, i.e. all P processors run a single simulation. If this switch is used, the P processors are split into separate partitions and each partition runs its own simulation. The arguments to the switch specify the number of processors in each partition. Arguments of the form MxN mean M partitions, each with N processors. Arguments of the form N mean a single partition with N processors. The sum of processors in all partitions must equal P. Thus the command "-partition 8x2 4 5" has 10 partitions and runs on a total of 25 processors.
The input script specifies what simulation is run on which partition; see the variable and next commands. This howto section gives examples of how to use these commands in this way. Simulations running on different partitions can also communicate with each other; see the temper command.
-in file
Specify a file to use as an input script. This is an optional switch when running SPPARKS in one-partition mode. If it is not specified, SPPARKS reads its input script from stdin - e.g. spk_linux < in.run. This is a required switch when running SPPARKS in multi-partition mode, since multiple processors cannot all read from stdin.
-log file
Specify a log file for SPPARKS to write status information to. In one-partition mode, if the switch is not used, SPPARKS writes to the file log.spparks. If this switch is used, SPPARKS writes to the specified file. In multi-partition mode, if the switch is not used, a log.SPPARKS file is created with hi-level status information. Each partition also writes to a log.SPPARKS.N file where N is the partition ID. If the switch is specified in multi-partition mode, the hi-level logfile is named "file" and each partition also logs information to a file.N. For both one-partition and multi-partition mode, if the specified file is "none", then no log files are created. Using a log command in the input script will override this setting.
-screen file
Specify a file for SPPARKS to write its screen information to. In one-partition mode, if the switch is not used, SPPARKS writes to the screen. If this switch is used, SPPARKS writes to the specified file instead and you will see no screen output. In multi-partition mode, if the switch is not used, hi-level status information is written to the screen. Each partition also writes to a screen.N file where N is the partition ID. If the switch is specified in multi-partition mode, the hi-level screen dump is named "file" and each partition also writes screen information to a file.N. For both one-partition and multi-partition mode, if the specified file is "none", then no screen output is performed.
-var name value
Specify a variable that will be defined for substitution purposes when the input script is read. "Name" is the variable name which can be a single character (referenced as $x in the input script) or a full string (referenced as ${abc}). The value can be any string. Using this command-line option is equivalent to putting the line "variable name index value" at the beginning of the input script. Defining a variable as a command-line argument overrides any setting for the same variable in the input script, since variables cannot be re-defined. See the variable command for more info on defining variables and this section for more info on using variables in input scripts.
As SPPARKS reads an input script, it prints information to both the screen and a log file about significant actions it takes to setup a simulation. When the simulation is ready to begin, SPPARKS performs various initializations and prints the amount of memory (in MBytes per processor) that the simulation requires. An example output is shown here, for the examples/in.potts script run on 4 processors.
SPPARKS (11 Dec 2015) Created box = (0 0 0) to (20 20 20) 1 by 2 by 2 processor grid Creating sites ... 8000 sites 8000 sites have 26 neighbors Setting site values ... 8000 settings made for site Setting up run ... Memory usage per processor = 4.375 Mbytes
During the run itself, statistical information is printed periodically, for every delta of simulation time, as specified by the stats commmand. When the run concludes, SPPARKS prints final statistical info and a total run time for the simulation.
Time Naccept Nreject Nsweeps CPU Energy
0 0 0 0 0 205912
10.01 88437 7919563 1001 0.195 72506
20 94828 15905172 2000 0.379 57038
30 98345 23901655 3000 0.565 49948
40 101449 31898551 4000 0.749 44316
50.01 103978 39904022 5001 0.933 39334
60.01 105578 47902422 6001 1.12 36902
70.01 106938 55901062 7001 1.3 34428
80 108491 63891509 8000 1.49 31668
90 110211 71889789 9000 1.67 27994
100 112074 79887926 10000 1.86 21894
Loop time of 1.86084 on 4 procs
It then appends statistics about the breakdown of CPU time for the simulation.
Solve time (%) = 1.52001 (81.6842) Update time (%) = 0 (0) Comm time (%) = 0.245275 (13.1809) Outpt time (%) = 0.0892967 (4.79874) App time (%) = 0 (0) Other time (%) = 0.00625533 (0.336157)