Documentation: Code Management

In this file we list some of the techniques that may be used to increase one's efficiency when developing PETSc application codes. We have learned to use these techniques ourselves, and they have improved our efficiency tremendously.

Editing and Compiling

The biggest time sink in code development is generally the cycle of EDIT-COMPILE-LINK-RUN. We often see users working in a single window with a cycle such as:

• Edit a file with emacs.
• Exit emacs.
• Run make and see some error messages.
• Start emacs and try to fix the errors; often starting emacs hides the error messages so that users cannot remember all of them and thus do not fix all the compiler errors.
• Run make generating a bunch of object (.o) files.
• Link the executable (which also removes the .o files). Users may delete the .o files because they anticipate compiling the next time on a different machine that uses a different compiler.
• Run the executable.
• Detect some error condition and restart the cycle.

In addition, during this process the user often moves manually among different directories for editing, compiling, and running.

Several easy ways to improve the cycle:

• emacs has a feature to allow the user to compile using make and have the editor automatically jump to the line of source code where the compiler detects an error, even when not currently editing the erroneous file.
• The etags feature of emacs enables one to search quickly through a group of user-defined source files (and/or PETSc source files) regardless of the directory in which they are located. Etags usage tips (where M- denotes the "meta" key in emacs):
  • To access the predefined PETSc tags in emacs, type M-x visit-tags-table and specify the file ${PETSC_DIR}/TAGS
  • To move to where a PETSc function is defined, enter M-. This is useful for viewing manpages and locating source code.
  • To search for a string and move to the first occurrence, use M-x tags-search Then, to locate later occurrences, type M-, In particular, this is useful for quickly locating examples of usage of certain PETSc routines.
  • Another quite helpful command is M-x tags-query-replace which replaces occurrences of a string hroughout all tagged files (can retain any entries if desired); this enables quick and painless revisions of source code, particularly when routines are spread throughout many files and directories.
• Also, emacs easily enables:
  • editing files that reside in any directory and retaining one's place within each of them
  • searching for files in any directory as one attempts to load them into the editor

Admittedly, it takes a while to get the hang of using emacs. But we've found that the increase in our programming efficiency has been well worth the time spent learning to use this editor. We seriously believe that we could not have developed PETSc as it is today if we had not switched to emacs.

vi and Eclipse also provide tools for quickly finding things and searching in PETSc source code, see the Emacs, VI/Vim and Eclipse sections of the users manual. GNU Global is a richer alternative to etags, see the user's manual for more details.

Debugging

Most code development for PETSc codes should be done on one processor; hence, using a standard debugger such as dbx, gdb, xdbx, etc. is fine. For debugging parallel runs we suggest Totalview if it is available on your machine. Otherwise, you can run each process in a separate debugger; this is not the same as using a parallel debugger, but in most cases it is not so bad. The PETSc run-time options -start_in_debugger [-display xdisplay:0] will open separate windows and debuggers for each process. You should debug using the debugging versions of the libraries (run ./configure with the additional option --with-debugging (the default)).

It really pays to learn how to use a debugger; you will end up writing more interesting and far more ambitious codes once it is easy for you to track down problems in the codes.

Other suggestions

• Choose consistent and obvious names for variables and functions. (Short variable names may be faster to type, but by using longer names you don't have to remember what they represent since it is clear from the name.)
• Use informative comments.
• Leave space in the code to make it readable.
• Line things up in the code for readability. Remember that any code written for an application will be visited over and over again, so spending an extra 20 percent of effort on it the first time will make each of those visits faster and more efficient.
• Realize that you will have to debug your code. No one writes perfect code, so always write code that may be debugged and learn how to use a debugger. In most cases using the debugger to track down problems is much faster than using print statements.
• Never hardwire a large problem size into your code. Instead, allow a command line option to run a small problem. We've seen people developing codes who have to wait 15 minutes after starting a run to reach the crashing point; this is an inefficient way of developing code.
• Develop your code on the simplest machine to which you have access. We have accounts on a variety of large parallel machines, but we write and initially test all our code on laptops or workstations because the user interface is friendlier, and the turn-around time for compiling and running is much faster than for the parallel machines. We use the parallel machines only for large jobs. Since PETSc code is completely portable, switching to a parallel machine from our laptop/workstation development environment simply means logging into another machine -- there are no code or makefile changes.
• Never develop code directly on a multi-node computing system; your productivity will be much lower than if you developed on a well-organized workstation.
• Keep your machines' operating systems and compilers up-to-date (or force your systems people to do this :-). You should always work with whatever tools are currently the best available. It may seem that you are saving time by not spending the time upgrading your system, but, in fact, your loss in efficiency by sticking with an outdated system is probably larger than then the time required to keep it up-to-date.

Fortran notes

Since Fortran77 does not provide type checking of routine input/output parameters, we find that many errors encountered within PETSc Fortran77 programs result from accidentally using incorrect calling sequences. Thus, if your Fortran program is crashing, one of the first things to check is that all subroutines are being called with the correct arguments.

When passing floating point numbers into Fortran subroutines, always make sure you have them marked as double precision (e.g., pass in 10.d0 instead of 10.0). Otherwise, the compiler interprets the input as a single precision number, which can cause crashes or other mysterious problems. Make sure to declare all variables (do not use the implicit feature of Fortran). In fact, we highly recommend using the implicit none option at the begining of each Fortran subroutine you write.

Fortran 90 provides the ability to do some type checking of subroutine calls by using Fortran 90 interfaces or modules. PETSc now provides interfaces and modules for Fortran 90; see the manual page for how to use them.