Bereichsnavigation

Programming considerations

Timers

 Compute Node Linux supports the following timing functions:

  • CPU timers: The Fortran intrinsic subroutine cpu_time(time) returns the processor time, where time is real4 or real8 type. Call cpu_time() before and after a section of code: the difference between the two times is the amount of CPU time used by the program (in seconds).
  • Wall clock timers: The Fortran intrinsic function system_clock(), and the MPI timing functions MPI_Wtime() and MPI_Wtick() are supported. MPI_Wtime() returns the elapsed time, and MPI_Wtick() returns the resolution of MPI_Wtime() in seconds.

The etime() function is not supported. The dclock() function that was available under Catamount is not available under Compute Node Linux.

Glibc Functions

Compute Node Linux glibc does not support the getgrgid(), getgrnam(), gethostbyname(), getpwnam(), or getpwuid() functions. There is limited suport of the glibc process control functions, such as popen(), fork(), exec(), and system().

Performance considerations

Huge pages support

On the Cray XT3 system Palu under the Catamount operating system there was an option available to use small pages (4KB) rather than the default 2 MB pages. This was done by passing the -small_pages option to the job launcher command (yod -small_pages). On Rosa, under Compute Node Linux, however, the default page size is 4 KB. For applications that use a large amount of virtual memory, 4 KB pages can put a heavy load on the virtual memory subsystem. The use of huge pages could provide a significant performance increase for such applications. If you want to to use huge pages (2 MB) then you need perform the following three steps:

  1. Direct the loader to load the the hugetlbfs.a library by passing it the -lhugetlbfs option: 

    ftn -c example.f

    ftn -o example.x example.o -lhugetlbfs

  2. At run-time, in your PBS batch script, set the huge pages environment variable: 

    export HUGETLB_MORECORE=yes

  3. Finally, tell the job launcher (aprun) to use huge pages with the option -msizeh, where size is the size of huge pages to be allocated to each processing element. All nodes use as much huge page memory as they are able to allocate and afterwards use 4 KB pages.

For example, the following aprun line would request 1600 MB of huge pages per core, or 6400 MB per processor on quad-core nodes: 

 > aprun -m1600h -n 128 -N 4 ./example.x