Gnu compiling environment features

Tero Kivinen <kivinen@hut.fi>

Tik-76.184 Seminar in Programming Languages

1. Configure

1.1 Gnu-autoconfigure 2.1

1.1.1 Features

• Produces independent shell scripts.
• Requires no manual user intervention when ran.
• Tests each feature individually, no need to maintain lists of supported features of each os.
• Can support cross-compiling.
• Does not automatically create Makefile with all of the standard targets.
• Does not provide replacements of missing standard library functions.
• Autoconf requires at least gnu-m4 version 1.1, autoscan needs perl, configure-scripts needs only /bin/sh and standard unix tools.
• Caches test results, so rerunning configure is fast.
• Supports separate build directory (needs VPATH).
• Support for automatic binary name transformations.
• No restrictions on distributing configure scripts (in autoconf version 1 configure scripts ware under GPL).

1.1.2 Creating autoconf input files

• Autoconf takes configure.in and creates configure from it. Configure takes Makefile.in and config.h.in and creates Makefile and config.h from them.
• Additional feature tests can be added to aclocal.m4, and acsite.m4.
• Autoheader can be used to create config.h.in from acconfig.h.

  

  

1.1.2.1 configure.in

1.1.2.1.1 Automatic tools

• Autoscan checks source file in the directory and searches common portability problems and creates configure.scan which is a preliminary configure.in for that package.
• ifnames searches all preprocessor conditionals that are used in the files given to it. This can be used to fill in the gap between autoscan generated configure.scan and source that contain some portability ifdefs.

1.1.2.1.2 Using macros

• Autoconf macros already exist to check for many features.
• Configure.in needs AC_INIT(file) at the beginning and AC_OUTPUT(...) at the end. Otherwise the order of macros is not important.
• One macro call in each line.
• No blank space between the macro name and the opening parenthesis (if the macro takes arguments).
• Multiline arguments must be enclosed within the quote characters '[' and ']'.
• Usually a backslash at the end of line can be used to continue long lines.
• Use m4-builtin dnl to insert comments. It discards text up through the next newline.

1.1.2.1.3 Suggested order of calling autoconf macros

• AC_INIT(file)
• checks for programs
• checks for libraries
• checks for header files
• checks for typedefs
• checks for structures
• checks for compiler characteristics
• checks for library functions
• checks for system services
• AC_OUTPUT([FILE...])

1.1.2.1.4 Macro overwiew

• AC_INIT(unique-file-in-source-dir)
• AC_OUTPUT([file ...] [, extra-cmds] [, init-cmds])
• AC_CONFIG_HEADER(header-to-create ...)
• AC_CONFIG_SUBDIRS(dir ...)
• AC_PREFIX_DEFAULT(prefix)
• AC_PREFIX_PROGRAM(program)
• AC_REVISION($Revision$)
• Program macros
  • AC_PROG_AWK
  • AC_PROG_CC
  • AC_PROG_CPP
  • AC_PROG_CXX,
  • AC_PROG_CXXCPP
  • AC_PROG_INSTALL
  • AC_PROG_LEX
  • AC_PROG_LN_S
  • AC_PROG_RANLIB
  • AC_PROG_YACC
• AC_CHECK_PROG(variable, prog-to-check-for, value-if-found [, value-if-not-found)
• AC_CHECK_PROGS(variable, progs-to-check-for [, value-if-not-found])
• AC_PATH_PROG{,S}(...)
• AC_CHECK_LIB(library, function [, action-if-found [, action-if-not-found [, other-libraries]]]): adds HAVE_LIBXXX to DEFS
• Function macros
  • AC_FUNC_ALLOCA
  • AC_FUNC_CLOSEDIR_VOID
  • AC_FUNC_GETLOADAVG
  • AC_FUNC_GETMNTENT
  • AC_FUNC_MEMCMP
  • AC_FUNC_MMAP
  • AC_FUNC_SETVBUF_REVERSED
  • AC_FUNC_STRCOLL
  • AC_FUNC_STRFTIME
  • AC_FUNC_UTIME_NULL
  • AC_FUNC_VFORK
  • AC_FUNC_VPRINTF
  • AC_FUNC_WAIT3
• AC_CHECK_FUNC(functions... [, action-if-found [, action-if-not-found]]): adds HAVE_XXX to DEFS.
• AC_CHECK_FUNC(function, action-if-found [, action-if-not-found])
• Header file macros
  • AC_DECL_SYS_SIGLIST
  • AC_HEADER_DIRENT
  • AC_HEADER_MAJOR
  • AC_HEADER_STDC
  • AC_HEADER_SYS_WAIT
  • AC_MEMORY_H
  • AC_UNISTD_H
• AC_CHECK_HEADERS(header-file... [, action-if-found [, action-if-not-found]]) : adds HAVE_XXX_H ot DEFS.
• AC_CHECK_HEADER(header-file, action-if-found [, action-if-not-found])
• Structure macros
  • AC_HEADER_STAT
  • AC_HEADER_TIME
  • AC_STRUCT_ST_BLKSIZE
  • AC_STRUCT_ST_BLOCKS
  • AC_STRUCT_ST_RDEV
  • AC_STRUCT_TM
  • AC_STRUCT_TIMEZONE
• Type macros. If type is missing define it to be its default value (off_t=long, pid_t=int etc).
  • AC_TYPE_GETGROUPS
  • AC_TYPE_MODE_T
  • AC_TYPE_OFF_T
  • AC_TYPE_PID_T
  • AC_TYPE_SIGNAL.SIZE_T
  • AC_TYPE_UID_T
• AC_CHECK_TYPE(type, default): define type to be default if not found.
• Compiler characteristics
  • AC_C_BIGENDIAN
  • AC_C_CONST
  • AC_C_INLINE
  • AC_C_CHAR_UNSIGNED
  • AC_C_LONG_DOUBLE
  • AC_C_INT_16_BITS
  • AC_C_LONG_64_BITS
• AC_CHECK_SIZEOF(type): AC_CHECK_SIZEOF(int *) will define SIZEOF_INT_P to be sizeof(int *).
• System feature macros
  • AC_SYS_INTERPRETER
  • AC_PATH_X
  • AC_PATH_XTRA
  • AC_SYS_LONG_FILE_NAMES
  • AC_SYS_RESTARTABLE_SYSCALLS
• Special system type macros
  • AC_AIX
  • AC_ISC_POSIX
  • AC_MINIX
• AC_TRY_COMPILE(includes, function-body, action-if-found [,action-if-not-found])
• AC_TRY_LINK(includes, function-body, action-if-found [, action-if-not-found])
• AC_TRY_RUN(program, action-if-true [, action-if-false [, action-if-cross-compiling]])
• AC_C_CROSS
• AC_LANG_C, AC_LANG_CPLUSPLUS, AC_LANG_SAVE, AC_LANG_RESTORE
• AC_DEFINE(variable [, value]), AC_DEFINED_UNQUOTED(variable [, value])
• AC_SUBST(variable), AC_SUBST_FILE(variable)
• AC_MSG_CHECKING(feature-description)
• AC_MSG_RESULT(result-description)
• AC_MSG_ERROR(error-description)
• AC_MSG_WARN(problem-description)
• AC_ARG_WITH(package, help-string, action-if-true [, action-if-false])
• AC_ARG_ENABLE(feature, help-string, action-if-true [, action-if-false])
• AC_ARG_PROGRAM

1.1.2.2 Makefile.in output variables

• exec_prefix: installation prefix for architecture-dependent files.
• prefix: installation prefix for architecture-independent files.
• srcdir: directory containing the source code.
• CFLAGS: debugging and optimization options for the C compiler.
• CPPFLAGS: header file search directory (-IDIR) options.
• CXXFLAGS: debugging and optimization optios for the C++ compiler.
• DEFS: -D options ot pass the C compiler. If AC_CONFIG_HEADER is called then it is replaced with -DHAVE_CONFIG_H.
• LDFLAGS: linker flags.
• LIBS: -l and -L options to the linker.
• program variables: AWK, CC, GCC, CPP, CXX, GXX, CXXCPP, INSTALL, LEX, LEXLIB, LN_S, RANLIB, YACC.

1.1.2.3 config.h.in

• Create (or use autoheader) config.h.in that contains the defines you want.
• for #ifs use "#define HAVE_UNISTD_H 0".
• for #ifdefs use "#undef HAVE_UNISTD_H".

1.1.3 Running configure

• Usually ./configure (or ../src/configure if using object directory) is enough.
• --prefix=/usr/local/gnu: set the installation prefix.
• --host=mips-dec-ultrix: set the host type (usually not needed).
• --target=i386-mach: set the target system type (for cross-compiling).
• --with-PACKAGE=arg, --without-PACKAGE: compile with/without PACKAGE.
• --enable-FEATURE=arg, --disable-FEATURE: enable/disable features.
• --help: show list of options (including valid with-* and enable-* options).
• --program-prefix=PREFIX, --program-suffix=SUFFIX.
• --srcdir=DIR: usually not needed, as configure can determine that automatically.

1.1.4 Sample files

1.1.4.1 configure.in

1.1.4.2 Makefile.in

1.2 Metaconfig 3.0 (originally by Larry Wall)

1.2.1 Features

• Configure-scripts are not copyrighted and belongs to the public domain.
• Four parts: Configure-script generator (metaconfig), Makefile.SH generator (Jmakefile + jmake), package generator (makedist, builds shell archives etc), patch generator (pat, generates patches).
• Metaconfig needs at least perl-4.0, Configure-scripts needs only /bin/sh and standard unix tools.
• Configure-scripts are interactive (ask users all kind of things, with motto "When in doubt, set a default and ask, Don't assume anything").
• jmake is like imake but generates Makefile.SH instead of Makefile. jmkmf is a similar wrapper to jmake as xmkmf is to imake.
• Operates from set of units which define everything that metaconfig knows about portability.
• Units are self-contained with cryptic syntax.
• Metaconfig figures needed units automatically and concatenates them to Configure-script.

1.3 Imake

1.3.1 Features

• Common database of host specific data.
• Special purpose (mainly for X11).
• Uses cpp, which is not fully programmable and that causes limitations.
• Imakefile variables are in mixed case and are expanded when imake is run. Make variables are written to makefile for later interpretation by make and are all upper case.
• Site specific stuff in site.def, vendor specific in *.cf file, project specific stuff in Project.tmpl.
• xmkmf -a; make will usually work (runs imake; make Makefiles; make includes; make depend).

2. Make

2.1 Special make features

• -include filenames

  Include filenames to makefile. Dash at the begining prevents error messages incase the file does not exist (gnu,pmake,cmumake).

• %:

  (gnu)

  all: Makeile: ; DIRS=lib utils dist.tar: clean tar cf dist.tar . %: force for i in $(DIRS); do; cd $$i; @$(MAKE) $@; cd ..; done force: ;

• builtin functions (widcard, patsubst etc)

  (gnu)

  objects := $(patsubst %.c,%.o,$(wildcard *.c))

• VPATH

  Specify the path where source files are search from (gnu, SysV, 4.3BSD, cmumake).

• .PHONY : clean

  Tell make to run commands regardless of wheatever the target is up to date (gnu). Another way to do this is:

  clean: FORCE -$(RM) *.o FORCE: which works as long as there is no file named FORCE (all).

• foo:=$(foo) more

  Simply expanded variables (gnumake, pmake)

2.2 Standard targets (gnu programs)

all

Compile the entire program (usually default target).

install

Compile the program and copy the executables, libraries, and so on to the file names where they should reside for actual use.

uninstall

Delete all the installed files that the `install' target would create.

clean

Delete all files from the current directory that are normally created by building the program. Don't delete the files that record the configuration.

distclean

Delete all files from the current directory that are created by configuring or building the program.

mostlyclean

Like `clean', but may refrain from deleting a few files that people normally don't want to recompile.

realclean

Delete everything from the current directory that can be reconstructed with this Makefile.

TAGS

Update a tags table for this program.

info

Generate any Info files needed.

dvi

Generate DVI files for all TeXinfo documentation.

dist

Create a distribution tar file for this program.

check

Perform self-tests (if any).

2.3 Additional commonly used targets

clobber

Usually same as distclean or realclean.

print

Print listings of the source files that have changed.

tar

Create a tar file of the source files.

shar

Create a shell archive (shar file) of the source files.

test

Same as check.

3. Version control systems

3.1 RCS

3.1.1 Features

• Store and retrieve multiple revisions of text. RCS saves all old revisions in a space efficient way.
• Maintain a complete history of changes.
• Resolve access conflicts.
• Maintain a tree of revisions (branches).
• Merge revisions and resolve conflicts.
• Control releases and configurations.
• Minimize secondary storage.
• Keywords (Author, Date, Header, Id, Locker, Log, RCSfile, Revision, Source, State).
• State attribute (Exp, Stab, Rel).

3.1.2 Usinging rcs

• Creating RCS directory : mkdir RCS
• Check-in file: ci file.c
• Check-out file: co file.c
• Check-in file without destroying it: ci -u file.c
• Check-out file for editing (take lock): co -l file.c
• Revisions can be given in -r[rev] format: co -r1.1 file.c
• Dates can be given in free format (the last example is date(1) format): co -d'8:00 pm' f.c; co -d'1995/01/01 22:40:11' f.c; co -d'Thu Jan 11 20:00:00 GMT 1990' f.c
• Comparing two revisions: rcsdiff f.c (current file against last checked-in), rcsdiff -r1.1 ...
• Using state attribute (default is Exp (experimental)):
  • rcs -sStab *.[ch]
    mark current version as stable
  • co -u -sStab *.[ch]; rcs -sRel *.[ch]
    fetch last stable version and mark it as released

3.2 cvs

3.2.1 Features

• Front end to the rcs.
• Extends the notion of revision control from a collection of files in a single directory to a hierarchical collection of directories.
• Keeps a single copy of the master sources (repository).
• Takes lots of disk space because every programmer have private copy of sources (often multiple copies :-).
• Organizational changes in directory structure (moving files etc) is quite difficult.
• Problems in branch support.
• Can check-out/check-in complete trees in one command.

3.2.2 Usinging cvs

• Setting repository: CVSROOT=/src/CVS; export CVSROOT
• Checking out module: cvs checkout modules...
• Updating your private source directory from changes that others have done to the repository: cvs update
• Committing your changes to repository and make them public: cvs commit file...
• Adding new files: cvs add file.c
• Removing files: cvs remove file.c
• Comparing modifications: cvs diff.
• Exporting source files for shipment off site: cvs export modules...
• Import source files from off-site into the source repository: cvs import.
• Accept same kind of dates and revisions as rcs.

4. CTAGS, ETAGS

• Scans source files and finds functions and typedefs (-t) and creates file that can be used in editor to jump to specific function or typedef.
• Ctags (tags) is for ex and vi users: ctags -t *.[chylfp].
• Etags (TAGS) is for emacs users: etags -t *.[chylfp].
• Newer versions (tags++) can handle more languages (C, Yacc, Lex, C++, (La)TeX, assembler, elisp/common lisp, scheme etc).
• In emacs you can use M-. to find tag. M-x tags-search will search string from all files mentioned in TAGS file (M-, will continue search after match). M-x tags-query-replace will query-replace string in all files in TAGS file.

5. Yacc/bison/byacc

5.1. Special yacc/bison/byacc features

• Bison parsers can only be used in GPL-software. Byacc parsers are freely distributable.
• Bison: %pure_parser (reentrant).
• Bison: multiple parsers in the same program: -p option.
• Bison: debugging #define YYDEBUG 1, add yydebug=1 in main or set it in debugger. Yacc: no need to define YYDEBUG. Byacc: if compiled with debugging code setting environment variable YYDEBUG to 1 will enable it.

6. lex/flex

6.1 Special lex/flex features

• flex: Interactive scanners (flex -I).
• flex: 8-bit scanner (flex -8).
• flex: EOF-handling.
• flex: multiple lexers (flex -Pprefix).
• flex: C++-scanners (flex++ or flex -+).
• flex: freely distributable (not under GPL).

7. Profiling tools

7.1. prof

• Compile with: cc -p.
• Run program (generates mon.out that containing pc-sampling data).
• Generate profiling raport: prof myprog mon.out.
• Sample output (Alpha OSF/1 V3.0): Profile listing generated Mon Feb 6 06:28:23 1995 with: prof a.out mon.out ---------------------------------------------------------------------------- * -p[rocedures] using pc-sampling; * * sorted in descending order by total time spent in each procedure; * * unexecuted procedures excluded * ---------------------------------------------------------------------------- Each sample covers 4.00 byte(s) for 0.0042% of 23.0068 seconds %time seconds cum % cum sec procedure (file) 54.5 12.5439 54.5 12.54 reply_4 (teaser.c) 44.0 10.1299 98.6 22.67 reply_3 (teaser.c) 1.3 0.3086 99.9 22.98 reply_5 (teaser.c) 0.1 0.0176 100.0 23.00 show_pairs (teaser.c) 0.0 0.0029 100.0 23.00 init_pair (teaser.c) 0.0 0.0020 100.0 23.00 reply_2 (teaser.c) 0.0 0.0020 100.0 23.01 get_result (teaser.c)
• On some systems (at least OSF/1 V3.0) can do basic-block counting (pixie) with: cc -o myprog -p ...; pixie myprog; myprog.pixie; prof -pixie myprog myprog.Addrs myprog.Counts.
• Sample output file for prof -pixie (cleaned some paths, and truncated basic block listing ): Profile listing generated Mon Feb 6 06:32:35 1995 with: prof -pixie a.out a.out.Addrs a.out.Counts ---------------------------------------------------------------------------- * -p[rocedures] using basic-block counts; * * sorted in descending order by the number of cycles executed in each * * procedure; unexecuted procedures are excluded * ---------------------------------------------------------------------------- 1369174353 cycles cycles %cycles cum % cycles bytes procedure (file) /call /line 697393829 50.94 50.94 697393829 32 reply_3 (teaser.c) 650050967 47.48 98.41 650050967 47 reply_4 (teaser.c) 20120041 1.47 99.88 20120041 47 reply_5 (teaser.c) 1169890 0.09 99.97 233978 38 show_pairs (teaser.c) 157546 0.01 99.98 157546 35 get_result (teaser.c) 157517 0.01 99.99 157517 25 init_pair (teaser.c) 121711 0.01 100.00 121711 32 reply_2 (teaser.c) 2234 0.00 100.00 2234 23 init_primes (teaser.c) 169 0.00 100.00 169 6 __start (/sucl/crt/alpha/crt0.s) 77 0.00 100.00 77 31 __prof_error (/sucl/libprof/prof_common.c) 63 0.00 100.00 63 30 _call_remove_gp_range (/sucl/libexc/call.c) 60 0.00 100.00 60 16 main (teaser.c) 44 0.00 100.00 44 19 _prof_monstartup (/sucl/libprof/prof.c) 41 0.00 100.00 41 21 _call_add_gp_range (/sucl/libexc/call.c) 29 0.00 100.00 29 30 _prof_init (/sucl/libprof/prof_common.c) 28 0.00 100.00 28 10 _prof_moncontrol (/sucl/libprof/prof.c) 24 0.00 100.00 24 20 __prof_calloc (/sucl/libprof/prof_common.c) 23 0.00 100.00 23 16 reply_1 (teaser.c) 18 0.00 100.00 18 16 allocate_buffers (/sucl/libprof/prof.c) 16 0.00 100.00 16 15 __check_profdir (/sucl/libprof/profdir.c) 14 0.00 100.00 14 19 _call_add_pc_range_table (/sucl/libexc/call.c) 12 0.00 100.00 12 16 _call_remove_pc_range_table (/sucl/libexc/call.c) ---------------------------------------------------------------------------- * -p[rocedures] using invocation counts; * * sorted in descending order by number of calls per procedure; * * unexecuted procedures are excluded * ---------------------------------------------------------------------------- 26 invocations total calls %calls cum% bytes procedure (file) 5 19.23 19.23 336 show_pairs (teaser.c) 1 3.85 23.08 108 allocate_buffers (/sucl/libprof/prof.c) 1 3.85 26.92 204 _call_add_gp_range (/sucl/libexc/call.c) 1 3.85 30.77 124 init_pair (teaser.c) 1 3.85 34.62 180 init_primes (teaser.c) 1 3.85 38.46 92 reply_1 (teaser.c) 1 3.85 42.31 188 reply_2 (teaser.c) 1 3.85 46.15 288 reply_3 (teaser.c) 1 3.85 50.00 464 reply_4 (teaser.c) 1 3.85 53.85 464 reply_5 (teaser.c) 1 3.85 57.69 48 _call_remove_pc_range_table (/sucl/libexc/call.c) 1 3.85 61.54 240 get_result (teaser.c) 1 3.85 65.38 240 main (teaser.c) 1 3.85 69.23 56 _call_add_pc_range_table (/sucl/libexc/call.c) 1 3.85 73.08 88 __check_profdir (/sucl/libprof/profdir.c) 1 3.85 76.92 412 __start (/sucl/crt/alpha/crt0.s) 1 3.85 80.77 96 __prof_calloc (/sucl/libprof/prof_common.c) 1 3.85 84.62 296 _call_remove_gp_range (/sucl/libexc/call.c) 1 3.85 88.46 396 __prof_error (/sucl/libprof/prof_common.c) 1 3.85 92.31 296 _prof_init (/sucl/libprof/prof_common.c) 1 3.85 96.15 132 _prof_moncontrol (/sucl/libprof/prof.c) 1 3.85 100.00 240 _prof_monstartup (/sucl/libprof/prof.c) ---------------------------------------------------------------------------- * -h[eavy] using basic-block counts; * * sorted in descending order by the number of cycles executed in each * * line; unexecuted lines are excluded * ---------------------------------------------------------------------------- procedure (file) line bytes cycles % cum % reply_4 (teaser.c) 178 132 521596521 38.10 38.10 reply_3 (teaser.c) 138 32 422775936 30.88 68.97 reply_3 (teaser.c) 139 152 273426084 19.97 88.94 reply_4 (teaser.c) 183 160 127819080 9.34 98.28 reply_5 (teaser.c) 204 132 15849154 1.16 99.44 reply_5 (teaser.c) 209 160 4128980 0.30 99.74 show_pairs (teaser.c) 229 104 1150980 0.08 99.82 reply_3 (teaser.c) 137 8 1067616 0.08 99.90 reply_4 (teaser.c) 177 8 511038 0.04 99.94 init_pair (teaser.c) 60 84 157311 0.01 99.95 reply_5 (teaser.c) 201 44 107811 0.01 99.96 reply_3 (teaser.c) 135 44 107811 0.01 99.97 get_result (teaser.c) 240 44 107811 0.01 99.97 reply_4 (teaser.c) 175 44 107811 0.01 99.98 reply_2 (teaser.c) 117 40 61380 0.00 99.99 reply_2 (teaser.c) 118 108 60125 0.00 99.99 get_result (teaser.c) 242 120 49520 0.00 99.99 reply_5 (teaser.c) 208 68 17408 0.00 99.99 reply_5 (teaser.c) 203 8 16236 0.00 100.00 reply_3 (teaser.c) 136 12 16176 0.00 100.00 show_pairs (teaser.c) 224 96 11880 0.00 100.00 reply_4 (teaser.c) 182 68 8568 0.00 100.00 reply_4 (teaser.c) 176 12 7743 0.00 100.00 show_pairs (teaser.c) 230 48 5940 0.00 100.00 init_primes (teaser.c) 85 60 1170 0.00 100.00 init_primes (teaser.c) 80 40 990 0.00 100.00 show_pairs (teaser.c) 228 8 990 0.00 100.00 reply_5 (teaser.c) 202 12 246 0.00 100.00 get_result (teaser.c) 239 8 198 0.00 100.00 init_pair (teaser.c) 59 8 198 0.00 100.00 reply_2 (teaser.c) 116 8 198 0.00 100.00 reply_3 (teaser.c) 134 8 198 0.00 100.00 reply_4 (teaser.c) 174 8 198 0.00 100.00 reply_5 (teaser.c) 200 8 198 0.00 100.00 init_primes (teaser.c) 83 24 48 0.00 100.00 show_pairs (teaser.c) 225 24 30 0.00 100.00 ....

7.2. gprof

• Shows profiling data for each routines including the time of their call graph descendants.
• Also displays similar display for functions direct call-graph parents.
• Sample output (SunOS 4.1.3C): @(#)callg.blurb 1.5 88/02/08 SMI call graph profile: The sum of self and descendents is the major sort for this listing. function entries: index the index of the function in the call graph listing, as an aid to locating it (see below). %time the percentage of the total time of the program accounted for by this function and its descendents. self the number of seconds spent in this function itself. descendents the number of seconds spent in the descendents of this function on behalf of this function. called the number of times this function is called (other than recursive calls). self the number of times this function calls itself recursively. name the name of the function, with an indication of its membership in a cycle, if any. index the index of the function in the call graph listing, as an aid to locating it. parent listings: self* the number of seconds of this function's self time which is due to calls from this parent. descendents* the number of seconds of this function's descendent time which is due to calls from this parent. called** the number of times this function is called by this parent. This is the numerator of the fraction which divides up the function's time to its parents. total* the number of times this function was called by all of its parents. This is the denominator of the propagation fraction. parents the name of this parent, with an indication of the parent's membership in a cycle, if any. index the index of this parent in the call graph listing, as an aid in locating it. children listings: self* the number of seconds of this child's self time which is due to being called by this function. descendent* the number of seconds of this child's descendent's time which is due to being called by this function. called** the number of times this child is called by this function. This is the numerator of the propagation fraction for this child. total* the number of times this child is called by all functions. This is the denominator of the propagation fraction. children the name of this child, and an indication of its membership in a cycle, if any. index the index of this child in the call graph listing, as an aid to locating it. * these fields are omitted for parents (or children) in the same cycle as the function. If the function (or child) is a member of a cycle, the propagated times and propagation denominator represent the self time and descendent time of the cycle as a whole. ** static-only parents and children are indicated by a call count of 0. cycle listings: the cycle as a whole is listed with the same fields as a function entry. Below it are listed the members of the cycle, and their contributions to the time and call counts of the cycle. granularity: each sample hit covers 2 byte(s) for 0.01% of 76.69 seconds called/total parents index %time self descendents called+self name index called/total children <spontaneous> [1] 84.1 0.00 64.47 start [1] 0.01 64.46 1/1 _main [2] 0.00 0.00 1/1 _on_exit [29] 0.00 0.00 1/1 _exit [142] ----------------------------------------------- 0.01 64.46 1/1 start [1] [2] 84.1 0.01 64.46 1 _main [2] 32.08 4.57 1/1 _reply_4 [3] 25.50 0.00 1/1 _reply_3 [4] 0.04 1.42 5/5 _show_pairs [8] 0.82 0.00 1/1 _reply_5 [11] 0.01 0.00 1/1 _get_result [19] 0.01 0.00 1/1 _init_pair [20] 0.01 0.00 1/1 _reply_2 [21] 0.00 0.00 1/1 _init_primes [145] 0.00 0.00 1/1 _reply_1 [150] .... @(#)flat.blurb 1.6 88/02/08 SMI flat profile: % the percentage of the total running time of the time program used by this function. cumulative a running sum of the number of seconds accounted seconds for by this function and those listed above it. self the number of seconds accounted for by this seconds function alone. This is the major sort for this listing. calls the number of times this function was invoked, if this function is profiled, else blank. self the average number of milliseconds spent in this ms/call function per call, if this function is profiled, else blank. total the average number of milliseconds spent in this ms/call function and its descendents per call, if this function is profiled, else blank. name the name of the function. This is the minor sort for this listing. The index shows the location of the function in the gprof listing. If the index is in parenthesis it shows where it would appear in the gprof listing if it were to be printed. granularity: each sample hit covers 2 byte(s) for 0.01% of 88.63 seconds % cumulative self self total time seconds seconds calls ms/call ms/call name 36.2 32.08 32.08 1 32082.49 36652.84 _reply_4 [3] 28.8 57.58 25.50 1 25501.98 25501.98 _reply_3 [4] 13.5 69.54 11.95 mcount (66) 9.7 78.13 8.59 mul_8bit [5] 5.2 82.70 4.57 12996060 0.00 0.00 .mul [6] 3.2 85.53 2.84 mul_4bit [7] 0.9 86.35 0.82 1 820.06 820.06 _reply_5 [11] 0.7 86.96 0.61 zero_divide [12] ... Index by function name [6] .mul [20] _init_pair [149] _putfreehdr [31] .rem [145] _init_primes [150] _reply_1 [24] .udiv [146] _insert [21] _reply_2 [32] .umul [147] _ioctl [4] _reply_3 [30] .urem [26] _isatty [3] _reply_4 [10] __doprnt [17] _localeconv [11] _reply_5 [22] __findbuf [2] _main [141] _sbrk [23] __wrtchk [27] _malloc [8] _show_pairs [14] __xflsbuf [13] _memchr [15] _write [139] _demote [16] _moncontrol (66) mcount [142] _exit [28] _morecore [7] mul_4bit [143] _free [29] _on_exit [5] mul_8bit [19] _get_result [18] _openlocale [25] noerr [140] _getfreehdr [9] _printf [12] zero_divide [144] _getpagesize [148] _profil

7.5. Atom (Alpha OSF/1 V3.0)

7.5.1. Features

• General programing analysis tool generator.
• Easy to use, add -Wl,-r to link flags to generate unrelocated output file that can be feeded to atom.
• ATOM provides long and growing list of standard analysis and performance tools:

  iprof

  instruction profiling tool

  liprof

  instruction profiling tool at basic block level

  pipe

  pipeline stall tool

  lpipe

  pipeline stall at basic block level

  syscall

  system call summary tool

  memsys

  memory system bandwidth tool

  lmemsys

  low level memory system bandwidth tool

  io

  input/output summary tool

  io2

  new input/output summary tool

  unalign

  byte and short data profile

  gprof

  gprof profiling tool

  3rd

  memory checker and leak finder (a-la-Purify)

  pixie

  superset of pixie

  heapcheck

  leak detection tool

• Applying tool: atom foo.rr -tool syscall -o foo.syscall.
• Not limited to standard tools, its easy to write new tools.

7.5.2. Writing own tools

• Write two files: foo.inst.c and foo.anal.c. The first one is instrumentation file that adds calls to analysis functions in the program and the another is the analysis file that is called when the program is running.
• Sample ptrace.inst.c file: #include <stdio.h> #include <instrument.h> Instrument() { Proc *proc; AddCallProto("ProcTrace(char *)"); for (proc = GetFirstProc(); proc != NULL; proc = GetNextProc(proc)) { AddCallProc(proc,ProcBefore, "ProcTrace",ProcName(proc)); } }
• Sample ptrace.anal.c file: #include <stdio.h> void ProcTrace(char *name) { fprintf(stderr,"%s\n",name); }
• Compiling with own tool: cc -Wl,-r -non_shared foo.o -o foo.rr
• Applying ptrace tool: atom foo.rr ptrace.inst.c ptrace.anal.c -o foo.ptrace.
• Sample output: __start main init_pair init_primes reply_1 reply_2 reply_3 reply_4 reply_5 get_result printf _doprnt __getmbcurmax strchr memcpy memcpy strchr memcpy memcpy strchr strlen memcpy fwrite _wrtchk _findbuf __geterrno __isatty __ioctl __seterrno memcpy memchr _xflsbuf __write exit __ldr_atexit __ldr_context_atexit _cleanup fclose fflush __close fclose fflush __close fclose __close

7.5.3. Standard tools

• Sample output of iprof tool: Procedure Calls Instructions Percentage __start 1 116 0.000 init_pair 1 157517 0.012 init_primes 1 2234 0.000 reply_1 1 23 0.000 reply_2 1 121711 0.009 reply_3 1 697393829 50.979 reply_4 1 650050967 47.518 reply_5 1 20120041 1.471 get_result 1 157546 0.012 main 1 40 0.000 exit 1 25 0.000 printf 1 38 0.000 .... Total 1368006045
• Sample output of syscall tool: Using model DEC3000_300 platform parameters System Call Calls Time(ms) context write 501 437.2 27 close 3 0.1 0 ioctl 1 0.5 0 Total 437.8 97
• Sample output of gprof tool (truncated): gprof: V0.6, 14 June 1994 1 time unit = 1,000 cycles Total time = 1,977,637 time units callTime called/total parents %time self descendents called+self *name callTime called/total children 100 1,977,637 1/1 top level procedure 100.0 0 1,977,637 1+0 *__start 100 1,977,620 1/1 main 0 17 1/1 exit ----------------------------------------------------------------------------- 100 1,977,620 1/1 __start 100.0 0 1,977,620 1+0 *main 55 1,093,889 1/1 reply_4 43 856,328 1/1 reply_3 1 26,710 1/1 reply_5 0 328 1/1 get_result 0 193 1/1 reply_2 0 167 1/1 init_pair 0 3 1/1 init_primes 0 0 1/1 reply_1 ----------------------------------------------------------------------------- 55 1,093,889 1/1 main 55.3 1,093,889 0 1+0 *reply_4 ----------------------------------------------------------------------------- 43 856,328 1/1 main 43.3 856,328 0 1+0 *reply_3 ----------------------------------------------------------------------------- 1 26,710 1/1 main 1.4 26,710 0 1+0 *reply_5 ----------------------------------------------------------------------------- 0 328 1/1 main 0.0 196 131 1+0 *get_result 0 131 1/1 printf ----------------------------------------------------------------------------- 0 193 1/1 main 0.0 193 0 1+0 *reply_2 ----------------------------------------------------------------------------- 0 167 1/1 main 0.0 167 0 1+0 *init_pair ----------------------------------------------------------------------------- ....
• Sample output of io2: Using model DEC3000_300 platform parameters reads writes lseeks fcntls read(bytes) write(bytes) read(bps) write(bps) File 0 0 0 0 0 0 0.00 0.00 stdin 0 1 0 0 0 19 0.00 41167.90 stdout 0 0 0 0 0 0 0.00 0.00 stderr
• Sample output of pipe: Procedure Instructions Dual Packed Cycles CPIPipe Stall CPct SPct __start 116 91 166 1.43 75 0.00 0.00 init_pair 157517 127816 167317 1.06 39501 0.01 0.00 init_primes 2234 1457 3025 1.35 1568 0.00 0.00 reply_1 23 14 23 1.00 9 0.00 0.00 reply_2 121711 88285 193516 1.59 105231 0.01 0.01 reply_3 697393829 536127770 856328203 1.23 320200433 43.30 16.19 reply_4 650050967 490465853 1093889198 1.68 603423345 55.32 30.51 reply_5 20120041 15128024 26710593 1.33 11582569 1.35 0.59 get_result 157546 118038 196750 1.25 78712 0.01 0.00 main 40 37 61 1.52 24 0.00 0.00 exit 25 20 38 1.52 18 0.00 0.00 ... --------------------------------------------------------------------------- totals 1368006045 1042058974 1977491194 1.45 935432220 47.30
• Sample output of memsys: Using model DEC3000_300 platform parameters imisses(ms) drmisses(ms) dwrites(ms) bmisses(ms) bvictims(ms) total(ms) init_pair 0.0 0.0 0.2 0.6 0.0 0.8 reply_2 0.0 0.0 0.2 0.0 0.0 0.2 reply_3 0.0 0.1 116.0 0.0 0.0 116.2 reply_4 0.0 385.4 0.3 0.0 0.0 385.7 reply_5 0.0 12.3 0.7 0.0 0.0 13.0 get_result 0.0 0.1 0.0 0.1 0.0 0.2 Other 0.0 0.0 0.0 0.2 0.1 0.3 Summary 0.0 398.0 117.4 0.8 0.1 516.4

8. Debuggers (GDB)

8.1. Useful features

• Watchpoints (watch EXPR).
• Thread support on some systems.
• M-x gdb in emacs.
• Command/variable completion (tab, second time will show possible completions). Very useful for completing overloaded C++-methods.
• Debugging an already-running process (attach pid, detach).
• Conditional breakpoints (when setting breakpoint (break ... if COND), adding later (condition BNUM EXPR)).
• Temporary breakpoints (tbreak).
• Breakpoints to functions matching regular expression (rbreak).
• Running commands when your program stops due to that breakpoint (commands [BNUM]
  ... COMMAND-LIST ...
  end). Useful for setting variables at the beginning of main (add "break main; commands ; set yydebug = 1; continue; end" to .gdbinit).
• Skip loops using u instead of s (u, until = continue running until a source line past the current line is reached).
• Debugging assembler (display/i $pc; si).
• Printing args and local variables (info args, info locals).
• Artificial arrays (p *array@len).
• Value history ($ = previous value, $$ = value before that, $$2 = value before that etc, $1 = value of 1st value printed etc). They can also be used in complex prints (p *$, p *$.next etc).
• Printing type information (whatis [EXPR], ptype TYPENAME, ptype EXPR).
• Calling your own debugging functions (pretty print of data structures etc). Gcc have .gdbinit that defines adds all kind of pretty-prints to debugger (pr (set debug_rtx ($)), pt (set debug_tree($)), etc.).

9. C-code generation

9.1. Examples

9.1.1. Multilingual printf

• Finds all strings from c-files that are marked with /**/ and calculates hash value for all of them.
• Tranlations of these strings can be added to allstrs file containing hash-value:language:string triples.
• ml_gentable.c then takes this allstrs file and generates c-code that will make ml_entry_table. This table contains hash values, strings in all languages and a next pointer. A ml_hash_table as also added to ml_table.c. It contains precalculated hash-table (using lower 11 bits of 32-bit crc) that points directly to ml_entry_table. Ml_entry_table contains linked list of all entries with same 11 bit hash value. This list is initialized to be pointers to ml_entry_table itself.
• ml_table.c is then compiled to shared library which is then used by all programs. This way new translations of strings can be added dynamically to all programs without recompiling them.
• Sample part of ml_table.c (automatically generated): /* THIS FILE IS AUTOMATICALLY GENERATED BY ML_GENTABLE * DO NOT EDIT. * * Generated by <kivinen@openet.freenet.hut.fi> at Thu Nov 24 22:50:34 1994 * */ #include <stdio.h> #include "ml_param.h" #include "ml_support.h" const ml_language_entry_t ml_language_table[ML_MAX_NUMBER_OF_LANGUAGES] = { { "eng" }, { "fin" }, { "swe" } }; const ml_entry_t ml_entry_table[ML_MAX_NUMBER_OF_STRINGS] = { { 0xb8360, { "ttopen can't get special chars", "ttopen can't get special chars", "ttopen can't get special chars" }, &(ml_entry_table[841]) }, { 0x3cfc70, { "Don't Use the Pager", "Älä käytä sivuttajaa", NULL }, &(ml_entry_table[50]) }, { 0x50f055, { "Received interrupt. Exiting...\n", "Saatu keskeytys. Lopetetaan...\n", NULL }, NULL }, { 0xa1bbe1, { "can't find list of remote files, oops\n", "etätiedostojen luetteloa ei löydy\n", NULL }, NULL }, .... { 0xff0bba0a, { "%s/ Search forward in the file for a pattern\n", "%s/ Mallin eteenpäinhaku tiedostosta\n", NULL }, NULL } }; const ml_entry_t *const ml_hash_table[ML_HASH_TABLE_SIZE] = { NULL, &(ml_entry_table[985]), &(ml_entry_table[364]), &(ml_entry_table[846]), &(ml_entry_table[265]), NULL, &(ml_entry_table[813]), NULL, &(ml_entry_table[106]), .... &(ml_entry_table[807]), &(ml_entry_table[686]), &(ml_entry_table[952]) }; /*EOF*/
• Part of Makefile: ml_table.c: allstrs ml_support.c: ml_param.h ml_param.h: ml_gentable allstrs ./ml_gentable allstrs ml_table.c ml_param.h > /dev/null maketable: ml_gencrcs -$(RM) strs find $(FINDPATH) -name '*.[chly]' -exec cat {} \; > strs /p/gnu/bin/emacs -batch -load getstrings.el strs \ -f getstrings -f save-buffer -quit cat additional-strings >> strs ./ml_gencrcs strs strs-2 touch allstrs cat strs-2 allstrs | sort -u > allstrs.new mv allstrs allstrs~ mv allstrs.new allstrs -$(RM) strs strs-2 -diff -u allstrs~ allstrs > allstrs.diff

9.1.2. db_print_types

• Used in ns532 mach kernel to add general structure printing to ddb.
• Input file db_types.c contains includes that define types.
• Input file is compiled to assembler with debugging information (gcc -S -g).
• The assembler file is then read to printtypes.c that will parse .stabs debugging information and create c-source file that can print structures defined in the debugging information.
• This file is then compiled to the mach-kernel so those structure printing functions can be called from debugger.
• No need to update structure printing when type definitions changes, it is updated automatically.
• Needs stabs debugging symbols (gcc can produce that for most systems, even if native debugging format is different).

10. Diff and Patch

10.1. Features

• Make context diff: diff -cb foo_orig.c foo.c
• Make recursive diff: diff -cbr orig_src src
• Gnu uni-diff: diff -u foo_orig.c foo.c
• Patch changes to sources (silent): patch -s < file.patch.
• -p1 option can be used to strip parts of pathname away.