xGCC - A program analysis tool based on a modified GNU - Compiler

Overview:

We want to develop an environment based on a modified GCC which allows analysis of programs written in various languages.
We want one environment for heterogenous software projects
We want to find low-level errors that are the typical basis for security holes (memory access, stack access)
We have modified GCC 3.1.1 to generate an extension of RTL called XRTL
We developed suitable datatypes for XRTL interpretation
We apply well known analysis techniques

The modified compiler has been tested for Linux/x86 and Solaris/Sparc but it should work for any other architecture, since only the language and machine independent part has been modified! If you are using FreeBSD don't forget to apply the given patches (see /usr/ports/lang/gcc31/files/) afterwards.

Files:

Extract the source code of gcc 3.1.1 and replace the following files.
The modified compiler accepts the option -dZ which produces two files {name}.32.xrtl.rtl and {name}.33.xrtl.stack.
You should use the -ggdb option and turn off any optimizations.

gcc/calls.c (last modified: 29.10.2003)
gcc/function.c (last modified: 27.05.2004)
gcc/output.h (last modified: 13.05.2003)
gcc/print-rtl.c (last modified: 27.05.2004)
gcc/rtl.h (last modified: 13.05.2003)
gcc/toplev.c (last modified: 13.05.2003)
gcc/varasm.c (last modified: 11.06.2003)

Changelog:

print-rtl.c, 27.05.2004: increased offset for scratch and CC registers.
function.c, 27.05.2004: few fixes.
function.c, 01.03.2004: Alignment fix (alignment 8 for temporary stack variables)
calls.c, 29.10.2003: Additional information about parameters, FreeBSD fix
function.c, 29.10.2003: Introduced dynamic stack register
print_rtl.c, 21.05.2003: Register renaming (bijection): (reg:CC* NO)-> (reg:SI NO+offset); (scratch:MODE) -> (reg:MODE 256)
varasm.c, 11.06.2003: modified XRTL output: (array) -> (record/union)
calls.c, 10.07.2003: added mode to register arguments

Examples:

example.c

int main()
{
  int i = (int )&i;
  double d = 0.9187;

  *((((int *)(i-4))+(int )(((int )d/0.25)+2.9))) = 7;

  return 0;
}

gcc -dZ -g -c example.c

example.c.32.xrtl.rtl

(note 0 0 1 ("example.c") NOTE_INSN_FUNCTION_NAME ("main"))
(note 1 0 2 ("example.c") 2)
(note 2 1 6 NOTE_INSN_DELETED)
(note 3 12 4 NOTE_INSN_FUNCTION_BEG)
(note 4 3 13 NOTE_INSN_DELETED)
(note 13 4 14 NOTE_INSN_BLOCK_BEG)
(note 14 13 15 NOTE_INSN_DELETED)
(note 15 14 16 ("example.c") 3)
(insn 16 15 18 (parallel[ 
             (set (reg:SI 61) (plus:SI (reg:SI 54) (const_int -4)))
             (clobber (reg:SI 274))
        ]))
(insn 18 16 19 (set (mem:SI (plus:SI (reg:SI 54) (const_int -4))) (reg:SI 61)))
(note 19 18 21 ("example.c") 4)
(insn 21 19 22 (set (mem:DF (plus:SI (reg:SI 54) (const_int -16))) (const_double:DF 0.9187)))
(note 22 21 24 ("example.c") 6)
(insn 24 22 26 (set (reg:DF 63) (mem:DF (plus:SI (reg:SI 54) (const_int -16)))))
(insn 26 24 27 (set (reg:SI 62) (fix:SI (reg:DF 63))))
(insn 27 26 29 (set (reg:DF 64) (float:DF (reg:SI 62))))
(insn 29 27 30 (set (reg:DF 66) (mem:DF (symbol_ref:SI ("*.LC0")))))
(insn 30 29 32 (set (reg:DF 65) (div:DF (reg:DF 64) (reg:DF 66))))
(insn 32 30 33 (set (reg:DF 68) (mem:DF (symbol_ref:SI ("*.LC1")))))
(insn 33 32 35 (set (reg:DF 67) (plus:DF (reg:DF 65) (reg:DF 68))))
(insn 35 33 37 (set (reg:SI 69) (fix:SI (reg:DF 67))))
(insn 37 35 39 (set (reg:SI 70) (reg:SI 69)))
(insn 39 37 41 (parallel[ 
             (set (reg:SI 71) (ashift:SI (reg:SI 70) (const_int 2)))
             (clobber (reg:SI 274))
        ]))
(insn 41 39 43 (parallel[ 
             (set (reg:SI 72) (plus:SI (reg:SI 71) (mem:SI (plus:SI (reg:SI 54) (const_int -4)))))
             (clobber (reg:SI 274))
        ]))
(insn 43 41 45 (parallel[ 
             (set (reg:SI 73) (plus:SI (reg:SI 72) (const_int -4)))
             (clobber (reg:SI 274))
        ]))
(insn 45 43 46 (set (mem:SI (reg:SI 73)) (const_int 7)))
(note 46 45 47 ("example.c") 8)
(note 47 46 48 NOTE_INSN_DELETED)
(note 48 47 50 NOTE_INSN_DELETED)
(insn 50 48 51 (set (reg:SI 58) (const_int 0)))
(jump_insn 51 50 52 (set (pc) (label_ref 56)))
(barrier 52 51 53)
(note 53 52 54 NOTE_INSN_BLOCK_END)
(note 54 53 55 NOTE_INSN_FUNCTION_END)
(note 55 54 59 ("example.c") 9)
(insn 59 55 60 (clobber (reg:SI 0)))
(insn 60 59 56 (clobber (reg:SI 58)))
(code_label 56 60 58 1 "" "")
(insn 58 56 61 (set (reg:SI 0) (reg:SI 58)))
(insn 61 58 0 (use (reg:SI 0)))

example.c.33.xrtl.stack

(xinsn (stackvar) (line 3) (size:SI 4) (align 32))
(xinsn (stackvar) (line 4) (size:DF 8) (align 64))
(xinsn (stackvar) (line 9) (size:HI 2) (align 0))
(xinsn (stackvar) (line 9) (size:HI 2) (align 0))
(xinsn (stackvar) (line 9) (size:SI 4) (align 0))
(xinsn (stack) (incomingp 53) (stackp 54) (hstackp 7) (outgoingp 56))
(xinsn (stack) (line 9) (size 24))
(xinsn (function) (line 2) (name "main"))
(xinsn (constant) (line 9) (name "LC0") (begin))
(xinsn (real) (line 9) (set (const:DF "LC0") (0.25)))
(xinsn (constant) (line 9) (name "LC0") (end))
(xinsn (constant) (line 9) (name "LC1") (begin))
(xinsn (real) (line 9) (set (const:DF "LC1") (2.9)))
(xinsn (constant) (line 9) (name "LC1") (end))
(xinsn (constant) (line 9) (name "LC2") (begin))
(xinsn (real) (line 9) (set (const:SF "LC2") (nan)))
(xinsn (constant) (line 9) (name "LC2") (end))

The analysis is performed on XRTL using the files example.c.32.xrtl.rtl and example.c.33.xrtl.stack to reconstruct the stack and its allocation.

Analysis:

The analysis part has not been integrated in the compiler, but it exists as a seperate tool, which will be available soon.

Impressum

Comments to Werner Backes