Monday, March 07, 2011

Illegal stack operations

Illegal Stack Operations

Illegal stack operations can lead to hard to detect crashes. This typically takes place when a program passes a pointer of the wrong type to a function. The example given below shows a case of a function expecting an integer pointer and the caller passes a pointer to a character.

char pointer/int pointer mixup
char count;
// The routine expects a int pointer but a char pointer has been passed
// Older compilers and non ANSI C compilers do not catch this error
// The called function was expecting an int (say 4 byte) variable. It was
// however passed a char pointer with one byte space. GetCount will still
// write four bytes, thus corrupting local variables or parameters on the
// stack

bool GetCount(int *pCount)
. . .
*pCount = returnValue;
return true;