[grsec] a new plugin: size_overflow

[PaX Team]

hello folks,

as of last night we have a new plugin from Emese Revfy (author of the constify
plugin as well) that i'd like to briefly introduce to you now (while she's working
on her blog post ;).

first some background: almost 3 years ago we had introduced a small hack into
grsecurity to handle a specific bug class, namely, (unintended) integer overflows
occuring in calculating certain function parameters, such as those used to
allocate memory or copy data between the kernel and userland (the bug class is
obviously not specific to the kernel, userland programs have had their fair
share of them over the years ;). the effect of such bugs is that due to the
finite precision of integers, the integer overflow ends up producing a value
much smaller than the program logic assumes. this in turn can cause memory
underallocation and a buffer overflow on later accesses, among others.

spender's hack attempted to handle a specific but also quite typical instance
of these bugs where the calculation is a simple multiplication of two variables,
with at least one under the attacker's control, e.g., void *p = kmalloc(a*b).
by (ab)using C operator precedence rules and some macro magic, he could turn
this into a kmalloc((intoverflow_t)a*b) call where the intoverflow_t type was
specifically created to have double width precision of what the original
function argument had. this would in turn force the compiler to compute the
multiplication with the same double width precision and thus the integer overflow
could be detected by a simple range check on the result.

now this approach has numerous drawbacks (more complex calculations and wrappers
around the interesting functions can't be handled without more invasive patching)
and it was clear from the beginning that a better way is needed - enter gcc plugins.
since the compiler sees the entire calculation step-by-step and can also see through
function calls it is possible to modify the internal representation of all the
calculations leading up to the final interesting function argument to use a double
width integer type.

now there's a lot of devil in the details as her blog post will explain it, but
for now what you need to be aware of is that you should consider this plugin as
experimental. in particular, there's a possibility for false positive reports
(the runtime detection logic logs integer overflow events with a

   "size overflow detected in function ..."

message) that we'd like to know about whenever you see them. of course if you catch
a real size overflow alive we'd like to know about that too ;). note that the result
of such an event will be a SIGKILL to the current task with all that it brings with
it (user account lockout or kernel panic ;).

another message you should not but still may see during compilation is

   "Function ... is missing from the size_overflow hash table ..."

please report them to her (re.emese at gmail.com) and us directly along with your
kernel config and exact patch version.

last but not least, simple performance tests showed some puzzling results in that
i measured a speedup of 2% on a specific load that exercised the kernel almost
exclusively ('du -s' on a directory containing many files). while we hope you won't
see any major performance regressions either, we're still interested in any numbers
you can get on your workloads.

that's all for now, if you have further questions, let us know. also don't forget
to thank Emese for her work, she's been busting at it since last August and produced
a plugin that's bigger and more complex than all the others we've had so far, that's
no small feat!

cheers,
  PaX Team