[grsec] Urgent problem: denied resource overstep

[Martijn Grendelman]

Hi,

I have a problem that needs a solution fast and I can't figure it out. I 
hope someone here can help.

Users that try to use our FTP server to upload files to their home 
directory, experience problems. Data transfers get cut off and lots of other 
annoying stuff, that makes using FTP virtually impossible. All this started 
a couple of days ago.

In my syslog, I find the follow

Jul  8 12:36:19 daffy kernel: grsec: From 84.35.16.78: denied resource 
overstep by requesting 328 for RLIMIT_NPROC against limit 256 for 
/usr/sbin/in.ftpd[in.ftpd:28746] uid/euid:0/1001 gid/egid:0/100, parent 
/usr/sbin/inetd[inetd:16359] uid/euid:0/0 gid/egid:0/0

My (pretty wild) guess is, that this error occurs when ftpd tries to fork a 
process for handling the data connection.

Unfortunately, I don't really understand why this is happening. Where does 
that limit 256 come from? I don't use customised resource limits, and a 
simple 'ulimit -u' returns 7168. And if 256 is the limit, how can the system 
at all get up to 328? Root had about 325 processes running at the time this 
error occurred, so I just assume that this is what's happening.

Is there a way to switch off these checks in grsecurity (no, 
kernel.grsecurity.execve_limiting doesn't do it), or even better: how can I 
increase the limit grsecurity uses?

Thanks,
Martijn.