we have few small wordpress sites nothing much server is powerful one its Xeon E3
i was extremely surprised about what is causing all this, what i did was i started with eaccelerator and when i realised the issue i removed that and installed xcache and with xcache the situation got worse, and at the end remove all those cache related things but still the same.
here you can find the screenshot of memory usage at that time:
http://i42.tinypic.com/k9esew.jpg
server support tried hard they tune up apache but didn't helped there were lots of nobody process running and consuming good amount of memory.