Diagnosing out-of-memory issues

• most people are misled by the used and free lines in the output from top for ram because of the file cache.

• once linux loads a file into ram, it doesn’t necessarily remove it from ram when a program is done with it. if there is ram available, linux will cache the file in ram so that if the program accesses the file again, it can do so much more quickly.

• if the system does need ram for active processes, it won’t cache as many files.

• even if swap is being used it’s not and indicator of a problem.

• if a process becomes idle, linux will often page its memory to swap to free up ram for other processes

• a good way to tell whether a server is running out of ram is to look at the file cache

• if your actual used memory minus the file cache is high, and the swap usage is also high, then there is most likely a memory problem.

• when the linux kernel is almost out of memory then OOM (out of memory) killer will kick in and start killing processes. sometimes it will kill the process that is using the most memory but that is not guaranteed.