MongoDB memory usage analysis

MongoDB is a database based on distributed file storage. Written by C++ language. Designed to provide scalable, high-performance data storage solutions for WEB applications.

MongoDB is a product between relational database and non-relational database. It is the most functional and relational database like non-relational database. The data structure it supports is very loose and is in bson format similar to json, so it can store more complex data types. The biggest feature of Mongo is that the query language it supports is very powerful. Its syntax is somewhat similar to that of object-oriented query language. It can almost realize most of the functions of a relational database single table query, and it also supports data indexing.

So let's look at ps 1.

$ ps aux|grep mongod
mongo    26994  9.0 20.0 797264324 13243052 ?  Sl   May16 117:03 /path/to/mongodb/bin/mongod

A total of 760G more virtual memory, but only 12.6G physical memory. This machine has 64G memory, so it looks like MongoDB doesn't use much memory at all.

Now look at free.

$ free -m
             total       used       free     shared    buffers     cached
Mem:         64544      64279        265          0        134      60413
-/+ buffers/cache:       3731      60813
Swap:        31999          0      31999

That's a lot of memory. It's mostly cached, or file system cache. MongoDB is an mmap way for the operating system to handle persistence and caching. Each data file is mapped directly to a virtual memory address. If the page is not in memory at the time of access, the system will attempt to load the page. This memory is all in cache. In the official documentation of mongodb, there is a statement that the RSIZE segment in top or ps shows the total memory size of the machine, because mongodb USES as much memory as possible. But in fact, these caches don't count. Therefore, the actual memory usage of MongoDB is not visible in top or ps. While free can see the memory usage of the system, it is impossible to determine how much of this memory is actually used by MongoDB.

It's a good thing someone made vmtouch. You can check the status of the file in the cache, or you can load the file directly into the cache or kick it out. All you need to do is check all the data files of MongoDB for 1 cache load to see how much data MongoDB is caching.

$ vmtouch -m4G /path/to/mongodb/data/
           Files: 256
     Directories: 3
  Resident Pages: 15465901/100219772  58G/382G  15.4%
         Elapsed: 4.072 seconds

Here -m4G is the file size limit that vmtouch checks. The data files for MongoDB are larger and usually exceed the default of 500M. In this case, the cache USES 58G, which is about right. The number to the left of Resident Pages is the number of pages, and the number of pages times the file system page size is the memory usage. The page size can be passed

getconf PAGESIZE 

Check, usually 4096, which is 4KB.

MongoDB runs on the NUMA machine, and there is a warning when the memory is fixed to an node

WARNING: You are running on a NUMA machine. 
We suggest launching mongod like this to avoid performance problems: 
numactl  � interleave=all mongod [other options]

Maybe you think that only one node of memory will be used in this situation. But the MongoDB cache is managed by the operating system. NUMA doesn't seem to have an effect. When the memory is not too small, MongoDB itself can hardly use up the memory of 1 node. In this case, whether or not to turn on numactl, interleave=all has little effect. Maybe all you can do is add memory, sharding, or switch to ssd.

Reference: http: / / xiezhenye com / 2013/05 / mongodb - % e5 ad e5 % % % 86% 85% 98% e4 e7 bf bd % % % % 94% a8. The html