Resolved the problem of MongoDB sorting exceeding memory limits

  • 2020-06-23 02:11:10
  • OfStack

Performing a large sort operation on a collection (such as aggregation), the following error occurred: (test version: MongoDB 3.0.6)


> db.bigdata.aggregate(
 {$group : {_id : "$range", total : { $sum : 1 }}},
 {$sort : {total : -1}}
);
#...
 aggregate failed
 at Error (<anonymous>)
 at doassert (src/mongo/shell/assert.js:11:14)
 #...
 Error: command failed: {
 "errmsg" : "exception: Sort exceeded memory limit of 104857600 bytes, 
 but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.",
 "code" : 16819,
 "ok" : 0
 }

The solution

Reference: Memory Restrictions

In MongoDB, the maximum memory limit for in-sort is 100M, and if you perform a larger sort, you need to use the allowDiskUse option to write the data to a temporary file to sort.

Add the allowDiskUse option to the query:


db.bigdata.aggregate(
[
 {$group : {_id : "$range", total : { $sum : 1 }}},
 {$sort : {total : -1}}
],
 {allowDiskUse: true}
);

conclusion


Related articles: