Resolved the problem of MongoDB sorting exceeding memory limits
- 2020-06-23 02:11:10
- OfStack
Performing a large sort operation on a collection (such as aggregation), the following error occurred: (test version: MongoDB 3.0.6)
> db.bigdata.aggregate(
{$group : {_id : "$range", total : { $sum : 1 }}},
{$sort : {total : -1}}
);
#...
aggregate failed
at Error (<anonymous>)
at doassert (src/mongo/shell/assert.js:11:14)
#...
Error: command failed: {
"errmsg" : "exception: Sort exceeded memory limit of 104857600 bytes,
but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.",
"code" : 16819,
"ok" : 0
}
The solution
Reference: Memory Restrictions
In MongoDB, the maximum memory limit for in-sort is 100M, and if you perform a larger sort, you need to use the allowDiskUse option to write the data to a temporary file to sort.
Add the allowDiskUse option to the query:
db.bigdata.aggregate(
[
{$group : {_id : "$range", total : { $sum : 1 }}},
{$sort : {total : -1}}
],
{allowDiskUse: true}
);
conclusion