Of Stack

Tutorial on replicating databases and collections between different hosts by MongoDB

  • 2020-05-27 07:30:46
  • OfStack

1. db.cloneCollection()
db.cloneCollection(from, collection, query)
Copying data between different instances of mongodb, db.cloneCollection is an externality of the cloneCollection database command. 


function (from, collection, query) {
 assert( isString(from) && from.length );
 assert( isString(collection) && collection.length );
 collection = this._name + "." + collection;
 query = query || {};
 return this._dbCommand( { cloneCollection:collection, from:from, query:query
} );
}

Parameters: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents

db.cloneCollection () does not allow copying tables through mongos, but only through mongod instances.
Example:
192.168.11.51 mongod instance mydb library, bar collection: 


{ "_id" : ObjectId("53687d9df433cf04b788c6d1"), "name" : "dog" }
{ "_id" : ObjectId("53687ff1f433cf04b788c6d2"), "name" : "cat" }
{ "_id" : ObjectId("53687ff4f433cf04b788c6d3"), "name" : "tiger" }

The local mongod instance mydb library copies the documents in the bar collection of the remote host that meet the query criteria: 


db.cloneCollection("192.168.11.52", "bar", {"name" : "tiger"})
db.bar.find();
{ "_id" : ObjectId("53687ff4f433cf04b788c6d3"), "name" : "tiger" }

2. db.cloneDatabase()
db.cloneDatabase("hostname")
Copy the database of the remote host to the local. This command assumes that the remote mongodb instance has the same database name as the local. 


hostname  string   Contains the database that needs to be replicated mongodb Instance host name

db.cloneDatabase is an external representation of the clone database command. 


function (from) {
 assert( isString(from) && from.length );
 return this._dbCommand( { clone: from } );
}

Example:
192.168.11.51 mongod instance mydb library,
Local mongodb instance: 


use mydb
db.dropDatabase();
db.cloneDatabase("192.168.11.52");

3. db.copyDatabase()
db.copyDatabase(fromdb, todb, fromhost, username, password)
Copy the database from the remote host to the local, or copy the database from the local to the remote host. 


db.copyDatabase is copydb database-ordered 1 An external manifestation. 
function (fromdb, todb, fromhost, username, password) {
 assert( isString(fromdb) && fromdb.length );
 assert( isString(todb) && todb.length );
 fromhost = fromhost || "";
 if ( username && password ) {
  var n = this._adminCommand( { copydbgetnonce : 1, fromhost:fromhost } );

  return this._adminCommand( { copydb:1, fromhost:fromhost, fromdb:fromdb,
 todb:todb, username:username, nonce:n.nonce, key:this.__pwHash( n.nonce, userna
me, password ) } );
 } else {
  return this._adminCommand( { copydb:1, fromhost:fromhost, fromdb:fromdb,
 todb:todb } );
 }
}

Parameters: 


fromdb  string   Source database name 
todb  string   Target database name 
fromhost string   Optionally, the host name of the source database. If is the same 1 Host, ignore this option 
username string   Optionally, the source hostname username 
password string   Optionally, the source hostname username corresponds to the password

Properties:
(1) db.copyDatabase () runs on the mongod instance of the target host.
(2) db.copyDatabase () creates the target database if it does not already exist.
(3) db.copyDatabase () requires sufficient space on the target machine for replication.
(4) db.copyDatabase () does not produce an instant snapshot of the target database. If a read or write operation occurs in the source or target library during replication, the database will be different.
(5) db.copyDatabase () does not lock the target host during the operation, so temporary interruptions may occur during replication to complete other operations.
Source database (fromdb) :
mongodb2.6 requires the following permissions on the source and target hosts to execute copydb.
(1) if the source host database is not admin, you must ensure that you have the following permissions:
{ resource: { db: "mySourceDB", collection: "" }, actions: [ "find" ] }
{ resource: { db: "mySourceDB", collection: "system.js" }, actions: [ "find" ] }
If the source host is a remote host, you must ensure that you have the following permissions: 


{ resource: { db: "mySourceDB", collection: "system.indexes" }, actions: [ "find" ] }
{ resource: { db: "mySourceDB", collection: "system.namespaces" }, actions: [ "find" ] }

(2) if the source host database is admin, you must ensure that you have the following permissions: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
0

If the source host is a remote host, you must ensure that you have the following permissions: 


{ resource: { db: "admin", collection: "system.indexes" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.namespaces" }, actions: [ "find" ] }

(3) the source database is on the remote host
If a database is replicated from a remote host that has user authentication, a user authentication with the appropriate permissions is required.
Target database (todb) :
A. If the target host database is not admin, you must ensure that you have the following permissions: 


{ resource: { db: "myTargetDB", collection: "" }, actions: [ "insert", "createIndex" ] }
{ resource: { db: "myTargetDB", collection: "system.js" }, actions: [ "insert" ] }

B. If the target host database is admin, you must ensure that you have the following permissions: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
3

Example:
192.168.11.51 mongod instance mydb library,
Copy to local newmydb library: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
4

4. cloneCollection
Copies the collection from the remote mongodb instance to the current mongodb instance. The collection name is 1: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
5

cloneCollection has the following field values: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
6

Example:
192.168.11.51 mongod instance mydb library, bar collection: 


{ "_id" : ObjectId("53687d9df433cf04b788c6d1"), "name" : "dog" }
{ "_id" : ObjectId("53687ff1f433cf04b788c6d2"), "name" : "cat" }
{ "_id" : ObjectId("53687ff4f433cf04b788c6d3"), "name" : "tiger" }

Local mongod instance: 


db.runCommand({cloneCollection : "mydb.bar", from : "192.168.11.52:27017", query : {"name" : "tiger"}})
use mydb
db.bar.find()
{ "_id" : ObjectId("53687ff4f433cf04b788c6d3"), "name" : "tiger" }

cloneCollectionAsCapped can create a new capped collection from non-cpped collections that exist in the database, with no side effects to the original collection.
Instruction syntax:
{ cloneCollectionAsCapped: <existing collection>, toCollection: <capped collection>, size: <capped size> }
 The new set name is unique in the database. If you want to convert an existing normal set into an cpped set, you can use the convertToCapped command. During the replication process, the cloneCollectionAsCapped instruction shows the following behavior:
mongodb traverses the documents in the collection in a natural order.
If size is smaller than the previous collection size, the previous document is deleted with the FIFO rule.
Example: 


from  string   Containing the tables that need to be replicated mongodb Instance host name 
collection string   The name of the table that needs to be replicated in the data instance. This command can only be replicated remotely mongodb A table with the same database name on the instance 
query  document  Optional options. Standard query statements filter out unwanted documents
9

5. clone
The clone command copies a database from the remote server mongodb instance to the current mongodb instance as follows: 


{ clone: "db1.example.net:27017" }

A few points to note:
(1) clone cannot operate the non-master node of slave node or replica set.
(2) clone does not support the database snapshot function. If a client has updated the data, the result may be different.
(3) the clone command must run on the target node.
(4) in the process of clone, the target host is not locked, so temporary interruption may occur in the replication process to complete other operations.

6. copydb
Copy the database from the remote host to the local, or copy the database from the local to the remote host.
Run the following command syntax in the local admin library: 


{ copydb: 1,
 fromhost: <hostname>,
 fromdb: <database>,
 todb: <database>,
 slaveOk: <bool>,
 username: <username>,
 nonce: <nonce>,
 key: <key> }

Options: 


fromhost  string   run mongodb The remote source host of the instance can be ignored if it is local 
fromdb   string   Source database name 
todb   string   Target database name 
slaveOk   boolean   Optionally, set to true Allows you to copy libraries from and from libraries 
username  string   Optionally, the user name of the remote host. 
nonce   string   Optionally, the remote host's Shared key 
key    string   Optionally, the remote host's authentication password is hashed

Properties:
(1) copydb() runs on the mongod instance of the target host.
(2) copydb() creates the target database if it does not already exist.
(3) copydb() requires sufficient space on the target machine for replication.
(4) copydb() does not produce an instant snapshot of the target database. If a read or write operation occurs in the source or target library during replication, the database will be different.
(5) copydb() does not lock the target host during the operation, so temporary interruptions may occur during replication to complete other operations.
mongodb2.6 requires the following permissions on the source and target hosts to execute copydb.
(1) if the source host database is not admin, you must ensure that you have the following permissions: 


{ resource: { db: "mySourceDB", collection: "" }, actions: [ "find" ] }
{ resource: { db: "mySourceDB", collection: "system.js" }, actions: [ "find" ] }

If the source host is a remote host, you must ensure that you have the following permissions: 


{ resource: { db: "mySourceDB", collection: "system.indexes" }, actions: [ "find" ] }
{ resource: { db: "mySourceDB", collection: "system.namespaces" }, actions: [ "find" ] }

(2) if the source host database is admin, you must ensure that you have the following permissions: 


{ resource: { db: "admin", collection: "" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.js" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.users" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.roles" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.version" }, actions: [ "find" ] }

If the source host is a remote host, you must ensure that you have the following permissions: 


{ resource: { db: "admin", collection: "system.indexes" }, actions: [ "find" ] }
{ resource: { db: "admin", collection: "system.namespaces" }, actions: [ "find" ] }

(3) the source database is on the remote host
If a database is replicated from a remote host that has user authentication, a user authentication with the appropriate permissions is required.
Target database (todb) :
If the target host database is not admin, you must ensure that you have the following permissions: 


{ resource: { db: "myTargetDB", collection: "" }, actions: [ "insert", "createIndex" ] }
{ resource: { db: "myTargetDB", collection: "system.js" }, actions: [ "insert" ] }

B. If the target host database is admin, you must ensure that you have the following permissions: 


resource: { db: "myTargetDB", collection: "" }, actions: [ "insert", "createIndex" ] },
{ resource: { db: "myTargetDB", collection: "system.js" }, actions: [ "insert" ] },
{ resource: { db: "myTargetDB", collection: "system.users" }, actions: [ "insert" ] },
{ resource: { db: "myTargetDB", collection: "system.roles" }, actions: [ "insert" ] },
{ resource: { db: "myTargetDB", collection: "system.version" }, actions: [ "insert" ] }

Certification:
If the remote host requires security authentication, username,nonce, and key are required for authentication.
nonce is a one-time password by running the copydbgetnonce command: 


use admin
mynonce = db.runCommand( { copydbgetnonce : 1, fromhost: <hostname> } ).nonce

If you run the copydbgetnonce command directly from a remote host, you can ignore the fromhost option.
A hash key is generated as follows: 


hex_md5(mynonce + username + hex_md5(username + ":mongo:" + password))

Replica set: set slaveOk to true to run copydb on slave nodes.
Sharding: do not run copydb on an mongos instance; Do not copy libraries that contain sharded collections.

Example:
(1) copydb running on the same host

(2) copydb copied from the remote host 


db._adminCommand({
 copydb : 1,
 fromdb : "mydb",
 todb : "mydbtwo",
 formhost : "192.168.11.52"
})
{ "ok" : 1 }

(3) copy copydb from the remote host that requires security verification
Remote host setup user test:caoqing/mydb 


use admin
mynonce = db.runCommand( { copydbgetnonce : 1, fromhost: "192.168.11.51:27017" } ).nonce
mykey = hex_md5(mynonce + "test" + hex_md5("test" + ":mongo:" + "caoqing"))
db._adminCommand({
 copydb: 1,
 fromdb: "mydb",
 todb: "mydbthree",
 fromhost: "192.168.11.51",
 username: "test",
 nonce: mynonce,
 key: mykey
})
{ "ok" : 1 }

Related articles: