MongoDB realizes automatic backup of the whole process record

preface

Some time ago, MongoDB crashed because the server was running out of disk space. After cleaning up some useless data, I restarted MongoDB, but it couldn't start. After a lot of fidgeting (20 minutes), I finally fixed it. This made me realize that although it is a small personal project, it must be backed up regularly, otherwise it would be embarrassing to lose the data.

twitter

I am not a master of MongoDB, and I chose MongoDB because the requirements were not clear -- MongoDB is good for developing scenarios with unclear requirements.

Personally, I am more familiar with Elasticsearch, and Elasticsearch is also suitable for business development with uncertain requirements. But personal servers only have 1G memory, so using Elasticsearch will cost you a lot of money to upgrade the server.

MongoDB backup is easy, just use the built-in mongodump, the command format is as follows:

[

mongodump-h {mongodb}:{port} -ES28en {account} -ES29en {password} -ES30en {database name} -ES31en {store path}

]

However, the password I set for MongoDB is with special characters, such as @#$\ and so on. Executing the command directly in the above form will report an error!

Therefore, the author chooses to execute the command without the -ES39en parameter, namely:

[

mongodump-h {mongodb}:{port} -ES46en {account} -ES47en {database name} -ES48en {store path}

]

The command prompt then prompts for a password. For example,

# mongodump -h 127.0.0.1 -u user -d itmuch -o /tmp/somepathEnter password: 

You have now implemented a backup of MongoDB. However, the manual backup is still quite troublesome, how to achieve backup automation.

Automatic backup

Normally, automatic backup is relatively simple -- just make the manual backup command into an Shell script and set the timer task. In my scenario, however, commands require interactive password entry.

How can I enter my password automatically? expect is here -- a tool that provides automatic interaction.

Install expect

yum install -y expect 

Write the expect script

The expect syntax is very simple, almost the same as Shell. The author's script is as follows:

#!/bin/expect 
# spawn is expect Before executing the command, add the word  
set DATE [exec date "+%Y-%m-%d"] 
set DIR /xxxxx/dbbak-$DATE 
spawn rm -rf $DIR 
spawn echo 'removing...$DIR' 
spawn mongodump -h {host:port} -u {user} -d {dbname} -o $DIR 
#  Interactive gets whether or not to return password: The keyword  
expect "password:" 
#  Send the password to the past, note that the last line must not be less, otherwise the manual input enter.  
send " password \r" 
#  If you stay at the remote console, you will return to the local console without this line shell after  
interact 

The notes are very comprehensive, so you can read them intelligently without pressure. The final backup files will be stored in the /xxxxx/dbbak- backup date directory.

Automatic backup

The author uses Linux timing task to realize automatic execution.

crontab -e 

Add the following to the new window:

0 0 1 * * ? /usr/bin/expect  The above expect shell Full path of  

This was supposed to be timed, but it didn't work.

After Baidu, the script will be modified as follows, finally can be normally executed.

#!/bin/expect 
# spawn is expect Before executing the command, add the word  
set DATE [exec date "+%Y-%m-%d"] 
set DIR /xxxxx/dbbak-$DATE 
spawn rm -rf $DIR 
spawn echo 'removing...$DIR' 
spawn mongodump -h {host:port} -u {user} -d {dbname} -o $DIR 
#  Interactive gets whether or not to return password: The keyword  
expect "password:" 
#  Send the password to the past, note that the last line must not be less, otherwise the manual input enter.  
send " password \r" 
 
set timeout 120 
expect eof 
 
exit 

conclusion

There are no difficult points in this article, just some details --

Because passwords contain special characters, they need to be entered interactively. Because the password is interactively entered, expect is used

In addition, expect is a universal tool that provides automatic interaction, which is used to realize the automatic login of ssh, sftp, mysql, etc. The script pattern is basically similar to the structure shown in this article.

conclusion