Detailed explanation of the solution to the problem of excessive output of nohup log under linux

I recently ran an spark streaming program on the 1hadoop test cluster and then used nohup./execute. sh & Execute let the program run in the background, only a few days on the log G, if there is a problem want to view the log, obviously open the file is a very troublesome thing, so I think of a way to reduce the file size:

1. nohup command interpretation:

a, Syntax: nohup [command] [args] [ & ]

b, Description: The nohup command runs the command specified by the Command parameter and any associated Arg parameter, ignoring all hang-up signals. After logging out, use the nohup command to run the program in the background. To run the nohup command in the background, add & (Symbol for "and") to the end of the command. If no redirection is specified, the log is output to the nohup. out file in the current directory by default,

1 Submit as follows: nohup./execute. sh & In this way, the log or output is currently running. nohup. out

Redirection: nohup./execute. sh > /home/xxx/log.log 2 > & 1 & This will redirect the log to the specified directory

2. Split nohup. out without letting it grow indefinitely

I use the 1-type commit command here: nohup./execute. sh & In this way, there are nohup. out files in the current directory. At this time, you can find a way to cut nohup. out into several small files at regular intervals, but at the same time, you should make nohup. out not grow indefinitely (1. Under normal circumstances, the program cannot be interrupted):

a, every day (set the time according to the needs), regularly segment the log of the previous day (for example, if it is about 1g every day, you can segment about 100m every time).

b, after the segmentation of the nohup. out file, to ensure that the new output log will continue to output to nohup. out

The above is in shell

current_date=`date -d "-1 day" "+%Y%m%d"`

split-b 65535000-d-a 4 nohup. out. /log/log_ ${current_date} _ The split command is used here to segment the nouhup file according to the specified size (65535000b is about more than 60 M, you can customize the size) and divide it into the specified format (-d-a 4 is suffixed in the form of 4 digits to start from 0000, which can be used by Baidu split command). The final output format is

cat /dev/null > nohup. out (this command instantly empties the nohup. out file, which will continue to be written later), directs the log to/dev/null

You can do this by using redirected output 1, just replace it with the redirected file name

These commands are defined in an shell file to run regularly every day, so that the daily log will be divided into several copies, and it is convenient to check, and if the log backlog is too large. You can delete the historical log regularly and keep it for a few days

The overall code is as follows:

this_path=$(cd `dirname $0`;pwd)
 
cd $this_path
echo $this_path
current_date=`date -d "-1 day" "+%Y%m%d"`
echo $current_date
split -b 65535000 -d -a 4 /home/.../nohup.out  /home/.../log/log_${current_date}_
 
cat /dev/null > nohup.out