The python script implements the data export excel format in a simple way recommended by of

During the internship, a senior server asked me to help sort out the log data of the server. Finally, I used Python to extract the data and export it in Excel format. The following is the source code of my Python implementation, can automatically traverse a certain 1 file directory under all text files, and the total data exported to Excel file, exported to Excel format so that more convenient statistics.

// to achieve the directory of all the file format for.txt file traversal statistics, if it is another format directly changed the following.txt format suffix you need to be able to, more convenient.

// the process is to extract and write the contents of all the files into a new file, then extract the data from the new file, and finally write the data into the Excel file

from pyExcelerator import *
import os
currentpath = os.getcwd() 
testlog = open('test.mak','w') 
os.mkdir(r'Excel') 
print "currentpath: ",currentpath 
for file in os.listdir(currentpath):
if os.path.isfile(os.path.join(currentpath,file))==True:
if file.find('.txt')>0:  // If it is another format directly put the following .txt Change the suffix to the format you need 
file_ = open(file,'r')
content = file_.read()  
file_.close()  
testlog.write( content ) 
print 1
os.popen('log_parse.exe test.mak >> shuju.log')
print 2
for _file in os.listdir(currentpath):
if os.path.isfile(os.path.join(currentpath,_file))==True:
if _file.find('.log')>0:
work = Workbook() 
works = work.add_sheet('Sheet1') 
print 3
file_object = open(_file)
for i in range(0,2):
works.col(i).width = 10000
i = 0
for line in file_object:
line = line.rstrip('\n')
print 4
if not line.split():
i = i + 1
if line.strip():
array = line.split(':')
lineleft = array[0]
lineright = array[1]
works.write(i,0,lineleft)
works.write(i,1,lineright)
i = i + 1
_file = _file.rstrip('.log')
_file = 'Excel\%s.xls' % _file
work.save(_file)

// print 1, 2, 3, 4 is what I called log and if you don't want it you can just delete it. With this Python implementation, you can simply save the above code to a file called test.py.

In addition, an extract executable file log_parse.exe is used in the middle, which is placed below. Use it in the same directory as test.py.

If it is convenient, you can create a.bat file and write it as the command line. Click 1 directly to automatically complete all the work, as follows:

echo
python test.py

My own implementation is about 150M files run 1 and a half minutes to produce the results, I think it is still relatively ideal.