Efficient implementation of large file copy function by java

  • 2021-07-24 10:58:44
  • OfStack

In java, there are one optimization method in the FileChannel class that can improve the efficiency of transmission. Among them, transferTo () and transferFrom () methods allow one channel to be cross-connected to another channel without passing data through a buffer. Only the FileChannel class has these two methods, so channel 1 in the channel-to-channel transport must be FileChannel. Data cannot be transferred between sock channels, but the socket channel implements the WritableByteChannel and ReadableByteChannel interfaces, so the contents of a file can be transferred to an socket channel using the transferTo () method, or data can be read directly from an socket channel to a file using the transferFrom () method.

The Channel-to-channel transport can be extremely fast, especially when the underlying operating system provides local support. Some operating systems can transfer data directly without transferring data through user space. For a large amount of data transmission, this will be a great help.

Note: If the file to be copied is larger than 4G, the method of Channel-to-channel cannot be used directly. Instead, use ByteBuffer, read from the original file channel to ByteBuffer, and then write ByteBuffer to the target file channel.

The following is the code to realize fast copy of large files:


import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
 
public class BigFileCopy {
 
 
 /**
 *  Pass channel To channel Direct transmission 
 * @param source
 * @param dest
 * @throws IOException
 */
 public static void copyByChannelToChannel(String source, String dest) throws IOException {
 File source_tmp_file = new File(source);
 if (!source_tmp_file.exists()) {
 return ;
 }
 RandomAccessFile source_file = new RandomAccessFile(source_tmp_file, "r");
 FileChannel source_channel = source_file.getChannel();
 File dest_tmp_file = new File(dest);
 if (!dest_tmp_file.isFile()) {
 if (!dest_tmp_file.createNewFile()) {
 source_channel.close();
 source_file.close();
 return;
 }
 }
 RandomAccessFile dest_file = new RandomAccessFile(dest_tmp_file, "rw");
 FileChannel dest_channel = dest_file.getChannel();
 long left_size = source_channel.size();
 long position = 0;
 while (left_size > 0) {
 long write_size = source_channel.transferTo(position, left_size, dest_channel);
 position += write_size;
 left_size -= write_size;
 }
 source_channel.close();
 source_file.close();
 dest_channel.close();
 dest_file.close();
 }
 
 
 public static void main(String[] args) {
 try {
 long start_time = System.currentTimeMillis();
 BigFileCopy.copyByChannelToChannel("source_file", "dest_file");
 long end_time = System.currentTimeMillis();
 System.out.println("copy time = " + (end_time - start_time));
 } catch (IOException e) {
 e.printStackTrace();
 }
 }
 
}

Related articles: