How to invoke Linux system commands using Java

Directory Java Invokes Linux System Command java Executes Linux Command Supports Wildcard Characters (*)

Java calls Linux system commands

Sometimes, when we use Java to do some operations, the performance may not achieve our satisfactory results. Take a scene encountered in recent work as an example, it is necessary to merge a large number of small files into a large file.

The original idea was to use Java for file operation. Traverse all small files and then write to one file (which can be written concurrently), but find that there is a problem in the operation process. It takes several 10 seconds to write more than 1,000 small files under the native Windows, and nearly 10 seconds even for highly configured machines under the Linux environment, which obviously has an important impact on the response time of the interface. How to optimize this piece?

We all know that large files can be divided and merged under Linux, using split and cat commands respectively, so we made an experiment, cutting the same 1G file into 1000 small files under Linux, and then merging more than 1,000 small files. The effect is amazing! ! ! It can be synthesized in an instant! This further strengthened my idea that I should use system commands to merge small files in batches.

Here we encapsulate a class to invoke system commands and get the return result of the system call.

We first encapsulated a return result class:

package com.majing.learning.fileupload.common.process;
 
public class ProcessResult {
 private boolean success = false;
 private String errorMessage;
 private String outputMessage;
 public boolean isSuccess() {
  return success;
 }
 public void setSuccess(boolean success) {
  this.success = success;
 }
 public String getErrorMessage() {
  return errorMessage;
 }
 public void setErrorMessage(String errorMessage) {
  this.errorMessage = errorMessage;
 }
 public String getOutputMessage() {
  return outputMessage;
 }
 public void setOutputMessage(String outputMessage) {
  this.outputMessage = outputMessage;
 }
 
}

Then we give the encapsulated system call implementation class:

package com.majing.learning.fileupload.common.process;
 
import java.io.IOException;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
 
import org.apache.commons.lang3.StringUtils;
 
public class CommandUtils {
 
 public static ProcessResult runCmdTest(ExecutorService executorService, String command) throws IOException, InterruptedException {
  StringBuilder queryInputResult = new StringBuilder();
  StringBuilder queryErroInputResult = new StringBuilder();
  ProcessResult processResult = new ProcessResult();
  String[] cmd = { "/bin/sh", "-c", command};
  Process pro = Runtime.getRuntime().exec(cmd);
  CountDownLatch lock = new CountDownLatch(2);
  executorService.submit(new ProcessCheckTask(queryInputResult, lock, pro.getInputStream()));
  executorService.submit(new ProcessCheckTask(queryErroInputResult, lock, pro.getErrorStream()));
  boolean done = false;
  while (!done) {
   lock.await();
   done = true;
  }
  processResult.setOutputMessage(queryInputResult.toString());
  processResult.setErrorMessage(queryErroInputResult.toString());
  processResult.setSuccess(StringUtils.isBlank(processResult.getErrorMessage()));
  return processResult;
 }
}

Where the ProcessCheckTask class is as follows:

package com.majing.learning.fileupload.common.process;
 
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.concurrent.CountDownLatch;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
import com.majing.learning.fileupload.common.ConstValues;
 
public class ProcessCheckTask implements Runnable {
 
 private static Logger logger = LoggerFactory.getLogger(ProcessCheckTask.class);
 
 /**  Lock  */
 private CountDownLatch lock;
 
 /**  Execute the result input stream  */
 private InputStream inputStream;
 
 /**  Character mosaic  */
 private StringBuilder queryInputResult;
 
 public ProcessCheckTask(StringBuilder queryInputResult, CountDownLatch lock, InputStream inputStream) {
  super();
  this.lock = lock;
  this.inputStream = inputStream;
  this.queryInputResult = queryInputResult;
 }
 
 @Override
 public void run() {
  try {
   BufferedReader bf = new BufferedReader(new InputStreamReader(inputStream));
   String line = null;
   while ((line = bf.readLine()) != null && line.length() > 0) {
    queryInputResult.append(line).append("\n");
   }
  } catch (Exception e) {
   logger.error(ConstValues.EXCEPTION_OCCURED, e);
  } finally {
   lock.countDown();
  }
 }
}

The above is a simple implementation, but there may be a problem, that is, if the time for executing system commands is relatively long, if you don't want to wait until the system commands are executed, you will directly consider failure if you don't return for a period of time, so you need to increase the consideration of expiration time. Here, I encapsulate the above method of calling system commands into an Callable object with the help of Future framework.

package com.majing.learning.fileupload.common.process;
 
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
 
public class CommandTask implements Callable<ProcessResult>{
 
 private ExecutorService executorService;
 
 private String command;
 
 public CommandTask(ExecutorService executorService, String command){
  this.executorService = executorService;
  this.command = command;
 }
 
 @Override
 public ProcessResult call() throws Exception {
  return CommandUtils.runCmdTest(executorService, command);
 }
 
}

Then, on the basis of the above CommandUtils, another layer is encapsulated to become CommandHelper, and the specific implementation is as follows:

package com.majing.learning.fileupload.common.process; 
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit; 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory; 
import com.majing.learning.fileupload.common.ConstValues;
 
 public class CommandHelper {
 private static Logger logger = LoggerFactory.getLogger(CommandHelper.class);
 private static ExecutorService executorService=Executors.newFixedThreadPool(50);
 private static long default_timeout = 8000;
 public static ProcessResult process(String command){
  return process(command, default_timeout, TimeUnit.MILLISECONDS);
 }
 
 public static ProcessResult process(String command, long timeout, TimeUnit unit){
  CommandTask commandTask = new CommandTask(executorService, command);
  Future<ProcessResult> processResult = executorService.submit(commandTask);
  ProcessResult result = null;
  try{
   result = processResult.get(timeout, unit);
  }catch(Exception e){
   logger.error(ConstValues.EXCEPTION_OCCURED, e);
  }
  return result;
 } 
}

At this point, we can directly call CommandHelper. process (command) when we need to call system commands, and then get the return result ProcessResult. I also make a record by myself, and friends who need it can use it directly.

By the way, when this encapsulated class completes the same tasks above, the time is all on the same machine, and the time consumption is instantly reduced from the original 10s to less than 200ms, which shows how important it is to call system commands in appropriate scenarios.

java executes the Linux command and supports wildcard characters (*)

java executes linux or windows commands, which is a common requirement.

However, if Runtime. getRuntime (). exec (cmd) is used; You will find that if cmd contains wildcard characters, it cannot be executed, such as cp/dira/*. txt/dirb

It can be performed in the following ways:

String[] cmdArr = new String[3];
        cmdArr[0] = "/bin/sh";
        cmdArr[1] = "-c";
        cmdArr[2] = command;
process = Runtime.getRuntime().exec(cmdArr);