Of Stack

Android uses AsyncTask to realize multithreaded breakpoint continuous transmission

  • 2021-09-11 21:24:10
  • OfStack

The previous blog "AsyncTask realizes breakpoint continuous transmission" explains how to realize breakpoint continuous transmission under single thread, that is, only one thread downloads a file.

For large files, multi-threaded downloading is 1% faster than single-threaded downloading. Multi-threaded downloads are a little more complicated than single-threaded downloads. This blog post will explain in detail how to use AsyncTask to implement multi-threaded breakpoint downloads.

1. Implementation principle

Multi-threaded downloading begins with the total number of downloading threads per file (I set 5 here) to determine where each thread is responsible for downloading. 


 long blockLength = mFileLength / DEFAULT_POOL_SIZE;
  for (int i = 0; i < DEFAULT_POOL_SIZE; i++) {
   long beginPosition = i * blockLength;// The starting position of each thread download 
   long endPosition = (i + 1) * blockLength;// End position of download for each thread 
   if (i == (DEFAULT_POOL_SIZE - 1)) {
    endPosition = mFileLength;// If the size of the entire file is not an integer multiple of the number of threads, the last 1 The end position of threads is the total length of the file 
   }
   ......
  }

It should be noted here that the file size is often not an integer multiple of the number of threads, so the end position of the last thread needs to be set to the file length.

After determining the download start and end location of each thread, you need to set the http request header to download the specified location of the file: 


  // Set the location of downloaded data beginPosition Byte to endPosition Byte 
  Header header_size = new BasicHeader("Range", "bytes=" + beginPosition + "-" + endPosition);
  request.addHeader(header_size);

The above is the principle of multi-threaded download, but to realize breakpoint resumption, it is necessary to record the downloaded size of each thread after each pause, and start downloading from the location after the last download when continuing downloading next time. 1 projects will be stored in the database, I here for the sake of simplicity directly exists in SharedPreferences, has downloaded url and thread number as key value. 


@Override
  protected void onPostExecute(Long aLong) {
   Log.i(TAG, "download success ");
   // Download complete removal record 
   mSharedPreferences.edit().remove(currentThreadIndex).commit();
  }
  @Override
  protected void onCancelled() {
   Log.i(TAG, "download cancelled ");
   // Record downloaded size current
   mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
  }

When downloading, first get the downloaded location. If it has been downloaded, start downloading from the location after the last download: 


 // Get the information saved by the previous download and continue the download from the location where it ended before 
  // Judgment is added here file.exists() If the file has not been downloaded, but has been deleted by the user, the file is downloaded again 
  long downedPosition = mSharedPreferences.getLong(currentThreadIndex, 0);
  if(file.exists() && downedPosition != 0) {
   beginPosition = beginPosition + downedPosition;
   current = downedPosition;
   synchronized (mCurrentLength) {
    mCurrentLength += downedPosition;
   }
  }

2. Complete code 


package com.bbk.lling.multithreaddownload;
import android.app.Activity;
import android.content.Context;
import android.content.SharedPreferences;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.Environment;
import android.os.Handler;
import android.os.Message;
import android.util.Log;
import android.view.View;
import android.widget.ProgressBar;
import android.widget.TextView;
import android.widget.Toast;
import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicHeader;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.RandomAccessFile;
import java.net.MalformedURLException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
public class MainActivity extends Activity {
 private static final String TAG = "MainActivity";
 private static final int DEFAULT_POOL_SIZE = 5;
 private static final int GET_LENGTH_SUCCESS = 1;
 // Download path 
 private String downloadPath = Environment.getExternalStorageDirectory() +
   File.separator + "download";
// private String mUrl = "http://ftp.neu.edu.cn/mirrors/eclipse/technology/epp/downloads/release/juno/SR2/eclipse-java-juno-SR2-linux-gtk-x86_64.tar.gz";
 private String mUrl = "http://p.gdown.baidu.com/c4cb746699b92c9b6565cc65aa2e086552651f73c5d0e634a51f028e32af6abf3d68079eeb75401c76c9bb301e5fb71c144a704cb1a2f527a2e8ca3d6fe561dc5eaf6538e5b3ab0699308d13fe0b711a817c88b0f85a01a248df82824ace3cd7f2832c7c19173236";
 private ProgressBar mProgressBar;
 private TextView mPercentTV;
 SharedPreferences mSharedPreferences = null;
 long mFileLength = 0;
 Long mCurrentLength = 0L;
 private InnerHandler mHandler = new InnerHandler();
 // Create a thread pool 
 private Executor mExecutor = Executors.newCachedThreadPool();
 private List<DownloadAsyncTask> mTaskList = new ArrayList<DownloadAsyncTask>();
 @Override
 protected void onCreate(Bundle savedInstanceState) {
  super.onCreate(savedInstanceState);
  setContentView(R.layout.activity_main);
  mProgressBar = (ProgressBar) findViewById(R.id.progressbar);
  mPercentTV = (TextView) findViewById(R.id.percent_tv);
  mSharedPreferences = getSharedPreferences("download", Context.MODE_PRIVATE);
  // Start downloading 
  findViewById(R.id.begin).setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View v) {
    new Thread() {
     @Override
     public void run() {
      // Create a storage folder 
      File dir = new File(downloadPath);
      if (!dir.exists()) {
       dir.mkdir();
      }
      // Get the file size 
      HttpClient client = new DefaultHttpClient();
      HttpGet request = new HttpGet(mUrl);
      HttpResponse response = null;
      try {
       response = client.execute(request);
       mFileLength = response.getEntity().getContentLength();
      } catch (Exception e) {
       Log.e(TAG, e.getMessage());
      } finally {
       if (request != null) {
        request.abort();
       }
      }
      Message.obtain(mHandler, GET_LENGTH_SUCCESS).sendToTarget();
     }
    }.start();
   }
  });
  // Pause download 
  findViewById(R.id.end).setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View v) {
    for (DownloadAsyncTask task : mTaskList) {
     if (task != null && (task.getStatus() == AsyncTask.Status.RUNNING || !task.isCancelled())) {
      task.cancel(true);
     }
    }
    mTaskList.clear();
   }
  });
 }
 /**
  *  Start downloading 
  *  Calculate the download location of each thread according to the file size to be downloaded, and create AsyncTask
  */
 private void beginDownload() {
  mCurrentLength = 0L;
  mPercentTV.setVisibility(View.VISIBLE);
  mProgressBar.setProgress(0);
  long blockLength = mFileLength / DEFAULT_POOL_SIZE;
  for (int i = 0; i < DEFAULT_POOL_SIZE; i++) {
   long beginPosition = i * blockLength;// The starting position of each thread download 
   long endPosition = (i + 1) * blockLength;// End position of download for each thread 
   if (i == (DEFAULT_POOL_SIZE - 1)) {
    endPosition = mFileLength;// If the size of the entire file is not an integer multiple of the number of threads, the last 1 The end position of threads is the total length of the file 
   }
   DownloadAsyncTask task = new DownloadAsyncTask(beginPosition, endPosition);
   mTaskList.add(task);
   task.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR, mUrl, String.valueOf(i));
  }
 }
 /**
  *  Update progress bar 
  */
 synchronized public void updateProgress() {
  int percent = (int) Math.ceil((float)mCurrentLength / (float)mFileLength * 100);
//  Log.i(TAG, "downloading " + mCurrentLength + "," + mFileLength + "," + percent);
  if(percent > mProgressBar.getProgress()) {
   mProgressBar.setProgress(percent);
   mPercentTV.setText(" Download progress: " + percent + "%");
   if (mProgressBar.getProgress() == mProgressBar.getMax()) {
    Toast.makeText(MainActivity.this, " Download End ", Toast.LENGTH_SHORT).show();
   }
  }
 }
 @Override
 protected void onDestroy() {
  for(DownloadAsyncTask task: mTaskList) {
   if(task != null && task.getStatus() == AsyncTask.Status.RUNNING) {
    task.cancel(true);
   }
   mTaskList.clear();
  }
  super.onDestroy();
 }
 /**
  *  Downloaded AsyncTask
  */
 private class DownloadAsyncTask extends AsyncTask<String, Integer , Long> {
  private static final String TAG = "DownloadAsyncTask";
  private long beginPosition = 0;
  private long endPosition = 0;
  private long current = 0;
  private String currentThreadIndex;
  public DownloadAsyncTask(long beginPosition, long endPosition) {
   this.beginPosition = beginPosition;
   this.endPosition = endPosition;
  }
  @Override
  protected Long doInBackground(String... params) {
   Log.i(TAG, "downloading");
   String url = params[0];
   currentThreadIndex = url + params[1];
   if(url == null) {
    return null;
   }
   HttpClient client = new DefaultHttpClient();
   HttpGet request = new HttpGet(url);
   HttpResponse response = null;
   InputStream is = null;
   RandomAccessFile fos = null;
   OutputStream output = null;
   try {
    // Local file 
    File file = new File(downloadPath + File.separator + url.substring(url.lastIndexOf("/") + 1));
    // Get the information saved by the previous download and continue the download from the location where it ended before 
    // Judgment is added here file.exists() If the file has not been downloaded, but has been deleted by the user, the file is downloaded again 
    long downedPosition = mSharedPreferences.getLong(currentThreadIndex, 0);
    if(file.exists() && downedPosition != 0) {
     beginPosition = beginPosition + downedPosition;
     current = downedPosition;
     synchronized (mCurrentLength) {
      mCurrentLength += downedPosition;
     }
    }
    // Set the location of downloaded data beginPosition Byte to endPosition Byte 
    Header header_size = new BasicHeader("Range", "bytes=" + beginPosition + "-" + endPosition);
    request.addHeader(header_size);
    // Execute the request to get the download input stream 
    response = client.execute(request);
    is = response.getEntity().getContent();
    // Create a file output stream 
    fos = new RandomAccessFile(file, "rw");
    // From the file's size Start writing in the future position, but you don't need to write it directly. Sometimes multithreaded downloads require 
    fos.seek(beginPosition);
    byte buffer [] = new byte[1024];
    int inputSize = -1;
    while((inputSize = is.read(buffer)) != -1) {
     fos.write(buffer, 0, inputSize);
     current += inputSize;
     synchronized (mCurrentLength) {
      mCurrentLength += inputSize;
     }
     this.publishProgress();
     if (isCancelled()) {
      return null;
     }
    }
   } catch (MalformedURLException e) {
    Log.e(TAG, e.getMessage());
   } catch (IOException e) {
    Log.e(TAG, e.getMessage());
   } finally{
    try{
     /*if(is != null) {
      is.close();
     }*/
     if (request != null) {
      request.abort();
     }
     if(output != null) {
      output.close();
     }
     if(fos != null) {
      fos.close();
     }
    } catch(Exception e) {
     e.printStackTrace();
    }
   }
   return null;
  }
  @Override
  protected void onPreExecute() {
   Log.i(TAG, "download begin ");
   super.onPreExecute();
  }
  @Override
  protected void onProgressUpdate(Integer... values) {
   super.onProgressUpdate(values);
   // Update Interface Progress Bar 
   updateProgress();
  }
  @Override
  protected void onPostExecute(Long aLong) {
   Log.i(TAG, "download success ");
   // Download complete removal record 
   mSharedPreferences.edit().remove(currentThreadIndex).commit();
  }
  @Override
  protected void onCancelled() {
   Log.i(TAG, "download cancelled ");
   // Record downloaded size current
   mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
  }
  @Override
  protected void onCancelled(Long aLong) {
   Log.i(TAG, "download cancelled(Long aLong)");
   super.onCancelled(aLong);
   mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
  }
 }
 private class InnerHandler extends Handler {
  @Override
  public void handleMessage(Message msg) {
   switch (msg.what) {
    case GET_LENGTH_SUCCESS :
     beginDownload();
     break;
   }
   super.handleMessage(msg);
  }
 }
}

The layout file and the previous blog "AsyncTask Breakpoint Continued Transmission" layout file is a sample, so no code will be posted here.

The above code is available for pro-test, and hundreds of M large files are no problem.

3. Pits encountered

Problem description: When using the above code to download the file http://ftp. neu. edu. cn/mirrors/eclipse/technology/downloads/release/juno/SR2/eclipse-java-SR2-linux-x86_64.tar-x86_64.tar-x86_64. onCancel.

(true) is not executed to cancel the download task, and the location of the thread download is not recorded. And when you click Download again, each of the five Task executes only the onPreEexcute () method, not the doInBackground () method at all. Downloading other files does not have this problem.

This problem tossed me for a long time, and it didn't report any anomalies, and debugging couldn't come out. Look at the source code of AsyncTask, and there is no reason for stackoverflow. When I saw this website (https://groups.google.com/forum/#! topic/android-developers/B-oBiS7npfQ), I really thought it was an bug of AsyncTask.

After a hundred twists and turns, the problem actually appears in line 239 of the above code (annotated here). For some reason, when this sentence is executed, the thread will be blocked there, so doInBackground () method 1 will not end, and onCancel () method will certainly not be executed. At the same time, because the thread pool Executor is used, the number of threads is 5, and after clicking Cancel, all 5 threads are blocked, so when clicking Download again, only onPreEexcute () method is executed, and there are no idle threads to execute doInBackground () method. It's really a huge pit. . .

Although the problem is solved, why do some files download to is. close () and some threads block while others don't? This is still a mystery. If any great God knows what the reason is, please give me some advice!

Source download: https://github.com/liuling07/MultiTaskAndThreadDownload

Summarize

Related articles: