Android uses AsyncTask to realize multithreaded breakpoint continuous transmission
- 2021-09-11 21:24:10
- OfStack
The previous blog "AsyncTask realizes breakpoint continuous transmission" explains how to realize breakpoint continuous transmission under single thread, that is, only one thread downloads a file.
For large files, multi-threaded downloading is 1% faster than single-threaded downloading. Multi-threaded downloads are a little more complicated than single-threaded downloads. This blog post will explain in detail how to use AsyncTask to implement multi-threaded breakpoint downloads.
1. Implementation principle
Multi-threaded downloading begins with the total number of downloading threads per file (I set 5 here) to determine where each thread is responsible for downloading.
long blockLength = mFileLength / DEFAULT_POOL_SIZE;
for (int i = 0; i < DEFAULT_POOL_SIZE; i++) {
long beginPosition = i * blockLength;// The starting position of each thread download
long endPosition = (i + 1) * blockLength;// End position of download for each thread
if (i == (DEFAULT_POOL_SIZE - 1)) {
endPosition = mFileLength;// If the size of the entire file is not an integer multiple of the number of threads, the last 1 The end position of threads is the total length of the file
}
......
}
It should be noted here that the file size is often not an integer multiple of the number of threads, so the end position of the last thread needs to be set to the file length.
After determining the download start and end location of each thread, you need to set the http request header to download the specified location of the file:
// Set the location of downloaded data beginPosition Byte to endPosition Byte
Header header_size = new BasicHeader("Range", "bytes=" + beginPosition + "-" + endPosition);
request.addHeader(header_size);
The above is the principle of multi-threaded download, but to realize breakpoint resumption, it is necessary to record the downloaded size of each thread after each pause, and start downloading from the location after the last download when continuing downloading next time. 1 projects will be stored in the database, I here for the sake of simplicity directly exists in SharedPreferences, has downloaded url and thread number as key value.
@Override
protected void onPostExecute(Long aLong) {
Log.i(TAG, "download success ");
// Download complete removal record
mSharedPreferences.edit().remove(currentThreadIndex).commit();
}
@Override
protected void onCancelled() {
Log.i(TAG, "download cancelled ");
// Record downloaded size current
mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
}
When downloading, first get the downloaded location. If it has been downloaded, start downloading from the location after the last download:
// Get the information saved by the previous download and continue the download from the location where it ended before
// Judgment is added here file.exists() If the file has not been downloaded, but has been deleted by the user, the file is downloaded again
long downedPosition = mSharedPreferences.getLong(currentThreadIndex, 0);
if(file.exists() && downedPosition != 0) {
beginPosition = beginPosition + downedPosition;
current = downedPosition;
synchronized (mCurrentLength) {
mCurrentLength += downedPosition;
}
}
2. Complete code
package com.bbk.lling.multithreaddownload;
import android.app.Activity;
import android.content.Context;
import android.content.SharedPreferences;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.Environment;
import android.os.Handler;
import android.os.Message;
import android.util.Log;
import android.view.View;
import android.widget.ProgressBar;
import android.widget.TextView;
import android.widget.Toast;
import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicHeader;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.RandomAccessFile;
import java.net.MalformedURLException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
public class MainActivity extends Activity {
private static final String TAG = "MainActivity";
private static final int DEFAULT_POOL_SIZE = 5;
private static final int GET_LENGTH_SUCCESS = 1;
// Download path
private String downloadPath = Environment.getExternalStorageDirectory() +
File.separator + "download";
// private String mUrl = "http://ftp.neu.edu.cn/mirrors/eclipse/technology/epp/downloads/release/juno/SR2/eclipse-java-juno-SR2-linux-gtk-x86_64.tar.gz";
private String mUrl = "http://p.gdown.baidu.com/c4cb746699b92c9b6565cc65aa2e086552651f73c5d0e634a51f028e32af6abf3d68079eeb75401c76c9bb301e5fb71c144a704cb1a2f527a2e8ca3d6fe561dc5eaf6538e5b3ab0699308d13fe0b711a817c88b0f85a01a248df82824ace3cd7f2832c7c19173236";
private ProgressBar mProgressBar;
private TextView mPercentTV;
SharedPreferences mSharedPreferences = null;
long mFileLength = 0;
Long mCurrentLength = 0L;
private InnerHandler mHandler = new InnerHandler();
// Create a thread pool
private Executor mExecutor = Executors.newCachedThreadPool();
private List<DownloadAsyncTask> mTaskList = new ArrayList<DownloadAsyncTask>();
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mProgressBar = (ProgressBar) findViewById(R.id.progressbar);
mPercentTV = (TextView) findViewById(R.id.percent_tv);
mSharedPreferences = getSharedPreferences("download", Context.MODE_PRIVATE);
// Start downloading
findViewById(R.id.begin).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
new Thread() {
@Override
public void run() {
// Create a storage folder
File dir = new File(downloadPath);
if (!dir.exists()) {
dir.mkdir();
}
// Get the file size
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(mUrl);
HttpResponse response = null;
try {
response = client.execute(request);
mFileLength = response.getEntity().getContentLength();
} catch (Exception e) {
Log.e(TAG, e.getMessage());
} finally {
if (request != null) {
request.abort();
}
}
Message.obtain(mHandler, GET_LENGTH_SUCCESS).sendToTarget();
}
}.start();
}
});
// Pause download
findViewById(R.id.end).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
for (DownloadAsyncTask task : mTaskList) {
if (task != null && (task.getStatus() == AsyncTask.Status.RUNNING || !task.isCancelled())) {
task.cancel(true);
}
}
mTaskList.clear();
}
});
}
/**
* Start downloading
* Calculate the download location of each thread according to the file size to be downloaded, and create AsyncTask
*/
private void beginDownload() {
mCurrentLength = 0L;
mPercentTV.setVisibility(View.VISIBLE);
mProgressBar.setProgress(0);
long blockLength = mFileLength / DEFAULT_POOL_SIZE;
for (int i = 0; i < DEFAULT_POOL_SIZE; i++) {
long beginPosition = i * blockLength;// The starting position of each thread download
long endPosition = (i + 1) * blockLength;// End position of download for each thread
if (i == (DEFAULT_POOL_SIZE - 1)) {
endPosition = mFileLength;// If the size of the entire file is not an integer multiple of the number of threads, the last 1 The end position of threads is the total length of the file
}
DownloadAsyncTask task = new DownloadAsyncTask(beginPosition, endPosition);
mTaskList.add(task);
task.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR, mUrl, String.valueOf(i));
}
}
/**
* Update progress bar
*/
synchronized public void updateProgress() {
int percent = (int) Math.ceil((float)mCurrentLength / (float)mFileLength * 100);
// Log.i(TAG, "downloading " + mCurrentLength + "," + mFileLength + "," + percent);
if(percent > mProgressBar.getProgress()) {
mProgressBar.setProgress(percent);
mPercentTV.setText(" Download progress: " + percent + "%");
if (mProgressBar.getProgress() == mProgressBar.getMax()) {
Toast.makeText(MainActivity.this, " Download End ", Toast.LENGTH_SHORT).show();
}
}
}
@Override
protected void onDestroy() {
for(DownloadAsyncTask task: mTaskList) {
if(task != null && task.getStatus() == AsyncTask.Status.RUNNING) {
task.cancel(true);
}
mTaskList.clear();
}
super.onDestroy();
}
/**
* Downloaded AsyncTask
*/
private class DownloadAsyncTask extends AsyncTask<String, Integer , Long> {
private static final String TAG = "DownloadAsyncTask";
private long beginPosition = 0;
private long endPosition = 0;
private long current = 0;
private String currentThreadIndex;
public DownloadAsyncTask(long beginPosition, long endPosition) {
this.beginPosition = beginPosition;
this.endPosition = endPosition;
}
@Override
protected Long doInBackground(String... params) {
Log.i(TAG, "downloading");
String url = params[0];
currentThreadIndex = url + params[1];
if(url == null) {
return null;
}
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
HttpResponse response = null;
InputStream is = null;
RandomAccessFile fos = null;
OutputStream output = null;
try {
// Local file
File file = new File(downloadPath + File.separator + url.substring(url.lastIndexOf("/") + 1));
// Get the information saved by the previous download and continue the download from the location where it ended before
// Judgment is added here file.exists() If the file has not been downloaded, but has been deleted by the user, the file is downloaded again
long downedPosition = mSharedPreferences.getLong(currentThreadIndex, 0);
if(file.exists() && downedPosition != 0) {
beginPosition = beginPosition + downedPosition;
current = downedPosition;
synchronized (mCurrentLength) {
mCurrentLength += downedPosition;
}
}
// Set the location of downloaded data beginPosition Byte to endPosition Byte
Header header_size = new BasicHeader("Range", "bytes=" + beginPosition + "-" + endPosition);
request.addHeader(header_size);
// Execute the request to get the download input stream
response = client.execute(request);
is = response.getEntity().getContent();
// Create a file output stream
fos = new RandomAccessFile(file, "rw");
// From the file's size Start writing in the future position, but you don't need to write it directly. Sometimes multithreaded downloads require
fos.seek(beginPosition);
byte buffer [] = new byte[1024];
int inputSize = -1;
while((inputSize = is.read(buffer)) != -1) {
fos.write(buffer, 0, inputSize);
current += inputSize;
synchronized (mCurrentLength) {
mCurrentLength += inputSize;
}
this.publishProgress();
if (isCancelled()) {
return null;
}
}
} catch (MalformedURLException e) {
Log.e(TAG, e.getMessage());
} catch (IOException e) {
Log.e(TAG, e.getMessage());
} finally{
try{
/*if(is != null) {
is.close();
}*/
if (request != null) {
request.abort();
}
if(output != null) {
output.close();
}
if(fos != null) {
fos.close();
}
} catch(Exception e) {
e.printStackTrace();
}
}
return null;
}
@Override
protected void onPreExecute() {
Log.i(TAG, "download begin ");
super.onPreExecute();
}
@Override
protected void onProgressUpdate(Integer... values) {
super.onProgressUpdate(values);
// Update Interface Progress Bar
updateProgress();
}
@Override
protected void onPostExecute(Long aLong) {
Log.i(TAG, "download success ");
// Download complete removal record
mSharedPreferences.edit().remove(currentThreadIndex).commit();
}
@Override
protected void onCancelled() {
Log.i(TAG, "download cancelled ");
// Record downloaded size current
mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
}
@Override
protected void onCancelled(Long aLong) {
Log.i(TAG, "download cancelled(Long aLong)");
super.onCancelled(aLong);
mSharedPreferences.edit().putLong(currentThreadIndex, current).commit();
}
}
private class InnerHandler extends Handler {
@Override
public void handleMessage(Message msg) {
switch (msg.what) {
case GET_LENGTH_SUCCESS :
beginDownload();
break;
}
super.handleMessage(msg);
}
}
}
The layout file and the previous blog "AsyncTask Breakpoint Continued Transmission" layout file is a sample, so no code will be posted here.
The above code is available for pro-test, and hundreds of M large files are no problem.
3. Pits encountered
Problem description: When using the above code to download the file http://ftp. neu. edu. cn/mirrors/eclipse/technology/downloads/release/juno/SR2/eclipse-java-SR2-linux-x86_64.tar-x86_64.tar-x86_64. onCancel.
(true) is not executed to cancel the download task, and the location of the thread download is not recorded. And when you click Download again, each of the five Task executes only the onPreEexcute () method, not the doInBackground () method at all. Downloading other files does not have this problem.
This problem tossed me for a long time, and it didn't report any anomalies, and debugging couldn't come out. Look at the source code of AsyncTask, and there is no reason for stackoverflow. When I saw this website (https://groups.google.com/forum/#! topic/android-developers/B-oBiS7npfQ), I really thought it was an bug of AsyncTask.
After a hundred twists and turns, the problem actually appears in line 239 of the above code (annotated here). For some reason, when this sentence is executed, the thread will be blocked there, so doInBackground () method 1 will not end, and onCancel () method will certainly not be executed. At the same time, because the thread pool Executor is used, the number of threads is 5, and after clicking Cancel, all 5 threads are blocked, so when clicking Download again, only onPreEexcute () method is executed, and there are no idle threads to execute doInBackground () method. It's really a huge pit. . .
Although the problem is solved, why do some files download to is. close () and some threads block while others don't? This is still a mystery. If any great God knows what the reason is, please give me some advice!
Source download: https://github.com/liuling07/MultiTaskAndThreadDownload
Summarize