Pit filling records for displaying pictures and playing videos in Android ViewPager

Introduction to ViewPager

The function of ViewPager is to slide the view, just like Lanucher slides left and right.

ViewPager is used to realize the switching effect of multiple pages. This class exists in the compatibility package android-support-v4.jar of Google.

ViewPager:

1) The ViewPager class inherits directly from the ViewGroup class, so it is a container class in which other view classes can be added.

2) The ViewPager class requires an PagerAdapter adapter class to provide it with data.

3) ViewPager is commonly used with Fragment1, and special FragmentPagerAdapter and FragmentStatePagerAdapter classes are provided for use with ViewPager in Fragment.

4) When writing the application of ViewPager, we also need to use two component classes, namely PagerTitleStrip class and PagerTabStrip class. PagerTitleStrip class directly inherits from ViewGroup class, while PagerTabStrip class inherits PagerTitleStrip class, so these two classes are also container classes. One thing to note, however, is that when defining layout of XML, these two classes must be child tags of ViewPager tag, otherwise an error will occur.

This article will introduce in detail the related contents of displaying pictures and playing videos to fill holes in Android ViewPager, and share them for your reference and study. I won't say much below, let's take a look at the detailed introduction.

1. Requirement sources and implementation ideas

1. In recent project requirements, there are functions that need to play video and display pictures in ViewPager. Video is local video. The initial implementation idea is to initialize PhotoView and SurfaceView according to whether the current position of item corresponds to pictures or videos in ViewPager, and to judge the removal of PhotoView and SurfaceView according to the position of item during destruction.

2. The above method can be realized, but there are two problems. First, the life cycle of MediaPlayer is not easy to control and there is a memory leak problem. Second, when three consecutive item are videos, the bug of the last frame of the previous video will appear in the process of sliding back and forth.

3. The user experience is not improved. Before the initialization of the video player is completed, the first frame picture of the video will be covered. However, it is found that the first frame picture is inconsistent with the information of the first frame of the video, and the solution will be given through the code later.

4. How to fit the sizes of pictures and videos to ensure that they are not deformed.

2. Pits to be filled

1. The essential reason why the life cycle of MediaPlayer is not easy to control is that there is only one player in this implementation idea, and frequent initialization and destruction cause problems, so I changed the implementation mode later, and one video of item corresponds to one player.

2. For bug, which was found to appear in the last frame of the previous video during sliding, it was found that it was caused by surfaceView, and then the problem was perfectly solved by replacing the playing carrier with TextureView.

3. Essential similarities and differences between SurfaceView and TextureView

1: Both can be drawn and rendered in separate threads, greatly improving rendering performance in dedicated GPU threads.

2: SurfaceView provides an embedded view level drawing interface. Developers can control the form of the interface such as Size, which can ensure the correct position of the interface on the screen. But there are limitations:
1. Because it is an independent 1-layer View, it is more like an independent Window, and animation, translation and scaling cannot be added;
2. Two SurfaceView cannot overwrite each other.

3: Texture is more like View, which can be scaled, translated and animated like TextView. TextureView can only be used in Window with hardware acceleration turned on, and consumes more memory than SurfaceView with a delay of 1-3 frames.

4: When the screen is locked, SurfaceView will be destroyed and rebuilt, but TextureView will not!

3. Implement the core code concretely

1. Initialization of ViewPager

mAdapter = ImageBrowseFragmentPagerAdapter(supportFragmentManager, this, imgs)
 imgs_viewpager.offscreenPageLimit = 1
 imgs_viewpager.adapter = mAdapter
 imgs_viewpager.currentItem = mPosition


 // In order to deal with the problem of video playback when clicking for the first time, 
 val message = Message.obtain()
 message.what = START_PLAY_VIDEO
 mHandler.sendMessageDelayed(message, 200)

2. Handler processing messages

private val START_PLAY_VIDEO = 0
private var DELETE_VIDEO = 1
private var DELETE_VIDEO_START_PLAY = 2
private var mHandler = Handler(Handler.Callback { msg ->
  when (msg.what) {
   // Start playing video 
   START_PLAY_VIDEO -> NotifyDispatch.dispatch(PreviewPlayVideoEvent(mPosition))
   // Refresh when video is deleted ui
   DELETE_VIDEO -> {
    mAdapter?.setImgs(imgs)
   }
   // Resolve when deleting a video and then jump to another 1 A item, The problem of not continuing to play when it is video 
   DELETE_VIDEO_START_PLAY -> NotifyDispatch.dispatch(PreviewPlayVideoEvent(mDeletePosition))
  }
  true
 })

3. Delete the processing logic of video or picture

 private fun deletePhotos(position: Int) {
  if (imgs!!.isEmpty()) {
   return
  }
  ThreadDispatch.right_now.execute({
   var file: File?
   file = File(imgs.get(position))
   if (file != null && file?.exists()!!) {
    val intent = Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE)
    val uri = Uri.fromFile(file)
    intent.data = uri
    sendBroadcast(intent)
    file?.delete()
    imgs.removeAt(position)
   }


   if (position == imgs.size) {
    mDeletePosition = position - 1
   } else {
    mDeletePosition = position
   }
   val message = Message.obtain()
   message.what = DELETE_VIDEO
   mHandler.sendMessage(message)
   NotifyDispatch.dispatch(DeletePreviewPhotoEvent(imgs))
   val message1 = Message.obtain()
   message1.what = DELETE_VIDEO_START_PLAY
   mHandler.sendMessageDelayed(message1, 200)
   if (imgs.isEmpty()) {
    finish()
   }
  })
// }
 }

4. Adapter corresponding to ViewPager

package com.immomo.camerax.gui.view.adapter;
import android.content.Context;
import android.os.Bundle;
import android.support.v4.app.Fragment;
import android.support.v4.app.FragmentManager;
import android.support.v4.app.FragmentStatePagerAdapter;
import android.support.v4.app.FragmentTransaction;
import android.view.View;
import android.view.ViewGroup;
import com.immomo.camerax.gui.fragment.PreviewImgFragment;
import com.immomo.camerax.gui.fragment.PreviewVideoFragment;
import java.util.ArrayList;
import java.util.List;

/**
 * Created by liuxu on 2018/3/26.
 */

public class ImageBrowseFragmentPagerAdapter extends FragmentStatePagerAdapter {
 private Context mContext;
 private List<String> datas;
 private int mCurrentSelectedPosition = -1;
 private FragmentManager mFragmentManager;
 private FragmentTransaction mFragmentTransaction;
 private ArrayList<Fragment> mFragments = new ArrayList<>();

 public ImageBrowseFragmentPagerAdapter(FragmentManager fm, Context context, List<String> datas) {
  super(fm);
  mFragmentManager = fm;
  mContext = context;
  this.datas = datas;
 }

 public void removeContext(){
  mContext = null;
 }

 @Override
 public void setPrimaryItem(ViewGroup container, int position, Object object) {
  mCurrentSelectedPosition = position;
 }

 @Override
 public void startUpdate(ViewGroup container) {
  super.startUpdate(container);
 }

 public void setImgs(List<String> imgs) {
  this.datas = imgs;
  notifyDataSetChanged();
 }

 // Invalid processing update ---- Delete an entry 
 @Override
 public int getItemPosition(Object object) {
  return POSITION_NONE;
 }

 public int getPrimaryItemPosition() {
  return mCurrentSelectedPosition;
 }

 public ImageBrowseFragmentPagerAdapter(FragmentManager fm) {
  super(fm);
 }

 @Override
 public boolean isViewFromObject(View view, Object object) {
  return view == ((Fragment) object).getView();
 }

 @Override
 public Fragment getItem(int position) {
  Bundle bundle = new Bundle();
  bundle.putString("url", datas.get(position));
  bundle.putInt("position", position);
  if (datas.get(position).endsWith(".jpg")) {
   PreviewImgFragment previewImgFragment = new PreviewImgFragment();
   previewImgFragment.setArguments(bundle);
   return previewImgFragment;
  } else {
   PreviewVideoFragment previewVideoFragment = new PreviewVideoFragment();
   previewVideoFragment.setArguments(bundle);
   return previewVideoFragment;
  }
 }

 @Override
 public int getCount() {
  return datas == null ? 0 : datas.size();
 }

 @Override
 public Object instantiateItem(ViewGroup container, int position) {
  return super.instantiateItem(container,position);
 }

 @Override
 public void destroyItem(ViewGroup container, int position, Object object) {
  super.destroyItem(container,position,object);
 }
}

5 Display the corresponding Fragment of the picture

package com.immomo.camerax.gui.fragment;
import android.os.Bundle;
import android.support.annotation.Nullable;
import android.support.v4.app.Fragment;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import com.bumptech.glide.Glide;
import com.immomo.camerax.R;
import com.immomo.camerax.foundation.util.StatusBarUtils;
import com.immomo.camerax.gui.view.ResizablePhotoView;
/**
 * Created by liuxu on 2018/3/27.
 */
public class PreviewImgFragment extends Fragment {
 @Nullable
 @Override
 public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
  View view = inflater.inflate(R.layout.fragment_preview_photo, null);
  ResizablePhotoView resizablePhotoView = view.findViewById(R.id.customPhotoView);
  String url = getArguments().getString("url");
  Glide.with(getContext()).load(url).into(resizablePhotoView);
  resizablePhotoView.setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View view) {
    getActivity().finish();
   }
  });
  return view;
 }

 @Override
 public void onPause() {
  super.onPause();
 }

 @Override
 public void onResume() {
  super.onResume();
 }

 @Override
 public void onDetach() {
  super.onDetach();
 }

 @Override
 public void onDestroyView() {
  super.onDestroyView();
 }

 @Override
 public void onStart() {
  super.onStart();
 }

 @Override
 public void onDestroy() {
  super.onDestroy();
 }

 @Override
 public void setUserVisibleHint(boolean isVisibleToUser) {
  super.setUserVisibleHint(isVisibleToUser);
 }
}

6. Custom View for pictures to fit height according to width

package com.immomo.camerax.gui.view;
import android.content.Context;
import android.graphics.drawable.Drawable;
import android.util.AttributeSet;
import com.github.chrisbanes.photoview.PhotoView;

/**
 * Created by liuxu on 2018/4/7.
 */

public class ResizablePhotoView extends PhotoView {
 public ResizablePhotoView(Context context) {
  super(context);
 }

 public ResizablePhotoView(Context context, AttributeSet attr) {
  super(context, attr);
 }

 public ResizablePhotoView(Context context, AttributeSet attr, int defStyle) {
  super(context, attr, defStyle);
 }

 @Override
 protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
  Drawable d = getDrawable();
  if (d != null){
   int width = MeasureSpec.getSize(widthMeasureSpec);
   // The height is calculated by filling the screen with the width of the picture 
   int height = (int) Math.ceil((float) width * (float) d.getIntrinsicHeight() / (float) d.getIntrinsicWidth());
   setMeasuredDimension(width, height);
  }else {
   super.onMeasure(widthMeasureSpec, heightMeasureSpec);
  }
 }
}

7. Play the Fragment corresponding to the video

/**
 * Created by liuxu on 2018/3/27.
 */

public class PreviewVideoFragment extends Fragment {
 private ImageView mPhotoView;
 private TextureView mTextureView;
 private String mUrl;
 private int mPosition;
 private AndroidMediaPlayer mIjkVodMediaPlayer;
 private boolean mIsSelected;
 private boolean mIsFirstPrepared;
 private PreviewPlayVideoSubscriber mPreviewPlayVideoSubscriber = new PreviewPlayVideoSubscriber() {
  @Override
  public void onEventMainThread(PreviewPlayVideoEvent event) {
   super.onEventMainThread(event);
   MDLog.e("liuxu",event.getPosition()+"");
   if (event != null && event.getPosition() == mPosition) {
    // Description is the current entry 
    if (mIjkVodMediaPlayer != null && !mIjkVodMediaPlayer.isPlaying()) {
     if (mTextureView != null) {
      mIjkVodMediaPlayer.setSurface(mSurface);
      mIjkVodMediaPlayer.prepareAsync();
      mPhotoView.setVisibility(View.VISIBLE);
     }
    }
    mIsSelected = true;
   } else {
    if (mIjkVodMediaPlayer != null && mIjkVodMediaPlayer.isPlaying()) {
     mIjkVodMediaPlayer.pause();
     mIjkVodMediaPlayer.stop();
    }

    if (mPhotoView != null) {
     mPhotoView.setVisibility(View.VISIBLE);
    }
    mIsSelected = false;
   }

  }
 };
 private String mWidth;
 private String mHeight;

 @Nullable
 @Override
 public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
  mPreviewPlayVideoSubscriber.register();
  View view = inflater.inflate(R.layout.fragment_preview_video, null);
  mPhotoView = view.findViewById(R.id.photoView);
  mTextureView = view.findViewById(R.id.surfaceView);
  mUrl = getArguments().getString("url");
  mPosition = getArguments().getInt("position");
  layoutPlayer();
  loadVideoScreenshot(getContext(), mUrl, mPhotoView, 1);
  view.setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View view) {

    release();
    getActivity().finish();
   }
  });
  initTextureMedia();
  return view;
 }

 private void initTextureMedia() {
  mTextureView.setSurfaceTextureListener(mSurfaceTextureListener);
 }

 private void play(String url) {
  try {
   mIjkVodMediaPlayer = new AndroidMediaPlayer();
   mIjkVodMediaPlayer.reset();
   mIjkVodMediaPlayer.setDataSource(url);
   // Jean MediaPlayer And TextureView Combine video pictures 
   mIjkVodMediaPlayer.setSurface(mSurface);
   // Set up listening 
   mIjkVodMediaPlayer.setOnBufferingUpdateListener((mp, percent) -> {

   });
   mIjkVodMediaPlayer.setOnCompletionListener(mp -> {
    mp.seekTo(0);
    mp.start();
   });
   mIjkVodMediaPlayer.setOnInfoListener((mp1, what, extra) -> {
      if (what == IMediaPlayer.MEDIA_INFO_VIDEO_RENDERING_START) {
       mPhotoView.setVisibility(View.GONE);
       mIsFirstPrepared = true;
      }
      return false;
     });
   mIjkVodMediaPlayer.setOnErrorListener((mp, what, extra) -> false);
   mIjkVodMediaPlayer.setOnPreparedListener(mp -> {
    mp.start();
    if (!mIsFirstPrepared){
    }else {
     mPhotoView.setVisibility(View.GONE);
    }
   });
   mIjkVodMediaPlayer.setScreenOnWhilePlaying(true);// Keep the screen highlighted while the video is playing 
   if (mIsSelected){
    // Asynchronous preparation 
    mIjkVodMediaPlayer.prepareAsync();
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 private Surface mSurface;
 private TextureView.SurfaceTextureListener mSurfaceTextureListener = new TextureView.SurfaceTextureListener() {
  @Override
  public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
   mSurface = new Surface(surface);
   play(mUrl);
  }

  @Override
  public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {

  }

  @Override
  public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
   if (mSurface != null){
    mSurface.release();
    mSurface = null;
   }
   if (mTextureView != null){
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
     mTextureView.releasePointerCapture();
    }
   }
   release();
   return false;
  }
  @Override
  public void onSurfaceTextureUpdated(SurfaceTexture surface) {

  }
 };


 @Override
 public void onStart() {
  super.onStart();
 }
 @Override
 public void onAttach(Context context) {
  super.onAttach(context);
 }

 @Override
 public void onDetach() {
  super.onDetach();
 }

 @Override
 public void onPause() {
  MDLog.e("liuxu", "onPause" + mPosition);
  // Player stops playing while handling screen lock 
  if (mIjkVodMediaPlayer != null && mIjkVodMediaPlayer.isPlaying()){
   mIjkVodMediaPlayer.pause();
   mIjkVodMediaPlayer.stop();
  }
  super.onPause();
 }

 // Replay when the screen is open 
 @Override
 public void onResume() {
  MDLog.e("liuxu", "onResume" + mPosition);
  super.onResume();
  if (mIsSelected && mIjkVodMediaPlayer != null && !mIjkVodMediaPlayer.isPlaying()) {
   mIjkVodMediaPlayer.prepareAsync();
  }
 }

 @Override
 public void onDestroy() {
  MDLog.e("liuxu", "onDestroy");
  release();
  if (mPreviewPlayVideoSubscriber.isRegister()) {
   mPreviewPlayVideoSubscriber.unregister();
  }
  super.onDestroy();
 }

 private void release() {
  if (mIjkVodMediaPlayer == null) {
   return;
  }
  if (mIjkVodMediaPlayer.isPlaying()) {
   mIjkVodMediaPlayer.stop();
  }
  mIjkVodMediaPlayer.release();
  mIjkVodMediaPlayer = null;
 }

 @Override
 public boolean getUserVisibleHint() {
  return super.getUserVisibleHint();
 }

 /**
  *  Dynamic setting of video width and height information 
  */
 private void layoutPlayer() {
  // Get video aspect ratio 
  getPlayInfo(mUrl);
  float ratio = Float.parseFloat(mHeight) / Float.parseFloat(mWidth);
  MDLog.e("type", mPosition + "ratio" + ratio);
  int type = 0;
  // Add fault tolerant value 
  if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_1_1().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
    && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_1_1().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
   type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_11();
  } else if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_4_3().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
    && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_4_3().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
   type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_43();
   MDLog.e("type", "43");
  } else if (ratio < MediaConstants.INSTANCE.getASPECT_RATIO_16_9().toFloat() + MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()
    && ratio > MediaConstants.INSTANCE.getASPECT_RATIO_16_9().toFloat() - MediaConstants.INSTANCE.getSCREEN_DEFAULT_VALUE()) {
   type = ScreenAdapterUtils.INSTANCE.getSCALE_TYPE_169();
   MDLog.e("type", "169");
  }

  FrameLayout.LayoutParams layoutParams = (FrameLayout.LayoutParams) mTextureView.getLayoutParams();
  layoutParams.height = ScreenAdapterUtils.INSTANCE.getSurfaceHeight(type);
  mTextureView.setLayoutParams(layoutParams);
  FrameLayout.LayoutParams params = (FrameLayout.LayoutParams) mPhotoView.getLayoutParams();
  params.height = ScreenAdapterUtils.INSTANCE.getSurfaceHeight(type);
  mPhotoView.setLayoutParams(params);
  MDLog.e("params.height", params.height + "");
 }


 private void getPlayInfo(String mUri) {
  android.media.MediaMetadataRetriever mmr = new android.media.MediaMetadataRetriever();
  try {
   if (mUri != null) {
    mmr.setDataSource(mUri);
   } else {
    //mmr.setDataSource(mFD, mOffset, mLength);
   }

   // Width 
   mWidth = mmr.extractMetadata(android.media.MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH);
   // Gao 
   mHeight = mmr.extractMetadata(android.media.MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT);
//   mBitmap = mmr.getFrameAtTime(1 );

  } catch (Exception ex) {
  } finally {
   mmr.release();
  }

 }

 public static void loadVideoScreenshot(final Context context, String uri, ImageView imageView, long frameTimeMicros) {
  //  The time here is in microseconds 
  RequestOptions requestOptions = RequestOptions.frameOf(frameTimeMicros);
  requestOptions.set(FRAME_OPTION, MediaMetadataRetriever.OPTION_CLOSEST);
  requestOptions.transform(new BitmapTransformation() {
   @Override
   protected Bitmap transform(@NonNull BitmapPool pool, @NonNull Bitmap toTransform, int outWidth, int outHeight) {
    return toTransform;
   }

   @Override
   public void updateDiskCacheKey(MessageDigest messageDigest) {
    try {
     messageDigest.update((context.getPackageName() + "RotateTransform").getBytes("utf-8"));
    } catch (Exception e) {
     e.printStackTrace();
    }
   }
  });
  Glide.with(context).load(uri).apply(requestOptions).into(imageView);
 }
}

4. Conclusion

The author uses this way to achieve the project requirements, but because I contact with audio and video related content is less, all are constantly exploring and learning forward, if there are deficiencies, please comment and correct, thank you. Let's learn and make progress together.