Practice Guide for Camera development on Android platform

Article catalog

I. Camera Practice Guide

1.1 turn on the camera
1.2 turn off the camera
1.3 open Preview
1.4 close Preview
1.5 taking photos
1.6 start video recording
1.7 end video recording

II. Camera2 Practice Guide

2.1 turn on the camera
2.2 turn off the camera
2.3 open Preview
2.4 close Preview
2.5 taking photos
2.6 start video recording
2.7 end video recording

Android Camera API is also one of the most serious pieces of Android ecological fragmentation. First, Android has two sets of APIs, Camera below Android 5.0 and Camera 2 above Android 5.0. Moreover, when it is more serious, the support of Camera 2 from all mobile phone manufacturers is different, which leads us to spend a lot of energy in Camera development to deal with compatibility issues.

What is the general process of camera development? 🤔

1. Detect and access camera resource check whether camera resource exists in mobile phone, if so, request to access camera resource.
2. Create a preview interface, create a capture preview class that inherits from SurfaceView and implements the SurfaceHolder interface. With the shooting preview class, you can create a layout file, integrate the preview screen with the designed user interface controls, and display the preview image of the camera in real time.
3. Set up a photo listener, bind the listener to the user interface control, so that it can respond to the user's operation and start the photo taking process.
4. Take photos and save the files, convert the captured images into bitmap files, and finally output and save the pictures in various common formats.
5. Release camera resources. The camera is a shared resource. When the camera is used, it must be released correctly to avoid conflicts when other programs access and use it.

What should be paid attention to in camera development? 🤔

1. Version compatibility problem: Camera 2 is used for cameras under Android 5.0 and Android 5.0 or above, SurfaceView is used for Android 4.0 or above, TextureView is used for Android 4.0 or above, and camera and other runtime permissions are compatible for Android 6.0 or above.
2. For the compatibility of devices, the implementation and support of various features in Camera/Camera2 are different in some mobile phone manufacturers. This requires compatibility testing and a little bit of stepping on the ground.
3. The most common life cycle change problems in various scenes are background scenes and lock screen scenes. The application and release of camera resources in these two scenes and the creation and destruction of Surface will bring some problems, which will be analyzed in detail later.

About Camera/Camear2

Since to solve this compatibility problem, we need to use two sets in parallel. Is that a choice based on the version: Camera under Android 5.0, Camera 2 over Android 5.0? 🤔
In fact, this is not desirable. As mentioned earlier, different mobile phone manufacturers have different levels of support for Camera2. Even Android phones above 5.0 have very poor support for Camera2. At this time, we need to downgrade the use of camera. How to judge the level of support for camera will be discussed below.

About SurfaceView/TextureView

  • SurfaceView is a View with its own Surface. Interface rendering can be placed in a separate thread instead of the main thread. It's more like a Window, it can't do deformation and animation by itself.
  • TextureView also has its own Surface. But it can only be drawn in the Window with hardware acceleration layer. It is more like a normal View, which can do deformation and animation.

For more information about the difference between SurfaceView and TextureView, please refer to In this article, surface texture, TextureView, SurfaceView and GLSurfaceView in Android 5.0(Lollipop)

So how to select a solution for the version? 🤔

cameraview, the official open source library, gives a solution:

Since two sets are to be used together, it is necessary to define a unified interface, provide different implementations for different scenarios, and create different instances according to different scenarios.
It is not difficult to find that this interface generally needs to define the following functions:

  • Turn on camera
  • Turn off camera
  • Open Preview
  • close preview
  • photograph
  • Start video recording
  • End video recording

After defining the interface, we have an idea to implement the corresponding scheme according to the specific characteristics of the camera. Then another problem arises. In daily development, the camera generally exists as an SDK for each business party to call. How to design a highly customizable camera SDK with functions separated from the UI? 🤔
The answer is to use Fragment to encapsulate the corresponding functions of various click events (click to take photos, click to switch cameras, click to switch flash mode, etc.) in Fragment. When the business side uses Fragment, it can mask a layer of UI (of course, we need to provide a default implementation), so that the functions and UI can be separated and integrated very easily.
The camera SDK frame is as follows:

  • Camera activity: camera interface, mainly used to customize the UI. The actual function (click event) is completed by camera fragment.
  • CameraFragment: provide function interface to CameraActivity to complete the click event in CameraActivity, such as taking photos, recording videos, etc.
  • Camera lifecycle: deal with camera changes with Activity lifecycle, internal camera manager, camera initialization and release, preview creation and destruction, etc.
  • Camera Manager: the actual manager of the camera, who calls the camera API to operate the camera, take photos and record videos.
  • Camera/Camera2: camera API.

After understanding the overall architecture, we will analyze how to implement Camera/Camera2 for this architecture.

Camera Practice Guide

Camera API mainly involves the following key classes:

  • Camera: operate and manage camera resources, support camera resource switching, set preview and shooting size, set aperture, exposure and other related parameters.
  • SurfaceView: used to draw camera preview images and provide real-time preview images.
  • SurfaceHolder: an abstract interface used to control the Surface. It can control the size, format, pixel, etc. of the Surface and monitor the change of the Surface.
  • SurfaceHolder.Callback : interface for monitoring Surface state changes.

What's the difference between a surface View and a normal View? 🤔

The common View shares a Surface, and all the drawing is carried out in the UI thread. Because the UI thread has to deal with other logic, the update speed and drawing frame rate of the View cannot be guaranteed. This is obviously not suitable for camera real-time preview, so SurfaceView holds a separate Surface, which is responsible for managing the format, size and display position of the Surface. Its Surface painting is also carried out in a separate thread, so it has higher rendering efficiency and frame rate.

SurfaceHolder.Callback Three functions are defined in the interface:

  • surfaceCreated(SurfaceHolder holder); when the Surface is first created, it can be called in this method camera.open(), camera.setPreviewDisplay() to open the Camera and connect the Camera to the Surface
    Etc.
  • surfaceChanged(SurfaceHolder holder, int format, int width, int height); when the size and format of the Surface change, it can be called in this method camera.startPreview() open preview.
  • Surfacecreated (surfaceholder holder); when the Surface is destroyed, it can be called in this method camera.stopPreview(), camera.release() and other methods to realize end preview and release

1.1 turn on the camera

Before opening the camera, we need to obtain the relevant information of the system camera.

 //How many cameras are there
 numberOfCameras = Camera.getNumberOfCameras();

for (int i = 0; i < numberOfCameras; ++i) {
    final Camera.CameraInfo cameraInfo = new Camera.CameraInfo();

    Camera.getCameraInfo(i, cameraInfo);
    //Rear camera
    if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_BACK) {
        faceBackCameraId = i;
        faceBackCameraOrientation = cameraInfo.orientation;
    } else if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
    	//front camera 
        faceFrontCameraId = i;
        faceFrontCameraOrientation = cameraInfo.orientation;
    }
}

Knowing the camera information, you can open the camera through the camera ID.

camera = Camera.open(cameraId);

In addition, after you turn on the Camera, you will get a camera object, from which you can get and set various parameters of the camera.

//Get camera parameters
camera.getParameters();
//Set camera parameters
camera.getParameters();

Common parameters are as follows.
Flash configuration parameters, which can be accessed through Parameters.getFlashMode() interface get.

  • Camera.Parameters.FLASH_MODE_AUTO auto mode, when the light is dark, the flash will be turned on automatically;
  • Camera.Parameters.FLASH_MODE_OFF turns off the flash;
  • Camera.Parameters.FLASH_ MODE_ Flash when taking pictures on;
  • Camera.Parameters.FLASH_MODE_RED_EYE flash parameters, anti red eye mode.

Focus mode configuration parameters can be set through Parameters.getFocusMode() interface get.

  • Camera.Parameters.FOCUS_MODE_AUTO auto focus mode, special mode for small white photography;
  • Camera.Parameters.FOCUS_MODE_FIXED focus mode of fixed fixed, shooting old driver mode;
  • Camera.Parameters.FOCUS_MODE_EDOF depth of field mode is the favorite mode for young women of literature and art;
  • Camera.Parameters.FOCUS_MODE_INFINITY vision mode, shooting mode of scenic area;
  • Camera.Parameters.FOCUS_MODE_MACRO micro focus mode, special mode for photographing small flowers, grass and ants;

Scene mode configuration parameters can be configured through Parameters.getSceneMode() interface get.

  • Camera.Parameters.SCENE_MODE_BARCODE scanning scenario, NextQRCode project will judge and set as this scenario;
  • Camera.Parameters.SCENE_MODE_ACTION action scene is used to capture fast athletes, cars and other scenes;
  • Camera.Parameters.SCENE_MODE_AUTO automatically selects the scene;
  • Camera.Parameters.SCENE_MODE_HDR high dynamic contrast scene is usually used to take photos with clear light and shade, such as sunset;
  • Camera.Parameters.SCENE_MODE_NIGHT scene;

1.2 turn off the camera

It's easy to turn off the camera, just release it.

camera.release();

1.3 open Preview

When previewing Camera, it is done through SurfaceHolder of SurfaceView. First, it is done through, specifically:

private void startPreview(SurfaceHolder surfaceHolder) {
    try {
        final Camera.CameraInfo cameraInfo = new Camera.CameraInfo();
        Camera.getCameraInfo(currentCameraId, cameraInfo);
        int cameraRotationOffset = cameraInfo.orientation;

        //Get camera parameters
        final Camera.Parameters parameters = camera.getParameters();
        //Set focus mode
        setAutoFocus(camera, parameters);
        //Set flash mode
        setFlashMode(mCameraConfigProvider.getFlashMode());

        if (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_PHOTO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)
            turnPhotoCameraFeaturesOn(camera, parameters);
        else if (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_PHOTO)
            turnVideoCameraFeaturesOn(camera, parameters);

        final int rotation = ((WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();
        int degrees = 0;
        switch (rotation) {
            case Surface.ROTATION_0:
                degrees = 0;
                break; // Natural orientation
            case Surface.ROTATION_90:
                degrees = 90;
                break; // Landscape left
            case Surface.ROTATION_180:
                degrees = 180;
                break;// Upside down
            case Surface.ROTATION_270:
                degrees = 270;
                break;// Landscape right
        }

        //Set the preview direction according to the difference between the front camera and the rear camera, otherwise the preview image will reverse.
        if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
            displayRotation = (cameraRotationOffset + degrees) % 360;
            displayRotation = (360 - displayRotation) % 360; // compensate
        } else {
            displayRotation = (cameraRotationOffset - degrees + 360) % 360;
        }
        this.camera.setDisplayOrientation(displayRotation);

        if (Build.VERSION.SDK_INT > 13
                && (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_VIDEO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)) {
//                parameters.setRecordingHint(true);
        }

        if (Build.VERSION.SDK_INT > 14
                && parameters.isVideoStabilizationSupported()
                && (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_VIDEO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)) {
            parameters.setVideoStabilization(true);
        }

        //Set preview size
        parameters.setPreviewSize(previewSize.getWidth(), previewSize.getHeight());
        parameters.setPictureSize(photoSize.getWidth(), photoSize.getHeight());

        //Set camera parameters
        camera.setParameters(parameters);
        //Set surfaceHolder
        camera.setPreviewDisplay(surfaceHolder);
        //Open Preview
        camera.startPreview();

    } catch (IOException error) {
        Log.d(TAG, "Error setting camera preview: " + error.getMessage());
    } catch (Exception ignore) {
        Log.d(TAG, "Error starting camera preview: " + ignore.getMessage());
    }
}

1.4 close Preview

Close preview is very simple, call directly camera.stopPreview() is enough.

camera.stopPreview();

1.5 taking photos

This is done by calling Camera's takePicture() method,

takePicture(ShutterCallback shutter, PictureCallback raw, PictureCallback postview, PictureCallback jpeg)

The method has four parameters:

  • ShutterCallback shutter: it is recalled at the moment of taking a picture. Here, you can usually play the sound effect of "clicking" for taking a picture.
  • PictureCallback raw: returns uncompressed image data.
  • PictureCallback postview: returns image data of type postview
  • PictureCallback jpeg: returns JPEG compressed image data.

We usually use the last one to implement the last PictureCallback.

camera.takePicture(null, null, new Camera.PictureCallback() {
        @Override
        public void onPictureTaken(byte[] bytes, Camera camera) {
            //Store the returned image data
            final File pictureFile = outputPath;
            if (pictureFile == null) {
                Log.d(TAG, "Error creating media file, check storage permissions.");
                return;
            }
            try {
                FileOutputStream fileOutputStream = new FileOutputStream(pictureFile);
                fileOutputStream.write(bytes);
                fileOutputStream.close();
            } catch (FileNotFoundException error) {
                Log.e(TAG, "File not found: " + error.getMessage());
            } catch (IOException error) {
                Log.e(TAG, "Error accessing file: " + error.getMessage());
            } catch (Throwable error) {
                Log.e(TAG, "Error saving file: " + error.getMessage());
            }
        }
 });

Call if you want to continue taking photos after taking photos camera.startPreview() continue to open preview, otherwise close preview and release camera resources.

1.6 start video recording

The video is recorded through MediaRecorder.

if (prepareVideoRecorder()) {
            mediaRecorder.start();
            isVideoRecording = true;
            uiHandler.post(new Runnable() {
                @Override
                public void run() {
                    videoListener.onVideoRecordStarted(videoSize);
                }
            });
}

MediaRecorder is mainly used to record audio and video. Before use, initialization and related parameter settings are required, as shown below:

protected boolean preparemediaRecorder() {
    mediaRecorder = new MediaRecorder();
    try {
        mediaRecorder.setAudioSource(MediaRecorder.AudioSource.CAMCORDER);
        mediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
        
        //Output format
        mediaRecorder.setOutputFormat(camcorderProfile.fileFormat);
        //Video frame rate
        mediaRecorder.setVideoFrameRate(camcorderProfile.videoFrameRate);
        //Video size
        mediaRecorder.setVideoSize(videoSize.getWidth(), videoSize.getHeight());
        //Video bitrate
        mediaRecorder.setVideoEncodingBitRate(camcorderProfile.videoBitRate);
        //Video encoder
        mediaRecorder.setVideoEncoder(camcorderProfile.videoCodec);
        
        //Audio coding rate
        mediaRecorder.setAudioEncodingBitRate(camcorderProfile.audioBitRate);
        //Audio Channel 
        mediaRecorder.setAudioChannels(camcorderProfile.audioChannels);
        //Audio sampling rate
        mediaRecorder.setAudioSamplingRate(camcorderProfile.audioSampleRate);
        //Audio encoder
        mediaRecorder.setAudioEncoder(camcorderProfile.audioCodec);
        
        File outputFile = outputPath;
        String outputFilePath = outputFile.toString();
        //Output path
        mediaRecorder.setOutputFile(outputFilePath);
        
        //Set the maximum size of video output
        if (mCameraConfigProvider.getVideoFileSize() > 0) {
            mediaRecorder.setMaxFileSize(mCameraConfigProvider.getVideoFileSize());
            mediaRecorder.setOnInfoListener(this);
        }
        
        //Set the maximum duration of video output
        if (mCameraConfigProvider.getVideoDuration() > 0) {
            mediaRecorder.setMaxDuration(mCameraConfigProvider.getVideoDuration());
            mediaRecorder.setOnInfoListener(this);
        }
        mediaRecorder.setOrientationHint(getVideoOrientation(mCameraConfigProvider.getSensorPosition()));
        
        //get ready
        mediaRecorder.prepare();

        return true;
    } catch (IllegalStateException error) {
        Log.e(TAG, "IllegalStateException preparing MediaRecorder: " + error.getMessage());
    } catch (IOException error) {
        Log.e(TAG, "IOException preparing MediaRecorder: " + error.getMessage());
    } catch (Throwable error) {
        Log.e(TAG, "Error during preparing MediaRecorder: " + error.getMessage());
    }
    releasemediaRecorder();
    return false;
}

It is worth mentioning that in the daily business, there are often requirements for the length or size of video shooting. This can be achieved through mediaRecorder.setOnInfoListener() to process, OnInfoListener will listen to the video being recorded, and then we
It can be handled in its callback method.

   @Override
public void onInfo(MediaRecorder mediaRecorder, int what, int extra) {
    if (MediaRecorder.MEDIA_RECORDER_INFO_MAX_DURATION_REACHED == what) {
        //Maximum time to reach
    } else if (MediaRecorder.MEDIA_RECORDER_INFO_MAX_FILESIZE_REACHED == what) {
        //Maximum size reached
    }
}

For more information about MediaRecorder, please refer to the official MediaRecorder documentation.

1.7 end video recording

The end of video recording is also very simple, just call mediaRecorder.stop() method.

mediaRecorder.stop();

In addition, if you no longer use the camera, also release the camera resources.

The above is the whole content of Camera2, which is relatively simple. Next, we will talk about the relevant content of Camera2, and pay attention to the difference between the two.

II. Camera2 Practice Guide

Camera2 API mainly involves the following key classes:

  • CameraManager: camera manager for turning system cameras on and off
  • CameraCharacteristics: to describe various features of the camera, we can get them through the getCameraCharacteristics(@NonNull String cameraId) method of CameraManager.
  • Camera device: describes the system camera, similar to the early camera.
  • CameraCaptureSession: Session class. When you need to take photos, preview and other functions, you need to create an instance of this class first, and then control it through the methods in the instance (for example: photo capture()).
  • CaptureRequest: describes an operation request. The CaptureRequest parameter needs to be passed in for photo taking, preview and other operations. The specific parameter control is also set through the member variable of CameraRequest.
  • CaptureResult: describes the result after taking pictures.

Camera2 photo taking process is as follows:

The developer creates a Capture request to the camera hair by creating a CaptureRequest. These requests are arranged in a queue for the camera to process. The camera wraps the results in CaptureMetadata and returns them to the developer. The whole process is established in a session of CameraCaptureSession.

2.1 turn on the camera

Before opening the camera, we first need to obtain the CameraManager, then obtain the camera list, and then obtain the parameters of each camera (mainly front camera and rear camera).

mCameraManager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
try {
    final String[] ids = mCameraManager.getCameraIdList();
    numberOfCameras = ids.length;
    for (String id : ids) {
        final CameraCharacteristics characteristics = mCameraManager.getCameraCharacteristics(id);

        final int orientation = characteristics.get(CameraCharacteristics.LENS_FACING);
        if (orientation == CameraCharacteristics.LENS_FACING_FRONT) {
            faceFrontCameraId = id;
            faceFrontCameraOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
            frontCameraCharacteristics = characteristics;
        } else {
            faceBackCameraId = id;
            faceBackCameraOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
            backCameraCharacteristics = characteristics;
        }
    }
} catch (Exception e) {
    Log.e(TAG, "Error during camera initialize");
}

Camera2 has the same concept of cameraId as camera. We use the mCameraManager.getCameraIdList() to get the cameraId list, and then use the mCameraManager.getCameraCharacteristics(id)
Get the parameters of each id corresponding to the camera.

The parameters in CameraCharacteristics are mainly used as follows:

  • LENS_FACING: LENS_FACING_FRONT or LENS_FACING_BACK).
  • SENSOR_ORIENTATION: camera direction.
  • FLASH_INFO_AVAILABLE: whether flash is supported.
  • C ameraCharacteristics.INFO_ SUPPORTED_ HARDWARE_ Level: gets the camera properties supported by the current device.

Note: in fact, on Android devices of various manufacturers, various features of Camera2 are not available, so it is necessary to pass the characteristics.get (C ameraCharacteristics.INFO_ SUPPORTED_ HARDWARE_ Level) method to obtain the level of support according to the return value, specifically:

  • INFO_SUPPORTED_HARDWARE_LEVEL_FULL: full hardware support, allowing manual control of Full HD camera, continuous shooting mode and other new features.
  • INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED: limited support. This needs to be queried separately.
  • INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY: all devices will support, which is consistent with the features supported by the outdated Camera API.

Use this INFO_SUPPORTED_HARDWARE_LEVEL parameter, we can judge whether to use camera or Camera2. The specific method is as follows:

@TargetApi(Build.VERSION_CODES.LOLLIPOP)
public static boolean hasCamera2(Context mContext) {
    if (mContext == null) return false;
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) return false;
    try {
        CameraManager manager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
        String[] idList = manager.getCameraIdList();
        boolean notFull = true;
        if (idList.length == 0) {
            notFull = false;
        } else {
            for (final String str : idList) {
                if (str == null || str.trim().isEmpty()) {
                    notFull = false;
                    break;
                }
                final CameraCharacteristics characteristics = manager.getCameraCharacteristics(str);

                final int supportLevel = characteristics.get(CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
                if (supportLevel == CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY) {
                    notFull = false;
                    break;
                }
            }
        }
        return notFull;
    } catch (Throwable ignore) {
        return false;
    }
}

For more parameters of ameraCharacteristics, see Official document of CameraCharacteristics

The main call to open the camera is mCameraManager.openCamera (currentnameraid, statecallback, backgroundhandler) method, as you can see, has three parameters:

  • String cameraId: unique ID of the camera.
  • CameraDevice.StateCallback Callback: relevant callback of camera opening.
  • Handler handler: StateCallback requires the handler to be called. Generally, we can use the handler of the current thread.
 mCameraManager.openCamera(currentCameraId, stateCallback, backgroundHandler);

We mentioned above CameraDevice.StateCallback , which is a callback of camera opening. It defines various callback methods such as opening, closing and error. We can
These callback methods do the corresponding operations.

private CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(@NonNull CameraDevice cameraDevice) {
        //Get CameraDevice
        mcameraDevice = cameraDevice;
    }

    @Override
    public void onDisconnected(@NonNull CameraDevice cameraDevice) {
        //Close CameraDevice
        cameraDevice.close();

    }

    @Override
    public void onError(@NonNull CameraDevice cameraDevice, int error) {
        //Close CameraDevice
        cameraDevice.close();
    }
};

2.2 turn off the camera

As described above, closing is simple.

//Close CameraDevice
cameraDevice.close();

2.3 open Preview

Camera2 is called by creating a request session, specifically:

1. Call mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW) method to create CaptureRequest
2. Call
mCameraDevice.createCaptureSession() method to create a CaptureSession.

CaptureRequest.Builder createCaptureRequest(@RequestTemplate int templateType)

The parameter templateType in the createCaptureRequest() method represents the request type, which is divided into six types, namely:

  • TEMPLATE_PREVIEW: request to create Preview
  • TEMPLATE_STILL_CAPTURE: create a request suitable for still image capture with image quality prior to frame rate.
  • TEMPLATE_RECORD: request to create a video recording
  • TEMPLATE_VIDEO_SNAPSHOT: request for screenshot when creating video recording
  • TEMPLATE_ZERO_SHUTTER_LAG: creates a request for zero shutter delay. Maximize image quality without affecting the preview frame rate.
  • TEMPLATE_MANUAL: create a basic capture request in which all automatic controls are disabled (auto exposure, auto white balance, auto focus).
createCaptureSession(@NonNull List<Surface> outputs, @NonNull CameraCaptureSession.StateCallback callback, @Nullable Handler handler)

The createCaptureSession() method contains three parameters:

  • List outputs: the list of surfaces we need to output to.
  • CameraCaptureSession.StateCallback Callback: session state related callback.
  • Handler handler: there can be multiple callbacks (from different threads). This handler is used to distinguish which callback should be called back. Generally, the handler of the current thread can be written.

about CameraCaptureSession.StateCallback Callback method in:

  • Onconfigured (@ nonnull cameracapturesession); after the camera is configured, it can process the Capture request.
  • Onconfigurefailed (@ nonnull cameracapturesession); camera configuration failed
  • Onready (@ nonnull cameracapturesession); the camera is in a ready state, and currently there is no request to process.
  • Onactive (@ nonnull cameracapturesession); the camera is processing the request.
  • Onclosed (@ nonnull cameracapturesession); session closed
  • Onsurfaceprepared (@ nonnull cameracapturesession, @ nonnull surface); surface ready

With this in mind, it's easy to create a preview request.

previewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
previewRequestBuilder.addTarget(workingSurface);

//Note that in addition to the preview Surface, we have also added imageReader.getSurface() it is responsible for obtaining data after taking photos
mCameraDevice.createCaptureSession(Arrays.asList(workingSurface, imageReader.getSurface()),
        new CameraCaptureSession.StateCallback() {
            @Override
            public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                cameraCaptureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler);
            }

            @Override
            public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                Log.d(TAG, "Fail while starting preview: ");
            }
        }, null);

As you can see, in onConfigured(), the cameraCaptureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler), so we can continue to preview.

Note: we added above imageReader.getSurface() it is used to get data after taking photos. The specific operation is to set an OnImageAvailableListener for the ImageReader, and then set the OnImageAvailableListener in its onImageAvailable()
Method.

mImageReader.setOnImageAvailableListener(mOnImageAvailableListener, mBackgroundHandler);

private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
            = new ImageReader.OnImageAvailableListener() {

        @Override
        public void onImageAvailable(ImageReader reader) {
            //Take pictures and save them when they are available
            mBackgroundHandler.post(new ImageSaver(reader.acquireNextImage(), mFile));
        }

 };

2.4 close Preview

To close preview is to close the current preview session. In combination with the content of opening preview above, the specific implementation is as follows:

if (captureSession != null) {
    captureSession.close();
    try {
        captureSession.abortCaptures();
    } catch (Exception ignore) {
    } finally {
        captureSession = null;
    }
}

2.5 taking photos

There are three steps to take photos:

1. Focus

try {
    //Camera focus
    previewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_START);
    //modify state
    previewState = STATE_WAITING_LOCK;
    //Send focus request
    captureSession.capture(previewRequestBuilder.build(), captureCallback, backgroundHandler);
} catch (Exception ignore) {
}

We defined a CameraCaptureSession.CaptureCallback To process the results returned by the focus request.

private CameraCaptureSession.CaptureCallback captureCallback = new CameraCaptureSession.CaptureCallback() {

    @Override
    public void onCaptureProgressed(@NonNull CameraCaptureSession session,
                                    @NonNull CaptureRequest request,
                                    @NonNull CaptureResult partialResult) {
    }

    @Override
    public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                   @NonNull CaptureRequest request,
                                   @NonNull TotalCaptureResult result) {
            //Waiting for focus
            final Integer afState = result.get(CaptureResult.CONTROL_AF_STATE);
            if (afState == null) {
                //Focus failed, take a picture directly
                captureStillPicture();
            } else if (CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED == afState
                    || CaptureResult.CONTROL_AF_STATE_NOT_FOCUSED_LOCKED == afState
                    || CaptureResult.CONTROL_AF_STATE_INACTIVE == afState
                    || CaptureResult.CONTROL_AF_STATE_PASSIVE_SCAN == afState) {
                Integer aeState = result.get(CaptureResult.CONTROL_AE_STATE);
                if (aeState == null ||
                        aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
                    previewState = STATE_PICTURE_TAKEN;
                    //Focus complete, take photos
                    captureStillPicture();
                } else {
                    runPreCaptureSequence();
                }
            }
    }
};

2. Take photos

We defined a captureStillPicture() to take photos.

private void captureStillPicture() {
    try {
        if (null == mCameraDevice) {
            return;
        }
        
        //Build CaptureRequest to take photos
        final CaptureRequest.Builder captureBuilder =
                mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
        captureBuilder.addTarget(imageReader.getSurface());

        //Use the same AR and AF modes as previews
        captureBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
        //Set direction
        captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, getPhotoOrientation(mCameraConfigProvider.getSensorPosition()));

        //Create session
        CameraCaptureSession.CaptureCallback CaptureCallback = new CameraCaptureSession.CaptureCallback() {
            @Override
            public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                           @NonNull CaptureRequest request,
                                           @NonNull TotalCaptureResult result) {
                Log.d(TAG, "onCaptureCompleted: ");
            }
        };
        //Stop continuous view
        captureSession.stopRepeating();
        //Capture photos
        captureSession.capture(captureBuilder.build(), CaptureCallback, null);

    } catch (CameraAccessException e) {
        Log.e(TAG, "Error during capturing picture");
    }
}

3. Defocus

After taking the picture, we also need to unlock the camera focus to restore the camera to the preview state.

try {
    //Reset autofocus
    previewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_CANCEL);
    captureSession.capture(previewRequestBuilder.build(), captureCallback, backgroundHandler);
    //Camera returns to normal preview state
    previewState = STATE_PREVIEW;
    //Turn on continuous viewfinder mode
    captureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler);
} catch (Exception e) {
    Log.e(TAG, "Error during focus unlocking");
}

2.6 start video recording

//Close the preview first, because you need to add a Surface to preview the output, that is mediaRecorder.getSurface()
closePreviewSession();

//Initialize MediaRecorder and set relevant parameters
if (preparemediaRecorder()) {

    final SurfaceTexture texture = Camera2Manager.this.texture;
    texture.setDefaultBufferSize(videoSize.getWidth(), videoSize.getHeight());

    try {
        //Build video recording aptureRequest
        previewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
        final List<Surface> surfaces = new ArrayList<>();

        //Set preview Surface
        final Surface previewSurface = workingSurface;
        surfaces.add(previewSurface);
        previewRequestBuilder.addTarget(previewSurface);

        //Set preview output Surface
        workingSurface = mediaRecorder.getSurface();
        surfaces.add(workingSurface);
        previewRequestBuilder.addTarget(workingSurface);

        mCameraDevice.createCaptureSession(surfaces, new CameraCaptureSession.StateCallback() {
            @Override
            public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                captureSession = cameraCaptureSession;

                previewRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
                try {
                    //Continue to send Capture requests for real-time preview.
                    captureSession.setRepeatingRequest(previewRequestBuilder.build(), null, backgroundHandler);
                } catch (Exception e) {
                }

                try {
                    //Start recording
                    mediaRecorder.start();
                } catch (Exception ignore) {
                    Log.e(TAG, "mediaRecorder.start(): ", ignore);
                }

                isVideoRecording = true;

                uiHandler.post(new Runnable() {
                    @Override
                    public void run() {
                        cameraVideoListener.onVideoRecordStarted(videoSize);
                    }
                });
            }

            @Override
            public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                Log.d(TAG, "onConfigureFailed");
            }
        }, backgroundHandler);
    } catch (Exception e) {
        Log.e(TAG, "startVideoRecord: ", e);
    }
}

When we talked about Camera on the media recorder, we already said that we will not repeat it here.
The above is the whole content of video recording, that is, simple API use, or relatively simple.

2.7 end video recording

The end of video recording is mainly to close the session and release some resources, specifically:

1. Close preview session
2. Stop mediaRecorder
3. Release mediaRecorder

//Close preview session
if (captureSession != null) {
    captureSession.close();
    try {
        captureSession.abortCaptures();
    } catch (Exception ignore) {
    } finally {
        captureSession = null;
    }
}

//Stop mediaRecorder
if (mediaRecorder != null) {
    try {
        mediaRecorder.stop();
    } catch (Exception ignore) {
    }
}

//Release mediaRecorder
try {
    if (mediaRecorder != null) {
        mediaRecorder.reset();
        mediaRecorder.release();
    }
} catch (Exception ignore) {

} finally {
    mediaRecorder = null;
}

The above is the relevant content of Camera/Camera2 practice.

Reprint Su CE's article and share how to develop Camera in Android

Tags: Android Session SurfaceView Mobile

Posted on Mon, 08 Jun 2020 22:51:00 -0400 by gelwa