The relationship among img, bitmap and mat in auto JS image retrieval

Yashu tutorial is simple and easy to understand

There are three data representations of autojs pictures

  • img: com.stardust.autojs.core.image.ImageWrapper
  • bitmap
  • mat

The three image data formats are converted to each other

img to bitmap and mat

img.bitmap
img.mat

bitmap to img

com.stardust.autojs.core.image.ImageWrapper.ofMat(bitmap)

mat to img

com.stardust.autojs.core.image.ImageWrapper.ofMat(mat)

bitmap to mat

org.opencv.android.Utils.bitmapToMat(bitmap, mat);

mat to bitmap

org.opencv.android.Utils.matToBitmap(mat, bitmap);

Will bitmap and mat be affected when img is recycled

Common function

// Print picture information
function printInfo(tag, img, bitmap, mat) {
  let arr = ["\n"];
  arr.push("tag = " + tag);
  try {
    arr.push(java.lang.String.format("%1$-9s: width = %2$s, height = %3$s", "img", img.width, img.height));
  } catch (e) {
    arr.push(java.lang.String.format("%1$-9s: %2$ss", "img", e));
  }
  arr.push(
    java.lang.String.format(
      "%1$-9s: width = %2$s, height = %3$s, allocationByteCount = %4$s",
      "bitmap",
      bitmap.width,
      bitmap.height,
      bitmap.allocationByteCount
    )
  );
  arr.push(java.lang.String.format("%1$-9s: width = %2$s, height = %3$s", "mat", mat.width(), mat.height()));
  log(arr.join("\n"));
}
// View img
function viewImg(img) {
  images.save(img, "/sdcard/1.jpg", "jpg", 50);
  app.viewFile("/sdcard/1.jpg");
}
// View mat
function viewMat(mat) {
  let mat2 = mat.clone();
  Imgproc.cvtColor(mat, mat2, Imgproc.COLOR_BGRA2RGBA);
  let tempFilePath = files.join(files.getSdcardPath(), "script", "mat.png");
  Imgcodecs.imwrite(tempFilePath, mat2);
  mat2.release();
  app.viewFile(tempFilePath);
}

Test code

let imgPath = files.path("./Tooth uncle square.png");
let img = images.read(imgPath);
let bitmap = img.bitmap;
let mat = img.mat;
printInfo("Original picture information", img, bitmap, mat);
img.recycle();
printInfo("Released img Picture information after", img, bitmap, mat);

journal

tag = Original picture information
img      : width = 564.0, height = 564.0
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 1272384.0
mat      : width = 564.0, height = 564.0
11-19 17:56:54.150 Script-56 Main [remote://789216b9dbb2c184ad7efa6ee6d2c830/main.js]/D:

tag = Released img Picture information after
img      : JavaException: java.lang.IllegalStateException: image has been recycleds
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 0.0
mat      : width = 0.0, height = 0.0

It can be seen from the log that after img recycling, although the width and height of bitmap are still the same, the size has changed to 0, and the width and height of mat is 0,

In other words, after img recycling, bitmap and mat must also be recycled

Will img and mat be affected when bitmap is recycled

Test code

let imgPath = files.path("./Tooth uncle square.png");
let img = images.read(imgPath);
let bitmap = img.bitmap;
let mat = img.mat;
printInfo("Original picture information", img, bitmap, mat);
bitmap.recycle();
// mat.release();
printInfo("Released bitmap Picture information after", img, bitmap, mat);

journal

tag = Original picture information
img      : width = 564.0, height = 564.0
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 1272384.0
mat      : width = 564.0, height = 564.0
11-19 18:00:35.347 Script-58 Main [remote://789216b9dbb2c184ad7efa6ee6d2c830/main.js]/D:

tag = Released bitmap Picture information after
img      : width = 564.0, height = 564.0
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 0.0
mat      : width = 564.0, height = 564.0

From the log, we can see that the size of bitmap has been reset to zero, so can img still be used? Let's test it with the graph color command

For example, get the color of a point

images.pixel(img, 100, 200)

Code error

Wrapped java.lang.IllegalStateException: Can't call getPixel() on a recycled bitmap

img doesn't work properly,

So is img recycled at this time?

At least mat still exists. Mat can be used normally, so img is not recycled,

Let's use the code to determine whether the picture is recycled

log(img.isRecycled()); // false

The code says that the picture is not recycled, so release the mat at this time. Is img recycled?

mat.release();
log(img.isRecycled()); // false

Or false, img is not marked as recycled

autojs img recycling method specific code

public synchronized void recycle() {
    Bitmap bitmap = this.mBitmap;
    if (bitmap != null) {
        bitmap.recycle();
    }
    this.mBitmap = null;
    Mat mat = this.mMat;
    if (mat != null) {
        OpenCVHelper.release(mat);
    }
    this.mMat = null;
    Image image = this.mediaImage;
    if (image != null) {
        image.close();
    }
    this.mediaImage = null;
    Debug.INSTANCE.noMemoryLeak(this.id);
    d<b> dVar = this.ref;
    if (dVar != null) {
        dVar.b = 0;
    }
    this.isRecycled = true;
}

It still looks a little complicated. What is mediaImage? I haven't used it. I don't care about it. Just know there's such a thing

Will img and bitmap be affected when mat is recycled

Test code

let imgPath = files.path("./Tooth uncle square.png");
let img = images.read(imgPath);
let bitmap = img.bitmap;
let mat = img.mat;
printInfo("Original picture information", img, bitmap, mat);
mat.release();
printInfo("Released mat Picture information after", img, bitmap, mat);

journal

tag = Original picture information
img      : width = 564.0, height = 564.0
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 1272384.0
mat      : width = 564.0, height = 564.0
11-19 18:24:44.707 Script-81 Main [remote://789216b9dbb2c184ad7efa6ee6d2c830/main.js]/D:

tag = Released mat Picture information after
img      : width = 564.0, height = 564.0
bitmap   : width = 564.0, height = 564.0, allocationByteCount = 1272384.0
mat      : width = 0.0, height = 0.0

It can be seen from the log that mat has been reset to zero and bitmap has not changed. Can img be used at this time? Let's test it with code
For example, get the color of a point

images.pixel(img, 100, 200); // -16777212

Normal color acquisition indicates that this color selection method can be used. Whether other methods can be used depends on whether mat is used,

If mat is used and it has been recycled, an error will be reported naturally

summary

IMG is used for image recycling. As long as img is recycled, bitmap and mat will be recycled;

Unless you use bitmap.copy or mat.clone to copy image data,

At this point, you have to recycle the image data you copied

Test environment for the above code

Mobile: Mi 11 Pro

Android version: 11

Autojs version: 9.0.11

Famous quotes

Thinking is the most important. Other Baidu, bing, stackoverflow, github, Android documents, autojs documents, and finally ask in the group
- Yashu tutorial

statement

Part of the content comes from the network
This tutorial is for learning purposes only and is prohibited for other purposes

bilibili

Yashu tutorial

WeChat official account

QQ group

747748653

Tags: OpenCV autojs bitmap

Posted on Sat, 20 Nov 2021 15:53:31 -0500 by reloj_alfred