Android Pit Guide, Gson collides with Kotlin for an unsafe operation

This article has authorized the original launch of the Hong Yang Public Number.

Recently, I found that WeChat has more album functions, which can aggregate a series of original articles. Just as I happen to encounter many students each week asking me a variety of questions, some of which are more meaningful. I will write demo verification in detail on the weekend, and simply expand it to write articles to share with you.

Of course, I don't encourage you to chat and ask questions in private. You can go to the planet to ask questions. When you answer "Planet" in the background of Public Number, you will see the entrance. There are more than 5,000 people there. After all, I still have work to do.

First look at a question

Let's look at a piece of code together:

public class Student  {
    private Student() {
        throw new IllegalArgumentException("can not create.");
    }
    public String name;
}

How do we create a Student object from Java code?

Let's first think about the approximate ways to create objects through Java:

  1. new Student()//Private
  2. Reflection Call Construction Method//throw ex
  3. Deserialization//Relevant serialization interfaces need to be implemented
  4. clone //clone-related interfaces need to be implemented
  5. ...

Okay, it's beyond my point of knowledge.

I cannot help but murmur:

The title is too biased and meaningless, and the title of the article is Android's Guide to Pits, which doesn't seem to matter at all

Yes, it's biased. Skip this issue. Let's look down and see how we encountered it in the Android development process. After that, the problem will be solved.

Source of the problem

Last week, a small partner of a group encountered a Bean written by Kotlin, and a problem that did not meet expectations occurred when Gson converted the string to a specific Bean object.

Because it's the code for their project, I don't post it and I've written a similar small example to replace it.

For Java Bean s, kotlin can use data class es, and there are many blogs on the Web that say:

In Kotlin, instead of writing a JavaBean yourself, you can use DataClass directly, and the DataClass compiler will silently help us generate some functions.

Let's start with a Bean:

data class Person(var name: String, var age: Int) {


}

This Bean is used to receive server data and convert it into objects via Gson.

Simplify the code to:

val gson = Gson()
val person = gson.fromJson<Person>("{\"age\":\"12\"}", Person::class.java)

We passed a json string, but it did not contain a value with key as name, and note:

The type of name in Person is String, that is, name=null is not allowed

So what happens when I run the code above?

  1. Error, after all, no value of name was passed;
  2. No error, name defaults to'';
  3. No error, name=null;

Feel 1 is most reasonable and complies with Kotlin's air safety check.

Verify, modify the code, and see the output:

val gson = Gson()
val person = gson.fromJson<Person>("{\"age\":\"12\"}", Person::class.java)
println(person.name )

Output results:

null

Isn't it strange to feel that I accidentally bypassed Kotlin's empty type check?

So the student who had the problem had a problem with the data since then, which made it difficult to solve the problem all the time.

Let's change the code again:

data class Person(var name: String, var age: Int): People(){

}

We inherit Person from the People class:

public class People {

    public People(){
        System.out.println("people cons");
    }

}

Print the log in the construction method of the People class.

We all know that, under normal circumstances, when you construct subclass objects, you will inevitably execute the construction method of the parent class first.

Run it:

The parent constructor was not executed, but the object was constructed

It can be guessed here that the construction of Person object is not a regular one and does not follow the construction method.

So how does it do it?

You can only go to Gson's source to find the answer.

Finding out how to do it is actually equivalent to answering the questions at the beginning of our article.

Trace Reason

Gson constructs an object like this, but does not construct it like a parent, which is extremely dangerous if it is.

This will make the program completely out of line with your expectations and leave some necessary logic behind.

So let's say in advance, you don't have to panic. It's not that Gson is prone to this. It happens to meet the above example. We'll make it clear later.

First we turn Person, a kotlin class, into Java to avoid hiding something behind it:

# Display after decompiling
public final class Person extends People {
   @NotNull
   private String name;
   private int age;

   @NotNull
   public final String getName() {
      return this.name;
   }

   public final void setName(@NotNull String var1) {
      Intrinsics.checkParameterIsNotNull(var1, "<set-?>");
      this.name = var1;
   }

   public final int getAge() {
      return this.age;
   }

   public final void setAge(int var1) {
      this.age = var1;
   }

   public Person(@NotNull String name, int age) {
      Intrinsics.checkParameterIsNotNull(name, "name");
      super();
      this.name = name;
      this.age = age;
   }

   // Some methods have been omitted.
}

You can see that Person has a two-parameter construction method with an empty security check for name.

That is, if a Person object is normally constructed with this construction method, there will be no null security issues.

Then you can only go to Gson's source code:

Gson's logic is typically based on what type is read, and then looks for the corresponding TypeAdapter to handle it. This example is a Person object, so it ends up in ReflectiVeTypeAdapterFactory.createThen a TypeAdapter is returned.

Let's take a look at its internal code:

# ReflectiveTypeAdapterFactory.create
@Override 
public <T> TypeAdapter<T> create(Gson gson, final TypeToken<T> type) {
	Class<? super T> raw = type.getRawType();
	
	if (!Object.class.isAssignableFrom(raw)) {
	  return null; // it's a primitive!
	}
	
	ObjectConstructor<T> constructor = constructorConstructor.get(type);
	return new Adapter<T>(constructor, getBoundFields(gson, type, raw));
}

Focus on the assignment of the constructor object, which knows at a glance that it is related to the constructed object.

# ConstructorConstructor.get
public <T> ObjectConstructor<T> get(TypeToken<T> typeToken) {
    final Type type = typeToken.getType();
    final Class<? super T> rawType = typeToken.getRawType();
	
	// ...omit some cache container-related code

    ObjectConstructor<T> defaultConstructor = newDefaultConstructor(rawType);
    if (defaultConstructor != null) {
      return defaultConstructor;
    }

    ObjectConstructor<T> defaultImplementation = newDefaultImplementationConstructor(type, rawType);
    if (defaultImplementation != null) {
      return defaultImplementation;
    }

    // finally try unsafe
    return newUnsafeAllocator(type, rawType);
  }

You can see that there are three processes for the return value of this method:

  1. newDefaultConstructor
  2. newDefaultImplementationConstructor
  3. newUnsafeAllocator

Let's start with the first newDefaultConstructor

private <T> ObjectConstructor<T> newDefaultConstructor(Class<? super T> rawType) {
    try {
      final Constructor<? super T> constructor = rawType.getDeclaredConstructor();
      if (!constructor.isAccessible()) {
        constructor.setAccessible(true);
      }
      return new ObjectConstructor<T>() {
        @SuppressWarnings("unchecked") // T is the same raw type as is requested
        @Override public T construct() {
            Object[] args = null;
            return (T) constructor.newInstance(args);
            
            // Some exception handling was omitted
      };
    } catch (NoSuchMethodException e) {
      return null;
    }
  }

As you can see, it's simple to try to get a parameterless constructor, and if you can find one, build the object using the newInstance reflection.

Following our Person code, there is actually only one two-parameter constructor in this class, not one-parameter construct, which hits NoSuchMethodException and returns null.

Returning null takes the newDefaultImplementationConstructor, which is the logic of some collection class related objects and skips directly.

So, in the end, you have to go: **newUnsafe Allocator ** method.

As you can see from the naming, this is an unsafe operation.

How did newUnsafe Allocator end up building an object insecure?

Looking down, the final execution is:

public static UnsafeAllocator create() {
// try JVM
// public class Unsafe {
//   public Object allocateInstance(Class<?> type);
// }
try {
  Class<?> unsafeClass = Class.forName("sun.misc.Unsafe");
  Field f = unsafeClass.getDeclaredField("theUnsafe");
  f.setAccessible(true);
  final Object unsafe = f.get(null);
  final Method allocateInstance = unsafeClass.getMethod("allocateInstance", Class.class);
  return new UnsafeAllocator() {
    @Override
    @SuppressWarnings("unchecked")
    public <T> T newInstance(Class<T> c) throws Exception {
      assertInstantiable(c);
      return (T) allocateInstance.invoke(unsafe, c);
    }
  };
} catch (Exception ignored) {
}
  
// try dalvikvm, post-gingerbread use ObjectStreamClass
// try dalvikvm, pre-gingerbread , ObjectInputStream

}

You can see that Gson passed theSun.misc.UnsafeAn object was constructed.

Note: Unsafe is not included in all Android versions, but is currently included in the new version, so there are three pieces of logic in Gson's method to generate objects, which you can think of as a three-insurance policy for different platforms.Test Device: Android 29 Simulator

For the time being, we will only discussSun.misc.Unsafe, the other one.

Sun.misc.UnsafeAnd license API s?

Unsafe is located atSun.miscA class under the package that mainly provides methods to perform low-level, unsafe operations, such as direct access to system memory resources, self-management of memory resources, etc. These methods have played a significant role in improving Java efficiency and enhancing the operational capabilities of Java language underlying resources.However, because the Unsafe class gives the Java language the ability to manipulate memory space like C language pointers, this undoubtedly increases the risk of pointer problems for programs.Excessive and incorrect use of the Unsafe class in a program increases the probability of program errors, making Java a safe language no longer "safe", so be careful with the use of Unsafe.
https://tech.meituan.com/2019/02/14/talk-about-java-magic-class-unsafe.html

Specifically, you can refer to this article by Meituan.

Okay, here's the truth.

The reason is that our Person doesn't provide a default construction method, and when Gson doesn't find one, it builds an object directly through the Unsafe method, bypassing the construction method.

Here we get:

  1. How does Gson build objects?
  2. When writing classes that require Gson to be converted to objects, we must remember to have a default construction method, otherwise it is not an error, but it is very unsafe!
  3. We know that there is also this Unsafe black technology that constructs objects.

Return to the question at the beginning of the article

How do you construct the following Student object in Java?

public class Student  {
    private Student() {
        throw new IllegalArgumentException("can not create.");
    }
    public String name;
}

We mimic Gson's code and write the following:

try {
    val unsafeClass = Class.forName("sun.misc.Unsafe")
    val f = unsafeClass.getDeclaredField("theUnsafe")
    f.isAccessible = true
    val unsafe = f.get(null)
    val allocateInstance = unsafeClass.getMethod("allocateInstance", Class::class.java)
    val student = allocateInstance.invoke(unsafe, Student::class.java)
    (student as Student).apply {
        name = "zhy"
    }
    println(student.name)
} catch (ignored: Exception) {
    ignored.printStackTrace()
}

Output:

zhy

Successfully built.

Not safe at all?

Perhaps the most rewarding thing to see here is that you understand Gson's process of building objects, and you'll be careful to provide default parameterized construction methods when writing beans later, especially when using Kotlin data class.

So what we just said about Unsafe is of no other practical use?

This class provides the ability to manipulate memory space like C language pointers.

It is well known that on Android P, Google restricts app access to the hidden API.

However, Google cannot restrict its access to the hidden API, right, so its own related classes allow access to the hidden API.

So how does Google differentiate between our app call or its own call?

With ClassLoader, if the ClassLoader is a BootStrapClassLoader, it is considered a system class, then it is released.

So, one of our ideas for breaking through P access restrictions is to create a class that replaces its lassLoader with a BootStrap ClassLoader so that it reflects any hidden api.

How do I change it?

Just set the class Loader member variable of this class to null.

Reference code:

private void testJavaPojie() {
	try {
	  Class reflectionHelperClz = Class.forName("com.example.support_p.ReflectionHelper");
	  Class classClz = Class.class;
	  Field classLoaderField = classClz.getDeclaredField("classLoader");
	  classLoaderField.setAccessible(true);
	  classLoaderField.set(reflectionHelperClz, null);
	} catch (Exception e) {
		  e.printStackTrace();
	}
}
//Come from:https://juejin.im/post/5ba0f3f7e51d450e6f2e39e0

However, there is a problem with this. The code above uses reflection to modify the classLoader member of a class, so suppose google someday completely restricts the reflection setting classLoader as well.

So what?ClassLoader works the same way, but instead of Java reflection, we use Unsafe:

@Keep
public class ReflectWrapper {
 
    //just for finding the java.lang.Class classLoader field's offset
    @Keep
    private Object classLoaderOffsetHelper;
 
    static {
        try {
            Class<?> VersionClass = Class.forName("android.os.Build$VERSION");
            Field sdkIntField = VersionClass.getDeclaredField("SDK_INT");
            sdkIntField.setAccessible(true);
            int sdkInt = sdkIntField.getInt(null);
            if (sdkInt >= 28) {
                Field classLoader = ReflectWrapper.class.getDeclaredField("classLoaderOffsetHelper");
                long classLoaderOffset = UnSafeWrapper.getUnSafe().objectFieldOffset(classLoader);
                if (UnSafeWrapper.getUnSafe().getObject(ReflectWrapper.class, classLoaderOffset) instanceof ClassLoader) {
                    Object originalClassLoader = UnSafeWrapper.getUnSafe().getAndSetObject(ReflectWrapper.class, classLoaderOffset, null);
                } else {
                    throw new RuntimeException("not support");
                }
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}
//From the author's section chief: A pure Java layer way to bypass Android P's private function call restrictions, article.

Unsafe gives us the ability to manipulate memory and complete code that normally only relies on C++.

Well, the problem encountered by a friend triggered a discussion throughout the article, and I hope you can get some results.

Thank You Guo lin, pale blue Wednesday, sky and other friends.

Tags: Java Android Google C

Posted on Mon, 08 Jun 2020 20:22:04 -0400 by razorsedgeuk