Extend deserialization from previous versions of Hessian to support jdk8 serializable lambda expressions

Try to extend deserialization from previous versions of Hessian to support jdk8 serializable lambda expressions

extraction

dubbo version 2.5.3
Built in default serialization hessian.io client serialization failed when calling a function interface with parameters passed.

// Fields that cannot be serialized
private Predict<T> bar = any -> true;

The reason is that Predict does not implement Serializable. After the custom implementation,

@FunctionalInterface
public interface SerializablePredict<T> extends Predict<T>, Serializable {}

The client does not report an error, and the server failed to deserialize.
The reason is that during serialization, the above objects will be serialized into an instance of serialized lambda. Obviously, hessian on the server side does not handle the deserialization implementation of the object, but uses the default deserializer JavaDeserializer, and finally reports an error.

Solve

1. Implement deserialization logic

gayhub searches for the latest hessian source code, which is expected to be copied directly. As a result, the search keyword Lambda in the source code doesn't see half a line of code (it may also be useful to the jdk default implementation, which is hidden in the default logic). Make your own.

In essence, the existence of functions in memory is serialized lambda. We need to deserialize the objects of this class to reuse the existing Java deserializer of hessian.

@Component
public class LambdaDeserializer extends AbstractDeserializer {
    
    private static JavaDeserializer serializableLambdaDeserializer =
            new JavaDeserializer(HessianAdaptSerializedLambda.class);

    @Override
    public Object readObject(AbstractHessianInput in) throws IOException {
        return serializableLambdaDeserializer.readObject(in);
    }


    @Override
    public Object readMap(AbstractHessianInput in) throws IOException {
        return serializableLambdaDeserializer.readMap(in);
    }

    @Override
    public Object readObject(AbstractHessianInput in, String[] fieldNames) throws IOException {
        return serializableLambdaDeserializer.readObject(in, fieldNames);
    }
    
}

The logic of deserializing pojo by hessian is to create a new one first. If the constructor needs to enter a parameter, it will pass in the default value, and then assign a value to the attribute.
This leads to two problems in deserialization of the class seralizedlambda:

  1. The construction method requires capturedArgs to be non null, but when hessian new passes the default value, that is, null in causes null pointer
  2. All properties have no assignment method

It's a final class, so I need to copy one myself

import lombok.Data;
import lombok.AllArgsConstructor;

import java.io.Serializable;
import java.lang.invoke.MethodHandleInfo;
import java.lang.invoke.SerializedLambda;
import java.lang.reflect.Method;
import java.security.AccessController;
import java.security.PrivilegedActionException;
import java.security.PrivilegedExceptionAction;


@Data
@AllArgsConstructor
public final class HessianAdaptSerializedLambda implements Serializable {
    private static final long serialVersionUID = 802502534576559999L;
    private final Class<?> capturingClass;
    private final String functionalInterfaceClass;
    private final String functionalInterfaceMethodName;
    private final String functionalInterfaceMethodSignature;
    private final String implClass;
    private final String implMethodName;
    private final String implMethodSignature;
    private final int implMethodKind;
    private final String instantiatedMethodType;
    private final Object[] capturedArgs;


	public void setCapturedArg(Object[] args) {
		capturedArgs = Objects.nonNull(args).clone();
	}

    public Object getCapturedArg(int i) {
        return capturedArgs[i];
    }

    public Object readResolve() throws ReflectiveOperationException {
        try {
            Method deserialize = AccessController.doPrivileged(new PrivilegedExceptionAction<Method>() {
                @Override
                public Method run() throws Exception {
                    Method m = capturingClass.getDeclaredMethod("$deserializeLambda$", SerializedLambda.class);
                    m.setAccessible(true);
                    return m;
                }
            });

            return deserialize.invoke(null, this.toReal());
        }
        catch (PrivilegedActionException e) {
            Exception cause = e.getException();
            if (cause instanceof ReflectiveOperationException)
                throw (ReflectiveOperationException) cause;
            else if (cause instanceof RuntimeException)
                throw (RuntimeException) cause;
            else
                throw new RuntimeException("Exception in SerializedLambda.readResolve", e);
        }
    }

    @Override
    public String toString() {
        String implKind= MethodHandleInfo.referenceKindToString(implMethodKind);
        return String.format("SerializedLambda[%s=%s, %s=%s.%s:%s, " +
                        "%s=%s %s.%s:%s, %s=%s, %s=%d]",
                "capturingClass", capturingClass,
                "functionalInterfaceMethod", functionalInterfaceClass,
                functionalInterfaceMethodName,
                functionalInterfaceMethodSignature,
                "implementation",
                implKind,
                implClass, implMethodName, implMethodSignature,
                "instantiatedMethodType", instantiatedMethodType,
                "numCaptured", capturedArgs.length);
    }


    public SerializedLambda toReal() {
        return new SerializedLambda(
                    capturingClass,
                    functionalInterfaceClass,
                    functionalInterfaceMethodName,
                    functionalInterfaceMethodSignature,
                    implMethodKind,
                    implClass,
                    implMethodName,
                    implMethodSignature,
                    instantiatedMethodType,
                    capturedArgs
        );
    }
}

This can deserialize normally.
Miraculously, the deserialized object can be directly recognized as the corresponding lambda by the jvm, so it can be returned directly.
The principle here remains to be studied.

2. For hessian in dubbo

The source code of hessian is endless, which roughly means that when SerializerFactory.getDeserializer() is taken from the cache, if not, it will judge the type to generate a new one, and then add the cache.
Its upper layer will judge the relationship between the type and the class of the object to be instantiated

 // Here, type = serializedlambda CL = serializeablepredict.class
 public Deserializer getObjectDeserializer(String type, Class cl)
    throws HessianProtocolException
  {
    Deserializer reader = getObjectDeserializer(type);
    
    if (cl == null
	|| cl.equals(reader.getType())
	|| cl.isAssignableFrom(reader.getType())
	|| HessianHandle.class.isAssignableFrom(reader.getType())) {
      return reader;
    }

    if (log.isLoggable(Level.FINE)) {
      log.fine("hessian: expected '" + cl.getName() + "' at '" + type + "' ("
	       + reader.getType().getName() + ")");
    }
    // Finally, I'll go here and take the default Java deserializer
    return getDeserializer(cl);
  }

Let's rewrite the code above

@Component
public class LambdaDeserializableSerializerFactory extends SerializerFactory {

    @Autowired
    private LambdaDeserializer lambdaDeserializer;

    @Override
    public Deserializer getObjectDeserializer(String type, Class cl) throws HessianProtocolException {
        if (type.equals(SerializedLambda.class.getName())) {
            return lambdaDeserializer;
        }
        return super.getObjectDeserializer(type, cl);
    }
}

Finally, on the dubbo side, there is a static attribute in the following class, which can be replaced after the spring container is started (to deal with the final modifier)

package com.alibaba.dubbo.common.serialize.support.hessian;

import com.alibaba.com.caucho.hessian.io.SerializerFactory;

public class Hessian2SerializerFactory extends SerializerFactory {
	// Replace with our implementation
	public static final SerializerFactory SERIALIZER_FACTORY = new Hessian2SerializerFactory();

	private Hessian2SerializerFactory() {
	}

	@Override
	public ClassLoader getClassLoader() {
		return Thread.currentThread().getContextClassLoader();
	}

}

Unsolvable problems

With the above enhancement, can the client write lambda to the server for execution?
The answer is no, the above can only ensure that there is a serializable function interface in the field that does not report an error in the default value or in the state of null or known implementation (such as enumeration defined by the assignment server). The reason is to figure out where the code to be executed is.
In the class that defines the function.

Anonymous function = anonymous inner class will be compiled into main class at compile time,

Serialized lambda = serialized SerializedLambda class, which records

captureClass = the main class of anonymous inner class. For example, if you write any - > true in A.java, its captureClass = A.class

There will be a $deserializeLambda $method in the main class to verify and restore the real anonymous function object

So you want to pass anonymous functions through rpc, provided that the server has this class, which is not available in most cases, so it will not succeed, and report NoClassDefException.

Why don't you serialize the main class class byte when serializing? Because you don't know how many classes are involved in the reference chain, so it's a bit tricky to serialize a jar package at last.

If I can serialize anonymous function code to the server, and if I can use bytecode technology to mock a main class, it will be saved. That's up to Daniel to realize.

Published 1 original article · praised 0 · visited 2
Private letter follow

Tags: Java Lambda Dubbo Attribute

Posted on Wed, 11 Mar 2020 02:16:04 -0400 by rrn