Service Reference for Dubbo Source Analysis 4

This article was translated from original text In the end
Catalog
1. Introduction
2. Principles of Service Reference
3. Source Code Analysis
3.1 Disposal Configuration
3.2 Reference Services
3.2.1 Create Invoker
3.2.2 Create proxy
4. Summary
supplement
Service Reference Call Chain

1. Introduction

In Dubbo, we can reference remote services in two ways.The first is to reference the service using a direct connection to the service, and the second is to reference it based on the registry.Service direct connections are only appropriate for scenarios where services are debugged or tested, not for online environments.Therefore, I will focus on the process of referencing services through the registry.Getting the service configuration from the registry is only one part of the service reference process. In addition, service consumers need to go through the steps of Invoker creation, proxy class creation, and so on.

2. Principles of Service Reference

The Dubbo service references two times:

The first is to reference the service when the Spring container calls the afterPropertiesSet method of the ReferenceBean.

The second is referenced when the service corresponding to the ReferenceBean is injected into another class.

The difference between the two referencing services is that the first is hungry and the second is lazy.By default, Dubbo uses lazy reference services.If you need to use Hungry Han style, you can turn it on by configuring the init property of <dubbo:reference>Let's start with the getObject method of ReferenceBean by following the Dubbo default configuration.When our services are injected into other classes, Spring calls the getObject method for the first time, and it executes the service reference logic.By convention, configuration checks and collection are required before specific work can be done.

Next, the way service y is referenced is determined based on the information collected:

The first is to reference local (JVM) services.

The second is to reference remote services through direct connections.

The third is to reference remote services through the registry.

Either way, you end up with an Invoker instance.If you have more than one registry and more than one service provider, you will get a set of Invoker instances, which you will need to merge into one instance through the cluster management class Cluster.The merged Invoker instance already has the ability to call local or remote services, but it cannot be exposed to users, which can cause intrusion into user business code.In this case, the framework also needs to generate proxy classes for service interfaces through the proxy factory class and have the proxy class call Invoker logic.It avoids the intrusion of Dubbo framework code into business code and makes the framework easier to use.

3. Source Code Analysis

The entry method for the service reference is the getObject method of the ReferenceBean, which is defined in the FactoryBean interface of Spring and is implemented by the ReferenceBean.The implementation code is as follows:

public Object getObject() throws Exception {
    return get();
}
public synchronized T get() {
    if (destroyed) {
        throw new IllegalStateException("Already destroyed!");
    }
    // Detect if ref is empty, or create it by init method
    if (ref == null) {
        // The init method is primarily used to handle configurations and to call createProxy to generate proxy classes
        init();
    }
    return ref;
}

3.1 Disposal Configuration

Dubbo provides a rich set of configurations for adjusting and optimizing framework behavior, performance, and more.When Dubbo references or exports services, these configurations are first checked and processed to ensure they are properly configured.Configuration parsing logic is encapsulated in the init method of ReferenceConfig, which is analyzed below.

private void init() {
    // Avoid duplicate initialization
    if (initialized) {
        return;
    }
    initialized = true;
    // Detect interface name validity
    if (interfaceName == null || interfaceName.length() == 0) {
        throw new IllegalStateException("interface not allow null!");
    }

    // Detect if consumer variable is empty, create if empty
    checkDefault();
    appendProperties(this);
    if (getGeneric() == null && getConsumer() != null) {
        // Set generic
        setGeneric(getConsumer().getGeneric());
    }

    // Detect if it is a generalized interface
    if (ProtocolUtils.isGeneric(getGeneric())) {
        interfaceClass = GenericService.class;
    } else {
        try {
            // Load Class
            interfaceClass = Class.forName(interfaceName, true, Thread.currentThread()
                    .getContextClassLoader());
        } catch (ClassNotFoundException e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
        checkInterfaceAndMethods(interfaceClass, methods);
    }
    
    // -----------------------------------------Split line 1------------------------------------------------------------------------------------------------------------------------

    // Get the property value corresponding to the interface name from the system variable
    String resolve = System.getProperty(interfaceName);
    String resolveFile = null;
    if (resolve == null || resolve.length() == 0) {
        // Get Parse File Path from System Properties
        resolveFile = System.getProperty("dubbo.resolve.file");
        if (resolveFile == null || resolveFile.length() == 0) {
            // Load profile from specified location
            File userResolveFile = new File(new File(System.getProperty("user.home")), "dubbo-resolve.properties");
            if (userResolveFile.exists()) {
                // Get absolute file path
                resolveFile = userResolveFile.getAbsolutePath();
            }
        }
        if (resolveFile != null && resolveFile.length() > 0) {
            Properties properties = new Properties();
            FileInputStream fis = null;
            try {
                fis = new FileInputStream(new File(resolveFile));
                // Load Configuration from File
                properties.load(fis);
            } catch (IOException e) {
                throw new IllegalStateException("Unload ..., cause:...");
            } finally {
                try {
                    if (null != fis) fis.close();
                } catch (IOException e) {
                    logger.warn(e.getMessage(), e);
                }
            }
            // Gets the configuration corresponding to the interface name
            resolve = properties.getProperty(interfaceName);
        }
    }
    if (resolve != null && resolve.length() > 0) {
        // Assign resolve to url
        url = resolve;
    }
    
    // -----------------------------------------Split Line 2--------------------------------------------------------------------------------------------------------------------
    if (consumer != null) {
        if (application == null) {
            // Get an Application instance from consumer, as follows
            application = consumer.getApplication();
        }
        if (module == null) {
            module = consumer.getModule();
        }
        if (registries == null) {
            registries = consumer.getRegistries();
        }
        if (monitor == null) {
            monitor = consumer.getMonitor();
        }
    }
    if (module != null) {
        if (registries == null) {
            registries = module.getRegistries();
        }
        if (monitor == null) {
            monitor = module.getMonitor();
        }
    }
    if (application != null) {
        if (registries == null) {
            registries = application.getRegistries();
        }
        if (monitor == null) {
            monitor = application.getMonitor();
        }
    }
    
    // Detecting Application Legality
    checkApplication();
    // Detect local stub configuration validity
    checkStubAndMock(interfaceClass);
    
	// -----------------------------------------Split line 3----------------------------------------------------------------------------------------------------------------------
    
    Map<String, String> map = new HashMap<String, String>();
    Map<Object, Object> attributes = new HashMap<Object, Object>();

    // Add side, protocol version information, timestamp and process number to map
    map.put(Constants.SIDE_KEY, Constants.CONSUMER_SIDE);
    map.put(Constants.DUBBO_VERSION_KEY, Version.getProtocolVersion());
    map.put(Constants.TIMESTAMP_KEY, String.valueOf(System.currentTimeMillis()));
    if (ConfigUtils.getPid() > 0) {
        map.put(Constants.PID_KEY, String.valueOf(ConfigUtils.getPid()));
    }

    // Non-Generalized Services
    if (!isGeneric()) {
        // Get Version
        String revision = Version.getVersion(interfaceClass, version);
        if (revision != null && revision.length() > 0) {
            map.put("revision", revision);
        }

        // Get a list of interface methods and add them to the map
        String[] methods = Wrapper.getWrapper(interfaceClass).getMethodNames();
        if (methods.length == 0) {
            map.put("methods", Constants.ANY_VALUE);
        } else {
            map.put("methods", StringUtils.join(new HashSet<String>(Arrays.asList(methods)), ","));
        }
    }
    map.put(Constants.INTERFACE_KEY, interfaceName);
    // Add field information for ApplicationConfig, ConsumerConfig, ReferenceConfig and other objects to the map
    appendParameters(map, application);
    appendParameters(map, module);
    appendParameters(map, consumer, Constants.DEFAULT_KEY);
    appendParameters(map, this);
    
	// ---------------------------------------Split line 4------------------------------------------------------------------------------------------------------------------------
    
    String prefix = StringUtils.getServiceKey(map);
    if (methods != null && !methods.isEmpty()) {
        // Traverse the MethodConfig list
        for (MethodConfig method : methods) {
            appendParameters(map, method, method.getName());
            String retryKey = method.getName() + ".retry";
            // Detecting if map contains methodName.retry
            if (map.containsKey(retryKey)) {
                String retryValue = map.remove(retryKey);
                if ("false".equals(retryValue)) {
                    // Configure methodName.retries by adding retries
                    map.put(method.getName() + ".retries", "0");
                }
            }
 
            // Add the Attributes field in MethodConfig to attributes
            // Such as onreturn, onthrow, oninvoke, and so on
            appendAttributes(attributes, method, prefix + "." + method.getName());
            checkAndConvertImplicitConfig(method, map, attributes);
        }
    }
    
	// -----------------------------------------Split line 5----------------------------------------------------------------------------------------------------------------------

    // Get service consumer ip address
    String hostToRegistry = ConfigUtils.getSystemProperty(Constants.DUBBO_IP_TO_REGISTRY);
    if (hostToRegistry == null || hostToRegistry.length() == 0) {
        hostToRegistry = NetUtils.getLocalHost();
    } else if (isInvalidLocalHost(hostToRegistry)) {
        throw new IllegalArgumentException("Specified invalid registry ip from property..." );
    }
    map.put(Constants.REGISTER_IP_KEY, hostToRegistry);

    // Store attributes in system context
    StaticContext.getSystemContext().putAll(attributes);

    // Create Proxy Class
    ref = createProxy(map);

    // Build ConsumerModel based on service name, ReferenceConfig, proxy class,
    // And save the ConsumerModel in the Application Model
    ConsumerModel consumerModel = new ConsumerModel(getUniqueServiceName(), this, ref, interfaceClass.getMethods());
    ApplicationModel.initConsumerModel(getUniqueServiceName(), consumerModel);
}

The code above is long and there are many things to do.Here the code is divided according to the code logic, so let's take a look at it.

The first is the code from the beginning of the method to Split Line 1.This code is used to detect the existence of a ConsumerConfig instance, create a new instance if it does not exist, and populate the ConsumerConfig field with system variables or a dubbo.properties configuration file.The next step is to detect the generalization configuration and set the value of the interfaceClass based on the configuration.

Next, look at the logic between Split Line 1 and Split Line 2.This logic loads the configuration corresponding to the interface name from the system properties or configuration files and assigns the resolving result to the url field.The url field is generally used for point-to-point calls.

Continuing down, the code between Split Line 2 and Split Line 3 detects if several core configuration classes are empty and attempts to get them from other configuration classes if they are empty.The code between Split Line 3 and Split Line 4 is mainly used to collect various configurations and store them in the map.

The code between Split Line 4 and Split Line 5 is used to process MethodConfig instances.This instance includes event notification configurations such as onreturn, onthrow, oninvoke, and so on.

The code splitting line 5 to the end of the method is mainly used to resolve the service consumer ip and call createProxy to create the proxy object.A detailed analysis of this method will be expanded in the following sections.

3.2 Reference Services

This section starts with createProxy.Literally, createProxy appears to be used only to create proxy objects.This is not the case, however; it also calls other methods to build and merge Invoker instances.Details are as follows.

private T createProxy(Map<String, String> map) {
    URL tmpUrl = new URL("temp", "localhost", 0, map);
    final boolean isJvmRefer;
    if (isInjvm() == null) {
        // url configuration is specified without local reference
        if (url != null && url.length() > 0) {
            isJvmRefer = false;
        // Detect whether local references are required based on parameters such as url's protocol, scope, and injvm
        // For example, if the user explicitly configures scope=local, isInjvmRefer returns true
        } else if (InjvmProtocol.getInjvmProtocol().isInjvmRefer(tmpUrl)) {
            isJvmRefer = true;
        } else {
            isJvmRefer = false;
        }
    } else {
        // Get injvm configuration value
        isJvmRefer = isInjvm().booleanValue();
    }

    // Local Reference
    if (isJvmRefer) {
        // Generate local reference URL, protocol is injvm
        URL url = new URL(Constants.LOCAL_PROTOCOL, NetUtils.LOCALHOST, 0, interfaceClass.getName()).addParameters(map);
        // Call refer method to build InjvmInvoker instance
        invoker = refprotocol.refer(interfaceClass, url);
        
    // Remote Reference
    } else {
        // The url is not empty, indicating that the user may want to make point-to-point calls
        if (url != null && url.length() > 0) {
            // When you need to configure more than one url, you can use a semicolon to split it, which is where it is split
            String[] us = Constants.SEMICOLON_SPLIT_PATTERN.split(url);
            if (us != null && us.length > 0) {
                for (String u : us) {
                    URL url = URL.valueOf(u);
                    if (url.getPath() == null || url.getPath().length() == 0) {
                        // Set interface fully qualified name to url path
                        url = url.setPath(interfaceName);
                    }
                    
                    // Detects whether the url protocol is a registry and, if so, indicates that the user wants to use the specified registry
                    if (Constants.REGISTRY_PROTOCOL.equals(url.getProtocol())) {
                        // Converts a map to a query string and adds the value as a refer parameter to the url
                        urls.add(url.addParameterAndEncoded(Constants.REFER_KEY, StringUtils.toQueryString(map)));
                    } else {
                        // Merge urls, remove some of the configuration of the service provider from the user-configured url properties,
                        // For example, thread pool related configuration.And keep some configuration of the service provider, such as version, group, timestamp, etc.
                        // Finally, the merged configuration is set to the url query string.
                        urls.add(ClusterUtils.mergeUrl(url, map));
                    }
                }
            }
        } else {
            // Load registry url
            List<URL> us = loadRegistries(false);
            if (us != null && !us.isEmpty()) {
                for (URL u : us) {
                    URL monitorUrl = loadMonitor(u);
                    if (monitorUrl != null) {
                        map.put(Constants.MONITOR_KEY, URL.encode(monitorUrl.toFullString()));
                    }
                    // Add refer parameter to url and Add url to urls
                    urls.add(u.addParameterAndEncoded(Constants.REFER_KEY, StringUtils.toQueryString(map)));
                }
            }

            // Registry not configured, exception thrown
            if (urls.isEmpty()) {
                throw new IllegalStateException("No such any registry to reference...");
            }
        }

        // Individual registry or service provider (direct service connection, same below)
        if (urls.size() == 1) {
            // Call RegistryProtocol's refer to build an Invoker instance
            invoker = refprotocol.refer(interfaceClass, urls.get(0));
            
        // Multiple registries, multiple service providers, or a mix of both
        } else {
            List<Invoker<?>> invokers = new ArrayList<Invoker<?>>();
            URL registryURL = null;

            // Get all Invoker s
            for (URL url : urls) {
                // Invoker is built by calling refer with refprotocol s, which are run-time
                // Loads the specified Protocol instance according to the url protocol header and calls the refer method of the instance
                invokers.add(refprotocol.refer(interfaceClass, url));
                if (Constants.REGISTRY_PROTOCOL.equals(url.getProtocol())) {
                    registryURL = url;
                }
            }
            if (registryURL != null) {
                // If the registry link is not empty, AvailableCluster will be used
                URL u = registryURL.addParameter(Constants.CLUSTER_KEY, AvailableCluster.NAME);
                // Create a StaticDirectory instance and merge multiple Invoker s by Cluster
                invoker = cluster.join(new StaticDirectory(u, invokers));
            } else {
                invoker = cluster.join(new StaticDirectory(invokers));
            }
        }
    }

    Boolean c = check;
    if (c == null && consumer != null) {
        c = consumer.isCheck();
    }
    if (c == null) {
        c = true;
    }
    
    // invoker availability check
    if (c && !invoker.isAvailable()) {
        throw new IllegalStateException("No provider available for the service...");
    }

    // Generate Proxy Class
    return (T) proxyFactory.getProxy(invoker);
}

There is a lot of code above, but the logic is clear.

First check whether it is called locally based on the configuration.

If so, the refer method of the InjvmProtocol is called to generate an InjvmInvoker instance.

If not, the direct connection configuration key or registry url is read, and the read url is stored in the urls.Then follow up based on the number of URLs elements.

If the number of urls elements is 1, the Invoker instance interface is built directly from the Protocol adaptive extension class.

If the number of URLs elements is greater than 1, there are multiple registries or service direct-connect urls, then the Invoker is built from the url first.

Then multiple Invokers are merged through Cluster, and ProxyFactory is called to generate the proxy class.The process of building Invoker and the process of proxy classes are important, so we will analyze them in two sections.

3.2.1 Create Invoker

Invoker is the core model of Dubbo and represents an executable entity.At the service provider, Invoker is used to invoke the service provider class.On the service consumer side, Invoker is used to perform remote calls.Invoker is built from the Protocol implementation class.There are many Protocol implementation classes, and the two most commonly used are RegistryProtocol and DubboProtocol, which are analyzed by others themselves.The refer method source for DubboProtocol is analyzed below.The following:

public <T> Invoker<T> refer(Class<T> serviceType, URL url) throws RpcException {
    optimizeSerialization(url);
    // Create DubboInvoker
    DubboInvoker<T> invoker = new DubboInvoker<T>(serviceType, url, getClients(url), invokers);
    invokers.add(invoker);
    return invoker;
}

The above method looks simpler, but here's a call that needs our attention: getClients.This method is used to get a client instance of type ExchangeClient.ExchangeClient does not actually have communication capabilities; it needs to communicate based on lower-level client instances.For example, NettyClient, MinaClient, and so on, by default, Dubbo uses NettyClient to communicate.Next, let's take a brief look at the logic of the getClients method.

private ExchangeClient[] getClients(URL url) {
    // Whether to share the connection
    boolean service_share_connect = false;
  	// Gets the number of connections, defaulting to 0, indicating no configuration
    int connections = url.getParameter(Constants.CONNECTIONS_KEY, 0);
    // Shared connection if connections are not configured
    if (connections == 0) {
        service_share_connect = true;
        connections = 1;
    }

    ExchangeClient[] clients = new ExchangeClient[connections];
    for (int i = 0; i < clients.length; i++) {
        if (service_share_connect) {
            // Get Shared Clients
            clients[i] = getSharedClient(url);
        } else {
            // Initialize a new client
            clients[i] = initClient(url);
        }
    }
    return clients;
}

This determines whether to get a shared client or create a new client instance based on the number of connections, which is used by default.The initClient method is also called in the getSharedClient method

private ExchangeClient getSharedClient(URL url) {
    String key = url.getAddress();
    // Get ExchangeClient with Reference Counting
    ReferenceCountExchangeClient client = referenceClientMap.get(key);
    if (client != null) {
        if (!client.isClosed()) {
            // Increase Reference Count
            client.incrementAndGetCount();
            return client;
        } else {
            referenceClientMap.remove(key);
        }
    }

    locks.putIfAbsent(key, new Object());
    synchronized (locks.get(key)) {
        if (referenceClientMap.containsKey(key)) {
            return referenceClientMap.get(key);
        }

        // Create ExchangeClient Client Client
        ExchangeClient exchangeClient = initClient(url);
        // Pass the ExchangeClient instance to the ReferenceCountExchangeClient, using decoration mode
        client = new ReferenceCountExchangeClient(exchangeClient, ghostClientMap);
        referenceClientMap.put(key, client);
        ghostClientMap.remove(key);
        locks.remove(key);
        return client;
    }
}

The above method accesses the cache first, and if the cache is not hit, creates a new ExchangeClient instance through the initClient method and passes it to the ReferenceCountExchangeClient construction method to create an ExchangeClient instance with reference counting.The internal implementation of the ReferenceCountExchangeClient is relatively simple and will not be analyzed.Let's look at the code for the initClient method again.

private ExchangeClient initClient(URL url) {

    // Get the client type, netty by default
    String str = url.getParameter(Constants.CLIENT_KEY, url.getParameter(Constants.SERVER_KEY, Constants.DEFAULT_REMOTING_CLIENT));

    // Add codec and heartbeat package parameters to url
    url = url.addParameter(Constants.CODEC_KEY, DubboCodec.NAME);
    url = url.addParameterIfAbsent(Constants.HEARTBEAT_KEY, String.valueOf(Constants.DEFAULT_HEARTBEAT));

    // Detect if client type exists, throw exception if none exists
    if (str != null && str.length() > 0 && !ExtensionLoader.getExtensionLoader(Transporter.class).hasExtension(str)) {
        throw new RpcException("Unsupported client type: ...");
    }

    ExchangeClient client;
    try {
        // Get the lazy configuration and determine the type of client to create based on the configuration value
        if (url.getParameter(Constants.LAZY_CONNECT_KEY, false)) {
            // Create lazy-loading ExchangeClient instance
            client = new LazyConnectExchangeClient(url, requestHandler);
        } else {
            // Create a normal ExchangeClient instance
            client = Exchangers.connect(url, requestHandler);
        }
    } catch (RemotingException e) {
        throw new RpcException("Fail to create remoting client for service...");
    }
    return client;
}

The initClient method first gets the client type configured by the user, defaulting to netty.It then detects if the client type configured by the user exists and throws an exception if it does not exist.Finally, depending on the lazy configuration, you decide what type of client to create.The LazyConnectExchangeClient code here is not very complex. This class creates an ExchangeClient client through Exchangers'connect method when the request method is called. The code for this class is not analyzed in this section.Let's take a look at Exchangers'connect method.

public static ExchangeClient connect(URL url, ExchangeHandler handler) throws RemotingException {
    if (url == null) {
        throw new IllegalArgumentException("url == null");
    }
    if (handler == null) {
        throw new IllegalArgumentException("handler == null");
    }
    url = url.addParameterIfAbsent(Constants.CODEC_KEY, "exchange");
    // Get an Exchanger instance, defaulting to HeaderExchangeClient
    return getExchanger(url).connect(url, handler);
}

As mentioned above, getExchanger loads the HeaderExchangeClient instance via SPI, which is a simple method. Let's take a look at it for yourself.Next, analyze the implementation of the HeaderExchangeClient.

public ExchangeClient connect(URL url, ExchangeHandler handler) throws RemotingException {
    // There are several calls, as follows:
    // 1. Create the HeaderExchangeHandler object
    // 2. Create DecodeHandler object
    // 3. Building Client Instances with Transporters
    // 4. Create the HeaderExchangeClient object
    return new HeaderExchangeClient(Transporters.connect(url, new DecodeHandler(new HeaderExchangeHandler(handler))), true);
}

There are many calls here, so let's focus on Transporters'connect method.The following:

public static Client connect(URL url, ChannelHandler... handlers) throws RemotingException {
    if (url == null) {
        throw new IllegalArgumentException("url == null");
    }
    ChannelHandler handler;
    if (handlers == null || handlers.length == 0) {
        handler = new ChannelHandlerAdapter();
    } else if (handlers.length == 1) {
        handler = handlers[0];
    } else {
        // Create a ChannelHandler Distributor if the number of handler s is greater than 1
        handler = new ChannelHandlerDispatcher(handlers);
    }
    
    // Get the Transporter adaptive extension class and call the connect method to generate a Client instance
    return getTransporter().connect(url, handler);
}

As mentioned above, the getTransporter method returns an adaptive extension class that loads the specified Transporter implementation class at runtime based on the client type.If the user does not configure the client type, NettyTransporter is loaded by default and the connect method of that class is called.The following:

public Client connect(URL url, ChannelHandler listener) throws RemotingException {
    // Create NettyClient object
    return new NettyClient(url, listener);
}

You won't be able to keep up with this anymore. The next step is to build Netty clients through the API s provided by Netty. Everyone is interested in seeing for themselves.At this point, the refer method for DubboProtocol is analyzed.Next, continue to analyze RegistryProtocol s refer method logic.

public <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException {
    // Take the registry parameter value and set it as the protocol header
    url = url.setProtocol(url.getParameter(Constants.REGISTRY_KEY, Constants.DEFAULT_REGISTRY)).removeParameter(Constants.REGISTRY_KEY);
    // Get Registry Instance
    Registry registry = registryFactory.getRegistry(url);
    if (RegistryService.class.equals(type)) {
        return proxyFactory.getInvoker((T) registry, type, url);
    }

    // Convert url query string to Map
    Map<String, String> qs = StringUtils.parseQueryString(url.getParameterAndDecoded(Constants.REFER_KEY));
    // Get group configuration
    String group = qs.get(Constants.GROUP_KEY);
    if (group != null && group.length() > 0) {
        if ((Constants.COMMA_SPLIT_PATTERN.split(group)).length > 1
                || "*".equals(group)) {
            // Load MergeableCluster instance through SPI and call doRefer to continue executing service reference logic
            return doRefer(getMergeableCluster(), registry, type, url);
        }
    }
    
    // Call doRefer to continue executing service reference logic
    return doRefer(cluster, registry, type, url);
}

The code above first sets the protocol header for the url, then loads the registry instance based on the URL parameter.Then get the group configuration to determine the type of the first parameter of the doRefer based on the group configuration.The emphasis here is on the doRefer method, as follows:

private <T> Invoker<T> doRefer(Cluster cluster, Registry registry, Class<T> type, URL url) {
    // Create RegistryDirectory Instance
    RegistryDirectory<T> directory = new RegistryDirectory<T>(type, url);
    // Set up a registry and protocol
    directory.setRegistry(registry);
    directory.setProtocol(protocol);
    Map<String, String> parameters = new HashMap<String, String>(directory.getUrl().getParameters());
    // Generate service consumer links
    URL subscribeUrl = new URL(Constants.CONSUMER_PROTOCOL, parameters.remove(Constants.REGISTER_IP_KEY), 0, type.getName(), parameters);

    // Register service consumers, new nodes in the consumers directory
    if (!Constants.ANY_VALUE.equals(url.getServiceInterface())
            && url.getParameter(Constants.REGISTER_KEY, true)) {
        registry.register(subscribeUrl.addParameters(Constants.CATEGORY_KEY, Constants.CONSUMERS_CATEGORY,
                Constants.CHECK_KEY, String.valueOf(false)));
    }

    // Subscribe to node data such as providers, configurators, routers
    directory.subscribe(subscribeUrl.addParameter(Constants.CATEGORY_KEY,
            Constants.PROVIDERS_CATEGORY
                    + "," + Constants.CONFIGURATORS_CATEGORY
                    + "," + Constants.ROUTERS_CATEGORY));

    // A registry may have multiple service providers, so you need to merge multiple service providers into one
    Invoker invoker = cluster.join(directory);
    ProviderConsumerRegTable.registerConsumer(invoker, url, subscribeUrl, directory);
    return invoker;
}

As mentioned above, the doRefer method creates a RegistryDirectory instance, then generates a service consumer link and registers it with the registry.Once registered, subscribe to data under providers, configurators, routers, and so on.When the subscription is complete, RegistryDirectory receives information about the child nodes under these nodes.Since a service may be deployed on multiple servers, resulting in multiple nodes in providers, Cluster is required to merge multiple service nodes into one and generate an Invoker.Regarding RegistryDirectory and Cluster, this article does not intend to do analysis, which will be expanded in subsequent articles.

3.2.2 Create proxy

When Invoker is created, the next thing to do is to generate proxy objects for the service interface.With a proxy object, remote calls can be made.The entry method generated by the proxy object is getProxy of ProxyFactory, which is analyzed next.

public <T> T getProxy(Invoker<T> invoker) throws RpcException {
    // Calling overloaded methods
    return getProxy(invoker, false);
}

public <T> T getProxy(Invoker<T> invoker, boolean generic) throws RpcException {
    Class<?>[] interfaces = null;
    // Get list of interfaces
    String config = invoker.getUrl().getParameter("interfaces");
    if (config != null && config.length() > 0) {
        // List of slicing interfaces
        String[] types = Constants.COMMA_SPLIT_PATTERN.split(config);
        if (types != null && types.length > 0) {
            interfaces = new Class<?>[types.length + 2];
            // Setting the service interface class and EchoService.class into interfaces
            interfaces[0] = invoker.getInterface();
            interfaces[1] = EchoService.class;
            for (int i = 0; i < types.length; i++) {
                // Loading interface classes
                interfaces[i + 1] = ReflectUtils.forName(types[i]);
            }
        }
    }
    if (interfaces == null) {
        interfaces = new Class<?>[]{invoker.getInterface(), EchoService.class};
    }

    // Provides support for generalized calls to http and hessian protocols, referring to pull request #1827
    if (!invoker.getInterface().equals(GenericService.class) && generic) {
        int len = interfaces.length;
        Class<?>[] temp = interfaces;
        // Create a new interfaces array
        interfaces = new Class<?>[len + 1];
        System.arraycopy(temp, 0, interfaces, 0, len);
        // Set GenericService.class to array
        interfaces[len] = GenericService.class;
    }

    // Calling overloaded methods
    return getProxy(invoker, interfaces);
}

public abstract <T> T getProxy(Invoker<T> invoker, Class<?>[] types);

As above, the large block of code above is used to get the interfaces array, so let's move on.GetProxy (Invoker, Class<?>[]) is an abstract method, so let's look at its implementation code in the JavassistProxyFactory class.

public <T> T getProxy(Invoker<T> invoker, Class<?>[] interfaces) {
    // Generate a Proxy subclass (Proxy is an abstract class).And call the newInstance method of the Proxy subclass to create a Proxy instance
    return (T) Proxy.getProxy(interfaces).newInstance(new InvokerInvocationHandler(invoker));
}

There is not much code above. First, the Proxy subclass is obtained through the Proxy's getProxy method, then an InvokerInvocationHandler object is created and passed to newInstance to generate a Proxy instance.InvokerInvocationHandler implements the InvocationHandler interface from JDK for the specific purpose of intercepting interface class calls.This type of logic is relatively simple and will not be analyzed here.Here we focus on the getProxy method of Proxy, as follows.

public static Proxy getProxy(Class<?>... ics) {
    // Calling overloaded methods
    return getProxy(ClassHelper.getClassLoader(Proxy.class), ics);
}

public static Proxy getProxy(ClassLoader cl, Class<?>... ics) {
    if (ics.length > 65535)
        throw new IllegalArgumentException("interface limit exceeded");

    StringBuilder sb = new StringBuilder();
    // Traverse interface list
    for (int i = 0; i < ics.length; i++) {
        String itf = ics[i].getName();
        // Detect whether the type is an interface
        if (!ics[i].isInterface())
            throw new RuntimeException(itf + " is not a interface.");

        Class<?> tmp = null;
        try {
            // Reload Interface Class
            tmp = Class.forName(itf, false, cl);
        } catch (ClassNotFoundException e) {
        }

        // Detect if interfaces are the same, where tmp may be empty
        if (tmp != ics[i])
            throw new IllegalArgumentException(ics[i] + " is not visible from class loader");

        // Fully qualified name of stitching interface with separator;
        sb.append(itf).append(';');
    }

    // Use stitched interface name as key
    String key = sb.toString();

    Map<String, Object> cache;
    synchronized (ProxyCacheMap) {
        cache = ProxyCacheMap.get(cl);
        if (cache == null) {
            cache = new HashMap<String, Object>();
            ProxyCacheMap.put(cl, cache);
        }
    }

    Proxy proxy = null;
    synchronized (cache) {
        do {
            // Get Reference<Proxy>instance from cache
            Object value = cache.get(key);
            if (value instanceof Reference<?>) {
                proxy = (Proxy) ((Reference<?>) value).get();
                if (proxy != null) {
                    return proxy;
                }
            }

            // Concurrency control to ensure that only one thread can perform subsequent operations
            if (value == PendingGenerationMarker) {
                try {
                    // Other threads are waiting here
                    cache.wait();
                } catch (InterruptedException e) {
                }
            } else {
                // Place flag bits in the cache and jump out of the while loop for subsequent actions
                cache.put(key, PendingGenerationMarker);
                break;
            }
        }
        while (true);
    }

    long id = PROXY_CLASS_COUNTER.getAndIncrement();
    String pkg = null;
    ClassGenerator ccp = null, ccm = null;
    try {
        // Create ClassGenerator Object
        ccp = ClassGenerator.newInstance(cl);

        Set<String> worked = new HashSet<String>();
        List<Method> methods = new ArrayList<Method>();

        for (int i = 0; i < ics.length; i++) {
            // Detect if interface access level is protected or privete
            if (!Modifier.isPublic(ics[i].getModifiers())) {
                // Get Interface Package Name
                String npkg = ics[i].getPackage().getName();
                if (pkg == null) {
                    pkg = npkg;
                } else {
                    if (!pkg.equals(npkg))
                        // Non-public level interfaces must be in the same package, or throw an exception
                        throw new IllegalArgumentException("non-public interfaces from different packages");
                }
            }
            
            // Add interface to ClassGenerator
            ccp.addInterface(ics[i]);

            // Traversal interface method
            for (Method method : ics[i].getMethods()) {
                // Get a method description, which can be interpreted as a method signature
                String desc = ReflectUtils.getDesc(method);
                // Ignore method description strings if they are already in worked.Considering this situation,
                // A and B interfaces contain exactly the same method
                if (worked.contains(desc))
                    continue;
                worked.add(desc);

                int ix = methods.size();
                // Get Method Return Value Type
                Class<?> rt = method.getReturnType();
                // Get a list of parameters
                Class<?>[] pts = method.getParameterTypes();

                // Generate Object[] args = new Object[1...N]
                StringBuilder code = new StringBuilder("Object[] args = new Object[").append(pts.length).append("];");
                for (int j = 0; j < pts.length; j++)
                    // Generate args[1...N] = ($w) ...N;
                    code.append(" args[").append(j).append("] = ($w)$").append(j + 1).append(";");
                // Generate invoker method call statements for the InvokerHandler interface as follows:
                // Object ret = handler.invoke(this, methods[1...N], args);
                code.append(" Object ret = handler.invoke(this, methods[" + ix + "], args);");

                // Return value is not void
                if (!Void.TYPE.equals(rt))
                    // Generate return statements, such as return (java.lang.String) ret;
                    code.append(" return ").append(asArgument(rt, "ret")).append(";");

                methods.add(method);
                // Add method name, access controller, parameter list, method code, etc. to ClassGenerator 
                ccp.addMethod(method.getName(), method.getModifiers(), rt, pts, method.getExceptionTypes(), code.toString());
            }
        }

        if (pkg == null)
            pkg = PACKAGE_NAME;

        // Build interface proxy class name: PKG +'.proxy'+ id, such as org.apache.dubbo.proxy0
        String pcn = pkg + ".proxy" + id;
        ccp.setClassName(pcn);
        ccp.addField("public static java.lang.reflect.Method[] methods;");
        // Generate private java.lang.reflect.InvocationHandler handler;
        ccp.addField("private " + InvocationHandler.class.getName() + " handler;");

        // Add constructors with InvocationHandler parameters to the interface proxy class, such as:
        // porxy0(java.lang.reflect.InvocationHandler arg0) {
        //     handler=$1;
    	// }
        ccp.addConstructor(Modifier.PUBLIC, new Class<?>[]{InvocationHandler.class}, new Class<?>[0], "handler=$1;");
        // Add default constructor for interface proxy class
        ccp.addDefaultConstructor();
        
        // Generate interface proxy class
        Class<?> clazz = ccp.toClass();
        clazz.getField("methods").set(null, methods.toArray(new Method[0]));

        // Build Proxy subclass names, such as Proxy1, Proxy2, and so on
        String fcn = Proxy.class.getName() + id;
        ccm = ClassGenerator.newInstance(cl);
        ccm.setClassName(fcn);
        ccm.addDefaultConstructor();
        ccm.setSuperClass(Proxy.class);
        // Generate implementation code for the abstract method newInstance of Proxy, such as:
        // public Object newInstance(java.lang.reflect.InvocationHandler h) { 
        //     return new org.apache.dubbo.proxy0($1);
        // }
        ccm.addMethod("public Object newInstance(" + InvocationHandler.class.getName() + " h){ return new " + pcn + "($1); }");
        // Generate Proxy Implementation Class
        Class<?> pc = ccm.toClass();
        // Create Proxy Instances from Reflection
        proxy = (Proxy) pc.newInstance();
    } catch (RuntimeException e) {
        throw e;
    } catch (Exception e) {
        throw new RuntimeException(e.getMessage(), e);
    } finally {
        if (ccp != null)
            // Release Resources
            ccp.release();
        if (ccm != null)
            ccm.release();
        synchronized (cache) {
            if (proxy == null)
                cache.remove(key);
            else
                // Write Cache
                cache.put(key, new WeakReference<Proxy>(proxy));
            // Wake up other waiting threads
            cache.notifyAll();
        }
    }
    return proxy;
}

The code above is complex and we've written a lot of comments.When you read this code, you need to understand the purpose of CCP and ccm, otherwise you will get confused.CCP is used to generate proxy classes for service interfaces, such as we have a DemoService interface, which is generated by ccp.CCM is used to generate subclasses for the org.apache.dubbo.common.bytecode.Proxy Abstract class, mainly to implement the abstract method of the Proxy class.Let's take the interface org.apache.dubbo.demo.DemoService as an example to see how the proxy class code for this interface looks (ignoring the EchoService interface).

package org.apache.dubbo.common.bytecode;

public class proxy0 implements org.apache.dubbo.demo.DemoService {

    public static java.lang.reflect.Method[] methods;

    private java.lang.reflect.InvocationHandler handler;

    public proxy0() {
    }

    public proxy0(java.lang.reflect.InvocationHandler arg0) {
        handler = $1;
    }

    public java.lang.String sayHello(java.lang.String arg0) {
        Object[] args = new Object[1];
        args[0] = ($w) $1;
        Object ret = handler.invoke(this, methods[0], args);
        return (java.lang.String) ret;
    }
}

The proxy class generation logic is analyzed here.The whole process is complicated and you need to be patient with it.

4. Summary

This article provides a more detailed analysis of the process of service references, and some logic, such as Directory and Cluster, is not yet addressed.These interfaces and implementation class functions are relatively independent and will be analyzed separately in future.For the time being, we can think of these as black boxes, as long as we know what they are used for.This is where the process of referencing a service is analyzed.

supplement

Service Reference Call Chain

ReferenceBean.getObject()
-->ReferenceConfig.get()  //Three variables Protocol refprotocol, Cluster cluster, ProxyFactory proxyFactory are initialized in ReferenceConfig
    -->ReferenceConfig.init()   //Configuration Check
   	   -->createProxy(map);   //Create proxy 
          -->refprotocol.refer(interfaceClass, urls.get(0)) //Generate invoker
              -->ExtensionLoader.getExtensionLoader(Protocol.class).getExtension("registry")
                 -->extension.refer(arg0, arg1)
                     -->ProtocolFilterWrapper.refer
                        -->RegistryProtocol.refer(type,url)  //refprotocol.refer ultimately executes the RegistryProtocol.refer method
                           -->registryFactory.getRegistry(url);//Connect with zk
                           -->doRefer(cluster, registry, type, url)
                               -->registry.register //Create a consumer zk node named dubbo/com.alibaba.dubbo.demo.DemoService/consumers
                               -->directory.subscribe //Subscribe to zk nodes, including providers, configurators, routers
                               -->cluster.join(directory)  //Multiple invokers masquerade as cluster invoker s until the invoker is created
          -->proxyFactory.getProxy(invoker)  //Generate proxy

Simply summarize in these steps:

ReferenceBean.getObject() --> ReferenceConfig.init() -->RegistryProtocol.doRefer--> cluster.join(directory) --> proxyFactory.getProxy(invoker)

55 original articles published. 8% praised. 130,000 visits+
Private letter follow

Tags: Dubbo Java Apache Netty

Posted on Sun, 12 Jan 2020 21:08:47 -0500 by itsjareds