Introduction to JSON Lib

Json lib is a commonly used Json Library in Java before. The last version is 2.4, which supports JDK 1.3 and 1.5 respectively. The last update time is December 14, 2010.

Although it has not been maintained for many years, search on search engines“ Java Json **"* * and other related keywords are found. It seems that someone has been introducing and using this library.

The official website of the project is:

http://json-lib.sourceforge.net/

One sentence conclusion

When Json lib parses every Json object through a string, it will substring the current parsing position to the end of the string.

Due to JDK7 and above substring  The intercepted content will be copied completely, so when large Json data is encountered and there are many objects, a large number of character array copying operations will be carried out, resulting in a large number of CPU and memory consumption, even serious Full GC problems.

problem analysis

One day, it was found that there were many full GCS problems in the online production server. After troubleshooting, it was found that when Full GC was generated, the volume of an old interface would increase, but the interface was not resolved Json In addition, the parsed data is stored in the cache.

It is suspected that it is related to the size of the interface request parameters. When logging, it is found that there is much larger Json data than the general request, but only about 1MB. To simplify this problem, write the following performance test code.

package net.mayswind;

import net.sf.json.JSONObject;
import org.apache.commons.io.FileUtils;

import java.io.File;

public class JsonLibBenchmark {
    public static void main(String[] args) throws Exception {
        String data = FileUtils.readFileToString(new File("Z:\\data.json"));
        benchmark(data, 5);
    }

    private static void benchmark(String data, int count) {
        long startTime = System.currentTimeMillis();

        for (int i = 0; i < count; i++) {
            JSONObject root = JSONObject.fromObject(data);
        }

        long elapsedTime = System.currentTimeMillis() - startTime;
        System.out.println(String.format("count=%d, elapsed time=%d ms, avg cost=%f ms", count, elapsedTime, (double) elapsedTime / count));
    }
}

After the above code is executed, it takes about 7 seconds to complete each parsing on average, as shown in the following figure.

In the Json file for testing, 34018 identical contents are omitted at "...". The whole Json data contains more than 30000 Json objects. The actual test data is shown in the figure below.

{
    "data":
    [
        {
            "foo": 0123456789,
            "bar": 1234567890
        },
        {
            "foo": 0123456789,
            "bar": 1234567890
        },
        ...
    ]
}

Use Java Mission Control to record the execution, as shown in the figure below. You can see that a large number of char [] arrays are allocated.

Look at the relevant source code, including jsonobject_ The main contents of the fromjsontokener method are as follows. You can see that it matches at the beginning of the code to see if it starts with "null.".

private static JSONObject _fromJSONTokener(JSONTokener tokener, JsonConfig jsonConfig) {
    try {
        if (tokener.matches("null.*")) {
            fireObjectStartEvent(jsonConfig);
            fireObjectEndEvent(jsonConfig);
            return new JSONObject(true);
        } else if (tokener.nextClean() != '{') {
            throw tokener.syntaxError("A JSONObject text must begin with '{'");
        } else {
            fireObjectStartEvent(jsonConfig);
            Collection exclusions = jsonConfig.getMergedExcludes();
            PropertyFilter jsonPropertyFilter = jsonConfig.getJsonPropertyFilter();
            JSONObject jsonObject = new JSONObject();
...

The matches method directly uses substring to intercept the string from the current position to the end, and then performs regular matching.

public boolean matches(String pattern) {
    String str = this.mySource.substring(this.myIndex);
    return RegexpUtils.getMatcher(pattern).matches(str);
}

String substring  A new String object is created by passing in an array of characters, a starting position, and a truncation length.

public String substring(int beginIndex) {
    if (beginIndex < 0) {
        throw new StringIndexOutOfBoundsException(beginIndex);
    }
    int subLen = value.length - beginIndex;
    if (subLen < 0) {
        throw new StringIndexOutOfBoundsException(subLen);
    }
    return (beginIndex == 0) ? this : new String(value, beginIndex, subLen);
}

In JDK7 and above, the intercepted data will be copied in the last line when the constructor is called, which is also the key to the whole problem.

public String(char value[], int offset, int count) {
    if (offset < 0) {
        throw new StringIndexOutOfBoundsException(offset);
    }
    if (count <= 0) {
        if (count < 0) {
            throw new StringIndexOutOfBoundsException(count);
        }
        if (offset <= value.length) {
            this.value = "".value;
            return;
        }
    }
    // Note: offset or count might be near -1>>>1.
    if (offset > value.length - count) {
        throw new StringIndexOutOfBoundsException(offset + count);
    }
    this.value = Arrays.copyOfRange(value, offset, offset+count);
}

Recommend to my blog to read more:

1.Java JVM, collection, multithreading, new features series

2.Spring MVC, Spring Boot, Spring Cloud series tutorials

3.Maven, Git, Eclipse, Intellij IDEA series tools tutorial

4.Latest interview questions of Java, backend, architecture, Alibaba and other large factories

Feel good, don't forget to like + forward!