Performance tool ANRdaemon

Performance tool ANRdaemon

1, README

ANRdaemon is a daemon that analyzes ANR problems caused by excessive CPU usage. The daemon uses debugfs to record information. The trace module to be captured is configured in / d/tracing in advance. Depending on the CPU usage level, trace can be accessed through the global switch / d/trace/trace_on to control. The original trace file is stored in / d/tracing/trace.

The following actions will cause the daemon to start:

$ anrd -t 9990 sched gfx am

This means that when the CPU utilization is above 99.90%, you will start to catch trace, and the catch module is sched gfx am

Using ANRdaemon_get_trace.sh [device serial]dump and get the compressed trace file.

You can use systrace to extract:

$ systrace.py --from-file=<path to compressed trace file>

Known issues:

In the systrace output, anrdaemon is displayed when the trace is not running. This is because the daemon turns off tracing when CPU usage drops. The last entry it leaves in the original trace file is that the scheduler switches from some other process to the daemon. Then, after a period of time (such as 20 seconds), when the CPU utilization becomes high and the daemon turns on tracing again, the first entry in / d/tracing/trace recorded by sched is switching from the daemon to some other process. Due to this mechanism, when systrace.py parses the original trace file, the daemon is displayed as running for the whole 20 seconds (because from the perspective of systrace, two sched trace entries with an interval of 20 seconds about the daemon mean that the daemon runs continuously for all 20 seconds). However, in the case of high CPU utilization, this does not affect the actually captured trace.

2, Use

anrd help information

emulator_x86_64:/ # anrd  -h                                                                                                        
usage: ANRdaemon [options] [categoris...]
Options includes:
   -a appname  enable app-level tracing for a comma separated list of cmdlines
   -t N        cpu threshold for logging to start (uint = 0.01%, min = 5000, max = 9999, default = 9990)
   -s N        use a trace buffer size of N KB default to 2048KB
   -h          show helps
Categoris includes:
   am         - activity manager
   sm         - sync manager
   input      - input
   dalvik     - dalvik VM
   audio      - Audio
   gfx        - Graphics
   rs         - RenderScript
   hal        - Hardware Modules
   irq        - kernel irq events
   sched      - kernel scheduler activity
   stack      - kernel stack
   sync       - kernel sync activity
   workq      - kernel work queues
Control includes:
   SIGQUIT: terminate the process
   SIGSTOP: suspend all function of the daemon
   SIGCONT: resume the normal function
   SIGUSR1: dump the current logging in a compressed form

Just use readme, which is to run a monitoring program. If the cpu exceeds the set value, he will catch trace.

:/ # anrd -t 5000 sched gfx am
:/ # ps -e |grep anr
root           5715      1 12346524  2064 __arm64_sys_nanosleep 0 S anrd
:/ # logcat |grep 5715
08-25 18:19:21.768  5715  5715 I anrdaemon: ANRdaemon starting

You think you can stop after you catch it and lead the trace out.

Log on device:

:/ # logcat |grep anrd
08-25 18:34:13.727  9882  9882 D anrdaemon: High cpu usage, start logging.
08-25 18:34:14.233  9882  9882 D anrdaemon: Usage back to low, stop logging.
08-25 18:38:07.754  9882  9882 I anrdaemon: Started to dump ANRdaemon trace.
08-25 18:38:16.081  9882  9882 I anrdaemon: Finished dump. Output file stored at: /data/misc/anrd/dump_of_anrdaemon.2021-08-25.18:38:07

Execute sh export trace on pc:

qiucheng@haha:~/aosp/system/extras/ANRdaemon$ ./ANRdaemon_get_trace.sh 
/data/misc/anrd/dump_of_anrdaemon.2021-08-25.18:38:07: 1 file pulled, 0 skipped. 27.1 MB/s (5279290 bytes in 0.186s)
SUCCEED!
Trace stored at /home/qiucheng/aosp/system/extras/ANRdaemon/dump_of_anrdaemon.2021-08-25.18:38:07

3, Source code

Code location:

system/extras/ANRdaemon/

ANRdaemon.cpp
ANRdaemon_get_trace.sh

There are only two files. One is cpp compiled into binary executable files on the device side, and the other is a shell script on the pc, which is used to export the captured trace

Let's look at the anrd executable first

anrd

system/extras/ANRdaemon/ANRdaemon.cpp

int main(int argc, char* argv[]) {
    if (get_options(argc, argv) != 0) return 1;

    if (daemon(0, 0) != 0) return 1;
    //Note 1: for signal monitoring, save the trace file when receiving the signal
    register_sighandler();
    //Note 2: new "/ d/tracing/trace" fd
    /* Clear any the trace log file by overwrite it with a new file */
    int fd = creat(dfs_trace_output_path, 0);
    if (fd == -1) {
        ALOGE("Faield to open and cleaup previous log");
        return 1;
    }
    close(fd);
    //Note 3: enable the dead loop monitoring cpu load monitoring signal, grab trace and save trace
    ALOGI("ANRdaemon starting");
    start();

    if (err) ALOGE("ANRdaemon stopped due to Error: %s", err_msg);

    ALOGI("ANRdaemon terminated.");

    return (err ? 1 : 0);
}
----------------------------------------------------------------
    static void start(void) {
    if ((set_tracing_buffer_size()) != 0) return;

    dfs_set_property(tag, apps, true);
    dfs_poke_binder();

    get_cpu_stat(&old_cpu);
    sleep(check_period);

    while (!quit && !err) {
        if (!suspend && is_heavy_load()) {//Note 4: check cpu load
            /*
             * Increase process priority to make sure we can stop logging when
             * necessary and do not overwrite the buffer
             */
            setpriority(PRIO_PROCESS, 0, -20);
            start_tracing();//Note 5: grab trace and save trace
            setpriority(PRIO_PROCESS, 0, 0);
        }
        sleep(check_period);
    }
    return;
}
--------------------------------------------------------
static void handle_signal(int signo) {//Note 6: intercept signals to react differently
    switch (signo) {
        case SIGQUIT:
            suspend = true;
            quit = true;
            break;
        case SIGSTOP:
            suspend = true;
            break;
        case SIGCONT:
            suspend = false;
            break;
        case SIGUSR1:
            request_dump_trace();

The principle is as simple as that in readme. Start an dead cycle, constantly check the cpu load, and catch the trace when it exceeds the set value

Let's take a look at the shell script that exports trace

ANRdaemon_get_trace.sh

#!/bin/bash

TRACE_DIR=/data/misc/anrd
TRACE_FILE_PATTEN=dump_of_anrdaemon

if [ $# -eq 1 ]; then
    DEVICE=$(echo "-s $1")
else
    DEVICE=""
fi
#Note 1: detect that anrd is running
PID=$(adb $DEVICE shell "ps | grep anrd")

if [ $? -ne 0 ]; then
    echo "FAILED. ADB failed or Daemon is not running."
    exit 1
fi
#Note 2: Send a signal to anrd to trigger the save of trace
PID=$(echo "$PID" | awk '{ print $2 }')
adb $DEVICE shell "kill -s SIGUSR1 $PID"
#Note 3: find the last generated trace file
TRACE_FILE=$(adb $DEVICE shell "ls $TRACE_DIR \
    | grep $TRACE_FILE_PATTEN | tail -n1" | tr -d '\r')
#Note 4: view the files opened by the anrd process
# Wiat the trace file generation to complete
adb $DEVICE shell "lsof -p $PID" | grep $TRACE_FILE > /dev/null
while [ $? -eq 0 ];
do
    sleep 1
    adb $DEVICE shell "lsof -p $PID" | grep "$TRACE_FILE" > /dev/null
done

if [ -z "$TRACE_FILE" ]; then
    echo "FAILED. Trace file not created"
fi
#Note 5: pull trace to pc
adb $DEVICE pull "${TRACE_DIR}/${TRACE_FILE}" ${TRACE_FILE}

CURRENT_DIR=$(pwd)
echo SUCCEED!
echo Trace stored at ${CURRENT_DIR}/${TRACE_FILE}

Simple sequential process. Find the anrd process in the device, then send the signal SIGUSR1 to trigger the signal interception in anrd, and then save the trace and pull the latest trace to the local pc.

4, There are bug s

anrd and ANRdaemon

The initial executable program is called ANRdaemon, which is later changed to anrd, but the writing method in usage and readme has not been changed. This needs to be corrected.

'/data/misc/anrd/': No such file or directory

After looking at the git log, the directory saved by trace has never been created on the file system. So you created it manually at the beginning of development? You also need to confirm whether to modify it and whether to change it in rc or anrd.

Debugging directories such as / data/misc/wmtrace / are created in init.rc during startup

system/core/rootdir/init.rc

mkdir /data/misc/wmtrace 0700 system system
on property:ro.debuggable=1
    # Give writes to anyone for the trace folder on debug builds.
    # The folder is used to store method traces.
    chmod 0773 /data/misc/trace
    # Give reads to anyone for the window trace folder on debug builds.
    chmod 0775 /data/misc/wmtrace

However, after manual creation, the program can be saved and exported successfully.

Tags: Android

Posted on Sun, 28 Nov 2021 15:22:41 -0500 by SteveMellor