Two week custom scripting language - day 4 for objects representing the program

Day 4 object to represent the program

After the program is divided into words, the next step is to construct an abstract syntax tree.

4.1 definition of abstract syntax tree

Abstract Syntax Tree (AST) is a kind of tree structure used to represent program structure.

Lexical analysis (segmentation of words) - > grammatical analysis (construction of abstract grammar tree)

The BinaryExpr object is used to represent a binocular expression.

Binocular expression refers to four operations and other operations that calculate new values through left and right values.

4.2 design node

All nodes of the abstract syntax tree are subclasses of the astroe

ASTLeaf class is the parent class of leaf node (node without branch) ASTList is the parent class of the node object with branches Other classes are subclasses of the ASTList or ASTLeaf classes

The NumberLiteral and Name classes are used to represent leaf nodes, and the BinaryExpr class is used to represent nodes with branches. They are subclasses of the above two classes respectively

Main methods of astrae class

ASTLeaf child(int i) //Return to the ith child node
int numChildren() //Returns the number of children (0 if there are no children)
Iterator<ASTree> children() //Returns an iterator for traversing child nodes

The BinaryExpr class also has left and right fields These two fields are not defined directly in the BinaryExpr class, but through the children field of its parent ASTList class. As shown in listing 4.6, the BinaryExpr class does not contain the left and right fields, but provides the left and right methods. These methods can respectively select from the astroe object saved in the children field, and return the corresponding left and right child nodes. The BinaryExpr class also does not have the operator fields shown in figures 4.1 and 4.2 to hold operators. The operator itself is a separate node (ASTLeaf object) that exists as a child of the BinaryExpr object. In other words, the BinaryExpr object contains three child nodes: left value, right value and operator. The BinaryExpr class does not have an operator field and provides an operator method. This method takes the word from the ASTLeaf object corresponding to the operator and returns the string in it.

Listing 4.1 astroe. Java
package stone.ast;
import java.util.Iterator;

public abstract class ASTree implements Iterable<ASTree> {
    public abstract ASTree child(int i);
    public abstract int numChildren();
    public abstract Iterator<ASTree> children();
    public abstract String location();
    public Iterator<ASTree> iterator() { return children(); }
}
Listing 4.2 ASTLeaf.java
package stone.ast;
import java.util.Iterator;
import java.util.ArrayList;
import stone.Token;

public class ASTLeaf extends ASTree {
    private static ArrayList<ASTree> empty = new ArrayList<ASTree>(); 
    protected Token token;
    public ASTLeaf(Token t) { token = t; }
    public ASTree child(int i) { throw new IndexOutOfBoundsException(); }
    public int numChildren() { return 0; }
    public Iterator<ASTree> children() { return empty.iterator(); }
    public String toString() { return token.getText(); }
    public String location() { return "at line " + token.getLineNumber(); }
    public Token token() { return token; }
}
Listing 4.3 ASTList.java
package stone.ast;
import java.util.List;
import java.util.Iterator;

public class ASTList extends ASTree {
    protected List<ASTree> children;
    public ASTList(List<ASTree> list) { children = list; }
    public ASTree child(int i) { return children.get(i); }
    public int numChildren() { return children.size(); }
    public Iterator<ASTree> children() { return children.iterator(); }
    public String toString() {
        StringBuilder builder = new StringBuilder();
        builder.append('(');
        String sep = "";
        for (ASTree t: children) {
            builder.append(sep);
            sep = " ";
            builder.append(t.toString());
        }
        return builder.append(')').toString();
    }
    public String location() {
        for (ASTree t: children) {
            String s = t.location();
            if (s != null)
                return s;
        }
        return null;
    }
}
Code listing 4.4 NumberLiteral.java
package stone.ast;
import stone.Token;

public class NumberLiteral extends ASTLeaf {
    public NumberLiteral(Token t) { super(t); }
    public int value() { return token().getNumber(); }
}
Code listing 4.5 Name.java
package stone.ast;
import stone.Token;

public class Name extends ASTLeaf {
    public Name(Token t) { super(t); }
    public String name() { return token().getText(); }
}
Listing 4.6 BinaryExpr.java
package stone.ast;
import java.util.List;

public class BinaryExpr extends ASTList {
    public BinaryExpr(List<ASTree> c) { super(c); }
    public ASTree left() { return child(0); }
    public String operator() {
        return ((ASTLeaf)child(1)).token().getText();
    }
    public ASTree right() { return child(2); }
}

4.3 BNF

BNF (Backus naurform)
EBNF(Extended BNF)

BNF can be used to express Noam Chomsky's context free grammar in the field of linguistics

Meta symbols used in BNF

{pat} pattern pat repeats at least 0 times

[pat] matches pattern pat with 0 or 1 recurrence

pat1 | pat2 matches with pat1 or pat2

{} treat parentheses as a complete pattern

Factor: factor

term: item

expression: expressions

It's a little difficult to understand just by looking at the concept. The examples in the book are clear

4.4 syntax analysis and abstract syntax tree

Slowly began to have difficulty, first according to the plan to read it. Some places may still have to be understood

Tags: Programming Java

Posted on Thu, 16 Jan 2020 10:22:28 -0500 by axon