[learn compiler from scratch] XVI. Summary of main points of MLIR ODS part I


stay [learn the compiler from scratch] 12. Learning notes of MLIR Toy Tutorials 1 As mentioned in, MLIR is used to unify different levels of IR through Dialect, that is, it is responsible for defining various operations (operators). Then, the definitions of Dialect and Operation are constructed through the TabelGen specification. The Operation definition of MLIR driven by TableGen is also called ODS (Operation Definition Specification). At present, we only have a brief understanding of how the collect and Operation of Toy Tutorials are defined through ODS, but we don't know much about the syntax and some restrictions of ODS itself, which leads to confusion when looking at the Operation definitions of some related projects. We don't know what the meaning of a field is or the definition of custom ops How to declare operands and Attr (for example, how to set the convolution groups parameter to an optional attribute).

So this article (this is the first and the next) The ODS document based on MLIR will explain some key points in ODS to help us better understand and get started with MLIR. I will break down the points needing attention in the official document into some small points. TableGen and ODS mentioned in the following article do not make special distinction, and the syntax in ODS is TableGen syntax. The key points introduced here are more or less used in ONEFLOW's docking with MLIR, which is interesting Interestingly, you can compare this part of ONEFLOW's source code. https://github.com/Oneflow-Inc/oneflow/blob/master/oneflow/ir/include/OneFlow/OneFlowOps.td In addition, all articles learned by MLIR and TVM are summarized in this warehouse: https://github.com/BBuf/tvm_mlir_learn At present, it has won about 170 stars. Welcome to pay attention. Thank you.

Due to space reasons and my tight time recently, this article mainly summarizes 10 key points. The remaining key points are given in the next article. Welcome to exchange and correct.

1. Why use ODS to define Operation

To define an Operation in MLIR, two methods are supported: direct definition in C + + and definition based on ODS framework. Direct definition in C + + requires us to inherit and rewrite some construction methods of the base class Op, and write a piece of C + + code for each Op. it is conceivable that the Op definition part of the whole system will be very redundant, generate a large number of repeatable code, and have poor readability. For example If an Operation is defined based on ODS, we only need to uniformly write the Op definition into a td file according to the ODS specification, and then use the code generation tool provided by MLIR to automatically generate the C + + definition of Operation. This complete auto codegen method gracefully implements the Operation definition, and the things that need users to worry about (that is, the syntax specification of ODS) are more intuitive.

ODS is the only choice for MLIR to define operations, so it is necessary for us to learn the syntax specification of ODS.

2. TableGen syntax

A TableGen file (ending with. td) contains the following syntax:

  • TableGen class is similar to C + + class. It can be used as a template or base class to generate subclasses.
  • TableGen def is an object similar to C + +. It is declared with a specialization of TableGen class. For example, def MyDef: MyClass <... >;, or def MyDef; can be used alone. It cannot be used as a template or as a base class to generate subclasses.
  • TableGen dag is a type specially used for directed acyclic graph elements. A dag type has an operator and zero or more parameters. The syntax is (operator arg0, arg1, argN.), where the operator can be any TableGen def. The parameters can be anything, including the dag itself. We can attach names to operators and parameters, such as (MyOp:op_name MyArg:arg_name).

To learn more about the types and expressions supported by tablegen, click this link: https://llvm.org/docs/TableGen/ProgRef.html .

3. Operation definition

MLIR defines several common structures to help define operations, and provides their semantics through tablegen backend: opdefinitionsgen. These common structures are defined in the file OpBase.td. They mainly include:

  • Op class: This is the main structure used when defining Operation. When specializing this class, specify all facts related to Operation with the help of the following structure.
  • Dialect class: operations belonging to the same logical group will be placed under the same dialect. Dialect contains dialect level information.
  • Optrain class and its subclasses: they are used to specify the special properties and constraints of the Operation, including whether the Operation has side effects, whether the output of the Op has the same shape as the input, etc.
  • ins/outs tag: These are two special tags built into the OpDefinitionsGen backend, which respectively guide the definition of operands / attributes and results.
  • TypeConstraint class and its subclasses: they are used to specify constraints on operands or results. A notable subclass is Type, which represents constraints of general C + + types.
  • AttrConstraint class and its subclasses: they are used to specify constraints on attributes. A notable subclass is Attr, which represents constraints on attributes with common values.

An Operation is defined through the specialized Op class. The specialized Op class contains the specific contents of all fields it needs. For example, tf.AvgPool is defined as follows:

def TF_AvgPoolOp : TF_Op<"AvgPool", [NoSideEffect]> {
  let summary = "Performs average pooling on the input.";

  let description = [{
Each entry in `output` is the mean of the corresponding size `ksize`
window in `value`.

  let arguments = (ins

    Confined<I64ArrayAttr, [ArrayMinCount<4>]>:$ksize,
    Confined<I64ArrayAttr, [ArrayMinCount<4>]>:$strides,
    TF_AnyStrAttrOf<["SAME", "VALID"]>:$padding,
    DefaultValuedAttr<TF_ConvertDataFormatAttr, "NHWC">:$data_format

  let results = (outs

  TF_DerivedOperandTypeAttr T = TF_DerivedOperandTypeAttr<0>;

All fields required to define an Operation are described below. For a complete list of supported fields, please refer to the definition of Op class (i.e. OpBase.td).

  • "Operation name": the name of the operation, such as tf.Add in tensorflow dialog.
  • "Operation documentation": the document description of operation, including summary and description. You can understand it after reading it. I won't say much.
  • "Operation arguments": the parameters of an operation. There are two kinds of parameters for an operation, one is operands, that is, operands, and the other is the attributes attribute parameter. The attribute parameters are divided into Natural attributes and Derived attributes. The former is a natural attribute, and the number of output channels such as convolution must be specified, and the latter is a derived attribute, such as the shape of output Tensor.

The operands and attributes are specified in arguments of dag type to boot in ins:

let arguments = (ins

Here, < type constraint > is a TableGen def from the TypeConstraint class level. Similarly, < attr constraint > is a TableGen def from the AttrConstraint class level. More details are provided in the Constraints section.

  • Variable operand. To define a variable operand, you need to wrap TypeConstraint with variadic <... >. Typically, an Operation has no or only one variable operand. In the latter case, dynamic variable operands can be easily derived from the definition of static variable operands. However, if an Operation has multiple variable length operands (optional or variable length), it is impossible to attribute the dynamic operands to the corresponding static variable length operand definition without further information from the Operation. Therefore, it is necessary to use the SameVariadicOperandSize or AttrSizedOperandSegments feature to indicate that all variable length operands have corresponding dynamic values.
  • Optional operands. To define an optional operand, you need to wrap TypeConstraint with optional <... >. The interpretation is the same as variable operands.
  • Optional properties. To define an optional attribute, you need to use optionalatr <... > to package AttrConstraint.
  • Optional properties with default values. Use DefaultValuedAttr <..., "..." > to package AttrConstraint. The second parameter to defaultvaluedatr should be a string containing the C + + default value. For example, the default value of a single precision floating-point array needs to be specified as "0.5f", and the default value of an integer array needs to be specified as "{1,2,3}".
  • "Confining attributes". Condensed is provided as a general mechanism to help further model the attribute constraints brought by value types. More primitive constraints can be combined into complex constraints through condensed. For example, the minimum value of a 32bit integer is 10, which can be expressed as defined < i32attr, [intminvalue < 10 >] >. There are other examples, such as intminvalue < n >: specifying an integer attribute greater than or equal to N, and so on.
  • "Operation results": similar to operands. The results are declared with tag type results and guided with outs.
let results = (outs
  • I haven't used "Operation regions" and "operation winners" yet. I don't know the application scenario for the time being.
  • "Operation traits and Constraints": features are operation attributes that affect syntax or semantics. Various features of MLIR C + + are in the MLIR:: optrain namespace. When the feature, interface or constraint of operation involves multiple operands / attributes / results, it shall be passed in as the second template parameter of Op class. They all need to inherit from the optrain class. See the Constraints section for details.

4. Default build method automatically generated by operation

After the Operation is defined, how do we build it?   for each Operation, some compilers will be automatically generated based on the parameters of the Operation and the return value of the Operation. For example, the following Operation definitions are given:

def MyOp : ... {
  let arguments = (ins


  let results = (outs

The following builders are generated:

// All result-types/operands/attributes have one aggregate parameter.
// All result types / operands / attributes are collected as an aggregate parameter.
static void build(OpBuilder &odsBuilder, OperationState &odsState,
                  ArrayRef<Type> resultTypes,
                  ValueRange operands,
                  ArrayRef<NamedAttribute> attributes);

// Each result-type/operand/attribute has a separate parameter. The parameters
// for attributes are of mlir::Attribute types.
// Each result type / operand / attribute is an independent parameter. The attribute parameter is of type mlir::Attribute
static void build(OpBuilder &odsBuilder, OperationState &odsState,
                  Type i32_result, Type f32_result, ...,
                  Value i32_operand, Value f32_operand, ...,
                  IntegerAttr i32_attr, FloatAttr f32_attr, ...);

// Each result-type/operand/attribute has a separate parameter. The parameters
// for attributes are raw values unwrapped with mlir::Attribute instances.
// (Note that this builder will not always be generated. See the following
// explanation for more details.)
// Each result type / operand / attribute is an independent parameter.
// Attribute parameters are raw values that are not wrapped by the mlir::Attribute instance.
// (note that this builder does not always generate. See the following explanation for more details.)
static void build(OpBuilder &odsBuilder, OperationState &odsState,
                  Type i32_result, Type f32_result, ...,
                  Value i32_operand, Value f32_operand, ...,
                  APInt i32_attr, StringRef f32_attr, ...);

// Each operand/attribute has a separate parameter but result type is aggregate.
// Each operand / attribute is an independent parameter. But the results are all collected into an aggregate type.
static void build(OpBuilder &odsBuilder, OperationState &odsState,
                  ArrayRef<Type> resultTypes,
                  Value i32_operand, Value f32_operand, ...,
                  IntegerAttr i32_attr, FloatAttr f32_attr, ...);

// All operands/attributes have aggregate parameters.
// Generated if return type can be inferred.
// This builder is generated only if the return value type can be inferred.
static void build(OpBuilder &odsBuilder, OperationState &odsState,
                  ValueRange operands, ArrayRef<NamedAttribute> attributes);

// (And manually specified builders depending on the specific op.)

The above code comment translation has explained the differences between these builders. And there may be some other builders, please refer to https://mlir.llvm.org/docs/OpDefinitions/#run -MLIR tblgen to see the generated content here.

5. Custom builder method

Assuming that the construction method in the C + + code generated above is not what we expect, we need to customize the builder method at this time. For example:

def MyOp : Op<"my_op", []> {
  let arguments = (ins F32Attr:$attr);

  let builders = [
    OpBuilder<(ins "float":$val)>

The builders field is a list of custom builders added to the Op class. In this example, we provide a convenient compiler that accepts floating-point values instead of attributes. In ODS using TableGen dag, many function declarations use the ins prefix. Followed by a comma separated list, each item in the list is a combination of type and name prefixed with $. The above definition will be converted into a builder in the following format:

class MyOp : /*...*/ {
  static void build(::mlir::OpBuilder &builder, ::mlir::OperationState &state,
                    float val);

Note that the builder has two additional pre parameters. These parameters are useful for building operations. In particular, in order to build an Operation through this method, the state must be filled with the attributes, operands, fields and return value types of the Operation. Builder can be used to build any IR object belonging to Op, such as type or nested Operation. When types and names are converted to C + + code, they should be valid C + + structures, a type (in the namespace of Op) and an identifier (for example, class is not a valid identifier). You can directly provide the implementation of builder in ODS, using the following code block of TableGen:

def MyOp : Op<"my_op", []> {
  let arguments = (ins F32Attr:$attr);

  let builders = [
    OpBuilder<(ins "float":$val), [{
      $_state.addAttribute("attr", $_builder.getF32FloatAttr(val));

_ Builder and_ State these two special parameters are equivalent to builder and state. The parameters in the ins section can be used directly, such as val. The C + + code implementation of the builder will be completed by replacing the special variables in ODS. It is necessary to ensure that other parts of the builder ODS implementation are effective c + + structures. Although there is no limit on the code size, we encourage you to inline only shorter defined builders in ODS and put the definitions of longer defined builders in C + + files. Finally, if some parameters require default values, they can be defined using carrg to specify the wrapper type and this value, as follows:

def MyOp : Op<"my_op", []> {
  let arguments = (ins F32Attr:$attr);

  let builders = [
    OpBuilder<(ins CArg<"float", "0.5f">:$val), [{
      $_state.addAttribute("attr", $_builder.getF32FloatAttr(val));

In the converted C + + code, the default parameters only appear in the declaration, not in the definition, which meets the requirements of C + +.

/// Header file.
class MyOp : /*...*/ {
  static void build(::mlir::OpBuilder &builder, ::mlir::OperationState &state,
                    float val = 0.5f);

/// Source file.
MyOp::build(::mlir::OpBuilder &builder, ::mlir::OperationState &state,
            float val) {
  state.addAttribute("attr", builder.getF32FloatAttr(val));

6. Declarative Assembly Format

The declaration instruction format of Operation can be specified in the declarative string matching Operation operands, attributes, etc. Have the ability to express additional information that needs to be parsed to build an Operation:

def CallOp : Std_Op<"call", ...> {
  let arguments = (ins FlatSymbolRefAttr:$callee, Variadic<AnyType>:$args);
  let results = (outs Variadic<AnyType>);

  let assemblyFormat = [{
    $callee `(` $args `)` attr-dict `:` functional-type($args, results)

It mainly consists of three parts:

  • "Directives: Directives". Instruction is a built-in function with optional parameters. The available instructions are attr dict, attr dict with keyword, operands, ref, and so on.
  • "Literals". Literal values are key values or punctuation marks wrapped with `. The following are valid punctuation sets::, =, <, >, (,), {,}, [,], - >,?, +, *\ n punctuation has the effect of starting another line. As follows:
let assemblyFormat = [{
  `{` `\n` ` ` ` ` `this_is_on_a_newline` `\n` `}` attr-dict
%results = my.operation {

Literals with empty content can be used to remove spaces after implicitly inserting some literal elements. For example) or] etc. For example,] may appear at the end of output, but it is not the last element in the format. In this example, you can use "] ` `" to delete the subsequent spaces.

  • "Variables". Variables are entities registered on an Operation, such as parameters (properties or operands), domains, results, successors, and so on. In CallOp, variables represent callee and args. Property variables display their respective value types. Unless the type of its value can be constructed, in this case, the value type of the attribute variable can be omitted.

7. Custom directives & optional groups

The declaration instruction format specification can handle most common scenarios when formatting an Operation. For OPS that want to specify a part of Operations in the format, declarative syntax is not supported. You can try to use custom instructions at this time.

In some cases, Operations may have "optional" information, such as properties or an empty set of variable parameter operands. In these cases, a part of the assembly format can be marked as optional according to the existence of this information.

These two parts are complex and I haven't used them yet, so I won't expand them here. If you are interested, please see the official documents.

8. Type inference

One requirement of format is that the types of operands and results must always exist. In some cases, the type of a variable can be inferred from type constraints or other available information. In these cases, the type of the variable can be omitted from the format.

  • Buildable Types. Some type constraints may have only one representation, allowing them to be built directly; For example, type I32 or Index. Types in ODS can mark themselves buildable by setting the builderCall field or inheriting from the BuildableType class.
  • "Trait Equality Constraints.". There are many Operations that have constraints that are registered on Operations as equal characteristics of known types; For example, the true, false, and result values of a select Operation usually have the same type. The assembly format can check these equivalent constraints to identify the types of missing variables. Currently supported features are: AllTypesMatch, TypesMatchWith, SameTypeOperands and SameOperandsAndResultType.
  • 「InferTypeOpInterface」 . Operations that implement infertypeopinterface can omit its result type in its assembly format because the result type can be inferred from the operand.
  • 「hasCanonicalizer」. This Boolean field indicates whether a normalization mode has been defined for this Operation. If it is 1, then:: getCanonicalizationPatterns() should be defined.
  • 「hasCanonicalizeMethod」. When this Boolean field is set to true, it means that the operation is a simple "matchAndRewrite" style normalization mode, which implements the canonicalize method. If hasCanonicalizer is 0, this function is called by the implementation of:: getCanonicalizationPatterns().
  • 「hasFolder」. This Boolean field indicates whether a generic collapse rule has been defined for this operation. If it is 1, then:: fold() should be defined.

9. Additional statements

One of the goals of table driven operation definition is to automatically generate as many logic and methods as possible for each operation. Having said that, there will always be long tail cases that cannot be covered. In this case, you can use extraClassDeclaration. The code in extraClassDeclaration is copied verbatim to the generated C++ op class.

Note that extraClassDeclaration is a mechanism for long tail cases for advanced users; For widely applicable cases that have not yet been implemented, it is desirable to improve infrastructure.

10. Generate C + + code

OpDefinitionsGen ( https://github.com/llvm/llvm-project/blob/main/mlir/tools/mlir-tblgen/OpDefinitionsGen.cpp )Process the Operation definition specification file (. td file) and generate two files containing corresponding C + + Code: one for declaration and the other for definition. The former is generated by the - gen op decls command line option, while the latter is generated by the - gen OP defs option.

The definition file contains all op method definitions, which can be defined by defining GET_OP_CLASSES to include and enable. For each operation, OpDefinitionsGen generates an operation class and an operand adapter class. In addition, it also contains a comma separated list of all defined Operations, which can be defined by defining GET_OP_LIST to include and enable these Operations.

  • 「Class name and namespaces」 .

For each Operation, the generated C + + class name is the name prefixed with TableGen def, and the Dialect prefix is deleted. First_ Used as a separator. For example, for def TF_AddOp, the C + + class name will be AddOp. We removed the TF prefix because it is more than one Operation scope. Other dialects can also define their own AddOps.

The namespace of the generated C + + class will come from the cppNamespace field of dialog. For example, if the namespace of a dialog is A::B, the Op of the dialog will be placed in namespace a {namespace B {...}}. If Dialect does not specify cppNamespace, we use the name of the Dialect as the namespace.

This means that the name of the generated C + + class does not necessarily exactly match the Operation name in the Operation name. This is to allow flexible naming to meet coding style requirements.

  • 「Operand adaptors」

For each Operation, MLIR automatically generates an operand adapter. This class solves the problem of accessing operands provided as list values without using the magic constant. The operand adapter references an array of values and provides methods with the same name as in the Operation class to access them. For example, for binary arithmetic operations, it can provide. lhs() to access the first operand and. rhs() To access the second operand. The operand adapter class is in the same namespace as the Operation class, and the name of the class consists of the name of the Operation class followed by an adapter.

The operand adapter can also be used to process the function template of Operation:

template <typename BinaryOpTy>
std::pair<Value, Value> zip(BinaryOpTy &&op) {
  return std::make_pair(op.lhs(), op.rhs());;

void process(AddOp op, ArrayRef<Value> newOperands) {

In OneFlow, we can see the generated UserOpAdaptor code, which provides a series of interfaces to access the operands and related properties of the Operation.

// ::mlir::oneflow::UserOp declarations

class UserOpAdaptor {
  UserOpAdaptor(::mlir::ValueRange values, ::mlir::DictionaryAttr attrs, ::mlir::RegionRange regions = {});
  UserOpAdaptor(UserOp &op);
  ::mlir::ValueRange getOperands();
  std::pair<unsigned, unsigned> getODSOperandIndexAndLength(unsigned index);
  ::mlir::ValueRange getODSOperands(unsigned index);
  ::mlir::ValueRange data_input();
  ::mlir::ValueRange ctrl_inputs();
  ::mlir::DictionaryAttr getAttributes();
  ::mlir::StringAttr op_name();
  ::mlir::BoolAttr trainable();
  ::mlir::StringAttr device_tag();
  ::mlir::ArrayAttr device_name();
  ::mlir::IntegerAttr scope_symbol_id();
  ::mlir::ArrayAttr hierarchy();
  ::mlir::DenseIntElementsAttr operand_segment_sizes();
  ::mlir::DenseIntElementsAttr result_segment_sizes();
  ::mlir::StringAttr op_type_name();
  ::mlir::ArrayAttr input_lbn_segment_keys();
  ::mlir::ArrayAttr input_lbn_segment_sizes();
  ::mlir::ArrayAttr output_lbn_segment_keys();
  ::mlir::ArrayAttr output_lbn_segment_sizes();
  ::mlir::ArrayAttr output_lbns();
  ::mlir::LogicalResult verify(::mlir::Location loc);

  ::mlir::ValueRange odsOperands;
  ::mlir::DictionaryAttr odsAttrs;
  ::mlir::RegionRange odsRegions;

11. Constraints

Constraint is a core concept in the definition of table driven Operation: Operation verification and graph Operation matching are based on satisfying constraints. Therefore, both the Operation definition and the rewrite rule are directly related to the write constraint. MLIR at opbase.td( https://github.com/llvm/llvm-project/blob/main/mlir/include/mlir/IR/OpBase.td )The constraint base class is defined in. The constraints of an Operation can cover different ranges, which may be:

  • Focus only on a single attribute (for example, a 32-bit integer greater than 5)
  • Multiple operands and results (for example, the shape of the first result must be the same as the first operand (which can be understood as Tensor))
  • The operation itself is inherent. (for example, if there are no side effects, refer to the case of transfer OP elimination)

We call them single entity constraints, multi-entity constraints and features respectively.

❝ I won't write a summary here. I'll write it after summarizing the next article. ❞

Posted on Fri, 03 Dec 2021 06:42:55 -0500 by neylitalo