This guide is intended for software engineers who want to create custom YAML-based rules for Migration Toolkit for Applications (MTA) tools.

See the Introduction to the Migration Toolkit for Applications for an overview and the CLI Guide for details.

This guide uses the <MTA_HOME> replaceable variable to denote the path to your MTA installation.

The mta-8.0.0-cli<OS>.zip* extracts a single binary called mta-cli.

When you encounter <MTA_HOME> in this guide, replace it with the actual path to your MTA installation.

The Migration Toolkit for Applications (MTA) contains rule-based migration tools (analyzers) that you can use to analyze the application user interfaces (APIs), technologies, and architectures used by the applications you plan to migrate.

An analyzer rule is a set of instructions used to analyze source code and detect issues that are problematic for migration.

The analyzer parses user-provided rules, applies them to applications' source code, and generates issues for matched rules.

A collection of one or more rules forms a ruleset. Creating rulesets provides a way of organizing multiple rules that achieve a common goal.

The analyzer CLI takes rulesets as input arguments.

You can use the MTA rules internally to perform the following tasks:

MTA builds a data model based on the rule execution results and stores component data and relationships in a graph database. This database can then be queried and updated as required by the migration rules and for reporting purposes.

MTA analyzer rules use the following rule pattern:

when(condition)
 message(message)
 tag(tags)

Rules are written in YAML. They consist of:

Rules instruct the analyzer to take specified actions when given conditions match.

A YAML rule file in MTA contains one or more YAML rules.

Rule metadata contains general information about the rule. The structure of metadata is as follows:

ruleID: "unique_id" 1
labels: 2
  # key=value pair
  - "label1=val1"
  # valid label with value omitted
  - "label2"
  # valid label with empty value
  - "label3="
  # subdomain prefixed key
  - "konveyor.io/label1=val1"
effort: 1 3
category: mandatory 4

Hyperlinks can be provided along with a message or tag action to provide relevant information about the issue, which has been discovered:

# links point to external hyperlinks
# rule authors are expected to provide
# relevant hyperlinks for quick fixes, docs and so on.
links:
  - url: "konveyor.io"
    title: "short title for the link"

Labels are key=val pairs specified for rules or rulesets as well as dependencies. For dependencies, a provider adds the labels to the dependencies when retrieving them. Labels on a ruleset are automatically inherited by all the rules that belong to it.

labels:
- "key1=val1"
- "key2=val2"

The key of a label can be subdomain-prefixed:

labels:
- "konveyor.io/key1=val1"

The value of a label can be empty:

labels:
- "konveyor.io/key="

The value of a label can be omitted. In that case, it is treated as an empty value:

labels:
- "konveyor.io/key"

Examples:

Currently, the analyzer adds the following labels to dependencies:

labels:
- konveyor.io/dep-source=internal
- konveyor.io/language=java

For example, the analyzer adds a konveyor.io/dep-source label to dependencies with a value that indicates whether the dependency is a known open source dependency.

To exclude incidents for all such open source dependencies, you can use --dep-label-selector as follows:

konveyor-analyzer …​ --dep-label-selector !konveyor.io/dep-source=open-source

The Java provider in the analyzer can also add an exclude label to a list of packages. To exclude all such packages, you can use --dep-label-selector and the ! operator as follows:

konveyor-analyzer …​ --dep-label-selector !konveyor.io/exclude

The analyzer engine enables multi-language source code analysis by using providers. The source code of a technology is analyzed by the provider.

The provider publishes what they can do with the source code in terms of capabilities.

The provider condition instructs the analyzer to use a specific provider and one of its capabilities. In general, it follows the <provider_name>.<capability> pattern.

when:
  <provider_name>.<capability>
    <input_fields>

The analyzer currently supports the following provider conditions:

Some providers have dependency_ capability. The dependency_capability means that the provider generates a list of dependencies for a given application.

You can use a dependency_condition to query this list and check whether a certain dependency, with a version range, exists for the application.

For example, to check if a Java application has a certain dependency, you create a java.dependency condition:

when:
  java.dependency:
    name: junit.junit
    upperbound: 4.12.2
    lowerbound: 4.4.0

The Analyzer currently supports the following providers:

The following table summarizes all the providers and their capabilities:

Following the example in the previous table, you can create the first part of the condition that does not contain any of the condition fields.

when:
  java.referenced:
    <fields>

The following table summarizes available providers, their capabilities and all of their fields:

when:
  java.referenced:
    location: PACKAGE
    pattern: org.jboss*

The builtin is an internal provider that can analyze various files and internal metadata generated by the engine. This provider has the following capabilities:

when:
  builtin.file:
    pattern: "<regex_to_match_filenames>"

when:
  builtin.filecontent:
    filePattern: "<regex_to_match_filenames_to_scope_search>"
    pattern: "<regex_to_match_content_in_the_matching_files>"

when:
  builtin.xml:
    xpath: "<xpath_expressions>" 1
    filepaths: 2
      - "/src/file1.xml"
      - "/src/file2.xml"

when:
  builtin.json:
    xpath: "<xpath_expressions>" 1

when:
  # when more than one tag is given, a logical AND is implied
  hasTags:  1
    - "tag1"
    - "tag2"

The java provider analyzes Java source code.

This provider has the following capabilities:

when:
  java.referenced:
    pattern: "<pattern>" 1
    location: "<location>" 2
    annotated: "<annotated>" 3

The go provider analyzes Go source code. This provider’s capabilities are referenced and dependency.

when:
  go.referenced: "<regex_to_find_reference>"

when:
  go.dependency:
    name: "<dependency_name>" 1
    upperbound: "<version_string>" 2
    lowerbound: "<version_string>" 3

The dotnet provider is an external provider used to analyze .NET and C# source code. Currently, the provider supports the referenced capability.

when:
  dotnet.referenced:
    pattern: "<pattern>" 1
    namespace: "<namespace>" 2

The java provider allows scoping the search down to certain source code locations.

java.referenced:
  pattern: org.apache.lucene.search*
  location: IMPORT

would match on each of these imports:

import org.apache.lucene.search.Query;
import org.apache.lucene.search.Sort;
import org.apache.lucene.search.SortField;

range of results, it is recommended to place it directly after the package, not after the dot:

java.referenced:
  pattern: org.apache.lucene.search*
  location: PACKAGE

would match on both the import and the fully qualified usage:

import org.apache.lucene.search.*;

public class Test {
  private org.apache.lucene.search.Query query;
}

(*) right after the package-separation dot (.) for better results.

For instance, looking for a method named “method” declared on org.konveyor.MyClass that returns a List of a type that extends java.lang.String and accepts a single parameter:

java.referenced:
  location: METHOD
  pattern: 'org.konveyor.Myclass.method(*) java.util.List<? extends java.lang.String>'

More information about the possibilities of these patterns can be found in the official Java documentaion, which contain all the information for building these patterns in the createPattern(String, int, int, int) section.

The supported locations are the following:

when:
  java.dependency:
    name: "<dependency_name>" 1
    upperbound: "<version_string>" 2
    lowerbound: "<version_string>" 3

You can add a query to match against specific annotations and their elements, for example:

when:
  java.referenced:
    location: METHOD
    pattern: org.package.MyApplication.runApplication(java.lang.String)
    annotated:
      pattern: org.framework.Bean
      elements:
      - name: url
        value: "http://www.example.com"

This would match against the runApplication method in the following Java code:

package org.package

import org.framework.Bean;

class MyApplication {

    @Bean(url = "http://www.example.com")
    public String runApplication(String str) {
        // ...
    }
}

The structure of the annotated YAML element is:

annotated:
  pattern: a Java regex to match the fully qualified name of the annotation (optional)
  elements: an array of elements to match within the annotation (optional)
  - name: the exact name of the element to match against
    value: a Java regex to match the value of the element

It is also possible to match an annotation with specific elements, without having to specify the symbol it annotates. The following example would also match on the @Bean annotation in the same code as the previous example:

when:
  java.referenced:
    location: ANNOTATION
    pattern: org.framework.Bean
    annotated:
      elements:
        - name: url
          value: "http://www.example.com"

The Language Server used by the Java provider is Eclipse’s JDTLS. Internally, the JDTLS uses the Eclipse Java Development Toolkit, which includes utilities for searching code in projects.

In the pattern element of a java.referenced condition, you can search through application code by using these utilities. For more details, see Class SearchPattern, which contains all the information for building these patterns for createPattern(String, int, int, int).

The analyzer provides two basic logical conditions, and and or, which you can use to aggregate results of other conditions and create more complex queries.

The and condition performs a logical AND operation on the results of an array of conditions.

The and condition matches when all of its child conditions match, for example:

when:
  and:
    - <condition1>
    - <condition2>

when:
  and:
    - java.dependency:
        name: junit.junit
        upperbound: 4.12.2
        lowerbound: 4.4.0
    - java.referenced:
        location: IMPORT
        pattern: junit.junit

The or condition performs a logical OR operation on the results of an array of conditions.

The or condition matches when any of its child conditions matches, for example:

when:
  or:
    - <condition1>
    - <condition2>

when:
  or:
  - java.dependency:
      name: junit.junit
      upperbound: 4.12.2
      lowerbound: 4.4.0
  - java.referenced:
      location: IMPORT
      pattern: junit.junit

You can use the output of one condition as the input for filtering another one in the and and or conditions. This is called condition chaining.

when:
 or:
  - builtin.xml:
      xpath: "//dependencies/dependency"
      filepaths: "{{poms.filepaths}}"
    from: poms
  - builtin.file:
      pattern: pom.xml
    as: poms
    ignore: true

In the above example, the output of the builtin.file condition is saved as poms:

+

[...]
      as: poms
[...]

The variables of builtin.file can then be used in the builtin.xml condition, by writing from and then using mustache templates in the provider_ condition block.

This is how this particular condition knows how to use the variable set to the name poms.

+

[...]
    from: poms
[...]

Then you can use the variables by setting them as mustached templates in any of the inputs to the provider condition.

+

[...]
      filepaths: "{{poms.filepaths}}"
[...]

[...]
    ignore: true
[...]

In the java provider, the filepaths variable must be uppercased. for example:

  when:
    and:
      - java.referenced:
          pattern: org.springframework.web.bind.annotation.RequestMapping
          location: ANNOTATION
        as: annotation
      - java.referenced:
          pattern: org.springframework.stereotype.Controller
          location: ANNOTATION
          filepaths: "{{annotation.Filepaths}}"

Conditions can also be nested within other conditions.

when:
  and:
  - and:
    - go.referenced: "*CustomResourceDefinition*"
    - java.referenced:
        pattern: "*CustomResourceDefinition*"
  - go.referenced: "*CustomResourceDefinition*"

Provider conditions can have associated custom variables. You can use custom variables to capture relevant information from the matched line in the source code. The values of these variables are interpolated with data matched in the source code. These values can be used to generate detailed template messages in a rule’s action (see Message actions). They can be added to a rule in the customVariables field:

- ruleID: lang-ref-004
   customVariables:
   - pattern: '([A-z]+)\.get\(\)' 1
      name: VariableName 2
    message: "Found generic call - {{ VariableName }}" 3
  when:
      java.referenced:
          location: METHOD_CALL
          pattern: com.example.apps.GenericClass.get

Rules can include the following types of actions:

Each rule includes either one of them or both of them.

- ruleID: test-rule
  when:
    <CONDITION>
  message: Test rule matched. Please resolve this migration issue.

Optionally, a message can include hyperlinks to external URLs that provide relevant information about the issue or a quick fix.

links:
  - url: "konveyor.io"
    title: "Short title for the link"

A message can also be a template to include information about the match interpolated through custom variables on the rule.

A tag action instructs the analyzer to generate one or more tags for the application when a match is found. Each string in the tag field can be a comma-separated list of tags. Optionally, you can assign categories to tags.

tag:
  - "tag1,tag2,tag3"
  - "Category=tag4,tag5"

Example

- ruleID: test-rule
  when:
    <CONDITION>
  tag:
  - Language=Golang
  - Env=production
  - Source Code

A tag can be a string or a key=val pair, where the key is treated as a tag category in MTA. Any rule that has a tag action is referred to as a “tagging rule” in this document.

A message action is employed to generate an issue with the specified message when a rule matches, for example:

# When a match is found, the analyzer generates incidents with the same message.
message: "helpful message about the violation"

You can also create a template message to include information about the match that has been interpolated through custom variables on the rule.

- ruleID: lang-ref-004
   customVariables:
   - pattern: '([A-z]+)\.get\(\)'
      name: VariableName
    message: "Found generic call - {{ VariableName }}"
  when:
    <CONDITION>

MTA YAML-based rules have the following basic structure:

when(condition)
 message(message)
 tag(tags)

This section guides you through the process of creating and testing your first MTA YAML-based rule. This assumes that you have already installed MTA. See Installing and running the CLI in the CLI Guide for installation instructions.

In this example, you will create a rule to discover instances where an application defines a jboss-web.xml file containing a <class-loading> element and to provide a link to the documentation that describes how to migrate the code.

$ mkdir /home/<USER>/rule.yaml

You can create custom rules for Golang (Go) applications by referring to this example.

You can use the following custom rule to check if the custom resource definition exposed to the Kubernetes API server has a valid format in the source code.

A set of rules forms a ruleset. MTA does not require every rule file to belong to a ruleset, but you can use rulesets to group multiple rules that achieve a common goal and to pass the rules to the rules engine.

You can create a ruleset by placing one or more YAML rules in a directory and creating a ruleset.yaml file at the directory root. When you pass this directory as input to the MTA CLI by using the --rules option, all rules in this directory are treated as a part of the ruleset defined by the ruleset.yaml file.

The ruleset.yaml file stores the metadata of the ruleset.

name: "Name of the ruleset" 1
description: "Description of the ruleset"
labels: #  2
  - key=val

To perform any application analysis, enter:

$ mta-cli analyze --input=<application_to_analyze> --output=<output_dir> --rules=<custom_rule_dir> --enable-default-rulesets=false

On initiation, the mta-cli tool determines the type of application and the provider needed for analysis. It then starts the provider in a container that has the required dependencies and tools. Finally, the provider uses the analyzer to execute a series of rulesets to analyze the source code.

If you want to group multiple similar rules, you can create a ruleset for them by placing their files in a directory and creating a ruleset.yaml file at the directory’s root. When you pass this directory as input to the MTA CLI using the --rules option, MTA treats all the files in the directory as belonging to the ruleset defined in the ruleset.yaml file.

To run an analysis, use the --rules option in the CLI.