Integrating EMF-IncQuery into EMF-based Application

EMF-IncQuery Pattern Matcher API Documentation

Outdated content

The contents of this page have been superseded by the following page on the Eclipse Wiki:

http://wiki.eclipse.org/EMFIncQuery/UserDocumentation/API

Concepts

There are two ways you can use the EMF-IncQuery Pattern Matcher in your application. Either you can use the generic pattern matcher components, or the pattern-specific generated components. In most cases you won’t need the generic pattern matcher, which is much more complex to use. However they conform to the same reflective interfaces, and there is no performance difference between the two. Here we will present a simple introduction to the generated components, which contains many features to help you to integrate it into your java application.

Most important classes and relationships

For every pattern a Match, a Matcher, a MatcherFactory, a Processor and optionally several Evaluator classes are generated. Let’s look into what these classes are responsible for:

Match: This represents a match of the pattern. Basically it is used to transfer data to and from the pattern matcher. The generated variables represent the pattern header parameters. You can use it to specify fixed input parameters to a query, and the results of you queries will be instances of this class. Note, that in each case the pattern parameters can be partially filled. It can be used in conjunction with the Matcher class.
Matcher: This is the main entry point in our API, with pattern-specific query methods. First of all it provides means to initialize a pattern matcher for a given EMF instance model which can either be a Resource, a ResourceSet, or an EObject (in this latter case, the scope of the matching will be the containment tree under the passed EObject). We recommend the use of ResourceSets if possible to avoid cross-reference related issues. After the initialization of the engine the Matcher provides getter methods to retrieve the contents of the match set anytime. For easy iteration over the match set it provides a convenience method (forEachMatch) as well, as this is the most frequent use case in our observation. Of course it contains other handy features (e.g.: countMatches, hasMatch) to help integration. Finally, it provides a DeltaMonitor which can be used to track the changes in the match set in an efficient, event-driven fashion.
MatcherFactory: A pattern-specific factory that can instantiate a Matcher class in a type-safe way. You can get an instance of it via the Matcher class’s factory() method. There are two ways to instantiate a Matcher, with a Notifier (e.g.: Resource, ResourceSet and EObject) as we mentioned already, or with an IncQueryEngine. In both cases if the pattern is already registered (with the same root in the case of the Notifier method) then only a lightweight reference is created which points to the existing engine.
Processor: The Matcher provides a function to iterate over the match set and invoke the process() method of the IMatchProcessor interface with every match. To help with the processing an abstract processor class is generated, which you can override to implement the logic you would like to use. The abstract class unpacks the match variables so it can be used directly in the process() method.
Evaluator: If your pattern contains check expressions an evaluator java code is generated from it. It is used by the engine during a query to evaluate the expression’s result. In most cases you don’t need to deal with these classes.

Lifecycle management

We have an EngineManager singleton class to orchestrate the lifecycle of the IncQueryEngines. There are two types of engines: managed and unmanaged. We recommend the use of managed engines, this is the default behavior, as these engines can share common indices and caches to save memory and cpu time. The EngineManager ensures that there will be no duplicated engine for the same root object. The managed engines can be disposed from the manager if needed. On the other hand creating an unmanaged engine will give you the power and responsibility to use it correctly. It will have no common part with other engines.

The IncQueryEngine is attached to an EMF resource (Resource, ResourceSet or EObject) and hosts the pattern matchers. It will listen on EMF update notifications stemming from the given model in order to maintain live results. Pattern matchers can be registered in the following ways:

Instantiate the specific matcher class generated for the pattern, by passing to the constructor either this engine or the EMF model root.
Use the matcher factory associated with the generated matcher class to achieve the same.
Use the GenericPatternMatcher or the GenericMatcherFactory instead of the various generated classes.

If you want to remove the matchers from the engine you can call the wipe() method on it. It discards any pattern matcher caches and forgets the known patterns. The base index built directly on the underlying EMF model, however, is kept in memory to allow reuse when new pattern matchers are built. If you don’t want to use it anymore call the dispose() instead, to completely disconnect and dismantle the engine.

Typical programming patterns

We recommend trying out the @Handler annotation first, if you’re unfamiliar with the use of the EMF-IncQuery! It generates a sample code with a handler and a dialog that shows the matches of the query in a selected file resource. However you will only need to write just a few lines of code to start working with the pattern matcher:

Using the MatchProcessor

With the MatchProcessor you can iterate over the matches of a pattern quite easily:

Using the DeltaMonitor

There are some usecases where you don’t want to follow every change of a pattern’s match, just gather them together and process them when you’re ready. The DeltaMonitor can do this for you in a convenient way. It is a monitoring object that connects to the rete network as a receiver to reflect changes since an arbitrary state acknowledged by the client.

If a new matching is found, it appears in the matchFoundEvents collection, and disappears when that particular matching cannot be found anymore. If the event of finding a match has been processed by the client, it can be removed manually. In this case, when a previously found matching is lost, the Tuple will appear in the matchLostEvents collection, and disappear upon finding the same matching again. "Matching lost" events can also be acknowledged by removing a Tuple from the collection. If the matching is found once again, it will return to matchFoundEvents.

EMF IncQuery Databinding Documentation

Outdated content

The contents of this page have been superseded by the following page on the Eclipse Wiki:

http://wiki.eclipse.org/EMFIncQuery/UserDocumentation/Databinding

Data binding overview

Data binding [1] is general technique that binds two data/information sources together and maintains synchronization of data. In UI data binding data objects are bound to UI elements and if the binding is done in the proper manner the changes in the data will be automatically reflected on the UI elements (for example a label will be automatically refreshed with new contents).

EMF-IncQuery provides a simple data binding facility that can be used to bind pattern match parameters to UI elements. The feature is mainly intended to be used to integrate EMF-IncQuery queries to newly developed user interfaces, however, the Query Explorer component also uses some related annotations

The scope of this document

This document is intended to be used mainly by developers but the section dealing with data binding related annotations may be useful for EMF-IncQuery end-users too.

Data binding related annotations

The following annotation can be used on patterns within the data binding context:

@QueryExplorer: the message parameter of the Query explorer annotation defines the label feature of the selected match.
@ObservableValue: allows the developer to customize the appearance of a match inside the Details panel. It defines an observable value (as defined in JFace databinding) which can be bound to an Eclipse/JFace UI. This annotation will also trigger the generation of an additional .databinding side-project next to your EMF-IncQuery project, which includes some helper classes that you can use to ease the integration of EMF-IncQuery into your user interface.
- Annotation parameter(s):
  - name (String): the name of the parameter
  - expression (String): the attribute definition without '$' marks. For example @ObservableValue(name = "Year", expression="Y.startingDate")
- The parameters of the pattern are considered the default observable values of the matcher. This means, it is not required to present values like @ObservableValue(name="Y" expression="Y"). However, it is possible to redefine these values by simply defining a new value, e.g. @ObservableValue(name="Y" expression="Y.startingDate").
- It is possible to use the annotation without parameters: in this case, it only represents that a default Observable matcher object should be generated with the default observable values.

The following example is from the school tutorial (see link on the bottom of this page under the 'Examples' section). Here, a pattern is given with various annotations.

The @QueryExplorer annotation will result that the match of the finalPattern (it has at most one match) pattern will have a label with the form of 'The busiest teacher $T.name$ taught the most sociable student $S.name$ in $Y.startingDate$' inside the Query Explorer. The attribute markers will be replaced with the appropriate attribute values based on the current pattern match.

For a more specific example on the @ObservableValue annotation see the next section.

Generated data binding plug-in

The .databinding side-project will only be generated if at least one pattern is annotated with @ObservableValue in your EMF-IncQuery project. In this case a $PatterName$DatabindingAdapter.java class will be generated which is a subclass of DatabindingAdapter.

In the finalPattern context mentioned above, the three @ObservableValue annotations will result that a FinalPatternDatabindingAdapter class will be generated in a .databinding side-project. The getParameterNames method call will return the array of ["Year","Teacher","Student"]. For each of these parameters an IObservableValue can be obtained based on the given attribute expression and a specific match of the pattern.

Please note that if you are binding an IObservableValue instance obtained from the above mentioned class, it is important to pay attention on the binding's update strategy as you should not use a two-way updating strategy (because it would modify the pattern match parameter). For example if you use an org.eclipse.core.databinding.DatabindingContext instance's bindValue method to create the binding, the suggested UpdateValueStrategy is the following:

UpdateValueStrategy.POLICY_NEVER in the UI to pattern match parameter binding
UpdateValueStrategy.POLICY_UPDATE in the pattern match parameter to UI binding

Also worth noting that you must take care of the IObservableValue instances' life-cycle as the pattern match may be removed from the match set of the EMF-IncQuery matcher. The best way to receive notification about the match disappearance is to register a listener on the matcher and upon callback, process the delta monitor's matchFoundEvents and matchLostEvents.

Example and useful resources

Please see the school example (https://viatra.inf.mit.bme.hu/incquery/new/examples/school) which demonstrates the usage of the above mentioned annotations.
The following tutorial is an in depth document about JFace/SWT data binding: http://www.vogella.com/articles/EclipseDataBinding/article.html
The org.eclipse.viatra2.emf.incquery.tooling.gui/src/org/eclipse/viatra2/emf/incquery/queryexplorer/content/detail/DetailObserver.java gives a hint how the generated data binding code can be used (in this case for the Query Explorer's Details view).

DetailObserver.java highlights

The DetailObserver class extends the AbstractObservableList class, thus can be used in data binding contexts. Within the EMF-IncQuery project it is used to server as the input for a TableViewer which diplays pattern match details. The TableViewer's content provider is ObservableListContentProvider, this way data binding is automatically created and maintained.

The cunstuctor initializes the data structures and registers the IValueChangleListener instance on all pattern match parameters which were declared with an @ObservableValue annotation, thus having a getter for its observable value in the appropriate DatabindingAdapter subclass.

The addDetail and removeDetail methods modifiy the contents inside the view and notify (with the fireListChange call) the UI element that the backing content has changed.

We have registered the IValueChangeListener instance on all pattern match parameters in the constructor. Upon attribute modification this listener will be called and the appropriate element can be changed in the details view.

[1] Data binding on wikipedia (http://en.wikipedia.org/wiki/Data_binding)

EMF-IncQuery Validation Framework Documentation

Validation framework overview

EMF-IncQuery provides facilities to create validation rules based on the pattern language of the framework.
These rules can be evaluated on various EMF instance models and upon violations of constraints, markers are automatically created in the Eclipse Problems View.

Example use case

The following scenario describes and illustrates the way to use the framework for validation purposes (see also the BPMN example):

Create an EMF-Incuery project with some patterns
Annotate some pattern with the @Constraint annotation - these will be the constraints. In particular, these patterns will be treated as queries that find the violations of the contraint.
Add the generated .validation project to your run configuration (along with the base EMF-IncQuery plug-in)
Initialize the validation framework on some instance model; use the UI context menu on an EMF editor to do so.
Upon constraint violation markers will be placed in the Problems view. Note that the markers are data-bound to corresponding model elements and the labels will be automatically refreshed (when the model changes).

Annotations

The @Constraint annotation can be used to mark an eiq pattern as a validation rule. If the framework finds at least one pattern with such annotation, an additional .validation project will be generated. This project will be used by the validation framework later in your runtime Eclipse configuration.

Annotation parameters:

location: The location of constraint represents the pattern parameter (the object) the constraint violation needs to be attached to.
message: The message to display when the constraint violation is found. The message may refer the parameter variables between $ symbols, or their EMF features, such as in $Param1.name$.
severity: "warning" or "error"
targetEditorId: An Eclipse editor ID where the validation framework should register itself to the context menu. Use "*" as a wildcard if the constraint should be used always when validation is started.

Generated validation plug-in

The generated .validation project will create a subclass of org.eclipse.viatra2.emf.incquery.validation.runtime.Constraint for each one of the patterns annotated with @Constraint.

Example and useful resources

Please see the BPMN example which demonstrates the usage of the @Constraint annotation.
The org.eclipse.viatra2.emf.incquery.validation.runtime.ConstraintAdapter.java and org.eclipse.viatra2.emf.incquery.validation.runtime.ConstraintViolation.java classes demonstrate the usage of the generated validation code inside the validation framework.

ConstraintAdapter and ConstraintViolation highlights

The validation framework collects all of the Constraints that applies to the constraint extension point schema (defined under org.eclipse.viatra2.emf.incquery.validation.runtime/schema/constraint.exsd). These constraints are initialized on the loaded instance models and upon constraint violation an appropriate error marker is placed in the runtime Eclipse's Problems View.

First for each collected constraint and instance model a ConstraintAdapter is created which will maintain the match set of the pattern (annotated with @Constraint); these matches are constraint violations, that the user needs to be informed about. For each match of the pattern a ConstraintViolation is instantiated, which is responsible for marker creation/update/deletion.

The ConstraintViolation class uses data binding facilities to register the appropriate callback methods on the location objects of the Constraint, this will result in marker text update when an attribute of some location object is modified.

EMF IncQuery Query-based Structural Features Documentation

EMF-IncQuery supports the definition of efficient, incrementally maintained, well-behaving derived features in EMF by using advanced model queries and incremental evaluation for calculating the value of derived features and providing automated code generation for integrating into existing applications.

Main scope of query-based features

Integrate model query results into EMF applications as structural features
Replace low performance derived feature implementations with incrementally evaluated model queries
Provide a flexible interlinking method for fragmented models
Support declarative definition of high-performance computed features with automatic code generation and validation

Overview

Derived features in EMF models represent information (attribute values, references) computed from the rest of the model, such as the number of elements in a given collection or the set of elements satisfying some additional conditions. Such derived features can ease the handling of models significantly, as they appear in the same way as regular features. However, in order to achieve complete transparency for derived features, the developer must ensure that proper change notifications are sent when model modifications cause changes in the value of the derived feature as well. Finally, since the value of the derived feature might be retrieved often, complete recalculation of the value may impact application performance. Therefore, it is better to keep a cached version of the value and update it incrementally based on changes in the model.

Usually, developers who use derived features in EMF have to manually solve each of these challenges for each derived feature they introduce into their model. Furthermore, although the derived features almost always represent the result of a model query (including type constraints, navigation, aggregation), they are implemented as imperative Java code.

In order to help developers in using derived features, EMF-IncQuery supports the definition of model queries that provide the results for the derived feature value calculation and includes out-of-the-box change notification and incremental maintenance of results. Additionally, the automatic generation of the glue code between the EMF model code and EMF-IncQuery offers easy integration into any existing EMF application.

Well-behaving structural features

The incremental approach of EMF-IncQuery relies on change notifications from every object and every feature in the model that is used in the query definitions. Therefore, a regular volatile feature that has no field, therefore there it does not store the current value of the feature and usually does not send proper change notifications (e.g. SET oldValue to newValue ). Such features are ignored by EMF-IncQuery, unless there is an explicit declaration, that the feature implementation sends proper change notifications at all times. These are called well-behaving structural features.

If your application uses volatile (and often derived) features, you provide proper notifications for them and would like to include them in query definitions, you can explicitly tell EMF-IncQuery that the feature is well-behaving. There is two ways to do this:

extend the org.eclipse.viatra2.emf.incquery.base.wellbehaving.derived.features extension point as described here
register your feature directly into the org.eclipse.viatra2.emf.incquery.base.comprehension.WellbehavingDerivedFeatureRegistry using the various registerX methods. Note that you must call this method before executing any queries (i.e. before the first getMatcher() or getEngine() call), since EMF-IncQuery checks the registry when it traverses the model.

Examples

For demonstration, we will use the school metamodel from the introductory example:

Example derived features in this metamodel could be the following:

Students enrolled in a given course: this feature would return the list of Students reached with the students reference from the SchoolClass that is connected by the schoolClass reference to the Course.
Number of specialization courses: this feature would count the number of SpecializationCourse objects in the courses reference of a given School.

Other examples

You can find examples using the EMF-IncQuery based derived features in the following locations:

Simple school example enhanced with derived features

Soft interconnections between models in different resources

Furthermore, we use such derived features in the snapshot models that are used for serializing result sets of EMF-IncQuery matchers.

How to use

EMF-IncQuery only provides the back-end for derived features, the developer must define the feature itself in the metamodel first. Once that is complete, the developer creates the query in a regular EMF-IncQuery project in a query definition file and adds a specific annotation with the correct parameters to invoke the code generation. These steps are detailed in the following:

Definition of the derived feature:
1. In the Ecore model (.ecore file), create the desired EAttribute or EReference in the selected EClass and set the name, type and multiplicity information correctly.
2. Use the following configuration for the other attributes of the created EStructuralFeature:
- derived = true (to indicate that the value of the feature is computed from the model)
- changeable = false (to remove setter methods)
- transient = true (to avoid persisting the value into file)
- volatile = true (to remove the field declaration in the object)
1. In the Generator model (.genmodel), right-click on the top-level element and select Reload, click Next, Load, and Finish to update the Generator model with the changes done in the Ecore model.
2. Right-click on the top-level element and select Generate Model Code to ensure that the getters are properly generated into the EMF model code. You can regenerate the Edit and Editor code as well, though those are not necessary here.
Definition of the model query:
1. Create an EMF-IncQuery project and query definition (.eiq) file as described in the cheat sheet or this tutorial.
2. Make sure that you imported your metamodel into the query definition. Create the EMF-IncQuery generator model, if necessary (.eiqgen file).
3. Make sure that the project containing the generated code is in the same workspace as the EMF-IncQuery project.
4. Create the query corresponding to your derived feature. For example, the students enrolled in a given course feature would look like this:
  
  Note that the first parameter of the pattern is the source of the derived feature and the second is the target. Although not mandatory, is is good practice to use the (This : EClass, Target) format to ease understanding. The @QueryBasedFeature annotation indicates to the code generator that the glue code has to be generated in the model code.
5. Save the query definition, which initiates the code generation. After it completes, you can open the implementation code to ensure that the new getter method is correctly created. Note that a well-behaving derived feature extension is also generated into the plugin.xml of the EMF-IncQuery project to indicate that the given derived feature correctly sends change notifications if the correct project is loaded.
Running the application:
1. Once the glue code is generated, you can use the derived features by including the EMF-IncQuery project into your runtime together with the model project.

Annotation parameters

The @QueryBasedFeatureannotation uses defaults for each possible parameters, which allows developers to avoid using any parameters if the query is correctly written.

In short, parameters are not needed, if the following conditions are satisfied:

The name of the pattern is the same as the name of the derived feature (comparison uses String.equals())
The first parameter is the defining EClass and its type is correctly given (e.g. This : Course)
The second parameter is the target of the derived feature
The derived feature value is a single EObject or a collection of EObjects

If the derived feature and its query does not satisfy the above conditions, the following parameters can be used in the annotation:

feature ="featureName" (default: pattern name) - indicates which derived feature is defined by the pattern
source ="Src" (default: first parameter) - indicates which query parameter (using its name) is the source EObject, the inferred type of this parameter indicates which EClass generated code has to be modified
target ="Trg" (default: second parameter) - indicates which query parameter (using its name) is the target of the derived feature
kind ="single/many/counter/sum/iteration" (default: feature.isMany?many:single) - indicates what kind of calculation should be done on the query results to map them to derived feature values
keepCache ="true/false" (default: true) - indicates whether a separate cache should be kept with the current value. Single and Many kind derived features can work without keeping an additional cache, as the EMF-IncQuery RETE network already keeps a cache of the current values.

Developer documentation

The JavaDoc can be found here.

Overview of the implementation

To support query-backed features captured as derived features, the outputs of the EMF-IncQuery engine need to be integrated into the EMF model access layer at two points: (1) query results are provided in the getter functions of derived features, and (2) query result deltas are processed to generate EMF Notification objects that are passed through the standard EMF API so that application code can process them transparently.

The application accesses both the model and the query results through the standard EMF model access layer -- hence, no modification of application source code is necessary. In the background, our novel derived feature handlers are attached to the EMF model plugin that integrate the generated query components (pattern matchers).

When an EMF application intends to read a soft link (B1), the current value is provided by the corresponding handler (B2) by simply retrieving the value from the cache of the related query. When the application modifies the EMF model (A1), this change is propagated to the generated query components of EMF-IncQuery along notifications (A2), which may update the delta monitors of the handlers (A3). Changes of derived features may in turn trigger further changes in the results sets of other derived features (A4).

Instantiating query-based feature handlers

The easiest way is to create a simple query-based feature and look at the generated code in the getter function.

If you need to create a handler for some reason, use the static getQueryBasedFeatureHandler() methods of the QueryBasedFeatureHelper class.

Example codes that were generated for the school example:

Accessing the current value of query-based features

The most straightforward way is to call the getter method of the feature itself. However, if for some reason that is not possible, you can access the values using the getter methods of the QueryBasedFeatureHandler object. Apart from the generic getValue, there are specific methods (getIntValue, getSingleReferenceValue etc.), each returning a properly typed target for a given source element.

Advanced issues

Creating an iteration query-based feature

It is possible to create a query-based feature that is not simply the result of the model query, but the value calculated by an iteration algorithm on the results of the query.

Important: the iteration algorithm must be able to compute the new value based on it's current value and the new or lost match of the used query.

In order to create your own iteration feature, you need to subclass QueryBasedFeature and implement the following methods:

newMatchIteration(IPatternMatch): based on the match (that just appeared) passed to the method, return a notification that represents the changes in the value of the feature. Note that you should not send out this notification, that is the responsibility of the feature handler.
lostMatchIteration(IPatternMatch): based on the match (that just disappeared) passed to the method, return a notification that represents the changes in the value of the feature. Note that you should not send out this notification, that is the responsibility of the feature handler.
getValueIteration(Object): based on the source element passed to the method, return the value for the feature.

Pitfalls

Code generation fails for derived feature query

Ensure that both the .genmodel file and the model project with the generated EMF model code is available in the same workspace as the EMF-IncQuery project with the query definitions.

Multiple results for a query used in a single (upper bound = 1) feature

If you define a query for a single feature that returns multiple results for a given source model element, the value of the derived feature will in most cases be the value from the last match that appeared. However, it is possible to change the values in a way that the feature will have no value, even though it might have exactly one. Therefore, it is important to define the queries for the feature in a way that only one result is possible. You can either make assumptions on your models and use other ways to ensure that there is only one match, or you can explicitly declare in the pattern, that it should only match once for a given source element. Additionally, you can use the Validation framework of EMF-IncQuery to create feedback for the user when the query would have multiple results indicating that the model is invalid.

The following is an example for a validated, ensured single feature:

UnsupportedOperationException during model editing, even after successful generation

If you have multiple inheritance in your metamodel, it is possible that the getter for a feature will be implemented in more than one place. The easy way to avoid this is to ensure, that query-based features are only inherited from one supertype and that supertype is used as the extension and not only as interface (i.e. that type must be the first in the values of the supertypes feature).

In the unfortunate case when you have query-based features in multiple supertypes, the generator will only override the getter in the implementation class of the defining EClass, so you will have to copy-paste the generated getter code and the handler into the subclass implementation as well.

Future versions of EMF-IncQuery may support the automatic generation into multiple implementation classes.