Using queries for derived features

We demonstrate how our high performance queries can be easily integrated with other EMF tools using an entirely new case study in which EMF-IncQuery is deeply integrated into the EMF modeling infrastructure to facilitate the incremental evaluation of derived EAttributes and EReferences.

Obtaining the example

  1. Eclipse Modeling or Classic (should include PDE, EMF transactions SDK and EMF validation SDK)
  2. EMF-IncQuery nightly svn and git update site
    1. install IncQuery runtime runtime from (http://viatra.inf.mit.bme.hu/update/incquery-ci/)
    2. install IncQuery validation feature from (http://viatra.inf.mit.bme.hu/update/nightly/)
  3. Indigo update site
    1. install SVN team provider
    2. after restart, install SVNkit or Native JavaHL connector
  4. Open perspective -> SVN Repository exploration
    1. New Repository Location
      1. https://viatra.inf.mit.bme.hu/svn/incquery
    2. go to trunk/examples/code and check out projects under school-incqDerived

Demonstration

  1. Open Perspective -> Plug-in Development
    1. Right click / Run As on schoolIncqDerived.validation.sample/IncqDerived.launch, select first option (IncqDerived)
  2. In the runtime Eclipse, select View -> Open perspective -> SVN Repository exploration
    1. New Repository Location
      1. https://viatra.inf.mit.bme.hu/svn/incquery
    2. trunk/example/models/school-derived
      1. check out school.derived.example
  3. Double-click on file My.schoolincqderived to open editor
    1. Click "EMF-IncQuery Validation Sample" action
    2. Switch to Problems view
    3. Modify the model using the tree editor to see incremental update in markers
      1. Add new teachers to school (affects numberOfTeachers)
      2. Add new year and set startingDate higher than any existing year, also add classes to that year to see changes in the problems view (affects lastYear)
      3. Modify existing year, either set current last year startingDate lower or increase startingDate on another year (affects lastYear)
      4. Add courses and set teachers for them, or modify teachers for existing courses (affects teachersWithMostCourses)

Development

In order to integrate IncQuery into your own application, there are three different steps you have to perform. First, you have to indicate that given derived features in your model will be either volatile or provide proper notification themselves in case of changes. Second, you have to create the queries that correspond to the value of the feature. Finally, in case you would like to use derived features in future queries, you must indicate that the use of derived features is allowed in a particular pattern.

Well-behaving derived features

Derived features in general do not ensure that their value is volatile or that they will send notifications upon changes. Therefore, EMF-IncQuery, by default, does not explore the parts of the model which are only reachable through a derived feature, since the incremental query engine would not work properly without correct notifications. However, in some cases the application developer can ensure that a given derived feature will actually adhere to the above limitations (volatile or notification facility). In EMF-IncQuery, we call these features "well-behaving", since they behave correctly in regards to the query engine.

Well-behaving features can be registered through an extension point (org.eclipse.viatra2.emf.incquery.wellbehaving.derived.features), where the package namespace URI, the name of the classifier containing the feature and the name of the feature must be given. These extensions are read when the EMF-IncQuery runtime plug-in is loaded for the first time.

Example:

</extension>
 <extension id="schoolIncqDerived.incquery.wellbehavedContribution" name=""
    point="org.eclipse.viatra2.emf.incquery.wellbehaving.derived.features">
  <wellbehaving-derived-feature
   classifier-name="School"
   feature-name="numberOfTeachers"
   package-nsUri="http:///schoolIncqDerived.ecore">
  </wellbehaving-derived-feature>
  <wellbehaving-derived-feature
   classifier-name="School"
   feature-name="teachersWithMostCourses"
   package-nsUri="http:///schoolIncqDerived.ecore">
  </wellbehaving-derived-feature>
  <wellbehaving-derived-feature
   classifier-name="School"
   feature-name="lastYear"
   package-nsUri="http:///schoolIncqDerived.ecore">
  </wellbehaving-derived-feature>
</extension>

Derived features in queries

Finally, if you would like to use the derived features in any query (e.g. teachersWithMostCourses from this example), you have to indicate to the EMF-IncQuery code generator, that the derived features in the query are safe to use in the query engine. For this, we provide an annotation that can be put on any query to override the prohibition on derived features.

The annotation is called CodeGeneration (since it affects the generation of query code) and for this specific case, the attribute "override = 'allowDerived'" can be used. The following example shows how a derived feature can be used in a query that returns the courses for the teachers:

@CodeGeneration(override='allowDerived')
@Constraint(mode="problem",location="Teacher",
 message="Teacher $Teacher.name$ has the most courses, including $Course.subject$")
pattern coursesOfTeacherWithMostCourses(School, Teacher, Course) = {
 'School'(School);
 'School'.teachersWithMostCourses(R,School,Teacher);
 'Teacher'(Teacher);
 'Teacher'.courses(R2,Teacher,Course);
 'Course'(Course);
}

Note, that the Constraint annotation in the above example is used for the validation framework (for more information, see our tutorial). 

Implementing well-behaving features in Java

Existing applications that already make use of derived features most likely has some kind of implementation. In EMF applications, the value of a derived features is usually calculated from the value of local features of the given classifier or features of classifiers accessible through navigation from the classifier. Therefore, it is possible to define (during design) the set of dependant features that the value of a given derived feature depends on. These features are either non-derived features (and well-behaving by definition) or well-behaving by implementation, thus they will send notifications about changes.

In order to ensure that a derived feature is well-behaving, the notifications of dependant features must be listened to and proper change notifications must be sent when the value of the given derived feature changes due to a change in the value of a dependant feature. Implementing such behavior for each derived feature independently would require a huge effort, thus we decided to implement an enhanced version of the derived attribute notifier found here.

EMF-IncQuery now contains a Derived Feature Handler that can be used in any EMF application for registering dependant features for any derived feature. It provides a few convenience methods for adding dependant features and sends notifications about changes on the value of the derived feature. These methods can be used for adding local and navigated features to a handler for a given derived feature.

For example, if the teachersWithMostCourses, lastYear and numberOfTeachures features from the above example are used, registering can be done in the following way in the constructor of the classifier:

new DerivedFeatureAdapter(this, SchoolPackage.Literals.SCHOOL__NUMBER_OF_TEACHERS,
  SchoolPackage.Literals.SCHOOL__TEACHERS);

The number of teachers derived attribute only depends on the teacher local feature of School.

new DerivedFeatureAdapter(this, SchoolPackage.Literals.SCHOOL__LAST_YEAR,
  SchoolPackage.Literals.SCHOOL__YEARS, SchoolPackage.Literals.YEAR__STARTING_DATE);

The last year depends on the starting date attribute of the years of the given School.

new DerivedFeatureAdapter(this, SchoolPackage.Literals.SCHOOL__TEACHERS_WITH_MOST_COURSES, 
  SchoolPackage.Literals.SCHOOL__TEACHERS, SchoolPackage.Literals.TEACHER__COURSES);

The teachers with the most courses are calculated from the courses feature of the teachers in a given school.

IncQuery based derived features

Queries that are intended for use as derived features are developed in the same way as regular EMF-IncQuery queries. The only difference is that the generated code should be in the same project with the model code. Once the pattern builders, matchers and signatures are generated for the features, there is a small glueing code required in order to make it work.

Example queries, the derived features will be numberOfTeachers, teacherWithMostCourses and lastYear:

pattern teachers(School,Teacher) = {
 'School'(School);
 'School'.teachers(R,School,Teacher);
 'Teacher'(Teacher);
}	
pattern years(School,Year) = {
 'School'(School);
 'School'.years(R,School,Year);
 'Year'(Year);
}	
pattern coursesOfTeacher(Teacher, Course) = {
 'Teacher'(Teacher);
 'Teacher'.courses(R,Teacher,Course);
 'Course'(Course);
}	
pattern startingDateOfYear(Year,Date) = {
 'Year'(Year);
 'EInt'(Date);
 'Year'.startingDate(R, Year, Date);
}
pattern teacherWithMostCourses(School, Teacher) = {
 find teachers(School,Teacher);
		
 neg 
  @SampleUI(mode="ignore")
  pattern moreCourses(School,Teacher) = {
   'Teacher'(Teacher);
   find coursesOfTeacher(Teacher,Course) # N;
   find teachers(School,Teacher2);
   find coursesOfTeacher(Teacher2,Course2) # M;
   check(N < M);
 }
}
pattern lastYear(School, Year) = {
 find years(School,Year);
 
 neg 
  @SampleUI(mode="ignore")
  shareable pattern laterYear(School, Year) = {
   find years(School,Year);
   find startingDateOfYear(Year,Date);
   find years(School,Year2);
   find startingDateOfYear(Year2,Date2);
   check(Date < Date2);
   Year =/= Year2;
 }
}

First, at the instantiation of the classifier with the derived feature, a derived feature handler is created with proper parameters:

protected SchoolImpl() {
 super();		
 numberOfSchoolsHandler = IncqueryFeatureHelper.createHandler(this,
  SchoolIncqDerivedPackage.Literals.SCHOOL__NUMBER_OF_TEACHERS,
  TeachersMatcher.FACTORY, "School", null, FeatureKind.COUNTER);
 lastYearHandler = IncqueryFeatureHelper.createHandler(this,
  SchoolIncqDerivedPackage.Literals.SCHOOL__LAST_YEAR,
  LastYearMatcher.FACTORY, "School", "Year", FeatureKind.SINGLE_REFERENCE);
 teachersWithMostCoursesHandler = IncqueryFeatureHelper.createHandler(this,
  SchoolIncqDerivedPackage.Literals.SCHOOL__TEACHERS_WITH_MOST_COURSES,
  TeacherWithMostCoursesMatcher.FACTORY, "School", "Teacher", FeatureKind.MANY_REFERENCE);
}

The IncqueryFeatureHandler.createHandler is a convenience method for instantiating and starting a feature handler. The first parameter is the source object, the second is the EStructuralFeature, the third is the factory usable for starting the matcher for the appropriate query. The fourth parameter identifies which parameter of the signature is the given object, while the fifth identifies the target of the feature in the query signature. The last parameter defines what kind of derived feature value is required, counter kind will track the number of matches for the query, single and multi reference kinds define whether a single target or a list of targets are possible.

The last thing to do is to implement the getter functions of the derived features. The feature handler created previously has a getValue method that returns the current value of the given feature. In order to provide more type-safe ways of accessing the current value, there are three other methods: getIntValue() that returns the value of counter kind features, getSingleReference() returns a single object, while getManyReference returns a collection of objects. Note that in the case of many reference, the getter usually returns an EList, which should be instantiated in the getter as an unmodifiable list.

Example getter methods:

public int getNumberOfTeachers() {
 if(numberOfSchoolsHandler != null) {
  return numberOfSchoolsHandler.getIntValue();
 } else {
  return 0;
 }
}
public EList<Teacher> getTeachersWithMostCourses() {
 if(teachersWithMostCoursesHandler != null) {
  Collection<Object> temp = teachersWithMostCoursesHandler.getManyReferenceValue();
  return new UnmodifiableEList<Teacher>(this,
   SchoolIncqDerivedPackage.Literals.SCHOOL__TEACHERS_WITH_MOST_COURSES, temp.size(), temp.toArray());
 } else {
  return new UnmodifiableEList<Teacher>(this,  
   SchoolIncqDerivedPackage.Literals.SCHOOL__TEACHERS_WITH_MOST_COURSES, 0, null);
 }
}
public Year basicGetLastYear() {
 if(lastYearHandler != null) {
  return (Year) lastYearHandler.getSingleReferenceValue();
 } else {
  return null;
 }
}