Add test cases for "completeness" checks

[goetzst]

We need to decide on test cases for the completeness checks.
We might want to track the steps separately but for now here's the list:

• identify which test to perform
  - for each test
  - create source model(s) + change + expected results

[mickaelclavreul]

From the case description, this is what we need to propose as scenarii:

1. • Null values
     -- source model (SampleClass?)
     -- [ ] create a change model in which we delete the value "owner" of Person::closestFriend
     -- expected result is that transformation runs fine (null access) and target model is produced, with a column "closestFrient" in the table "DefaultType?"
     -- even after multiple readings, explanation in the case is not as clear a it should be. We should provide an example as we do not have a lot of alternatives with the current metamodel...
2. • Dangling objects
     2.1. Class/Type
     -- source model (SampleClass?)
     -- [ ] create a change model in which we add a new Class ("House"?)
     -- expected result is that transformation runs fine (null access) and "House" becomes a new table in the target model
     2.2 Attribute/Column
     2.2.1 Single-valued
     --- source model (SampleClass?)
     --- [ ] create a change model in which we add a new attribute ("marries") which type is "Person" with no owner and which is not added to the "attr" reference of any object.
     --- expected result is that transformation runs fine (null access) and "marries" becomes a Column in table "Family" (first found?)
     2.2.2 Multi-valued
     --- source model (SampleClass?)
     --- [ ] create a change model in which we add a new multi-valued attribute ("cousins") which type is "Person" with no owner and which is not added to the "attr" reference of any object.
     --- expected result is that transformation runs fine (null access) and "cousins" becomes a new table with a pair "Person-Family (first found?)" as primary key
3. • Dangling references
     -- source model (SampleClass?)
     -- [ ] create a change model in which we delete the class "Person", Family::members type becomes a dangling reference
     -- expected result is that transformation runs fine (null access) and the primary key of the corresponding table "members" is a pair "Family-Default Value"

We can provide additional source models but we can use SampleClass to cover all scenarios. For scenario 3, we could test an alternative with dangling Datatypes.

Hope this will help :)

[sandragreiner]

Unfortunately many of the changes cannot be expressed with the change-metamodel so far.
I'm on it to update it and keep this thread updated on which changes we can perform eventually.

[sandragreiner]

I committed an initial change set which refer mostly to the tasks:

-- [ ] create a change model in which we delete the value "owner" of Person::closestFriend

setting the owner to null means that the Attribute is removed from the source model; I added this as special case

• more interestingly: i created a change which sets the type of the closesPerson Attribute null
• and further null values for the name of the Family-class (case 2) and the type of the firstNames (case 5)

[sandragreiner]

create a change model in which we add a new Class ("House"?)
done in case 8

2.2 Attribute/Column
done in case 9

create a change model in which we add a new attribute ("marries") which type is "Person" with no owner and which is not added to the "attr" reference of any object
done in case 10
(even though it is not called 'married' but 'addresses' and has no type at all)

[sandragreiner]

Dangling references
-- source model (SampleClass?)
-- [ ] create a change model in which we delete the class "Person", Family::members type becomes a dangling reference

I can create that change but i cannot apply it savely to the source model currently; even if i delete it here
with EcoreUtil.delete, it will still be referenced by the attribute members

[mickaelclavreul]

@sandragreiner could you explain how correctness folders numbering relates to numbers in the change set?
For instance, correctness1 looks like it contains 1), 2), 3), and 9) cases...
I wonder if we should provide one change model for each case and thus, propose only one change per correctness check?

[sandragreiner]

are you sure that your up-to-date on the main branch?

correctness1 refers only to case 1 of the description file
same for the rest.

[sandragreiner]

and yes, we thought the same, to have just one change per check :)
the only thing 'coupled change' which I added is called respectively;
that one inserts a class and deletes it afterwards.

but maybe we do not need that one? i have not listed it yet in the description for that reason that i was not sure if we should add/keep it

[mickaelclavreul]

Thanks Sandra for your reply, after an update it looks better ;)
We'll see about the "coupled change" if it's relevant or not at the end !

[mickaelclavreul]

Hi @sandragreiner and @TheoLeCalvar ,

I'm on correctness13 expected models and I wonder how strict we would like the version of expected (2). In the current source model, we have an attribute typed by Person but it is multi-valued, so the target model contains a table with foreign keys. Is it expected that this table is not removed by the change in expected 2? Is expected (2) the right description for it?
In my understanding, expected (2) should be related to an attribute of type Person in Family but single-valued, right?

[mickaelclavreul]

I'm about to finish correctness10 to 12 but need to review 13 and previous examples, especially for null values :)

[sandragreiner]

trying to anwering your previous comment:

good question.

I guess it may depend on what happens to the source model when the deletion operations is applied: i assumed that the type of the multi-valued members-reference is unset.
then, resorting to a default type of the column would be okay (behavior(2)) or
as you said - delete the table and create a column in Family-table (as another behavior)

I tried to apply the single change to the source model just now and found that the deletion of the class Person was not applied :/ so you best either counter check @TheoLeCalvar and @mickaelclavreul whether and how the change is applied to the source model. I'm unfortunately mostly busy this afternoon :/

[sandragreiner]

but I can have a look at it tomorrow

[TheoLeCalvar]

i assumed that the type of the multi-valued members-reference is unset.

We found an edge case the other about these behaviors. Depending on which metamodel you used (loaded from ecore or compiled) we had different behaviors for uninitialized attributes (multiValued which is EBoolean), one defaulted to null (it was using Boolean), the other user primite boolean and thus defaulted to false. So to make sure the behavior was not implementation dependent I added false as a default value for the attribute.

I tried to apply the single change to the source model just now and found that the deletion of the class Person was not applied :/

I guess that's because right now the ATOL transformation does not subscribe to change at the resource level (see here, we only go other the resource content and apply transformation rules), so added (or removed) elements in the resource are missed. We'll need to fix that.

[mickaelclavreul]

Done adding expected models for each correctness check (but scales).
Found behaviors with ATOL and ATL that I reported to Theo :)
I put some questions in the change set that we have to answer to provide a reference solution that does what we stated in the article...

[sandragreiner]

If I saw it correctly, @mickaelclavreul , you created the expected models in the way they include the questioned behavior, right?
and this is the behavior ATOL also implements, right? then i would say they should be implemented this way :)

[mickaelclavreul]

That was the idea @sandragreiner :)
Most of the expected models come from the ATL (batch) transformation since it handles multivalued attributes whereas ATOL does not. For cases with multiple possible behaviors, models come from ATL, ATOL, or have been modified manually according to the description you gave in the change set.