Skip to content

Commit

Permalink
feat(decorator): added new methods to extract vocab or non-vocab deco…
Browse files Browse the repository at this point in the history 
…rators from model (#888)

* feat(extractor): added new methods to extract vocab or non-vocab decorators from model and fixed some bugs

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: added description for action

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: static variable related issues

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: remove decorator issue

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: corrected the version number

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: updated method from public to private

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: updated param name

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: made action an optional param

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: map decorator issue and updated validateLocale method

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: added a null check on model decorator

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: code refactor and updated tests

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: test case changes

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* fix: used startwith instead of regex

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: minor code refactor

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: added comment to code for more clearity

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

* refactor: updated filterdecorators function

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>

---------

Signed-off-by: Dibyam Agrawal <dibyam.agrawal@gmail.com>
  • Loading branch information
@DibyamAgrawal
DibyamAgrawal authored Aug 7, 2024
1 parent e6e0771 commit ad780d0
Show file tree
Hide file tree
Showing 10 changed files with 532 additions and 45 deletions.
5 changes: 2 additions & 3 deletions packages/concerto-core/api.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,12 @@ class Concerto {
+ string getNamespace(obj)
}
+ object setCurrentTime()
class DecoratorExtractor {
+ void constructor(boolean,string,string,Object)
}
class DecoratorManager {
+ ModelManager validate(decoratorCommandSet,ModelFile[]) throws Error
+ ModelManager decorateModels(ModelManager,decoratorCommandSet,object?,boolean?,boolean?,boolean?,boolean?)
+ ExtractDecoratorsResult extractDecorators(ModelManager,object,boolean,string)
+ ExtractDecoratorsResult extractVocabularies(ModelManager,object,boolean,string)
+ ExtractDecoratorsResult extractNonVocabDecorators(ModelManager,object,boolean,string)
+ void validateCommand(ModelManager,command)
+ Boolean falsyOrEqual(string||,string[])
+ void applyDecorator(decorated,string,newDecorator)
Expand Down
5 changes: 5 additions & 0 deletions packages/concerto-core/changelog.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@
# Note that the latest public API is documented using JSDocs and is available in api.txt.
#

Version 3.17.3 {1c15de121dd80375cf531bc2244efdd0} 2024-07-26
- Added new methods to extract vocab or non-vocab decorators from model
- Fixed some minor bugs related to vocabulary parsing
- Exposed method to validate locale

Version 3.17.2 {eb3903401fdcf7c26cca1f3f8a029171} 2024-07-17
- Added new optimized methods for decorating models with decorator commandsets
- fixed other decorator command set bugs
Expand Down
104 changes: 79 additions & 25 deletions packages/concerto-core/lib/decoratorextractor.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,45 @@ const { MetaModelNamespace } = require('@accordproject/concerto-metamodel');
* Utility functions to work with
* [DecoratorCommandSet](https://models.accordproject.org/concerto/decorators.cto)
* @memberof module:concerto-core
* @private
*/
class DecoratorExtractor {
/**
* The action to be performed to extract all, only vocab or only non-vocab decorators
*/
static Action = {
EXTRACT_ALL: 0,
EXTRACT_VOCAB: 1,
EXTRACT_NON_VOCAB: 2
};

/**
* Create the DecoratorExtractor.
* @constructor
* @param {boolean} removeDecoratorsFromModel - flag to determine whether to remove decorators from source model
* @param {string} locale - locale for extracted vocabularies
* @param {string} dcs_version - version string
* @param {Object} sourceModelAst - the ast of source models
* @param {int} [action=DecoratorExtractor.Action.EXTRACT_ALL] - the action to be performed
*/
constructor(removeDecoratorsFromModel, locale, dcs_version, sourceModelAst) {
constructor(removeDecoratorsFromModel, locale, dcs_version, sourceModelAst, action = DecoratorExtractor.Action.EXTRACT_ALL) {
this.extractionDictionary = {};
this.removeDecoratorsFromModel = removeDecoratorsFromModel;
this.locale = locale;
this.dcs_version = dcs_version;
this.sourceModelAst = sourceModelAst;
this.updatedModelAst = sourceModelAst;
this.action = Object.values(DecoratorExtractor.Action).includes(action)? action : DecoratorExtractor.Action.EXTRACT_ALL;
}

/**
* Returns if the decorator is vocab or not
* @param {string} decoractorName - the name of decorator
* @returns {boolean} - returns true if the decorator is a vocabulary decorator else false
* @private
*/
isVocabDecorator(decoractorName) {
return decoractorName === 'Term' || decoractorName.startsWith('Term_');
}
/**
* Adds a key-value pair to a dictionary (object) if the key exists,
Expand Down Expand Up @@ -105,18 +127,24 @@ class DecoratorExtractor {
Object.keys(vocabObject).forEach(decl =>{
if (vocabObject[decl].term){
strVoc += ` - ${decl}: ${vocabObject[decl].term}\n`;
const otherProps = Object.keys(vocabObject[decl]).filter((str)=>str !== 'term' && str !== 'propertyVocabs');
}
const otherProps = Object.keys(vocabObject[decl]).filter((str)=>str !== 'term' && str !== 'propertyVocabs');
//If a declaration does not have any Term decorator, then add Term_ decorators to yaml
if(otherProps.length > 0){
if (!vocabObject[decl].term){
strVoc += ` - ${decl}: ${decl}\n`;
}
otherProps.forEach(key =>{
strVoc += ` ${key}: ${vocabObject[decl][key]}\n`;
});
}
if (vocabObject[decl].propertyVocabs && Object.keys(vocabObject[decl].propertyVocabs).length > 0){
if (!vocabObject[decl].term){
if (!vocabObject[decl].term && otherProps.length === 0){
strVoc += ` - ${decl}: ${decl}\n`;
}
strVoc += ' properties:\n';
Object.keys(vocabObject[decl].propertyVocabs).forEach(prop =>{
strVoc += ` - ${prop}: ${vocabObject[decl].propertyVocabs[prop].term}\n`;
strVoc += ` - ${prop}: ${vocabObject[decl].propertyVocabs[prop].term || ''}\n`;
const otherProps = Object.keys(vocabObject[decl].propertyVocabs[prop]).filter((str)=>str !== 'term');
otherProps.forEach(key =>{
strVoc += ` ${key}: ${vocabObject[decl].propertyVocabs[prop][key]}\n`;
Expand Down Expand Up @@ -205,6 +233,18 @@ class DecoratorExtractor {
dictVoc[decl.declaration].propertyVocabs[decl.property][extensionKey] = dcs.arguments[0].value;
}
}
else if (decl.mapElement !== ''){
if (!dictVoc[decl.declaration].propertyVocabs[decl.mapElement]){
dictVoc[decl.declaration].propertyVocabs[decl.mapElement] = {};
}
if (dcs.name === 'Term'){
dictVoc[decl.declaration].propertyVocabs[decl.mapElement].term = dcs.arguments[0].value;
}
else {
const extensionKey = dcs.name.split('Term_')[1];
dictVoc[decl.declaration].propertyVocabs[decl.mapElement][extensionKey] = dcs.arguments[0].value;
}
}
else {
if (dcs.name === 'Term'){
dictVoc[decl.declaration].term = dcs.arguments[0].value;
Expand All @@ -227,30 +267,54 @@ class DecoratorExtractor {
let vocabData = [];
Object.keys(this.extractionDictionary).forEach(namespace => {
const jsonData = this.extractionDictionary[namespace];
const patternToDetermineVocab = /^Term_/i;
let dcsObjects = [];
let vocabObject = {};
jsonData.forEach(obj =>{
const decos = JSON.parse(obj.dcs);
const target = this.constructTarget(namespace, obj);
decos.forEach(dcs =>{
if (dcs.name !== 'Term' && !patternToDetermineVocab.test(dcs.name)){
const isVocab = this.isVocabDecorator(dcs.name);
if (!isVocab && this.action !== DecoratorExtractor.Action.EXTRACT_VOCAB){
dcsObjects = this.parseNonVocabularyDecorators(dcsObjects, dcs, this.dcs_version, target);
}
else {
if (isVocab && this.action !== DecoratorExtractor.Action.EXTRACT_NON_VOCAB){
vocabObject = this.parseVocabularies(vocabObject, obj, dcs);
}
});
});
decoratorData = this.transformNonVocabularyDecorators(dcsObjects, namespace, decoratorData);
vocabData = this.transformVocabularyDecorators(vocabObject, namespace, vocabData);
if(this.action !== DecoratorExtractor.Action.EXTRACT_VOCAB){
decoratorData = this.transformNonVocabularyDecorators(dcsObjects, namespace, decoratorData);
}
if(this.action !== DecoratorExtractor.Action.EXTRACT_NON_VOCAB){
vocabData = this.transformVocabularyDecorators(vocabObject, namespace, vocabData);
}
});
return {
decoratorCommandSet: decoratorData,
vocabularies: vocabData
};
}
/**
* Filter vocab or non-vocab decorators
* @param {Object} decorators - the collection of decorators
* @returns {Object} - the collection of filtered decorators
* @private
*/
filterOutDecorators(decorators){
if(!this.removeDecoratorsFromModel){
return decorators;
}

if (this.action === DecoratorExtractor.Action.EXTRACT_ALL){
return undefined;
}
else if(this.action === DecoratorExtractor.Action.EXTRACT_VOCAB){
return decorators.filter((dcs) => !this.isVocabDecorator(dcs.name));
}
else{
return decorators.filter((dcs) => this.isVocabDecorator(dcs.name));
}
}
/**
* Process the map declarations to extract the decorators.
*
Expand All @@ -267,9 +331,7 @@ class DecoratorExtractor {
mapElement: 'KEY'
};
this.constructDCSDictionary(namespace, declaration.key.decorators, constructOptions);
if (this.removeDecoratorsFromModel){
declaration.key.decorators = undefined;
}
declaration.key.decorators = this.filterOutDecorators(declaration.key.decorators);
}
}
if (declaration.value){
Expand All @@ -279,9 +341,7 @@ class DecoratorExtractor {
mapElement: 'VALUE'
};
this.constructDCSDictionary(namespace, declaration.value.decorators, constructOptions);
if (this.removeDecoratorsFromModel){
declaration.value.decorators = undefined;
}
declaration.value.decorators = this.filterOutDecorators(declaration.value.decorators);
}
}
return declaration;
Expand All @@ -304,9 +364,7 @@ class DecoratorExtractor {
property: property.name
};
this.constructDCSDictionary(namespace, property.decorators, constructOptions );
}
if (this.removeDecoratorsFromModel){
property.decorators = undefined;
property.decorators = this.filterOutDecorators(property.decorators);
}
return property;
});
Expand All @@ -328,9 +386,7 @@ class DecoratorExtractor {
declaration: decl.name,
};
this.constructDCSDictionary(namespace, decl.decorators, constructOptions);
}
if (this.removeDecoratorsFromModel){
decl.decorators = undefined;
decl.decorators = this.filterOutDecorators(decl.decorators);
}
if (decl.$class === `${MetaModelNamespace}.MapDeclaration`) {
const processedMapDecl = this.processMapDeclaration(decl, namespace);
Expand All @@ -352,11 +408,9 @@ class DecoratorExtractor {
*/
processModels(){
const processedModels = this.sourceModelAst.models.map(model =>{
if ((model?.decorators.length > 0)){
if ((model?.decorators?.length > 0)){
this.constructDCSDictionary(model.namespace, model.decorators, {});
if (this.removeDecoratorsFromModel){
model.decorators = undefined;
}
model.decorators = this.filterOutDecorators(model.decorators);
}
const processedDecl = this.processDeclarations(model.declarations, model.namespace);
model.declarations = processedDecl;
Expand Down
55 changes: 47 additions & 8 deletions packages/concerto-core/lib/decoratormanager.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -424,15 +424,11 @@ class DecoratorManager {
return newModelManager;
}
/**
* @typedef decoratorCommandSet
* @type {object}
* @typedef vocabularies
* @type {string}
* @typedef ExtractDecoratorsResult
* @type {object}
* @property {ModelManager} modelManager - A model manager containing models stripped without decorators
* @property {decoratorCommandSet} object[] - Stripped out decorators, formed into decorator command sets
* @property {vocabularies} object[] - Stripped out vocabularies, formed into vocabulary files
* @property {*} decoratorCommandSet - Stripped out decorators, formed into decorator command sets
* @property {string[]} vocabularies - Stripped out vocabularies, formed into vocabulary files
*/
/**
* Extracts all the decorator commands from all the models in modelManager
Expand All @@ -449,15 +445,58 @@ class DecoratorManager {
...options
};
const sourceAst = modelManager.getAst(true);
const decoratorExtrator = new DecoratorExtractor(options.removeDecoratorsFromModel, options.locale, DCS_VERSION, sourceAst);
const decoratorExtrator = new DecoratorExtractor(options.removeDecoratorsFromModel, options.locale, DCS_VERSION, sourceAst, DecoratorExtractor.Action.EXTRACT_ALL);
const collectionResp = decoratorExtrator.extract();
return {
modelManager: collectionResp.updatedModelManager,
decoratorCommandSet: collectionResp.decoratorCommandSet,
vocabularies: collectionResp.vocabularies
};
}

/**
* Extracts all the vocab decorator commands from all the models in modelManager
* @param {ModelManager} modelManager the input model manager
* @param {object} options - decorator models options
* @param {boolean} options.removeDecoratorsFromModel - flag to strip out vocab decorators from models
* @param {string} options.locale - locale for extracted vocabulary set
* @returns {ExtractDecoratorsResult} - a new model manager with/without the decorators and vocab yamls
*/
static extractVocabularies(modelManager,options) {
options = {
removeDecoratorsFromModel: false,
locale:'en',
...options
};
const sourceAst = modelManager.getAst(true);
const decoratorExtrator = new DecoratorExtractor(options.removeDecoratorsFromModel, options.locale, DCS_VERSION, sourceAst, DecoratorExtractor.Action.EXTRACT_VOCAB);
const collectionResp = decoratorExtrator.extract();
return {
modelManager: collectionResp.updatedModelManager,
vocabularies: collectionResp.vocabularies
};
}
/**
* Extracts all the non-vocab decorator commands from all the models in modelManager
* @param {ModelManager} modelManager the input model manager
* @param {object} options - decorator models options
* @param {boolean} options.removeDecoratorsFromModel - flag to strip out non-vocab decorators from models
* @param {string} options.locale - locale for extracted vocabulary set
* @returns {ExtractDecoratorsResult} - a new model manager with/without the decorators and a list of extracted decorator jsons
*/
static extractNonVocabDecorators(modelManager,options) {
options = {
removeDecoratorsFromModel: false,
locale:'en',
...options
};
const sourceAst = modelManager.getAst(true);
const decoratorExtrator = new DecoratorExtractor(options.removeDecoratorsFromModel, options.locale, DCS_VERSION, sourceAst, DecoratorExtractor.Action.EXTRACT_NON_VOCAB);
const collectionResp = decoratorExtrator.extract();
return {
modelManager: collectionResp.updatedModelManager,
decoratorCommandSet: collectionResp.decoratorCommandSet
};
}
/**
* Throws an error if the decoractor command is invalid
* @param {ModelManager} validationModelManager the validation model manager
Expand Down
Loading

0 comments on commit ad780d0

Please sign in to comment.