Configuration Repository
Microdot uses a flexible configuration repository based on XML files, which are arranged in a hierarchy. This enables configuring each entry (key) with different values per level.
A configuration entry is a point in the configuration repository where some specific data is stored. Each entry has a key and a value. The key is the location of the entry in the XML (similar to its XPath), while the value is the actual data stored. For example, the following XML represent an entry with the key Logging.IncludeStackTrace and value true.
<configuration>
<Logging IncludeStackTrace="true" />
</configuration>As is the nature of XML, all entries sit in a tree-like structure. The keys are a flattened representation of the path leading to the entry. Each configuration entry is a leaf on the tree, and has a parent entry, which in turn can have its own parent entry, which finally will lead to the tree's root, <configuration> (which is always excluded from the entry's key). XML elements and attributes are treated equally (except attributes cannot have child entries), so a an entry's value can either be an attribute's value (Key="value") or an element's content (<Key>value</Key>) or any mix of the two.
Example of a configuration tree
A configuration level is a set of configuration values, which should take effect for services which are running under the specific level.
A service level is either static (such as a global/default level) or defined by the currently running environment. It can represent configuration that is specific to a certain service, data center, environment, etc.
It is possible to define different values for the same key at different levels. This is needed when the same service is installed in different environments that require different values. For instance, a service requires a feature to be toggled on by default, but in a specific environment it must be toggled off. Or if different data centers have different database connection strings, or if different environments (dev, staging, prod) require a different logging-level.
Each configuration level inherits values from the level above it, and possibly overrides them. This inheritance makes it possible to have some global configuration entries, which will contain default values for all levels, and some level-specific configurations for specific configuration entries on each level. A service will only see the the configuration after all the levels applied, which is the effective configuration.
The configuration repository is stored in XML files. Files are stored in folders, one folder for each configuration level.
The mapping of configuration levels to their matching folder is stored in a JSON file that usually named configPaths.json. The file's location can be set by an environment variable named GIGYA_CONFIG_PATHS_FILE (see Environment Variables for more info).
The file is a list of configuration folders, each representing a level, sorted by their priority. The lowest priority is considered to be the first level. Any level with a higher priority will override it.
[
{ "Pattern": "D:/Microdot/Config/_Global/*.config", "Priority" : 1 },
{ "Pattern": "D:/Microdot/Config/_Global/Services/$(appName)/*.config", "Priority": 2 },
{ "Pattern": "D:/Microdot/Config/DataCenters/%DC%/*.config", "Priority": 3 },
{ "Pattern": "D:/Microdot/Config/DataCenters/%DC%/Environments/%ENV%/*.config", "Priority": 4 }
]
In the example above, we use $(appName) which is a special token that is replaced by the name of your service, and we use %DC% and %ENV% which are environment variables. You can use any environment variable to customize your own configuration levels by enclosing the environment variable name between two percent signs (%). $(appName) and environment variables can also be used inside configuration values (but not keys).
As mentioned before, it is possible to specify configuration in different ways. Let's take two entries as an example:
- Key:
Https.Certificates.DbCertificate.CertificatePath, Value:LocalMachine\MY\db_cert - Key:
Https.Certificates.CreditCardProviderCertificate.CertificatePath, Value:LocalMachine\MY\cc_cert
In the following example, the configuration files are are equivalent, and represents the same two configuration entries.
Fully expanded XML elements with content (most verbose)
<configuration>
<Https>
<Certificates>
<DbCertificate>
<CertificatePath>LocalMachine\MY\db_cert</CertificatePath>
</DbCertificate>
<CreditCardProviderCertificate>
<CertificatePath>LocalMachine\MY\cc_cert</CertificatePath>
</CreditCardProviderCertificate>
</Certificates>
</Https>
</configuration>Fully expanded XML elements with attributes
<configuration>
<Https>
<Certificates>
<DbCertificate CertificatePath="LocalMachine\MY\db_cert" />
<CreditCardProviderCertificate CertificatePath="LocalMachine\MY\cc_cert" />
</Certificates>
</Https>
</configuration>Collapsed XML elements with attributes (most compact)
<configuration>
<Https.Certificates.DbCertificate CertificatePath="LocalMachine\MY\db_cert" />
<Https.Certificates.CreditCardProviderCertificate CertificatePath="LocalMachine\MY\db_cert" />
</configuration>Inconsistent mix just to illustrate a point
<configuration>
<Https.Certificates>
<DbCertificate CertificatePath="LocalMachine\MY\db_cert" />
<CreditCardProviderCertificate.CertificatePath>LocalMachine\MY\db_cert</CreditCardProviderCertificate.CertificatePath>
</Https.Certificates>
</configuration>All files that read by the configuration system are monitored for changes. When a change is detected, new values are loaded and the effective value is re-calculated by applying the overrides in the various levels. Any request made to the configuration system will then receive these updated values.
After a file change is detected, the configuration will wait for 5 seconds before it actually reloads the files, to prevent multiple file changes from causing repeated re-calculations.
During a reload, if any file has invalid XML (or cannot be successfully parsed for any reason), then the whole reload process is aborted and the existing configuration entries (before the reload) are used, and a warning is written to the log. Another attempt is made each time more changes are detected.
In order for change detection to work, all configuration locations specified in configPaths.json must be placed under the folder specified in the GIGYA_CONFIG_ROOT environment variable. See Environment Variables for more info.