|
| 1 | +# Developer Guide |
| 2 | + |
| 3 | +All the information required for Magic Modules (MM) to compile a specific |
| 4 | +product is usually contained within its folder under the [`products`](products/) |
| 5 | +folder. For example all the sources for the Google Compute Engine are inside the |
| 6 | +[`products/compute`](products/compute) folder. |
| 7 | + |
| 8 | +When compiling a product you specify the path to the products folder with the |
| 9 | +`-p` parameter. |
| 10 | + |
| 11 | + |
| 12 | +## Anatomy of a Product |
| 13 | + |
| 14 | +A product definition for a specific provider contains a few basic components: |
| 15 | + |
| 16 | + 1. Product definition: `api.yaml` |
| 17 | + 2. Provider dependent product definitions: `<provider>.yaml` |
| 18 | + 3. Examples |
| 19 | + 4. Tests |
| 20 | + |
| 21 | +### Product definition: api.yaml |
| 22 | + |
| 23 | +The `api.yaml` contains all the object definitions for the GCP product. It also |
| 24 | +contains the relationships between objects. |
| 25 | + |
| 26 | +It also includes other product specific information, such as product version and |
| 27 | +API endpoint. |
| 28 | + |
| 29 | +#### Resource |
| 30 | + |
| 31 | +A resource is an object defined and supported by the product. For example a |
| 32 | +virtual machine, a disk, a network, a container, a container cluster are all |
| 33 | +resources from MM's point of view. |
| 34 | + |
| 35 | +A resource defines some basic properties about the resource such as name, |
| 36 | +endpoint name, as well as its properties and parameters. |
| 37 | + |
| 38 | + - !ruby/object:Api::Resource |
| 39 | + name: 'Address' |
| 40 | + kind: 'compute#address' |
| 41 | + base_url: projects/{{project}}/regions/{{region}}/addresses |
| 42 | + exports: |
| 43 | + ... |
| 44 | + description: | |
| 45 | + ... |
| 46 | + parameters: |
| 47 | + ... |
| 48 | + properties: |
| 49 | + ... |
| 50 | + |
| 51 | +##### Resource / [ parameters | properties ] |
| 52 | + |
| 53 | +Specifies fields that the user can specify to define the object and its desired |
| 54 | +state. |
| 55 | + |
| 56 | +> The main difference between a parameter and a property is that a parameter is |
| 57 | +> not part of the object, and will be used by the user to convey data to the |
| 58 | +> generated code, usually to help locate the object. Every property will be |
| 59 | +> eventually persisted as fields of the remote object in GCP. |
| 60 | +
|
| 61 | +Required fields: |
| 62 | + |
| 63 | +- `type`: Specifies the type of the parameter / property |
| 64 | +- `name`: The user facing name of the parameter / property |
| 65 | +- `description`: The description for the parameter / property |
| 66 | + |
| 67 | +Optional fields: |
| 68 | + |
| 69 | +- `required`: true|false indicating if the field if required to be specified by |
| 70 | + the user |
| 71 | +- `input`: true|false indicating if the field will be used as "input", which |
| 72 | + means that the field will be used only during the _creation_ of the object |
| 73 | +- `output`: true|false indicating that the field is produced and controlled by |
| 74 | + the server. The user can use an output field to ensure the value matches |
| 75 | + what's expected. |
| 76 | +- `field`: In case we want the user facing name to be different from the |
| 77 | + corresponding API property we can use `field` to map the user facing name |
| 78 | + (specified by the `name` parameter) to the backend API (specified by the |
| 79 | + `field` parameter) |
| 80 | + |
| 81 | +Example: |
| 82 | + |
| 83 | + - !ruby/object:Api::Type::ResourceRef |
| 84 | + name: 'region' |
| 85 | + resource: 'Region' |
| 86 | + imports: 'name' |
| 87 | + description: | |
| 88 | + URL of the region where the regional address resides. |
| 89 | + This field is not applicable to global addresses. |
| 90 | + required: true |
| 91 | + |
| 92 | +> Please describe these fields from the user's perspective and not from the API |
| 93 | +> perspective. That will allow the generated documentation be useful for the |
| 94 | +> provider user that should not care about how the internals work. This is a |
| 95 | +> core strength of the MM paradigm: the user should neither need to know nor |
| 96 | +> care how the backend works. All he cares is how to describe in a high-level, |
| 97 | +> uniform and elegant way his dependencies. |
| 98 | +> |
| 99 | +> For example a in a virtual machine disk, do not say "disk: array or full URI |
| 100 | +> (self link) to the disks". Instead you should say "disks: reference to a list |
| 101 | +> disks to attach to the machine". |
| 102 | +> |
| 103 | +> Also avoid language or provider specific lingo in the product definitions. |
| 104 | +> |
| 105 | +> For example instead of saying "a hash array of name to datetime" say "a map |
| 106 | +> from name to timestamps". While the former may be correct for a provider in |
| 107 | +> Java it will not be in a Ruby or Python based output. |
| 108 | +
|
| 109 | + |
| 110 | +### Provider dependent product definitions: <provider>.yaml |
| 111 | + |
| 112 | +Each provider has their own product specific definitions that are relevant and |
| 113 | +necessary to properly build the product for such provider. |
| 114 | + |
| 115 | +For example a Puppet provider requires to specify the Puppet Forge URL for the |
| 116 | +module being created. That goes into the product `puppet.yaml`. Similarly a Chef |
| 117 | +provider requires metadata about dependencies of other cookbooks. That goes into |
| 118 | +the product `chef.yaml`. |
| 119 | + |
| 120 | +Detailed provider documentation: |
| 121 | + |
| 122 | +- [`puppet.yaml`][puppet-yaml] on how to create Puppet product definitions |
| 123 | +- [`chef.yaml`][chef-yaml] on how to create Chef product definitions |
| 124 | + |
| 125 | +### Examples |
| 126 | + |
| 127 | +It is strongly encouraged to provide live, working examples to end users. It is |
| 128 | +also good practice to test the examples to make sure they do not become stale. |
| 129 | + |
| 130 | +To test examples you can use our `[end2end](tests/end2end)` mini framework. |
| 131 | + |
| 132 | + |
| 133 | +## Types |
| 134 | + |
| 135 | +When defining a property you have to specify its type. Although some products |
| 136 | +may not care about types and may convert from/to strings, it is important to |
| 137 | +know the type so the compiler can do the best job possible to validate the |
| 138 | +input, ensure consistency, etc. |
| 139 | + |
| 140 | +Currently MM supports the following types: |
| 141 | + |
| 142 | +- `Api::Type::Array`: Represents an array of values. The type of the values is |
| 143 | + identified by the `item\_type`property. |
| 144 | +- `Api::Type::Boolean`: A boolean (true or false) value. |
| 145 | +- `Api::Type::Constant`: A constant that will be passed to the API. |
| 146 | +- `Api::Type::Double`: A double number. |
| 147 | +- `Api::Type::Enum`: Input is allowed only from a fixed list of values, |
| 148 | + specified by the `values` property. |
| 149 | +- `Api::Type::Integer`: An integer number. |
| 150 | +- `Api::Type::Long`: A long number |
| 151 | +- Api::Type::NameValues |
| 152 | +- `Api::Type::NestedObject`: A composite field, composed of inner fields. This |
| 153 | + is used for structures that are nested. |
| 154 | +- `Api::Type::ResourceRef`: A reference to another object described in the |
| 155 | + product. This is used to create strong relationship binding between the |
| 156 | + objects, where the generated code will make sure the object depended upon |
| 157 | + exists. A `ResourceRef` also specifies which property from the dependent |
| 158 | + object we are interested to fetch, by specifying the `resource` and `imports` |
| 159 | + fields. |
| 160 | +- `Api::Type::String`: A string field. |
| 161 | +- `Api::Type::Time`: An RFC 3339 representation of a time stamp. |
| 162 | + |
| 163 | + |
| 164 | +## Exports |
| 165 | + |
| 166 | +When data needs to be read from another object that we depend on we use exports. |
| 167 | + |
| 168 | +For example a virtual machine has many disks. When specifying the disks we do |
| 169 | +not want the user to worry about which representation Google developers chose |
| 170 | +for that object (is it the self\_link? is it just the disk name? is it the |
| 171 | +zone+name?). |
| 172 | + |
| 173 | +Instead we describe the relationship in `api.yaml` and let the generated code |
| 174 | +deal with that. The user will only say "vm X uses disks [A, B, C]". As GCP VM |
| 175 | +requires the disk self\_link the generated code will compute their values and |
| 176 | +pass along. |
| 177 | + |
| 178 | +All fields need to be explicitly exported before they can be imported and |
| 179 | +dependent upon by other objects. This allows us to track and verify the |
| 180 | +dependencies more reliably. |
| 181 | + |
| 182 | +The following exports are allowed: |
| 183 | + |
| 184 | +- `<property-name>`: By specifying the `name` of an existing property of the |
| 185 | + object, we're allowing other objects to import it. |
| 186 | +- `Api::Type::FetchedExternal`: Exports the version of the data that was |
| 187 | + returned from the API. That's useful when the user specification of the |
| 188 | + property is different from the representation coming from the API |
| 189 | +- `Api::Type::SelfLink`: URI that represents the object |
| 190 | + |
| 191 | +> To avoid overexposure and inefficient code, only export the fields you |
| 192 | +> actively require to access from dependent objects, and practice housecleaning |
| 193 | +> by removing exports if you change or remove an object that depends on it. |
| 194 | +
|
| 195 | + |
| 196 | +[puppet-yaml]: docs/puppet.yaml.md |
| 197 | +[chef-yaml]: docs/chef.yaml.md |
0 commit comments