Grid Migrations and why they are a pain.

[KevinJump]

This is a bit of brain dump after a day of thinking and tinkering with grid to block (a lot of this was in my head and why i wasn't starting this one!).

@soreng has started a grid to block grid migration and its very cool - but it sort of scratches at the surface as to why migrating grid to block list is hard.

the problems are many but one that is a real pain is that we don't know the formats of the target elements in the grid, to create the blocks.

within grid config/content you just get an alias for a git element e.g.

{
  "grid": 4,
  "allowAll": false,
  "allowed": [
    "quote",
    "embed",
    "macro",
    "media",
    "rte"
  ]
}

or

 "controls": [
  {
    "value": "Oooh la la",
    "editor": {
      "alias": "headline"
    },
    "styles": null,
    "config": null
  }
]

There are plenty of ways we can guess this, (for example RTE is a Rich text editor) - but beyond the basics it becomes hard, what about leblender, or doctypegrid editors - they leak some information but not always enough.

the true configuration of a grid lives in the grid config files on the source site - this comes mainly from one file grid.editor.config.js in the config folder - except it doesn't exclusively come from there it can be a merging together of several files from packages and config.

{
    "name": "Headline",
    "alias": "headline",
    "view": "textstring",
    "icon": "icon-coin",
    "config": {
        "style": "font-size: 36px; line-height: 45px; font-weight: bold",
        "markup": "<h1>#value#</h1>"
    }
},

So this would give us enough to say, we need to create a doctype with a text string in it called 'headline'

but uSync doesn't touch/have these files - so when we come to import we don't have all that.
So we can't reliably tell you what grid elements should be converted too. in terms of doctypes.

We have a few options

1. try and guess - which will work for 80% of people if they haven't done anything to fancy, with the grid, we can go, oh thats a RTE that's media etc. and we can with @soreng's PR go that's a DTGE - but what we can't reliably do is anything else.

2. validate - and pre-make content types. - We could as part of package validation go through the grid work out what the alias of content types should be and if they don't exist tell people they need to go create them?

again, we could do 80% of this create BlockRTE, BlockMedia etc. and say you just have to do the rest now.?

3. get people to upload their grid config files -

this might not work 100% because config might not contain all grid config, but if people uploaded it somewhere we could just go create all the grid elements from it.

4. only do conversions on the site that already has the grid, so for example on a v8/9/10 site we do have access to all the grid configs so in theory on those sites we can do a proper conversion! - and i think outside of migrations that might well be what we do - for people wanting to convert from NestedContent to BlockList in a current site as they go to v11.

none of this is truly satisfying just yet- but i am thinking about it, ways in which we can make the migration less painful (it might be this cannot be pain free!)

[soreng]

It seems like grid migration is more work than everything else combined 😄

i think the easiest would be to implement some new type of migrators, that would enable to create doctypes and datatypes.

If the migrators encounter a type that can’t be handled, it should just throw an error, and then we should describe how to implement a custom migrator for the unhandled conversion.

Setting up migration for the known types, “RTE”, “macro”, “embed” etc, could be done, but as you mention the implementation may only cover 80%.

Doing them for LeBlender and DTGE should also be possible, but they should be as light as possible.

[KevinJump]

Yes - that is what i am leaning towards.

the idea of nesting the migrators, so the grid migrator is basically parsing config/content and passing it on to 'Block/Grid' Migrators that can do things like create the doctypes, actually migrate the content.

I am a little torn about how we 'create' the new content types - we want these migrators to be able to add properties to them and this might make a generic AdditonalContentType method a little bit to basic, but then are we saying its up the migrator to create the xml, because that seems very wrong.

the separation of concerns between what the grid looks like and the properties inside it should in theory make the code simpler ! 🤞 .

we already have some validation methods, so we too can probably do something that would at least throw warnings if we get something we can't automagically resolve, and say to people "you need to fix this"

but yes the grid is often more work than anything else, it was the bulk of the translation stuff the first time around too :)

[soreng]

The AdditonalContentType is a bit more complex in the version I had looking into, but I don't have anything wokring just yet.

The grid itself output a new element (doctype) for each layout. In the code I have made, rows and layouts that are full width are ignored, since you can just insert at block at root instead (a block being a layout or content). This could be an issue.

For each type of content identified from the mapping of layouts, will then actually be able to have different settings based on it being a "root element" or inside a "full width layout". Just on more thing... to think about.

If there is a list of stuff being generated in order, it may be easier to structure the code.

Steps:
Map grid config settings
There are a lot of small bits and pieces that can be configured templates, layouts and controls. Only required if we want to migrate the settings
This step creates both datatypes and elements, required for layouts (rows / columns) and controls.

Map grid layouts and templates
Generate elements (doctypes) for layouts and settings for each layout. Layouts are just a name (no properties), but settings are full doctypes.

Map grid controls
Generate elements from the referenced grid editors, and the configured settings the same way as for layouts. This step is where we have the most unkowns, since the grid editors can be created in various ways (as mentioned above).

Grid Settings and Grid Editors
Both the settings and editors can be implemented by custom code. This means that we may need to be able to validate and migrate both types.