Migration

A migration moves a live site’s content, media and addresses into a Meridian install. The engine imports three kinds of file, each with a dry run, each safe to run again:

You can write these files with any tool. For WordPress and Sanity, the migration toolkit (meridian-migrate) writes them from the platform’s own export through a mapping file, and maps the old site’s addresses to the new documents. Organisations, people and memberships (on engine-dmo) come in through engine-dmo crm import, described in CRM.

engine-cms behaves the same for content, media and redirects.

The orderCopy Anchor Link

1. Configure the project’s types, routes and locales, and apply them: engine-dmo schema apply.
2. Import the media, so content can refer to it by id.
3. Import the content, in an order where a document comes after those it refers to; forward references in a second pass.
4. Review the drafts, then publish.
5. Import the redirects, once the documents they point at exist.

Every import writes through the same validation, permissions and audit as the admin, and nothing is published by an import.

Media manifestsCopy Anchor Link

{
  "format": "meridian.media.v1",
  "assets": [
    {
      "id": "6f1c2b9e-3d4a-5b6c-8d7e-9f0a1b2c3d4e",
      "file": "files/6f1c2b9e-3d4a-5b6c-8d7e-9f0a1b2c3d4e.jpg",
      "filename": "harbour-at-dawn.jpg",
      "alt": { "en-GB": "Fishing boats in the harbour at dawn" },
      "caption": { "en-GB": "The harbour in October" },
      "focal": { "x": 0.5, "y": 0.4 },
      "rights": { "holder": "Jo Smith", "licence": "CC-BY-4.0", "credit": "Photo: Jo Smith" },
      "source": "https://old.example.org/wp-content/uploads/2024/10/harbour.jpg",
      "tags": ["harbour"],
      "source_id": "attachment:1042"
    }
  ]
}

A manifest holds at most 20,000 assets, and unknown keys are refused.

engine-dmo media import --manifest out/media/manifest.json --dry-run --project .
engine-dmo media import --manifest out/media/manifest.json --project .

Each file goes through the media pipeline as an upload does: its type sniffed, its size checked, its metadata (camera location included) stripped. An id the library already holds is reported exists and left alone, so a rehearsal never undoes work done in the library since. The report counts the assets without a licence; the library’s No rights recorded filter is the queue to work through. See Media.

Content filesCopy Anchor Link

{
  "format": "meridian.content.v1",
  "site": "main",
  "type": "article",
  "documents": [
    {
      "id": "1b4e28ba-2fa1-51d2-883f-0016d3cca427",
      "locales": {
        "en-GB": {
          "values": {
            "title": "The harbour in winter",
            "slug": "harbour-in-winter",
            "summary": "What to see when the boats are in.",
            "publishedAt": "2024-12-02T09:00:00Z",
            "body": { "html": "<p>The boats come in at dusk.</p><img src=\"asset:6f1c2b9e-3d4a-5b6c-8d7e-9f0a1b2c3d4e\" alt=\"Boats at dusk\">" }
          }
        }
      }
    }
  ]
}

At most 2,000 documents a file. engine-dmo content export writes the same shape, so an export is the best template for your own files.

engine-dmo content import --site main --file out/content/0001.json --dry-run --project .
engine-dmo content import --site main --file out/content/0001.json --project .

Each locale of each document is create, update, unchanged or reject, with a reason for a rejection: an unknown type, a type not on the site, a locale the site does not enable, a missing or malformed id, an id that belongs to another type or site or is in the trash, field validation errors, or a permission the person running it lacks. Values are validated as a save in the admin is, and a slug another document already holds is a validation error. An import never publishes; a second run of the same file reports every locale unchanged.

Rich text as HTMLCopy Anchor Link

A portable_text field’s value may be {"html": "..."} instead of Portable Text blocks. The import converts it with the same converter the editor uses for pasted HTML, then normalises the blocks to what the field allows (its styles, lists and marks); every adjustment is a note on the row, so nothing changes silently.

• An <img> whose src is asset:<uuid> becomes the field’s image block pointing at that library asset, with the HTML’s alt text. Import the media first and write the ids into the HTML.
• Any other <img> is left out and counted in a note. The import never fetches a file from an address in an import file.
• Only top-level portable_text fields convert; a rich text field inside a blocks or repeater field must be given as Portable Text.

Publishing what you importedCopy Anchor Link

engine-dmo content publish --ids-from out/ids.txt --all-locales --project .

--ids-from reads one document id a line. --all-locales publishes every locale with a working revision, the source locale first, through the publish checks; a locale whose checks fail is skipped and named.

Redirect CSVsCopy Anchor Link

source,target,status,locale,note
/2019/05/harbour-walk,document:1b4e28ba-2fa1-51d2-883f-0016d3cca427,301,,from the old blog
/visit-us,/visit,301,,
/old-shop,,410,,the shop closed

Other columns are ignored.

engine-dmo redirects import --site main --file redirects.csv --dry-run --project .
engine-dmo redirects import --site main --file redirects.csv --project .
engine-dmo redirects list --site main --origin imported --project .
engine-dmo redirects rollback --site main --batch <batch-id> --project .

Every row is checked: the source and status, the locale, a duplicate in the file or in the site’s rules, that a target document exists, that a target path is one the site answers, and loops and chains through every rule the site has. Any error refuses the whole import unless --skip-invalid. A document target not yet published is a warning (the rule serves once it is), and a row whose document is already published at that address is unneeded. The import is one transaction with origin imported and a batch id, which rollback --batch disables. The import writes exact rules only; prefix and wildcard rules belong in routes.yaml. See Redirects and URLs.

The migration toolkitCopy Anchor Link

meridian-migrate run --mapping mapping.yaml --source <export> --out out [--dry-run] [--uploads dir]
meridian-migrate crawl --start https://old.example.org --out crawl.json [--delay 1s] [--max 10000]
meridian-migrate redirects --crawl crawl.json --urls out/urls.json --out redirects.csv --review redirects-review.csv

Every id it writes is a UUID derived from the mapping’s namespace and the source’s own id, so a second run writes the same files, a second media import finds every asset present, and a second content import reports every locale unchanged. A rehearsal can run any number of times against the same staging database.

The mapping fileCopy Anchor Link

format: meridian.migrate.v1
source: wordpress              # or sanity
namespace: example-org-2026    # seeds every id; keep it for every rehearsal and the cutover
site: main                     # the Meridian site the import files name
locales:
  default: en-GB               # a record with no locale, and the first locale a document is created in
  map: { en: en-GB, fr: fr-FR } # source code to Meridian code; a code not named is rejected
media:                         # dotted paths into each asset's metadata
  credit: creditLine
  holder: source.name
  licence: license
  source_url: source.url
  review_tag: rights-review    # optional: a tag key for every asset without a licence
types:
  - from: post                 # the source type: a WordPress post_type or taxonomy, a Sanity _type
    to: article                # the Meridian type
    statuses: [publish]        # the states imported; empty imports the published state only
    dedupe: [slug]             # slug, title or both: later duplicates are rejected
    fields:                    # a field of the Meridian type: a source field (dotted path), or { from, as }
      title: title
      slug: slug
      summary: excerpt
      publishedAt: { from: date, as: value }
      hero: { from: featured_image, as: asset }
      topics: { from: categories, as: reference }
      body: { from: content, as: html }
  - from: category
    to: topic
    fields:
      title: title
      slug: slug

The whole file is checked before anything runs, and every problem is listed together.

What a run writesCopy Anchor Link

--dry-run writes only rejects.csv and summary.json and downloads nothing. Start every migration with it, and treat each reject as a decision: map the type, change a status, or accept the loss.

Records of one translation group become the locales of one document, the default locale first. Within a Meridian type, a document whose first locale has the same slug (or normalised title) as an earlier one is rejected as a duplicate. Media are deduplicated by content: two files with the same SHA-256 import once.

WordPressCopy Anchor Link

source: wordpress reads a WXR export (Tools, Export, All content), versions 1.0 to 1.2.

• Posts, pages and custom post types are records of their post_type, with title, slug, content (HTML), excerpt, author (display name, as text), date and modified (UTC), parent, featured_image, categories, tags, each custom taxonomy by its name, and meta (post meta without a leading underscore). WordPress’s own types (menu items, revisions, templates) are skipped.
• Categories, tags and terms are records of type category, post_tag or the taxonomy, with title, slug, description and parent. An unmapped type is rejected, which is a decision: map it, or leave it.
• Attachments are assets, with alt text from _wp_attachment_image_alt and the caption from the excerpt. Originals are read from --uploads (a copy of wp-content/uploads) when given, else downloaded politely (one request at a time, --delay apart, default 1s, with --user-agent; add a contact address to it).
• HTML is cleaned for the engine’s HTML path: classic editor paragraphs restored, [caption] turned into the image and a paragraph, [embed] into a link, any other shortcode reported once per post and left as text. Each <img> of an imported attachment, including a resized copy (photo-300x200.jpg is photo.jpg), is rewritten to asset:<id>; any other image is reported.
• Multilingual plugins (WPML, Polylang) are not read: every record takes the default locale.

SanityCopy Anchor Link

source: sanity reads the output of sanity dataset export, the .tar.gz or its unpacked directory (data.ndjson, assets.json, images/, files/).

• _id is the source id (a drafts. prefix removed) and _type the source type; system documents are skipped. A draft of a published document is ignored in favour of the published version; a draft with no published version has status draft.
• slug.current is read where a text or slug field names a slug object; {_ref} anywhere is a reference by source id.
• Both asset forms are read, with originalFilename, altText, description (or title) as the caption, creditLine, source {name, id, url} and any custom metadata the mapping’s media paths name. The CDN address is recorded, never fetched.
• Portable Text passes through with its keys, styles, lists and marks; what is Sanity’s is mapped by the mapping, and anything it does not name is left out with a warning.
• Document-level internationalisation (@sanity/document-internationalization) and the two common field-level shapes (sanity-plugin-internationalized-array, and locale objects such as localeString) become locales.

Crawling the old site and matching addressesCopy Anchor Link

meridian-migrate crawl --start https://old.example.org --out crawl.json --user-agent "Example migration; ops@example.org"
meridian-migrate redirects --crawl crawl.json --urls out/urls.json --out redirects.csv --review redirects-review.csv

The crawl keeps to robots.txt (its Crawl-delay wins when longer than --delay), reads the sitemaps it names, /sitemap.xml and any --sitemap, and follows same-host links one request at a time: no nofollow links, no files, fragments and tracking parameters removed, query strings dropped unless --keep-query. It records each page’s status, title and canonical address, and each redirect with its target. --max (default 10,000) stops it and says so.

redirects maps every old address (the source’s own from urls.json, and each crawled page that answered 200) to a document:

• an address the source recorded for a document matches it outright, with confidence 1;
• any other is scored against every document of its locale by the similarity of its last path segment to the slug and of its page title to the title;
• two candidates within 0.05 of each other always go to review;
• the home page is left alone.

A match at or above --threshold (default 0.8) goes to redirects.csv with a document:<id> target and a confidence column; the rest go to the review file with the best candidate, the others and an empty decision column for a person to fill: redirect (copy the row into the redirects file), recreate, or 410. --locale-prefix /fr=fr-FR (repeatable) removes an old locale prefix from the source and sets the locale column.

The runbookCopy Anchor Link

Do every step against staging first, and rehearse the whole cutover at least twice.

Before DNS moves, a rollback is a database restore; a redirect batch alone is engine-dmo redirects rollback --batch <id>.

Not handledCopy Anchor Link

• Links inside migrated rich text that point at the old site stay as addresses; the redirect set serves them.
• Old media addresses (/wp-content/uploads/..., the Sanity CDN) get no redirects to the imported assets.
• Multilingual WordPress (WPML, Polylang).
• Sources other than WordPress and Sanity need their files written by your own tool, in the formats above.