Copy variants for A/B tests

A/B testing copy is not just about changing a button label. It means serving one group of users a specific string, serving another group a different string, and ensuring that each user consistently sees the same variant. When copy variants are managed by returning different strings from application conditionals, the variants are invisible to reviewers, untested, and mixed into product code that should not own content decisions.

Messagevisor keeps copy variants in the translation layer, where they can be reviewed in definitions, tested against the evaluation engine, and changed without application deployments. If your runtime provides experiment variation resolvers, for example through a Featurevisor integration, experiment conditions can gate translation overrides directly.

The model for A/B copy variants#

Copy variants in Messagevisor are overrides on a message. Each override can target a specific experiment variation using a conditions expression. At runtime, the SDK evaluates which override applies given the active context - which includes the current experiment variation the user is assigned to.

This means:

Both the control and treatment copy live in one message definition
Reviewers see all variants in a single YAML file
Tests can assert each variant independently with withVariations and withFlags
Changing copy in a variant does not require touching application code

Targeting experiment variations#

Experiments typically assign users to a control or treatment variation. The base translation serves as control copy. Add an override when users in treatment should see different text.

When feature or experiment resolvers are configured, override conditions can reference those variations:

messages/auth/ctaSignup.yml

description: Primary sign-up call to action
translations:
  en: Sign up
  nl: Aanmelden
overrides:
  - key: cta-test-treatment
    conditions:
      experiment: cta-copy-test
      operator: hasVariation
      value: treatment
    translations:
      en: Get started
      nl: Aan de slag

Users assigned to treatment in the cta-copy-test experiment see "Get started." Users in control, or anyone not in the experiment, see "Sign up."

Combining experiment conditions with other targeting#

Experiment conditions compose with attribute conditions and segments. This lets you serve a copy variant only to the subset of an experiment audience that also matches other criteria:

messages/checkout/submitOrder.yml

description: Submit order button label
translations:
  en: Place order
  nl: Bestelling plaatsen
overrides:
  - key: treatment-new-user
    conditions:
      and:
        - experiment: checkout-copy-test
          operator: hasVariation
          value: treatment
        - attribute: daysSinceSignup
          operator: lessThanOrEquals
          value: 30
    translations:
      en: Place your first order
      nl: Plaats je eerste bestelling
  - key: treatment-returning
    conditions:
      experiment: checkout-copy-test
      operator: hasVariation
      value: treatment
    translations:
      en: Express checkout
      nl: Snel afrekenen

New users in treatment see a more personal message. Returning users also in treatment see "Express checkout." Everyone in control, or outside the experiment, sees "Place order."

Targeting a feature flag without an experiment#

When you want to serve different copy for users who have a feature enabled rather than for a specific experiment variation:

messages/billing/upgradePrompt.yml

description: Upgrade prompt shown to users who do not have the new billing UI
translations:
  en: Upgrade your plan
  nl: Upgrade je abonnement
overrides:
  - key: new-billing-ui
    conditions:
      feature: new-billing-ui
      operator: isEnabled
    translations:
      en: Explore new billing options
      nl: Ontdek nieuwe betaalopties

Users with the new-billing-ui feature enabled see the new copy. Everyone else sees the existing copy. Turning off the feature flag reverts the copy without any application change. See Testing copy variants for how to assert both states in CI.

Passing experiment context from the application#

The SDK receives feature and experiment assignments through resolver functions. The exact implementation depends on your experiment system, but the pattern is:

JavaScript

import { createMessagevisor } from "@messagevisor/sdk";
import datafile from "./datafiles/messagevisor-web-en-US.json";
const m = createMessagevisor({
  datafile,
  context: {
    platform: "web",
    plan: currentUser.plan,
    daysSinceSignup: currentUser.daysSinceSignup,
  },
  /**
   * Resolvers
   */
  resolveFlag: (featureKey, context) => {
    return featurevisor.isEnabled(featureKey, context);
  },
  resolveVariation: (experimentKey, context) => {
    return featurevisor.getVariation(experimentKey, context);
  },
});
const ctaLabel = m.translate("auth.ctaSignup");

The exact integration shape depends on your application's experiment system. If you use Featurevisor, see your project's Featurevisor docs for how variation values flow into Messagevisor. If you do not, you can model the assigned variation as a normal Messagevisor attribute and use an attribute condition instead.

Testing copy variants#

Write explicit assertions for each variant to lock the copy and catch unintended changes during rollout. Message tests accept withVariations for experiment assignments and withFlags for feature flag state. Both map directly to the experiment and feature conditions on your overrides.

Experiment variations#

Use withVariations to simulate which variation a user is assigned to. The key is the experiment name; the value is the variation name, such as control or treatment.

tests/messages/auth/ctaSignup.spec.yml

message: auth.ctaSignup
assertions:
  - description: Control - no experiment
    locale: en
    target: web
    expectedTranslation: Sign up
  - description: Control variation
    locale: en
    target: web
    withVariations:
      cta-copy-test: control
    expectedTranslation: Sign up
  - description: Treatment variation
    locale: en
    target: web
    withVariations:
      cta-copy-test: treatment
    expectedTranslation: Get started
  - description: Dutch treatment variation
    locale: nl
    target: web
    withVariations:
      cta-copy-test: treatment
    expectedTranslation: Aan de slag

Feature flags#

Use withFlags to simulate whether a feature is enabled. The key is the feature name; the value is a boolean.

tests/messages/billing/upgradePrompt.spec.yml

message: billing.upgradePrompt
assertions:
  - description: Default - feature disabled
    locale: en
    target: web
    expectedTranslation: Upgrade your plan
  - description: Feature enabled
    locale: en
    target: web
    withFlags:
      new-billing-ui: true
    expectedTranslation: Explore new billing options
  - description: Feature explicitly disabled
    locale: en
    target: web
    withFlags:
      new-billing-ui: false
    expectedTranslation: Upgrade your plan

When an override depends on both a feature and an experiment, use withFlags and withVariations on the same assertion. These tests run in CI and catch accidental copy changes before they reach production.

Matrix for compact multi-locale variant coverage#

When you have many locales and several variants, the matrix assertion keeps the test file concise:

tests/messages/auth/ctaSignup.spec.yml

message: auth.ctaSignup
assertions:
  - description: Control copy for en
    locale: en
    target: web
    expectedTranslation: Sign up
  - description: Control copy for nl
    locale: nl
    target: web
    expectedTranslation: Aanmelden

Reviewing variants in the catalog#

The catalog renders all overrides for a message in an expandable view. Reviewers can see:

The base (control) translation
Each override key and its conditions
The variant translations per locale

This makes experiment copy review accessible to product managers and content teams without requiring them to read YAML directly. Deep-linkable override rows let you point a stakeholder at a specific variant for sign-off before the experiment launches.

Rolling back a copy variant#

Because translation changes go through pull requests, rolling back a copy variant that performed poorly or contained an error is the same as any other Git rollback:

Command

$ git revert <commit-sha-that-added-the-variant>

After the revert merges and CI publishes updated datafiles, the variant copy is gone. No application deployment needed. The experiment can continue running with the control copy while the team decides whether to create a new variant.