Sample catalog data ingestion

This sample data set emulates the catalog data for a fictional B2B2X Automobile conglomerate called Carvelo. When ingested, this sample data creates a single base catalog that can be configured and filtered to deliver custom catalogs for different sales channels, locales, and customer segments.

The sample catalog data ingestion process described here is a prerequisite for completing the Storefront and Catalog Administrator end-to-end use case that demonstrates how to create custom catalogs to support sales operations for a complex business organization.

About this repository

This repository provides the tools to ingest the sample data set into a Commerce Optimizer instance. The process uses the Commerce Optimizer SDK which is based on the Data Ingestion API.

Also, you can use this repository code as the basis for loading your own commerce data to create a single base catalog to power any Commerce front end:

• Update the metadata, products, price books, and prices data in the data files with your own commerce data.
• Run the script to load the sample data.
• Use the Merchandising GraphQL APIs to retrieve catalog data for your frontend.

What will you do?

You will ingest Product Metadata, Product, Price Book, and Price data for the Carvelo Automotive demo dataset.

Using our Commerce Optimizer Typescript SDK, we will ingest:

• Metadata for our Product attributes
• 1080 Products across our 3 brands (in batches of 100)
• 5 unique Price Books
• 6480 Prices across our 5 Price Books (in batches of 100)

After you ingest the data, setup Adobe Commerce Optimizer by defining catalog views and policies to filter the base catalog data. Then, you can integrate with an Adobe Commerce Edge Delivery Services storefront to deliver custom catalogs for your business use cases.

Environment setup

Make sure you have the following prerequisites installed in your local development environment:

• Node.js version 20.14.0 or higher
• Access to Adobe Commerce Optimizer instance
• Developer access to Adobe Developer Console to create IMS credentials
• Basic familiarity with Commerce Optimizer concepts

Install dependencies

1. Clone this repository to your local development environment.

2. Run the following command to install the necessary dependencies to run the sample data ingestion.

   npm install

Get credentials and tenant ID for your instance

You need the following values to authenticate requests to ingest data from the sample data set to your Commerce Optimizer instance.

• Tenant_ID—Identifies the Commerce Optimizer instance targeted for data ingestion.
• Adobe IMS client_id and client_secret credentials—These authentication credentials are required to authenticate API requests for data ingestion. You create these credentials from the Adobe Developer Console, which requires an Adobe account with developer access to the Adobe Commerce Optimizer.

Get your tenant ID

Find your tenant ID in the access URLs for your Commerce Optimizer instance in Cloud Manager.

1. Log in to your Adobe Experience Cloud account.

2. Under Quick access, click Commerce to open the Commerce Cloud Manager.

   The Commerce Cloud Manager displays a list of instances that are available in your Adobe IMS organization.

3. To view the access URLs including the base URL for the REST and GraphQL APIs, click the information icon next to the instance name.

   

4. Your tenant ID is included in the endpoint details. For example, you can see it in the Catalog Endpoint that follows this pattern:

   https://na1-sandbox.api.commerce.adobe.com/{tenantId}/v1/catalog

   Note: If you don't have access to the Commerce Cloud Manager, contact your system administrator.

Generate the IMS credentials for API authentication

You generate the client_ID and client_secret credentials from the Adobe Developer Console. You must have a system administrator or developer role for the Commerce Optimizer project to complete this configuration. See User Management in the Commerce Optimizer documentation.

1. Log in to the Adobe Developer Console.

2. Select the Experience Cloud Organization for the integration.

3. Create an API project.

   • Add the Adobe Commerce Optimizer Ingestion API to your project. Then, click Next.

   • Select the Default - Cloud Manager profile.

   • Click Save configured API.

4. In the Connected Credentials section, view API configuration details by selecting OAUTH Server-to-Server.

   

5. Copy the Client ID and the Client Secret values to a secure location.

Configure environment variables

The .env.dist file provides the configuration template to instantiate the SDK client and provide secure communication between the client and Commerce Optimizer.

1. Clone this repository to your local development environment.

2. Create environment file

   cp .env.dist .env

3. Edit the .env file to add your credentials.

   • Add the IMS client id and client secret credentials from your Adobe Developer project.

     CLIENT_ID=my-client-id
     CLIENT_SECRET=my-client-secret
     

   • Add the tenant Id for your Commerce Optimizer instance.

     TENANT_ID=my-tenant-id
     REGION=na1
     ENVIRONMENT=sandbox
     

4. Save your changes.

Ingest the data into Commerce Optimizer

The ingestion process handles data in the following order:

1. Product Metadata (10 items)
2. Products (1080 items in batches of 100)
3. Price Books (5 items)
4. Prices (6480 items in batches of 100)

Run the following command to ingest the Carvelo sample data found in the data directory.

node index.js

You should see output similar to the following:

> $ node index.js
Ingesting metadata batch 1 containing 10 items                  #  Ingesting the metadata for the product attributes

...

Metadata batch 1 response: { status: 'ACCEPTED', acceptedCount: 10 }
Successfully ingested 10 out of 10 items
Ingesting products batch 1 containing 100 products
Products batch 1 response: { status: 'ACCEPTED', acceptedCount: 100 }  # Ingesting product data
Ingesting products batch 2 containing 100 products

...

Successfully ingested 1080 out of 1080 products
Ingesting price books batch 1 containing 5 price books                    # Ingesting price books data
Price books batch 1 response: { status: 'ACCEPTED', acceptedCount: 5 }
Successfully ingested 5 out of 5 price books
Ingesting prices batch 1 containing 100 prices                         # Ingesting price data
Prices batch 1 response: { status: 'ACCEPTED', acceptedCount: 100 }
Ingesting prices batch 2 containing 100 prices
Prices batch 2 response: { status: 'ACCEPTED', acceptedCount: 100 }
Ingesting prices batch 3 containing 100 prices
Prices batch 3 response: { status: 'ACCEPTED', acceptedCount: 100 }

...

Successfully ingested 6480 out of 6480 prices

Reset the sample data

To reset the sample catalog data in your Commerce Optimizer instance, run the following script to delete the Carvelo catalog data loaded by the index.js ingestion script.

node reset.js

Configure Commerce Optimizer

Set up the catalog view and polices required to filter the base catalog data to create custom catalogs and pricing based on business use cases.

Create catalog views and policies

From the Commerce Optimizer user interface, create the catalog views and policies required to use the sample data with your storefront.

Note: See the documentation to learn more about catalog views, policies, and policy types policy types.

Create universal policies

Create four universal (STATIC) policies with the following configuration settings:

Policy Name	Attribute	Operator	Value source	Value
West Coast Inc brands	brand	IN	STATIC	Aurora, Bolt, Cruz
East Coast Inc brands	brand	IN	STATIC	Bolt, Cruz
Arkbridge part categories	part_category	IN	STATIC	tires, brakes, suspension
Kingsbluff part categories	part_category	IN	STATIC	tires, brakes

Create each universal policy

1. Log in to Commerce Optimizer.

2. Navigate to Catalog > Policies.

3. Click Add Policy.

4. Enter the policy name.

5. Configure the static filter: Click Add Filter.

6. Add the filter details

   • Click Add Filter
   • For Attribute, enter the attribute name from the table.
   • For Operator, select IN
   • For Value source, select STATIC
   • For value, add the values one at a time, then press enter.

   The modal should look like the screenshot below.

   

7. Apply configuration changes: Click Save.

8. Activate the policy you have just created by clicking the action dots (…) and selecting Enable.

9. Save the policy: Click Save.

Create exclusive (TRIGGER) policies

Create two exclusive policies with the following configuration settings:

Policy Name	Trigger - Name	Trigger - transport type	Attribute	Operator	Value source	Value
Brand	AC-Policy-Brand	HTTP_HEADER	model	IN	TRIGGER	AC-Policy-Brand
Model	AC-Policy-Model	HTTP_HEADER	model	IN	TRIGGER	AC-Policy-Model

Create each exclusive policy

Use the values from the table to create each policy.

1. From the Catalog > Policies page, click Add Policy

2. Enter the policy name.

3. Configure the Trigger details: Click Add Trigger.

   • For Name, add the trigger name value, for example AC-Policy-Brand
   • For Transport type, select HTTP_HEADER
   • Click Save

4. Add the filter details for the trigger: Click Add Filter

   • For Attribute, enter the attribute name from the table.
   • For Operator, select IN.
   • For Value source, select TRIGGER.
   • For Value, enter the trigger name, for example AC-Policy-Brand
   • Click Save.
   • Activate the policy by clicking on the action dots (…) and selecting Enable.

Verify policies

After you create the six policies, verify that the policy list page includes all the policies.

Important: Ensure that all of your policies have a status of Enabled.

Create Catalog Views

1. In the Commerce Optimizer interface, navigate to Catalog > Views.

2. Create three catalog views that use your newly created policies.

   • Click Add Catalog View.

   • Add the following details: Name: Global Catalog Sources: en-US (make sure you hit enter button after typing in this value) Policies: Brand, Model, West Coast Inc brands

     The modal should look like the screenshot below.

     

   • Click Save.

   Repeat the above steps to create two more catalog views. Use the following details:

   Name	Catalog Sources	Policies
   Arkbridge	en-US	Brand Model West Coast Inc brands Arkbridge part categories
   Kingsbluff	en-US	Brand Model East Coast Inc brands Kingsbluff part categories

At this point you have created three catalog views and six policies that reflect the business structure and sales operations flow for the sample Carvelo organization. You are now ready to set up your storefront.

Explore the SDK

Client

To get started with the Adobe Commerce Optimizer SDK, you first need to create the client. In order to do this, use the createClient function provided in the @adobe-commerce/aco-ts-sdk package. The createClient function accepts a client configuration object of type ClientConfig. The ClientConfig object requires the following:

• credentials: The credentials object contains the IMS fields needed to authenticate with the ACO APIs
  • clientId: This is your client id found in the Adobe Developer Console.
  • clientSecret: This is your client secret found in the Adobe Developer Console.
• tenantId: This is the identifier for your ACO instance.
• region: This is the region in which your ACO instance is deployed. Example: na1.
• environment: This is your ACO instance's environment type: sandbox or production

import {
  createClient,
  consoleLogger,
  Client,
  ClientConfig,
  Environment,
  LogLevel,
  Region,
} from "@adobe-commerce/aco-ts-sdk";

// Define your configuration
const config: ClientConfig = {
  credentials: {
    clientId: "my-client-id", // Your IMS client id from Dev Console
    clientSecret: "my-client-secret", // Your IMS client secret from Dev Console
  },
  tenantId: "my-tenant-id", // Your instance's tenant id found in Commerce Cloud Manager UI
  region: "na1" as Region, // Your instance's region found in Commerce Cloud Manager UI
  environment: "sandbox" as Environment, // Your instance's environment type: sandbox or production
  timeoutMs: 30000, // Optional. HTTP timeout override in ms. Default is 10000ms
  logger: consoleLogger(LogLevel.DEBUG), // Optional. Pass in your existing logger. If not provided, a default console logger is used. See Types -> Logger section below.
};

// Initialize the client instance
const client: Client = createClient(config);

Client Functions

Each entity provides a create, update, and delete function (ie. createPriceBooks). Using your Codespaces IDE, inspect each method and their parameter types to get a feel for the API.

Supported Entities:

• Products
• Product Metadata
• Price Books
• Prices

Product Operations

Create Products

import {
  FeedProduct,
  FeedProductStatusEnum,
  FeedProductVisibleInEnum,
} from "@adobe-commerce/aco-ts-sdk";

const product1: FeedProduct = {
  sku: "EXAMPLE-SKU-001",
  source: { locale: "en-US" },
  name: "Example Product 1",
  slug: "example-product-1",
  description: "This is an example product created via the SDK",
  status: FeedProductStatusEnum.Enabled,
  visibleIn: [
    FeedProductVisibleInEnum.Catalog,
    FeedProductVisibleInEnum.Search,
  ],
  attributes: [
    {
      code: "brand",
      values: ["Example Brand"],
    },
  ],
};

const product2: FeedProduct = {
  sku: "EXAMPLE-SKU-002",
  source: { locale: "en-US" },
  name: "Example Product 2",
  slug: "example-product-2",
  description: "This is another example product created via the SDK",
  status: FeedProductStatusEnum.Enabled,
  visibleIn: [
    FeedProductVisibleInEnum.Catalog,
    FeedProductVisibleInEnum.Search,
  ],
  attributes: [
    {
      code: "brand",
      values: ["Example Brand"],
    },
  ],
};

const response = await client.createProducts([product1, product2]);
// response.data: { status: 'ACCEPTED', acceptedCount: 2 }

Update Products

import { FeedProductUpdate } from "@adobe-commerce/aco-ts-sdk";

const productUpdate: FeedProductUpdate = {
  sku: "EXAMPLE-SKU-001",
  source: { locale: "en-US" },
  name: "Updated Product Name",
};

const response = await client.updateProducts([productUpdate]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Delete Products

import { FeedProductDelete } from "@adobe-commerce/aco-ts-sdk";

const productDelete: FeedProductDelete = {
  sku: "EXAMPLE-SKU-001",
  source: { locale: "en-US" },
};

const response = await client.deleteProducts([productDelete]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Product Metadata Operations

Create Product Metadata

import {
  FeedMetadata,
  FeedMetadataDataTypeEnum,
  FeedMetadataVisibleInEnum,
} from "@adobe-commerce/aco-ts-sdk";

const metadata: FeedMetadata = {
  code: "color",
  source: { locale: "en-US" },
  label: "Color",
  dataType: FeedMetadataDataTypeEnum.Text,
  visibleIn: [FeedMetadataVisibleInEnum.ProductDetail],
  filterable: true,
  sortable: true,
  searchable: true,
};

const response = await client.createProductMetadata([metadata]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Update Product Metadata

import { FeedMetadataUpdate } from "@adobe-commerce/aco-ts-sdk";

const metadataUpdate: FeedMetadataUpdate = {
  code: "color",
  source: { locale: "en-US" },
  label: "Updated Color Label",
};

const response = await client.updateProductMetadata([metadataUpdate]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Delete Product Metadata

import { FeedMetadataDelete } from "@adobe-commerce/aco-ts-sdk";

const metadataDelete: FeedMetadataDelete = {
  code: "color",
  source: { locale: "en-US" },
};

const response = await client.deleteProductMetadata([metadataDelete]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Price Book Operations

Create Price Books

import { FeedPricebook } from "@adobe-commerce/aco-ts-sdk";

const pricebook: FeedPricebook = {
  priceBookId: "default",
  name: "Default Price Book",
  currency: "USD",
};

const response = await client.createPriceBooks([pricebook]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Update Price Books

import { FeedPricebook } from "./src/types";

const pricebookUpdate: FeedPricebook = {
  priceBookId: "default",
  name: "Updated Price Book Name",
};

const response = await client.updatePriceBooks([pricebookUpdate]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Delete Price Books

import { FeedPricebook } from "@adobe-commerce/aco-ts-sdk";

const pricebookDelete: FeedPricebook = {
  priceBookId: "default",
};

const response = await client.deletePriceBooks([pricebookDelete]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Price Operations

Create Prices

import { FeedPrices } from "@adobe-commerce/aco-ts-sdk";

const price: FeedPrices = {
  sku: "EXAMPLE-SKU-001",
  priceBookId: "default",
  regular: 99.99,
};

const response = await client.createPrices([price]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Update Prices

import { FeedPricesUpdate } from "@adobe-commerce/aco-ts-sdk";

const priceUpdate: FeedPricesUpdate = {
  sku: "EXAMPLE-SKU-001",
  priceBookId: "default",
  regular: 99.99,
};

const response = await client.updatePrices([priceUpdate]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Delete Prices

import { FeedPricesDelete } from "@adobe-commerce/aco-ts-sdk";

const priceDelete: FeedPricesDelete = {
  sku: "EXAMPLE-SKU-001",
  priceBookId: "default",
};

const response = await client.deletePrices([priceDelete]);
// response.data: { status: 'ACCEPTED', acceptedCount: 1 }

Types

See the types.ts file for all exported type definitions.

Client Config

The ClientConfig object is required to be passed into the createClient function. It configures how the SDK client will interact with Adobe Commerce Optimizer services.

/**
 * Client configuration
 *
 * @param credentials - Adobe IMS credentials for authentication
 * @param tenantId - The tenant ID for the API requests
 * @param region - The region for the API endpoint (e.g., 'us', 'eu')
 * @param environment - The environment to use ('production' or 'sandbox')
 * @param timeoutMs - The timeout for the API requests
 * @param logger - Optional logger for customizing logging behavior
 */
export interface ClientConfig {
  credentials: AdobeCredentials;
  tenantId: string;
  region: Region;
  environment: Environment;
  timeoutMs?: number;
  logger?: Logger;
}

Logger

The Adobe Commerce Optimizer SDK provides flexible logging capabilities through the Logger interface. You can either use the default console logger or implement your own logger that matches the interface.

Default Logger

The default console logger that can be used like this:

import { consoleLogger, LogLevel } from "@adobe-commerce/aco-ts-sdk";

const config: ClientConfig = {
  // ... other config options ...
  logger: consoleLogger(LogLevel.INFO), // Uses default console logger
};

Custom Logger

You can implement your own logger by creating an object that implements the Logger interface. This is useful for integrating with your existing logging infrastructure and customizing log formats.

/**
 * Logger interface for customizing logging behavior
 *
 * @param debug - Log a debug message
 * @param info - Log an info message
 * @param warn - Log a warning message
 * @param error - Log an error message
 */
export interface Logger {
  debug(message: string, meta?: Record<string, unknown>): void;
  info(message: string, meta?: Record<string, unknown>): void;
  warn(message: string, meta?: Record<string, unknown>): void;
  error(message: string, error?: Error, meta?: Record<string, unknown>): void;
}

/**
 * Log level
 *
 * @param DEBUG - Debug log level
 * @param INFO - Info log level
 * @param WARN - Warning log level
 * @param ERROR - Error log level
 */
export enum LogLevel {
  DEBUG = 0,
  INFO = 1,
  WARN = 2,
  ERROR = 3,
}

Using Winston

The Winston Logger interface matches the SDK's Logger interface and is a drop-in logger override.

import winston from "winston";

const logger = winston.createLogger({
  level: "debug",
  format: winston.format.json(),
  transports: [new winston.transports.Console()],
});
// ...
const config: ClientConfig = {
  // ... other config options ...
  logger, // Uses winston logger from your application
};

Using Pino

The Pino Logger interface does not exactly match the SDK's Logger interface, but can be easily adapted to be provided as a logger override.

import pino from "pino";

// Create a simple adaptor for the Pino logger interface
const createPinoAdapter = (pinoInstance) => {
  const logWithMetadata = (level, message, ...args) => {
    const metadata = args[0];
    if (metadata && typeof metadata === "object" && metadata !== null) {
      pinoInstance[level](metadata, message);
    } else {
      pinoInstance[level](message, ...args);
    }
  };

  return {
    debug: (message, ...args) => logWithMetadata("debug", message, ...args),
    info: (message, ...args) => logWithMetadata("info", message, ...args),
    warn: (message, ...args) => logWithMetadata("warn", message, ...args),
    error: (message, error, ...args) => {
      if (error instanceof Error) {
        const metadata = args[0];
        if (metadata && typeof metadata === "object" && metadata !== null) {
          pinoInstance.error({ ...metadata, err: error }, message);
        } else {
          pinoInstance.error({ err: error }, message);
        }
      } else if (error && typeof error === "object") {
        pinoInstance.error(error, message);
      } else {
        pinoInstance.error(message);
      }
    },
  };
};
const logger = createPinoAdapter(pino({ level: "debug" }));
// ...
const config: ClientConfig = {
  // ... other config options ...
  logger, // Uses pino logger from your application
};