Skip to main content

​
Overview

The Migration API provides endpoints for bulk data migration to Synap from other memory systems, knowledge management platforms, or custom datastores. Whether you are replacing an existing memory layer or consolidating data from multiple sources, the Migration API is designed to handle large-scale data transfers with validation, progress tracking, and rollback support.
Migration API endpoints are currently under development. For now, use the Bootstrap Ingestion API (POST /v1/memories/batch) to bulk-load data into Synap. See the Memory API for details.

​
Current Migration Path

While the dedicated Migration API is being developed, you can perform data migrations using the existing batch ingestion endpoint.

​
Using the Batch Ingestion API

The POST /v1/memories/batch endpoint with BOOTSTRAP priority is the recommended approach for migrating data today. Bootstrap priority ensures high-throughput processing without impacting real-time ingestion. 
from maximem_synap import MaximemSynapSDK
from maximem_synap.memories.models import CreateMemoryRequest
import asyncio

async def migrate_data(sdk: MaximemSynapSDK, documents: list[dict]):
    """
    Migrate documents to Synap using the SDK batch ingestion.
    """
    batch_size = 100
    total = len(documents)

    for i in range(0, total, batch_size):
        batch = documents[i:i + batch_size]

        requests = [
            CreateMemoryRequest(
                document=doc["content"],
                document_type=doc.get("document_type", "document"),
                document_id=doc.get("id"),
                document_created_at=doc.get("created_at"),
                user_id=doc.get("user_id"),
                customer_id=doc.get("customer_id"),
                mode="long-range",
                metadata={
                    "source": "migration",
                    **doc.get("metadata", {})
                }
            )
            for doc in batch
        ]

        result = await sdk.memories.batch_create(
            documents=requests,
            fail_fast=False
        )

        progress = min(i + batch_size, total)
        print(f"Migrated {progress}/{total} — accepted: {len(result.results)}, rejected: {len(result.errors)}")

    print("Migration complete")

​
Migration Best Practices

Always include a document_id in the metadata for each migrated document. Use the original system’s identifier to ensure idempotency. If you need to re-run the migration (e.g., after fixing a transformation bug), documents with the same document_id will be updated rather than duplicated.
"metadata": {
    "document_id": f"legacy_{original_system}_{original_id}",
    "source": "migration"
}
Include the original created_at timestamp from the source system in the metadata. This helps maintain chronological context even though the memory’s Synap created_at will reflect the migration time.
"metadata": {
    "original_created_at": "2024-06-15T10:30:00Z",
    "source": "migration"
}
Before starting the migration, build a mapping from source system user/customer identifiers to Synap’s user_id and customer_id values. Consistent identity mapping ensures that migrated memories are correctly scoped and retrievable.
Run a small test batch (10-50 documents) first. Verify that:
  • Documents are processed successfully (check ingestion status)
  • Entity resolution produces expected results
  • Retrieval returns the migrated memories for relevant queries
  • Scoping is correct (user, customer, client levels)
For large migrations, implement progress tracking in your migration script. Log batch completion, track failures, and maintain a checkpoint so you can resume from the last successful batch if interrupted.

​
Future Migration API Features

The dedicated Migration API will include the following capabilities when released:

Data Validation Endpoints

Pre-migration validation that checks your data against Synap’s schema requirements, identifies potential issues (missing fields, encoding problems, oversized documents), and provides a detailed report before you commit to the migration.

Progress Tracking

Real-time progress tracking for long-running migrations. Query the migration status, see per-batch results, identify failed documents, and get estimated time to completion.

Rollback Support

The ability to roll back a migration if results are not as expected. Rollback removes all documents ingested as part of a specific migration job, including their vector embeddings and graph connections.

Schema Mapping

Declarative schema mapping that transforms source data formats into Synap’s document model without writing custom code. Define field mappings, transformations, and default values in a configuration file.
Want early access to the Migration API? Reach out via the Synap Dashboard with details about your migration use case and data volume.

​
Next Steps

Memory API

Current API endpoints for memory creation and batch operations.

Migration Overview

Conceptual guide to planning your migration.

Bootstrap Ingestion

Patterns for bulk-loading data into Synap.