Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.maximem.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Synap supports bulk data migration from other memory systems, knowledge management platforms, or custom datastores. Whether you are replacing an existing memory layer or consolidating data from multiple sources, the SDK’s batch ingestion handles large-scale transfers with idempotency and per-document error reporting.
Dedicated migration tooling (pre-flight validation, rollback, schema mapping) is on the roadmap. Today, sdk.memories.batch_create() with the BOOTSTRAP priority is the recommended approach for bulk data migration.

​
Current Migration Path

​
Using the SDK batch ingestion

sdk.memories.batch_create() with BOOTSTRAP priority is the recommended approach for migrating data today. Bootstrap priority ensures high-throughput processing without impacting real-time ingestion. 
from maximem_synap import MaximemSynapSDK
from maximem_synap.memories.models import CreateMemoryRequest
import asyncio

async def migrate_data(sdk: MaximemSynapSDK, documents: list[dict]):
    """
    Migrate documents to Synap using the SDK batch ingestion.
    """
    batch_size = 100
    total = len(documents)

    for i in range(0, total, batch_size):
        batch = documents[i:i + batch_size]

        requests = [
            CreateMemoryRequest(
                document=doc["content"],
                document_type=doc.get("document_type", "document"),
                document_id=doc.get("id"),
                document_created_at=doc.get("created_at"),
                user_id=doc.get("user_id"),
                customer_id=doc.get("customer_id"),
                mode="long-range",
                metadata={
                    "source": "migration",
                    **doc.get("metadata", {})
                }
            )
            for doc in batch
        ]

        result = await sdk.memories.batch_create(
            documents=requests,
            fail_fast=False
        )

        progress = min(i + batch_size, total)
        print(f"Migrated {progress}/{total} — accepted: {len(result.results)}, rejected: {len(result.errors)}")

    print("Migration complete")

​
Migration Best Practices

Always include a document_id in the metadata for each migrated document. Use the original system’s identifier to ensure idempotency. If you need to re-run the migration (e.g., after fixing a transformation bug), documents with the same document_id will be updated rather than duplicated.
"metadata": {
    "document_id": f"legacy_{original_system}_{original_id}",
    "source": "migration"
}
Include the original created_at timestamp from the source system in the metadata. This helps maintain chronological context even though the memory’s Synap created_at will reflect the migration time.
"metadata": {
    "original_created_at": "2024-06-15T10:30:00Z",
    "source": "migration"
}
Before starting the migration, build a mapping from source system user/customer identifiers to Synap’s user_id and customer_id values. Consistent identity mapping ensures that migrated memories are correctly scoped and retrievable.
Run a small test batch (10-50 documents) first. Verify that:
  • Documents are processed successfully (check ingestion status)
  • Entity resolution produces expected results
  • Retrieval returns the migrated memories for relevant queries
  • Scoping is correct (user, customer, client levels)
For large migrations, implement progress tracking in your migration script. Log batch completion, track failures, and maintain a checkpoint so you can resume from the last successful batch if interrupted.

​
Future Migration Features

Dedicated migration tooling will include the following capabilities when released:

Data Validation Endpoints

Pre-migration validation that checks your data against Synap’s schema requirements, identifies potential issues (missing fields, encoding problems, oversized documents), and provides a detailed report before you commit to the migration.

Progress Tracking

Real-time progress tracking for long-running migrations. Query the migration status, see per-batch results, identify failed documents, and get estimated time to completion.

Rollback Support

The ability to roll back a migration if results are not as expected. Rollback removes all documents ingested as part of a specific migration job, including their vector embeddings and graph connections.

Schema Mapping

Declarative schema mapping that transforms source data formats into Synap’s document model without writing custom code. Define field mappings, transformations, and default values in a configuration file.
Want early access to the dedicated migration tooling? Reach out via the Synap Dashboard with details about your migration use case and data volume.

​
Next Steps

Memory reference

SDK methods for memory creation and batch operations.

Migrate from Competitors

Mapping your existing memory system (Mem0, Zep, Letta, SuperMemory) to Synap.

Bootstrap Ingestion

Patterns for bulk-loading data into Synap.