(Destination) - Couchbase connector 🎉

[teetangh]

What

Implementation of a Couchbase destination connector that enables Airbyte to write data to Couchbase databases. This connector:

• Supports all Airbyte sync modes (append, append-dedup, overwrite)
• Handles schema validation and data type conversions
• Implements efficient batch writing with retry mechanisms
• Provides robust error handling and reporting

How

The implementation achieves its goals through several key mechanisms:

1. Connection Management:

   • Uses PasswordAuthenticator for secure connections
   • Implements connection testing with temporary collection creation
   • Configures appropriate timeouts for operations

2. Data Handling:

   • Implements batch processing with configurable batch sizes (max 1000)
   • Generates unique document IDs based on sync mode and primary keys
   • Validates records against JSON schema before writing
   • Handles null values intelligently based on schema requirements

3. Sync Modes:

   • Append: Generates UUID-based document IDs
   • Append-dedup: Uses primary key fields for document IDs
   • Overwrite: Clears existing collection data before sync

4. Error Handling:

   • Implements exponential backoff retry mechanism
   • Handles document conflicts based on sync mode
   • Provides detailed error reporting via AirbyteTraceMessages

Review guide

1. destination.py:
   • check(): Connection testing and validation
   • write(): Main data writing implementation
   • _prepare_record(): Record validation and document preparation
   • _flush_buffer(): Batch writing with retry logic
   • _setup_collection(): Collection management
   • Helper methods for ID generation, null handling, etc.

User Impact

Users gain the ability to:

• Write Airbyte data to Couchbase databases with configurable sync modes
• Validate data against schemas before writing
• Configure batch sizes for optimal performance
• Get detailed error reporting for troubleshooting

Side effects to consider:

• Overwrite mode clears existing collection data
• Primary key conflicts in append-dedup mode skip new records
• Schema validation may reject invalid records
• Performance depends on batch size configuration

Can this PR be safely reverted and rolled back?

• YES 💚

The implementation is self-contained within the destination-couchbase connector and doesn't modify any shared infrastructure. Rolling back would only affect this specific connector's functionality.