Skip to main content

Serialization Error Explanation

The Problem​

The error AttributeDeserializationError: Cannot deserialize 'status' attribute from type: BOOL occurs when trying to query the CardPaymentRecordTableMaster table.

Root Cause​

Schema Mismatch: The PynamoDB model definition doesn't match the actual data type stored in DynamoDB.

Model Definition (Expected)​

# In functions/callback/src/cardPaymentRecordTableMaster.py
status = BooleanAttribute(null=True)

This expects the status field to be stored as DynamoDB type BOOL (true/false).

Actual Data in DynamoDB (Reality)​

status: {'S': 'success'}  // Stored as STRING type, not BOOL

The actual data is:

  • Type: S (STRING)
  • Value: 'success' (a string, not a boolean)

Why It Fails​

When PynamoDB tries to deserialize the record:

  1. It reads status from DynamoDB as type S (STRING) with value 'success'
  2. It tries to deserialize STRING → BooleanAttribute
  3. BooleanAttribute expects either:
    • {'BOOL': true} or {'BOOL': false}
    • Or {'NULL': true} (for null values)
  4. Since it receives {'S': 'success'}, the deserialization fails

Evidence​

From the test run with charge ID chrg_prod_1143084777d1604dd41328c9a760e1eeb2fbe:

DEBUG: status field in DynamoDB: {'S': 'success'} (type keys: ['S'])
DEBUG: status is stored as S with value: success

Solutions​

Change the model to use UnicodeAttribute instead of BooleanAttribute:

# In functions/callback/src/cardPaymentRecordTableMaster.py
status = UnicodeAttribute(null=True)  # Changed from BooleanAttribute

This allows the model to accept string values like 'success', 'failed', etc.

Option 2: Migrate Data to Match Model​

Update all records in DynamoDB to store status as BOOL type:

  • Change 'success' → true (BOOL)
  • Change 'failed' → false (BOOL)
  • Change null → null (NULL)

This requires a data migration script to update existing records.

Option 3: Use Custom Attribute (Advanced)​

Create a custom attribute that handles both string and boolean values during deserialization.

Current Workaround​

The code now uses a workaround:

  1. When PynamoDB deserialization fails, it catches the exception
  2. Uses boto3 directly to query DynamoDB and get raw data
  3. Extracts only the needed fields (like orderId) that don't have schema issues
  4. Creates a minimal record object with just the needed data

This allows the function to work even with the schema mismatch, but it's not ideal for accessing all fields.

Impact​

  • Functionality: The notify function still works (orderId is extracted successfully)
  • Data Access: Other fields like status, tokenId, etc. from the card payment record may not be accessible via PynamoDB for records with this issue
  • Performance: Minimal impact - the workaround adds one extra boto3 query when deserialization fails
  • functions/callback/src/cardPaymentRecordTableMaster.py - Model definition
  • functions/callback/notify.py - Error handling workaround (lines 182-220)
  • DynamoDB table: payment3-card-payment-record-master