🐛Source-Stripe - Removed Updated col from customer_balance_transactions (Fix of PR 49811) #49976

williamkaper · 2024-12-20T18:56:33Z

What

This PR fixes a previous PR where I missed removing the updated column from the schema. 🐛Source-Stripe - Fix for Customer_Balance_Transaction records occasionally getting missed #49811.
Unrelated, there was an issue with the invoice line items and subscription items lazy loading with the updated cursor. The events were the parent events, and the logic was exposed to use the sub_attrib element to drill into lines and items respectively to pull out the correct nested object. For now, I've fixed it by just changing the stream to pull those non-lazy from the API when the parent is updated.

How

I've removed updated from schema

Review guide

metadata.yml - version bump
pyproject.toml - version bump
source-stripe/source.py
source-stripe/streams.py
source-stripe/schemas/customer_balance_transactions.json
stripe.md - Updated the documentation

User Impact

Schema should detect the dropped column

Can this PR be safely reverted and rolled back?

I believe so assuming column backfill is enabled.

YES 💚
NO ❌

vercel · 2024-12-20T18:56:37Z

@williamkaper is attempting to deploy a commit to the Airbyte Growth Team on Vercel.

A member of the Team first needs to authorize it.

williamkaper · 2024-12-20T19:02:00Z

@marcosmarxm @maxi297 I just realized I missed removing updated from the schema in the PR last week. It doesn't seem to cause any issues, but I don't want to leave this in a pigged up spot. I apologize for missing this last PR.

marcosmarxm · 2024-12-23T19:50:09Z

Thanks for the quick fix @williamkaper I triggered tests let's wait them.

williamkaper · 2025-01-02T17:21:20Z

Thanks for the quick fix @williamkaper I triggered tests let's wait them.

Thanks. I found a bug in one of @alafanechere changes on one of my older PRs that is already merged. I'm testing a fix for that now. If its tests out, I may just add it to this PR

williamkaper · 2025-01-03T14:05:05Z

airbyte-integrations/connectors/source-stripe/source_stripe/source.py

@@ -206,7 +206,7 @@ def streams(self, config: MutableMapping[str, Any]) -> List[Stream]:
            ],
            **args,
        )
-        subscription_items = UpdatedCursorIncrementalStripeLazySubStream(
+        subscription_items = ParentIncrementalStripeSubStream(


Changed subscription_items and invoice_line_items from UpdatedCursorIncrementalStripeLazySubStream to ParentIncrementalStripeSubStream because UpdatedCursorIncrementalStripeLazySubStream on update used the events from the parent, but the logic is missing to use sub_items_attr to dig out the elements from the invoice/subscription object that comes back from the events.

StripeLazy handles the digging properly
UpdatedCursorIncrementalStripeStream does not

williamkaper · 2025-01-03T14:06:30Z

airbyte-integrations/connectors/source-stripe/source_stripe/streams.py

@@ -698,6 +698,12 @@ def __init__(
            legacy_cursor_field=legacy_cursor_field,
            event_types=event_types,
            response_filter=response_filter,
+            record_extractor=UpdatedCursorIncrementalRecordExtractor(


This was added to mirror the change @alafanechere made below for StripeLazySubStream

marcosmarxm · 2025-01-21T17:59:59Z

@alafanechere and @maxi297 this is a small hotfix to solve a bug in previous contribution. Can we take a look at this?

CLAassistant · 2025-01-28T13:56:56Z

All committers have signed the CLA.

williamkaper · 2025-02-10T16:25:09Z

airbyte-integrations/connectors/source-stripe/source_stripe/streams.py

@@ -698,12 +700,6 @@ def __init__(
            legacy_cursor_field=legacy_cursor_field,
            event_types=event_types,
            response_filter=response_filter,
-            record_extractor=UpdatedCursorIncrementalRecordExtractor(


@ericvanbenschoten-dv01 why was this removed?

… invoices and customers

marcosmarxm · 2025-02-11T01:10:19Z

Hello 👋, I apologize if your contribution hasn't been reviewed or updated yet. I'm working to address the community contribution backlog. I created this GitHub project to help you understand when to expect a review. This week, my goal is to complete all contributions related to Marketplace and Manifest-only connectors. Next week, I will focus on API Certified Connectors, followed by DB Java Connector contributions. If you have any questions about your contribution, please feel free to send me a direct message in the Airbyte Community Slack (and include the link to your pull request).

…e-transactions-stream-fix-updated-col

marcosmarxm · 2025-02-18T23:29:57Z

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (7d4a6e8)

marcosmarxm · 2025-03-06T19:59:39Z

@ericvanbenschoten-dv01 need you to sign the CLA to continue the process here.

ericvanbenschoten-dv01 · 2025-03-06T21:02:03Z

@ericvanbenschoten-dv01 need you to sign the CLA to continue the process here.

@marcosmarxm thank you! CLA has been signed

…e-transactions-stream-fix-updated-col

vercel · 2025-03-13T15:07:50Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 13, 2025 3:14pm

Removed updated column from schema

ef9c9eb

octavia-squidington-iii added area/connectors Connector related issues community connectors/source/stripe labels Dec 20, 2024

williamkaper had a problem deploying to community-ci-auto December 20, 2024 18:56 — with GitHub Actions Error

williamkaper had a problem deploying to community-ci December 20, 2024 18:56 — with GitHub Actions Error

Version bump and readme update

3067333

octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Dec 20, 2024

williamkaper temporarily deployed to community-ci December 20, 2024 19:01 — with GitHub Actions Inactive

williamkaper temporarily deployed to community-ci-auto December 20, 2024 19:01 — with GitHub Actions Inactive

williamkaper temporarily deployed to community-ci December 20, 2024 19:01 — with GitHub Actions Inactive

Fixed incremental sync for UpdatedCursorIncrementalStripeLazySubStream

7300d96

williamkaper added 2 commits January 2, 2025 14:47

Changed stream type

0957c78

Changed subscription items

42d25a2

williamkaper temporarily deployed to community-ci-auto January 3, 2025 14:00 — with GitHub Actions Inactive

williamkaper had a problem deploying to community-ci January 3, 2025 14:00 — with GitHub Actions Error

williamkaper commented Jan 3, 2025

View reviewed changes

marcosmarxm requested review from alafanechere and maxi297 January 21, 2025 17:58

ericvanbenschoten-dv01 had a problem deploying to community-ci January 28, 2025 13:57 — with GitHub Actions Error

ericvanbenschoten-dv01 temporarily deployed to community-ci-auto January 28, 2025 13:57 — with GitHub Actions Inactive

ericvanbenschoten-dv01 force-pushed the williamkaper/source-stripe-customer-balance-transactions-stream-fix-updated-col branch from 11fa9ed to 12bdc4c Compare February 10, 2025 16:23

ericvanbenschoten-dv01 temporarily deployed to community-ci-auto February 10, 2025 16:23 — with GitHub Actions Inactive

ericvanbenschoten-dv01 had a problem deploying to community-ci February 10, 2025 16:23 — with GitHub Actions Error

williamkaper commented Feb 10, 2025

View reviewed changes

Adding custom connector for CustomerBalanceTransactions using parents…

568828d

… invoices and customers

ericvanbenschoten-dv01 force-pushed the williamkaper/source-stripe-customer-balance-transactions-stream-fix-updated-col branch from 12bdc4c to 568828d Compare February 10, 2025 16:43

ericvanbenschoten-dv01 had a problem deploying to community-ci February 10, 2025 16:44 — with GitHub Actions Error

ericvanbenschoten-dv01 temporarily deployed to community-ci-auto February 10, 2025 16:44 — with GitHub Actions Inactive

Merge branch 'master' into williamkaper/source-stripe-customer-balanc…

2df5cbb

…e-transactions-stream-fix-updated-col

marcosmarxm had a problem deploying to community-ci February 18, 2025 22:22 — with GitHub Actions Error

marcosmarxm temporarily deployed to community-ci-auto February 18, 2025 22:22 — with GitHub Actions Inactive

chore: auto-fix lint and format issues

7d4a6e8

octavia-approvington temporarily deployed to community-ci February 18, 2025 23:32 — with GitHub Actions Inactive

octavia-approvington temporarily deployed to community-ci-auto February 18, 2025 23:32 — with GitHub Actions Inactive

octavia-approvington temporarily deployed to community-ci February 18, 2025 23:32 — with GitHub Actions Inactive

Merge branch 'master' into williamkaper/source-stripe-customer-balanc…

51d24be

…e-transactions-stream-fix-updated-col

marcosmarxm temporarily deployed to community-ci March 13, 2025 15:07 — with GitHub Actions Inactive

marcosmarxm temporarily deployed to community-ci-auto March 13, 2025 15:07 — with GitHub Actions Inactive

marcosmarxm temporarily deployed to community-ci March 13, 2025 15:07 — with GitHub Actions Inactive

marcosmarxm requested review from davydov-d and removed request for alafanechere March 13, 2025 15:09

vercel bot deployed to Preview March 13, 2025 15:14 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛Source-Stripe - Removed Updated col from customer_balance_transactions (Fix of PR 49811) #49976

🐛Source-Stripe - Removed Updated col from customer_balance_transactions (Fix of PR 49811) #49976

williamkaper commented Dec 20, 2024 •

edited

Loading

vercel bot commented Dec 20, 2024

williamkaper commented Dec 20, 2024

marcosmarxm commented Dec 23, 2024

williamkaper commented Jan 2, 2025

williamkaper Jan 3, 2025 •

edited

Loading

williamkaper Jan 3, 2025

marcosmarxm commented Jan 21, 2025

CLAassistant commented Jan 28, 2025 •

edited

Loading

williamkaper Feb 10, 2025

marcosmarxm commented Feb 11, 2025

marcosmarxm commented Feb 18, 2025 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 6, 2025 •

edited

Loading

ericvanbenschoten-dv01 commented Mar 6, 2025

vercel bot commented Mar 13, 2025 •

edited

Loading

🐛Source-Stripe - Removed Updated col from customer_balance_transactions (Fix of PR 49811) #49976

Are you sure you want to change the base?

🐛Source-Stripe - Removed Updated col from customer_balance_transactions (Fix of PR 49811) #49976

Conversation

williamkaper commented Dec 20, 2024 • edited Loading

What

How

Review guide

User Impact

Can this PR be safely reverted and rolled back?

vercel bot commented Dec 20, 2024

williamkaper commented Dec 20, 2024

marcosmarxm commented Dec 23, 2024

williamkaper commented Jan 2, 2025

williamkaper Jan 3, 2025 • edited Loading

Choose a reason for hiding this comment

williamkaper Jan 3, 2025

Choose a reason for hiding this comment

marcosmarxm commented Jan 21, 2025

CLAassistant commented Jan 28, 2025 • edited Loading

williamkaper Feb 10, 2025

Choose a reason for hiding this comment

marcosmarxm commented Feb 11, 2025

marcosmarxm commented Feb 18, 2025 • edited by github-actions bot Loading

marcosmarxm commented Mar 6, 2025 • edited Loading

ericvanbenschoten-dv01 commented Mar 6, 2025

vercel bot commented Mar 13, 2025 • edited Loading

williamkaper commented Dec 20, 2024 •

edited

Loading

williamkaper Jan 3, 2025 •

edited

Loading

CLAassistant commented Jan 28, 2025 •

edited

Loading

marcosmarxm commented Feb 18, 2025 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 6, 2025 •

edited

Loading

vercel bot commented Mar 13, 2025 •

edited

Loading