Source Instagram: updating media and mediainsights fields

[hromit]

What

Updating media & mediainsights stream with new available fields.

How

Updated Manifest.yaml with new fields & updated schema.

Review guide

1. manifest.yaml, media_insights.json
2. updated unit test cases

User Impact

• What is the end result perceived by the user?
• users should be able to consume new fields.

Can this PR be safely reverted and rolled back?

• If unsure, leave it blank.

• YES 💚

[vercel]

Someone is attempting to deploy a commit to the Airbyte Growth Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[marcosmarxm]

Can you provide an issue links to Instagram documentation page showing these fields?

[hromit]

Can you provide an issue links to Instagram documentation page showing these fields?

Sure @marcosmarxm

Media Stream

MediaInsights Stream

Attaching screenshots as well.

[natikgadzhi]

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (471ead5)

[natikgadzhi]

@hromit thank you for this!

@DanyloGL can you please run a regression test on this too and see if there are any problems with it? Otherwise the code changes look fine to me.

[hromit]

@natikgadzhi @DanyloGL please let me know if anything pending at my end.

[natikgadzhi]

This looks good to me, but we should do a regression test or a progressive rollout, this is pretty sensitive.

And we should bump the version, I'll try this now.

[natikgadzhi]

TY for regenerating ERDs too!

[natikgadzhi]

/bump-version type="patch" changelog="New fields in media and mediainsights"

Bump Version job started... Check job output.

✅ Changes applied successfully. (68a4bd8)

[natikgadzhi]

@DanyloGL take this for a regression test please!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 13, 2025 1:20pm

[DanyloGL]

@DanyloGL take this for a regression test please!

Can't run regression as the branch is not in Airbyte repo. Will try to find out the way how to test it.

[natikgadzhi]

@hromit we're testing this with @DanyloGL internally and should be ready to move in a direction this week

[hromit]

thanks for the update @natikgadzhi , I have created two more streams, will raise new PR once this is merged.

[DanyloGL]

@hromit, I've made several regression runs and them showed that stream media_insights retrieves less records than in current prod version. It's not expected result, wright?
cc @natikgadzhi

[hromit]

Hey @DanyloGL,

Thanks for the update.

My changes are not condition based changes. All I did is added fileds available in the API which were not present in the streams.

So not so sure about the reason for your concern.

Ideally the changes would have no impact on the number of records fetched.

cc: @natikgadzhi

[hromit]

Hey @DanyloGL , @natikgadzhi

It would be great if you could let me know that how are you running the regression or give me more insights about it.

When I ran tests on my local, all tests were passing and i did not see any reduction in the number of records as such.

[DanyloGL]

testing #53214, don't merge #54138

Hi! We are running the same regression tests on current prod connector version and on new version. After this results are compared. In this case mentioned stream on your connector version retrieves less data then prod version.
Acceptance tests are passing because media_insights stream is excluded from read test as we have no data on our test account for this stream.

[hromit]

Hi @DanyloGL

Thanks for the response.

I connected with my two instagram accounts, I see no difference in the total number of records for media and media_insights. I also tried to match with the original version of the connector & still there was no difference.

Now looks like, to resolve this I need to run regression in my local system. Please suggest if there is some other way around.

It would be really great if you could you please help me out, how to reproduce this in our local environment.

Currently with the below command, when I am using the test suite in my our local, it is skipping regression test.

airbyte-ci --disable-update-check --disable-auto-update connectors --name=source-instagram test
Tried with below command as well, but no still regresion is getting skipped.

airbyte-ci --disable-update-check --disable-auto-update connectors --name=source-instagram test -only-step connector_live_tests

It would be really great if you could point me to some resource about how to perform regression testing at local. Is there some specific params needs to be passed while running test airbyte-ci test.

I am also trying to figure out, what account it connects for the regression testing. Would i able to connect to same account and check the regression records count.

[hromit]

Hey @natikgadzhi @DanyloGL

Could you help me move forward here.