Skip to content

Update landsat tutorial notebook #1152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 26, 2023

Conversation

jpolchlo
Copy link
Contributor

@jpolchlo jpolchlo commented Jun 7, 2023

Related Issue(s):

Description:
This updates the Landsat tutorial notebook to use collection 2 data, since the old collection 1 data no longer appears on AWS. This notebook sources its data from MS planetary computer, and so requires the planetary-computer python package to be installed.

PR Checklist:

  • pre-commit hooks pass locally
  • Tests pass (run scripts/test)
  • Documentation has been updated to reflect changes, if applicable
  • This PR maintains or improves overall codebase code coverage.
  • Changes are added to the CHANGELOG. See the docs for information about adding to the changelog.

@jpolchlo jpolchlo force-pushed the update/landsat-notebook branch from 5378d7c to c07ae76 Compare June 7, 2023 20:12
@codecov
Copy link

codecov bot commented Jun 7, 2023

Codecov Report

Patch coverage has no change and project coverage change: +0.07 🎉

Comparison is base (9b363db) 91.99% compared to head (e72410b) 92.06%.

❗ Current head e72410b differs from pull request most recent head 3021242. Consider uploading reports for the commit 3021242 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1152      +/-   ##
==========================================
+ Coverage   91.99%   92.06%   +0.07%     
==========================================
  Files          51       50       -1     
  Lines        6792     6708      -84     
  Branches     1000      990      -10     
==========================================
- Hits         6248     6176      -72     
+ Misses        368      357      -11     
+ Partials      176      175       -1     

see 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jpolchlo jpolchlo requested a review from pjhartzell June 8, 2023 17:06
@pjhartzell
Copy link
Collaborator

pjhartzell commented Jun 9, 2023

General comments:

  • The notebook was initially written for Landsat 8. This PR uses both Landsat 8 and 9 scenes. There are a number of references to "Landsat 8" in the notebook that should be updated to just "Landsat".
  • There is at least one mention of "AWS" that needs to be removed.
  • The stac-browser example at the end - you can't actually browse the STAC. Or at least I couldn't. It serves up the Collection, but you can't link to the Items. But maybe that's normal for a local catalog? I don't use stac-browser enough to know.
  • The notebook isn't formatted per black[jupyter]. pre-commit isn't set up quite right, so the notebooks are silently not being formatted. I'm opening a new issue for this. In the meantime, pip install black[jupyter] locally and pre-commit should apply the formatting when committing.

Macro comment:

  • Getting the extra raster and classification extension requirements in the notebook is good from the standpoint of modernizing to use the latest extensions. But it reminds me of a pet peeve of mine, which are "unit" tests that try to do too much, rather than focusing on one thing. I feel like an example/tutorial of how to create STAC should certainly show how to add extensions, but only one or two simple ones. A new user has less chance of getting lost (like me wandering around test code muttering about how I can't remember what is being tested anymore) if we keep it simple. I'm not opposed to keeping it all in there, as it makes for a more complete example. But wondering if any others (@gadomski?) have thoughts.

Copy link
Collaborator

@pjhartzell pjhartzell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Just a few comments of note.

@jpolchlo
Copy link
Contributor Author

On your macro comment: it feels to me that this notebook is a little special. It's not just an example of how to build a STAC, but how to build a real-world STAC, in some sense. There are already examples of how to build a Catalog and how to use extensions. This is specifically about talking through the process of taking a real data set and building a Catalog for it. For this data set, that means a complete job is going to be a bit heavier in terms of the extensions that are required. I agree that examples should get the point across parsimoniously, in general, but this isn't strictly an example about how to use the API, but what it takes to pack a real data set into a complete STAC.

If the argument is that there are better places to do this, then I might not disagree. Maybe the stactools-packages repos serve this function better. In which case, the question is do we remove this notebook entirely and just add a pointer to stactools-packages into the docs instead?

W.r.t. the STAC browser, I wonder if @gadomski has a pointer about why that's not working as expected. I looked at the endpoint and saw a catalog served, which I thought was the expected outcome.

@jpolchlo jpolchlo force-pushed the update/landsat-notebook branch from ecb79d3 to 6b617ed Compare June 15, 2023 20:50
@jpolchlo jpolchlo requested a review from pjhartzell June 15, 2023 20:51
@gadomski
Copy link
Member

gadomski commented Jun 15, 2023

A new user has less chance of getting lost (like me wandering around test code muttering about how I can't remember what is being tested anymore) if we keep it simple.

Agreed. IMO both raster and classification are a bit complex, since they live down in assets and have a hard-to-read object-in-list structure. proj is probably the best to demonstrate extensions w/o too much complexity.

The stac-browser example at the end - you can't actually browse the STAC. Or at least I couldn't. It serves up the Collection, but you can't link to the Items. But maybe that's normal for a local catalog? I don't use stac-browser enough to know.

You should be able to browse items as well, so if you can't something is mis-wired or stac-browser is broken. I'll take a look.

UPDATE: I took a look, and we should just point people to Radiant's browser instance, e.g.: https://radiantearth.github.io/stac-browser/#/external/http:/localhost:5555/LC80140322022305/LC80140322022305.json

@pjhartzell
Copy link
Collaborator

On your macro comment: it feels to me that this notebook is a little special. It's not just an example of how to build a STAC, but how to build a real-world STAC, in some sense. There are already examples of how to build a Catalog and how to use extensions. This is specifically about talking through the process of taking a real data set and building a Catalog for it. For this data set, that means a complete job is going to be a bit heavier in terms of the extensions that are required. I agree that examples should get the point across parsimoniously, in general, but this isn't strictly an example about how to use the API, but what it takes to pack a real data set into a complete STAC.

If the argument is that there are better places to do this, then I might not disagree. Maybe the stactools-packages repos serve this function better. In which case, the question is do we remove this notebook entirely and just add a pointer to stactools-packages into the docs instead?

I think your point is valid. Let's leave it all in there.

Copy link
Collaborator

@pjhartzell pjhartzell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a few changes to the stac-browser section. Feel free to edit as you see fit. One or two other minor comments, but looks pretty much good to go.

@jpolchlo jpolchlo force-pushed the update/landsat-notebook branch from 6407bd9 to 0731cea Compare June 20, 2023 16:36
@jpolchlo
Copy link
Contributor Author

Thanks for the changes to the stac-browser section, @pjhartzell ! I've incorporated the remaining comments that you offered, and I think that means this is GTG...

@jpolchlo jpolchlo requested a review from pjhartzell June 20, 2023 16:37
Copy link
Collaborator

@pjhartzell pjhartzell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need a quick fix for CI to pass.

pystac/validation/__init__.py:97: error: Unused "type: ignore" comment  [unused-ignore]

@jpolchlo
Copy link
Contributor Author

jpolchlo commented Jun 20, 2023

Fixing this in #1166

@jpolchlo jpolchlo force-pushed the update/landsat-notebook branch from 0731cea to 4b6d76a Compare June 21, 2023 14:12
@jpolchlo jpolchlo requested a review from pjhartzell June 21, 2023 20:43
@jpolchlo jpolchlo force-pushed the update/landsat-notebook branch from e72410b to 3021242 Compare June 23, 2023 18:07
@jpolchlo jpolchlo added this pull request to the merge queue Jun 26, 2023
Merged via the queue into stac-utils:main with commit 3bf0676 Jun 26, 2023
@jpolchlo jpolchlo deleted the update/landsat-notebook branch June 26, 2023 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update landsat example notebook
3 participants