Skip to content

Add a stac_io argument in Catalog.save? #666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fnattino opened this issue Nov 15, 2021 · 4 comments · Fixed by #751
Closed

Add a stac_io argument in Catalog.save? #666

fnattino opened this issue Nov 15, 2021 · 4 comments · Fixed by #751

Comments

@fnattino
Copy link
Contributor

It is nice how the STAC I/O has been refactored, with the possibility for the user to provide customised StacIO instances as argument to StacObject.save_object (or ItemCollection.save_object).

But shouldn't the catalog Catalog.save method also accept a stac_io argument to optionally specify the StacIO to recursively write out all children and items?

If you agree, I would be happy to contribute with a PR!

@fnattino
Copy link
Contributor Author

fnattino commented Nov 17, 2021

I see that if one reads a catalog with a user-defined StacIO instance this is stored internally as catalog._stac_io so that this is used in later calls to Catalog.save(dest_href). But what if one wants to use a different StacIO instance for writing? I guess this could happen if one is reading from/writing to different filesystem types.

@thomafred
Copy link
Contributor

StacIO's abstract member functions read_text and write_text are called by read_stac_object and save_json respectively. This allows for custom file formats, but is limiting as it does not enable the ability to use alternative protocols. This is actually mentioned in the docs:

# stac_io.py:324
This implementation uses :func:`open` and therefore can only write to the local
file system.

Personally, I think that StacIO should only contain two abstract functions:

# Essentially the same as current implementation
def read_stac_object(
    self,
    source: Union[str, "Link_Type"],
    root: Optional["Catalog_Type"] = None,
    *args: Any,
    **kwargs: Any,
) -> "STACObject_Type"

def write_stac_object(
        self,
        dest: Union[str, "Link_Type"],
        stac_object: "STACObject_Type",
        *args: Any,
        **kwargs: Any,
    ) -> None

This would create explicit entry-points for IO-operations and enable develops to easily create custom integrations for APIs and custom file protocols.

@duckontheweb
Copy link
Contributor

It is nice how the STAC I/O has been refactored, with the possibility for the user to provide customised StacIO instances as argument to StacObject.save_object (or ItemCollection.save_object).

But shouldn't the catalog Catalog.save method also accept a stac_io argument to optionally specify the StacIO to recursively write out all children and items?

If you agree, I would be happy to contribute with a PR!

@fnattino I would support adding a stac_io argument to Catalog.save and would be happy to review a PR! I agree that needing to read and write using different protocols/file systems is a common enough use case that we should make it convenient to do.

@fnattino
Copy link
Contributor Author

Apologies for the long delay with this - if still in time I can open a PR before the end of the week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants