Integrate codespell into CI process.

[lossyrob]

I, for one, am an admittedly horrible speller. My editor doesn't have spell check and I'm constantly letting bad spellings through.

One option to combat this for myself and others is to implement a codespell check on all *.rst, *.md and *.py files in the project. Codespell has an interactive mode and an ignore dictionary, so we could commit the ignore dict if there's any intentional spellings that block CI from passing.

[schwehr]

Initial investigations...

Looking at https://pre-commit.com/

e.g. https://github.com/ioos/ioos-python-package-skeleton/blob/12fbcaeed97bd33a9d4ba997a4809080933017d2/.pre-commit-config.yaml#L56 where they have:

- repo: https://github.com/codespell-project/codespell
  rev: v1.16.0
  hooks:
    - id: codespell
      args:
        - --quiet-level=2

From man codespell:

 -D DICTIONARY, --dictionary DICTIONARY
    Custom dictionary file that contains spelling corrections

[schwehr]

What am I doing wrong with codespell?

cat .ignore_words.txt
ACI
ALOS
NED
UE

And with -L

codespell -L LACI,ALOS,NED,UE -I .ignore_words.txt --quiet-level=2 CHANGELOG.md LICENSE README.md requirements-dev.txt setup.py docs pystac scripts tests 
tests/data-files/examples/gee-0.6.2/catalog.json:18: ACI ==> ACPI
tests/data-files/examples/gee-0.6.2/catalog.json:438: ALOS ==> ALSO
tests/data-files/examples/gee-0.6.2/catalog.json:443: ALOS ==> ALSO
tests/data-files/examples/gee-0.6.2/catalog.json:448: ALOS ==> ALSO
tests/data-files/examples/gee-0.6.2/catalog.json:453: ALOS ==> ALSO
tests/data-files/examples/gee-0.6.2/catalog.json:2218: NED ==> NEED
tests/data-files/examples/sentinel-0.6.0/sentinel-2-l1c/9/V/catalog.json:1: UE ==> USE, DUE

[schwehr]

This is where I'm at that doesn't totally work

cat .pre-commit-config.yaml

repos:

- repo: https://github.com/codespell-project/codespell
  rev: v1.17.1
  hooks:
  - id: codespell
    args:
      - --quiet-level=2
      - --ignore-words .ignore_words.txt

diff --git a/requirements-dev.txt b/requirements-dev.txt
index fe757be..e0289d8 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -1,10 +1,12 @@
+coverage==5.2.*
+distlib
+flake8==3.8.*
 jsonschema==3.2.0
+nbsphinx==0.4.3
+pre-commit
 pylint==1.9.1
 Sphinx==1.8.0
 sphinx-autobuild==0.7.1
 sphinxcontrib-fulltoc==1.2.0
 sphinxcontrib-napoleon==0.7
-flake8==3.8.*
 yapf==0.28.*
-nbsphinx==0.4.3
-coverage==5.2.*

Trying it out:

pip install --upgrade -r requirements-dev.txt .
pre-commit install
pre-commit run --all-files

Without --upgrade in the pip command, I get this:

pre-commit run --all-files
[INFO] Initializing environment for https://github.com/codespell-project/codespell.
[INFO] Installing environment for https://github.com/codespell-project/codespell.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/usr/local/google/home/schwehr/src/pystac-schwehr/ve/bin/python', '-mvirtualenv', '/usr/local/google/home/schwehr/.cache/pre-commit/repousztw9z0/py_env-python3.8', '-p', '/usr/local/google/home/schwehr/src/pystac-schwehr/ve/bin/python')
return code: 1
expected return code: 0
stdout:
    ImportError: cannot import name 'enquote_executable' from 'distlib.scripts' (/usr/local/google/home/schwehr/src/pystac-schwehr/ve/lib/python3.8/site-packages/distlib/scripts.py)

[schwehr]

Note that I only added pre-commit and distlib to the requirements-dev.txt. The rest is just sorting

[lossyrob]

@schwehr the current approach to the repo is to have linting and tests contained in the test script which is executed by the CI GitHub Action. I think it'd be better to keep to this workflow rather than introduce a pre-commit hook.

So the steps as I see them are:

• Add the requirement to codespell in requirements-dev.txt
• Add the proper execution of the command in scripts/test
• Document how to run spell check in the README (similarly to the lint sections)

And then it will be picked up by CI automatically.

This seems to work for me fo the ignore file:

> codespell -I .codespell_ignore tests/extensions/test_version.py

where .codespell_ignore has a word on a single line for a word I intentionally misspelled in test_version.py