nbdevAuto

Command line automation for the nbdev publish cycle, plus the dataset and image helpers the notebooks in this collection reuse.
Author

Benedict Thekkel

Unlike most sites in this collection, this one documents an installable library rather than a topic. nbdevAuto exists because the nbdev release cycle is a fixed sequence of commands typed in the same order every time: export the notebooks, run the tests, clean them, bump the version, commit, push, build the docs. Each step is one nbdev call, and forgetting one produces a confusing failure two steps later.

The library collapses that sequence into single-word shell commands. It is what just upload and just full_upload invoke under the hood for every submodule in the parent Knowledge repo.


Install

pip install nbdevAuto

Use

Every exported function is also a console script, so the common case is a bare word in the terminal at the root of an nbdev project:

upload -m "commit message"    # export, test, clean, then add/commit/push

Or import the helpers into a notebook:

from nbdevAuto.functions import download_search_images, classify_images
from nbdevAuto.pdf import PDFreader

Contents

Page Covers
Functions Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and graph, a graphviz.Digraph subclass preloaded with a rounded, filled style
Automate The command line surface: the prep/commit/push pipeline, GitHub and PyPI releases, reinstall and update, conda environment creation, and the help output that lists them all
PDF Reader PDFreader, a class that converts a PDF through pdf2image and renders page ranges inline when you slice it (pdf[0:5])

The Command Line Surface

Installing the package puts these on your PATH. They are ordinary functions in Automate, exposed as scripts through [project.scripts].

Command Does
prep Export, test, and clean the notebooks, rendering the README if it needs it
gacp git add, commit, and push, with -m for the message
status Show the working tree state
upload prep then gacp, the one you actually type
release Bump the version, then release to both GitHub and PyPI
gitrelease / piprelease The two halves of release, separately
reinstall / update Reinstall or update the package locally
everything upload, update, release, reinstall, in that order
create_env Create a conda environment for AI work, named fast by default
h Print the list of available commands

Both prep and upload take -p to choose which version part to increment.


Where It Is Used

The parent Knowledge repo’s justfile calls upload once per submodule. just upload runs it only where the working tree is dirty, just full_upload runs it everywhere. That is the main consumer, so a change to automate.py affects the publish path for every site in the collection.


Not Covered Yet

  • No test notebook. The functions are exercised only by being used, so a broken helper surfaces in a downstream repo rather than here.
  • Undeclared dependencies. pyproject.toml lists only graphviz, but the image helpers import fastai, the Kaggle shortcuts need the Kaggle CLI and its credentials, and PDFreader needs pdf2image with poppler behind it, plus matplotlib. Install those yourself.
  • core.py is an empty nbdev stub (a single foo), with no notebook behind it.
  • The docstrings are one-liners, so the rendered API pages are thin. The notebooks carry the real explanation.

Back to top