Skip to content

v0.1.1 Documentation Gap Analysis And Roadmap

This roadmap tracked the v0.1.1 branch during development and lists documentation and verification gaps that were addressed before publication.

This report reviews the current docs/ directory, README.md, tests, benchmarks, and source files under src/adaptshot/ for AdaptShot v0.1.0.

Gap Analysis

Gap: Empty Getting Started Pages

Location: docs/index.md, line 1; docs/getting-started/installation.md, line 1; docs/getting-started/quickstart.md, line 1
Issue: MkDocs navigation points to pages that were empty.
Impact: Beginners clicking "Home", "Installation", or "Quick Start" saw blank pages.
Suggested Fix: Add concise landing, install, and quickstart pages with runnable examples.

Gap: README Uses Non-Existent classes Constructor Argument

Location: README.md, line ~139 and repeated examples around lines ~344, ~378, ~454, ~476
Issue: Examples instantiate FewShotLearner(classes=[...], device="cpu"), but FewShotLearner.__init__ accepts config: Optional[AdaptShotConfig] = None, **kwargs, and AdaptShotConfig has no classes field.
Impact: Copy-paste examples raise TypeError: AdaptShotConfig.__init__() got an unexpected keyword argument 'classes'.
Suggested Fix: Use FewShotLearner(config=AdaptShotConfig(...)) or pass only valid AdaptShotConfig keyword arguments.

Gap: README Shows Unsupported Folder Loader

Location: README.md, line ~146
Issue: learner.load_support_images("dataset/train/", k=10) does not match the v0.1.0 signature load_support_images(image_paths: List[str], labels: List[Union[str, int]]).
Impact: Users may expect automatic folder scanning that is not implemented.
Suggested Fix: Replace with explicit image_paths and labels, or mark a folder loader as planned.

Gap: String-Label Correction Fails In Calibration Route

Location: src/adaptshot/training/feedback_router.py, line ~78
Issue: route_feedback() casts correction.predicted_label and correction.corrected_label with int(...) before updating calibration.
Impact: Tutorials using labels like "maize_blight" can predict successfully but fail when calling learner.correct().
Suggested Fix: Add a label encoder inside FewShotLearner/FeedbackRouter, or update calibration to accept already-encoded class ids while retaining display labels.

Gap: Benchmark Claims Exceed Current Evidence

Location: README.md, lines ~405-439; docs/getting-started/benchmarks.md, lines ~29-38 and ~46-53
Issue: README lists benchmark metrics for TinyImageNet, PlantVillage, CheXpert, RAM usage, and CI-enforced latency targets. The repository benchmark harness currently implements a CIFAR-10 smoke test and integration simulations.
Impact: Users and reviewers may treat unverified figures as measured release claims.
Suggested Fix: Restrict claims to metrics produced by benchmarks/run_benchmark.py or clearly label other numbers as future validation targets.

Location: docs/getting-started/benchmarks.md, lines ~89-91; docs/api/core.md, line ~200; docs/api/config.md, lines ~165-166
Issue: Links point to ui.md, ../../CONTRIBUTING.md, ../../CHANGELOG.md, or uppercase root filenames that are not MkDocs pages under docs/.
Impact: Rendered documentation has dead links or links that escape the docs tree.
Suggested Fix: Use relative docs paths such as ../api/core.md, ../contributing.md, and ../changelog.md. Add a UI page only when it exists.

Gap: API Reference Markdown Is Wrapped In Literal File Markers

Location: docs/api/core.md, line 1; docs/api/training.md, line 1; docs/api/config.md, line 1
Issue: Files start with generated wrapper text like # File ... and open a ```markdown fence.
Impact: MkDocs renders the actual documentation as a code block instead of as structured docs.
Suggested Fix: Remove wrapper lines and keep only normal Markdown content.

Gap: FewShotLearner.load() Needs Validation

Location: src/adaptshot/core/learner.py, lines ~275-299
Issue: save() writes {path}.head.pt, but load() attempts emb_path.with_suffix(".head.pt"), which resolves to a different filename. load() also does not rebuild _label_to_idx and _idx_to_label.
Impact: Persistence tutorials cannot honestly claim full save/load recovery for predictions.
Suggested Fix: Add tests for save/load round trips and rebuild label maps during load.

Gap: Beginner Concepts Need A Glossary

Location: docs/api/core.md, docs/api/training.md, and README.md research sections
Issue: Terms such as cosine similarity, ECE, temperature scaling, ACT, Fisher information, and EWC appear before beginner explanations.
Impact: Non-ML users may be blocked even when the API is simple.
Suggested Fix: Add docs/glossary.md and link first use of each concept.

Gap: Raspberry Pi Guidance Is Not Backed By ARM Benchmarks

Location: README.md, line ~373; docs/getting-started/benchmarks.md, line ~51
Issue: Raspberry Pi latency expectations are documented, but the current benchmarks target the local Python environment and do not include ARM validation scripts or stored results.
Impact: Edge users may plan around numbers that have not been reproduced in the repo.
Suggested Fix: Replace with a "how to profile on your device" guide until ARM results are committed.

Verification Checklist

  • [ ] Every issue above references a current file or source path.
  • [ ] API mismatches are checked against src/adaptshot/.
  • [ ] Performance concerns are checked against benchmarks/.
  • [ ] Speculative fixes are not described as existing features.

Proposed v0.1.1 Roadmap

Low Effort

  • [ ] Fix README quickstart signatures.
    Files: README.md, docs/getting-started/quickstart.md
    Why: Prevents immediate copy-paste failures from the unsupported classes= argument.

  • [ ] Remove generated wrapper fences from API docs.
    Files: docs/api/core.md, docs/api/training.md, docs/api/config.md
    Why: Makes MkDocs render API pages as documentation instead of code blocks.

  • [ ] Replace broken MkDocs links.
    Files: docs/getting-started/benchmarks.md, docs/api/core.md, docs/api/config.md, docs/api/training.md
    Why: Keeps users inside the published documentation tree.

  • [ ] Add calibration warm-up note to the quickstart.
    Files: docs/getting-started/quickstart.md, src/adaptshot/core/calibration.py
    Why: Initial current_ece is 0.0 until updates arrive, which can confuse users.

Medium Effort

  • [ ] Support string labels through correction routing.
    Files: src/adaptshot/core/learner.py, src/adaptshot/training/feedback_router.py, tests/test_feedback_router.py
    Why: Public examples and real users naturally use labels like "maize_blight".

  • [ ] Add save/load round-trip tests and fix head path restoration.
    Files: src/adaptshot/core/learner.py, new tests/test_learner_persistence.py
    Why: Enables honest persistence tutorials and checkpoint workflows.

  • [ ] Add a folder-based support-set helper or documented recipe.
    Files: src/adaptshot/utils/io.py, src/adaptshot/core/learner.py, docs under docs/getting-started/
    Why: Beginners often organize datasets by class folders and expect that workflow.

  • [ ] Add benchmark JSON examples generated by the release harness.
    Files: benchmarks/run_benchmark.py, docs/getting-started/benchmarks.md, results/ or benchmarks/results/
    Why: Keeps performance claims auditable.

High Effort

  • [ ] Add Raspberry Pi profiling guide with committed ARM results.
    Files: benchmarks/run_benchmark.py, new docs/deployment/raspberry-pi.md
    Why: Edge deployment is central to AdaptShot's mission, but ARM numbers need reproducible evidence.

  • [ ] Add PlantVillage benchmark loader.
    Files: benchmarks/run_benchmark.py, new benchmark utility module, docs/tutorials.md
    Why: Agriculture is a core use case, and users need a public-dataset path beyond synthetic examples.

  • [ ] Harden extract_embedding() model loading for repeated inference.
    Files: src/adaptshot/core/extractor.py, src/adaptshot/core/learner.py, tests/test_extractor.py
    Why: The current function constructs a backbone on each call, which may dominate latency and memory behavior.

⚠️ Planned for v0.1.1+ ONNX export, full mobile packaging, federated buffer sharing, and multilingual UI work should remain planned until code and tests exist.

Verification Checklist

  • [ ] Each roadmap item maps to a documented pain point.
  • [ ] Each roadmap item names likely source or docs files.
  • [ ] New algorithms are deferred unless implementation work is explicitly scoped.
  • [ ] Performance validation work points to benchmarks/.

Big Picture

v0.1.1 should focus on usability hardening: making the first install, first prediction, first correction, and first benchmark truthful and easy to reproduce. It should not add new research claims before the existing v0.1.0 APIs are documented accurately and backed by tests. The highest-impact product improvement is making human-readable labels work throughout correction routing, because that unlocks realistic agriculture, healthcare, and education tutorials. The highest-impact documentation improvement is replacing aspirational benchmark numbers with commands that produce local evidence.

Verification Checklist

  • [ ] v0.1.1 scope emphasizes documentation, correctness, and test-backed examples.
  • [ ] Speculative work is clearly marked as planned.
  • [ ] The summary does not claim unimplemented algorithms or unmeasured performance.

Created by Johnson Christopher Hassan
Connect on LinkedIn
Project: github.com/johnson2006christopher/adaptshot