A field report from building Pythonaibrain-Warden — an npm-inspired dependency and environment manager for Python with its own lock file, its own virtual environments, and a few opinions pip doesn't share.
The itch
Python's dependency story is not broken, exactly. It's plural. pip installs things. venv isolates things. pip-tools locks things. poetry and pdm try to do all three with opinions about how. If your project needs two different versions of the same library for two different scripts — a legacy report generator still pinned to numpy==1.24 sitting next to a new one that needs numpy>=1.26 — none of them have a clean answer. You get a second virtualenv, a README note nobody reads, and a slow drift toward "just don't touch that script."
I wanted something narrower and more opinionated: own the whole vertical slice — manifest, lock file, environments, and the install step — well enough that I could make specific, sharp decisions instead of general, safe ones. That became Warden.
This isn't a pitch. It's a walkthrough of what I actually built, and — more usefully — three bugs I caught by actually running the thing, not by reading the code and nodding.
The one rule that kept the codebase sane
Before any of the interesting stuff, one boring decision paid for itself repeatedly: the CLI layer is not allowed to think.
cli.py (argparse only)
│
▼
core.py ──────────────────────► everything else
Every warden subcommand is an argparse definition plus a two-line function that parses arguments and calls a core.* function. That's it. core.install() doesn't know it was invoked from a terminal — it takes a Manifest, a list of package names, a boolean, and returns an exit code. You can import warden.core as core and call it directly, no subprocess, no argv.
I didn't design this because it's a best practice I'd read about. I designed it because by the time Warden had grown past a dozen commands, I needed a reason to trust that "what does install actually do" had exactly one answer, not one answer per code path that happened to call it. It also meant every new feature — and there were a lot — slotted into the same shape: write the logic once in core.py, wire up a thin CLI wrapper, move on.
I didn't use venv.EnvBuilder. On purpose.
Every Python environment tool eventually calls venv.EnvBuilder or shells out to python -m venv. Warden doesn't. It builds the environment by hand:
- Find the real base interpreter (
sys._base_executable). - Create the directory layout itself.
- Symlink (or copy, on Windows) the interpreter in.
- Hand-write
pyvenv.cfg. - Bootstrap
pipviaensurepipdirectly.
This wasn't rebellion for its own sake — it's the same algorithm venv uses internally, just owned end to end instead of delegated to a stdlib module whose error messages and edge-case behavior I didn't control. It paid off almost immediately, because it's what led us straight into the first real bug.
Bug #1: "python" doesn't mean what you think it means on Windows
Early on, a Windows tester ran the most boring possible sequence:
warden init myapp
cd myapp
warden wenv default
warden install
warden run
And got:
D:\Program Files\python.exe: Error while finding module specification for 'myapp.main'
(ModuleNotFoundError: No module named 'myapp')
The venv existed. The package was installed into it. warden run even printed Running in env 'default' right before failing. So why was it launching D:\Program Files\python.exe — the system interpreter — instead of the one sitting three directories below it?
The answer is one of those platform facts that only bites you once you actually test on the platform: on Windows, subprocess.Popen(['python', ...], env=custom_env) does not use custom_env['PATH'] to resolve the bare name python. CreateProcess resolves the executable using the calling process's own PATH, before the child's environment ever comes into play. Prepending the venv's Scripts\ directory to a dict you're about to hand to subprocess does exactly nothing on Windows. (On POSIX it works fine, which is precisely why this kind of bug survives testing on a Mac and ships broken to everyone else.)
The fix wasn't a workaround bolted onto the Windows case — it was to stop relying on PATH search at all:
def resolve_executable(env_path: Path, name: str) -> str:
if name in ("python", "python3", "py"):
p = python_path(env_path)
if p.is_file():
return str(p)
...
return name # fall back to PATH search only for things the env doesn't provide
Every command Warden spawns now resolves to an absolute path inside the target environment before it's ever launched. Correct on Windows because there's no PATH ambiguity left to exploit; correct on POSIX because it always was, and now it's explicit instead of accidental.
Different files, different dependency versions, same project
Here's the feature I're most pleased with, and it's the direct answer to the "legacy script vs. new script" problem from the opening:
[targets]
"scripts/legacy_report.py" = { numpy = "==1.24.0" }
"scripts/new_report.py" = { numpy = "==1.26.4" }
myproject/
├── scripts/legacy_report.py → numpy==1.24.0
└── scripts/new_report.py → numpy==1.26.4
The important design choice here was resisting the urge to invent something clever. A "target" isn't a special sandboxed sub-concept — it's a completely ordinary Warden environment with an auto-generated name (target-scripts-legacy-report-py), its own real lock file, and it shows up in warden wenv list next to everything else. warden run-file scripts/legacy_report.py just looks up whether that file has an override, and if it does, routes to that environment — auto-provisioning it on first use if it isn't set up yet.
I tested this with two files pinned to two different click versions in the same project, with the project's own default environment sitting on a third version, just to make sure nothing was quietly sharing state:
$ warden run-file scripts/file1.py
! WARN Target env 'target-scripts-file1-py' isn't set up yet -- provisioning it now.
==> Resolving target 'scripts/file1.py' -> env 'target-scripts-file1-py'
✓ Locked 1 package(s)
✓ click==8.0.0 installed
file1.py running with click 8.0.0
$ warden run-file scripts/file2.py
! WARN Target env 'target-scripts-file2-py' isn't set up yet -- provisioning it now.
✓ click==8.4.2 installed
file2.py running with click 8.4.2
$ .warden/venvs/default/bin/python -c "import click; print(click.__version__)"
8.1.7
Three environments, three versions, one project, and nothing silently sharing site-packages with anything else.
The bug that only showed up completely offline
This one's our favorite, because it's a genuine two-layer bug and neither layer looks like a bug from the surface.
Warden can package a project into a .warden.wheel — source, lock file, and (by default) every dependency's actual .whl file bundled in, so the whole thing installs without touching the network. A tester built one, disconnected from the internet, and ran:
python -m warden install-wheel myapp-0.1.0.warden.wheel
Everything vendored installed fine. Then, right at the end:
==> Installing the project itself in editable mode
! WARN Could not editable-install the project itself: ...
error: subprocess-exited-with-error
pip subprocess to install build dependencies did not run successfully.
Followed, inevitably, by warden run failing with ModuleNotFoundError: No module named 'myapp' — because the package genuinely never got installed.
Here's the thing: I'd vendored every runtime dependency. So why did anything need the network at all?
pip install -e . doesn't just install your package. By default it uses PEP 517 build isolation — it spins up an ephemeral, throwaway environment and fetches setuptools and wheel into it from PyPI, every single time, to actually perform the build. Those two packages aren't runtime dependencies. They're not in warden.lock. I had never even thought to vendor them.
And the reason a freshly created environment couldn't just use its own copy: I checked.
$ python3 -m venv fresh
$ fresh/bin/python -c "import setuptools"
ModuleNotFoundError: No module named 'setuptools'
Modern CPython's ensurepip stopped bundling setuptools and wheel. It used to. It doesn't anymore. A brand-new venv today has exactly one thing in it: pip.
The fix needed both ends:
At build time, vendor the project's build-system requirements too — read [build-system].requires out of pyproject.toml, resolve and cache setuptools/wheel the same way I do runtime dependencies, and drop them in vendor/ alongside everything else.
At install time, before attempting the editable install, install those from the local vendor directory via pip --no-index --find-links, then run the editable install with --no-build-isolation — which tells pip "don't try to fetch anything, use what's already here."
def ensure_build_tools(python_exe, vendor_dir=None):
if run_cmd([python_exe, "-c", "import setuptools, wheel"]).ok:
return True
args = [python_exe, "-m", "pip", "install", "--disable-pip-version-check"]
if vendor_dir is not None:
args += ["--no-index", "--find-links", str(vendor_dir)]
return run_cmd(args + ["setuptools", "wheel"]).ok
I didn't trust this until I'd actually broken the network to test it — not "ran it with --no-vendor unset and assumed," but literally monkeypatched socket.getaddrinfo to raise for pypi.org and files.pythonhosted.org mid-run:
def blocked_getaddrinfo(host, *a, **kw):
if host in ('pypi.org', 'files.pythonhosted.org'):
raise socket.gaierror('simulated offline')
return _orig(host, *a, **kw)
socket.getaddrinfo = blocked_getaddrinfo
Then ran install-wheel and run against it. Both succeeded. That's the difference between "should work offline" and "works offline" — one of them I'd have shipped confidently and been wrong.
A quieter bug: brackets are not decoration
Adding progress bars and tables (via rich) was mostly straightforward, except for one thing I almost missed entirely. rich parses plain strings passed to Table cells and console.status() as markup by default — [green]text[/] becomes styled green text. Convenient, until you print something that legitimately contains square brackets for an unrelated reason:
>>> table.add_row('pkg[extra1,extra2]>=1.0')
renders as:
pkg>=1.0
The extras syntax — completely standard PEP 508 dependency specification — got silently eaten. In a package manager, silently corrupting a dependency specifier in displayed output is merely embarrassing. It's the kind of bug that's obvious the second you see it and invisible until you specifically go looking, because the common case (requests==2.31.0) never has a bracket in it.
The fix was mechanical once I knew to look for it: never hand rich a raw dynamic string. Always wrap it in Text(), which bypasses markup parsing entirely.
# wrong: table.add_row(spec_string)
# right:
from rich.text import Text
table.add_row(Text(spec_string))
I went back through every table, every status spinner, every place a package name or version spec touched rich output, and fixed each one — then wrote a one-line regression test I now run whenever this code path changes: feed it pkg[extra1,extra2]>=1.0 and confirm the brackets survive.
What I'd tell you if you're building something similar
Test the platform you're least likely to be sitting in front of. The Windows PATH bug and the "ensurepip doesn't bundle setuptools anymore" bug both share a shape: they're invisible if you only ever develop and test on the same machine, in the same network conditions, on the same OS. Neither was hard to fix once found. Both were completely invisible in code review.
A library's convenience default is not automatically your correctness default. rich's markup-everywhere behavior is genuinely nice for 95% of use cases. It's actively wrong for a tool whose entire job is displaying user-supplied strings verbatim. pip's build isolation is genuinely the right default for "run pip install on a random package." It's the wrong default when you've already made an explicit, hash-verified promise about what's installed and where it came from.
Reuse the abstraction you already trust instead of inventing a parallel one. Per-file dependency targets could have been a bespoke sandboxing feature. Instead it's "an environment, but the name is computed instead of typed" — which meant it inherited every bit of correctness work already put into environment creation, activation, and locking, instead of needing its own.
None of this is finished, and that's fine to say out loud. Warden's resolver is deliberately greedy, not backtracking — a real diamond-dependency conflict gets reported clearly instead of resolved cleverly, which is a trade I're happy with but which I say plainly in the docs rather than let people discover. Per-target locks re-resolve independently rather than sharing work across targets. Being specific about what a tool doesn't do is, I'd argue, more useful to the next person than a features list that implies it does everything.
Warden is npm-inspired, not npm-derived: its own manifest (warden.toml), its own hash-pinned lock file (warden.lock), and environments it builds and owns itself rather than delegating to venv. Full command reference in CLI-REFERENCE.md, internal design writeup in ARCHITECTURE.md.
Resources
PyPI: https://pypi.org/project/pythonaibrain-warden
GitHub: https://github.com/DivyanshuSinha136/pythonaibrain-warden
Architecture: https://github.com/DivyanshuSinha136/Pythonaibrain-Warden/blob/main/ARCHITECTURE.md
CLI Reference: https://github.com/DivyanshuSinha136/Pythonaibrain-Warden/blob/main/CLI-REFERENCE.md
Issues: https://github.com/DivyanshuSinha136/Pythonaibrain-Warden/issues
