# AI-Assisted Development in an Enterprise Context

> **Audience:** Technology / Architecture leadership
> **Purpose:** Examine the challenges of deploying an AI-assisted JavaScript codebase
> into an enterprise that does not have a dedicated JS team, with particular attention
> to security and architectural governance.

---

## Context

This project takes a deliberate hybrid approach: domain experts (Architect developers)
write the syntax-test specifications; AI writes the implementation. The syntax-tests
provide a strong correctness guarantee for *what the validators catch*. This document
examines what that guarantee does and does not cover, and what additional controls an
enterprise should put in place.

---

## The Core Tension

The TDD approach is genuinely sound as a correctness guarantee for validator output.
The syntax-tests tell you whether the right diagnostics are emitted for a given piece
of Architect code. They say almost nothing about *how the implementation is written* —
its security posture, its architectural coherence, or whether it will degrade
gracefully under conditions the tests do not cover.

---

## Specific Risks

### 1 · Security — Supply Chain

AI-generated code routinely reaches for `npm install` as the first solution. Each
dependency brings its own dependency tree, and without a JS-literate reviewer the
team has no way to evaluate whether a suggested package is trustworthy, maintained,
or carries known CVEs.

The current codebase has **no runtime npm dependencies beyond VS Code itself**. That
is a significant security asset that should be protected by policy, not just goodwill.

**Mitigation:**
- A written rule: *no new `dependencies` in `package.json` without explicit sign-off*
- A PR check that diffs the lockfile and flags any change to `dependencies`
- Azure Artifacts upstream proxy configured to block unlisted packages at the network
  level before they can be installed by any developer or build agent

---

### 2 · Security — The Vscode Shim

`cli/vscode-shim.js` uses `Module._load` interception — a well-established pattern,
but one that a non-JS reviewer would not immediately recognise as safe. It
monkey-patches the Node module loader for the entire process lifetime. If a future
contributor adds a module with a legitimate use of `vscode` at load time with
side-effects, the shim could silently misbehave.

This is a maintainability risk more than a direct security risk, but it is precisely
the kind of subtle problem that AI tends to generate and non-JS reviewers tend to miss.

**Mitigation:**
Phase 2 (formal npm workspace packages) eliminates the shim entirely. Once `core/`
is a proper package with no `vscode` in its `dependencies`, the package boundary
enforces the constraint at install time rather than at runtime. Prioritise Phase 2.

---

### 3 · Architecture Drift — AI Has No Memory of Intent

Each AI session starts fresh. Over time, AI-generated additions may:
- Duplicate logic that already exists elsewhere
- Introduce inconsistent patterns
- Silently add `require('vscode')` calls inside `core/` — undoing the decoupling
  without anyone noticing

The AI does not feel the architectural pain of this; it solves the immediate problem
in front of it.

**Mitigation:**
- The `dev-docs/` architecture documents are the guardrail. Include the relevant doc
  in the AI prompt context before any non-trivial implementation session.
- A CI assertion that `core/src/**` contains no `require('vscode')` calls catches
  structural drift automatically — see the pipeline snippet below.
- The syntax-test suite catches behavioural regressions but not structural ones;
  both are needed.

```yaml
# azure-pipelines.yml — structural integrity check
- script: |
    if grep -r "require('vscode')" core/src/; then
      echo "ERROR: core/ must not depend on vscode"
      exit 1
    fi
  displayName: Assert core has no vscode dependency
```

---

### 4 · Testability Gap — Syntax-Tests Only Cover Validator Output

The current test harness verifies that given a piece of Architect code, the right
diagnostics are emitted. It does **not** cover:

- CLI exit code behaviour
- JSON report schema correctness
- The vscode shim's accuracy
- Include resolver graph-traversal edge cases

These are exactly the components an AI is most likely to get subtly wrong and a
non-JS reviewer is least likely to catch manually.

**Mitigation:**
A small suite of CLI integration tests would dramatically increase confidence.
These can be written by the AI against a plain-English specification written by
the humans — applying the same TDD model already used for validators:

| Test | What it asserts |
|---|---|
| Clean folder | `node cli/index.js --folder test/clean` exits `0` |
| Folder with errors | `node cli/index.js --folder test/with-errors` exits `1` |
| `--fail-on warning` | Exits `1` when only warnings are present |
| `--fail-on none` | Exits `0` regardless of diagnostics found |
| Report schema | Output JSON contains `summary.filesScanned`, `files[*].diagnostics[*].ruleId` |

---

### 5 · Maintainability — JS Is Not the Team's Primary Language

When the extension breaks in six months — because VS Code updated an API, or a
Windows path separator causes a subtle bug — the team needs someone who can read and
fix JavaScript. If no such person exists, the team becomes dependent on AI to fix
what AI wrote, with no human capable of reviewing whether the fix is correct.

**Mitigation:**
This is the strongest argument for keeping the codebase **small and the dependency
surface minimal**. The current architecture — zero runtime npm dependencies, plain
Node.js, no bundling, no transpilation — is maximally readable to anyone who can
follow basic programming logic. This should be treated as a hard constraint, not an
accident.

---

## Summary — Practical Steps

| Concern | Practical step | Effort |
|---|---|---|
| No JS reviewer | Recruit one part-time JS reviewer (can be external). Scope is narrow: architectural boundaries and security surface. Correctness is covered by syntax-tests. | Low / ongoing |
| AI architecture drift | CI assertion: `core/src/**` must not `require('vscode')` | 1 hour |
| Dependency risk | PR policy flags `package.json` changes; Azure Artifacts upstream proxy blocks unlisted packages | Half day |
| Shim fragility | Complete Phase 2 (npm workspaces) — eliminates the shim permanently | Phase 2 scope |
| CLI untested | 5 integration tests: clean exits 0, errors exit 1, schema check | Half day |
| Knowledge concentration | Keep `dev-docs/` current; always include architecture docs in AI prompt context for new sessions | Ongoing discipline |

---

## What the Syntax-Test Model Gets Right

It is worth being clear about what the current approach *does* protect well.

The syntax-test suite means that **any regression in validator behaviour is caught
automatically**, regardless of who or what wrote the code. A new AI session that
accidentally breaks the `ACU-FNC-003` rule will be caught by the existing test. This
is the most important correctness guarantee and it is robust.

The gaps described above are all in areas *outside* the scope of what syntax-tests
can cover. The mitigations are mostly lightweight additions — CI assertions, policy
rules, a handful of integration tests — rather than fundamental changes to the
development model.

---

## Relationship to the CI/CD Architecture

The concerns above interact directly with the CI/CD deployment described in
[CICD_ARCHITECTURE.md](CICD_ARCHITECTURE.md):

- The **supply chain lockdown** (Azure Artifacts, pinned versions) described in
  Section 5 of that document is the same control that protects against unreviewed
  AI-introduced dependencies.
- The **structural CI assertion** above should be added to the same
  `azure-pipelines.yml` that runs the validator.
- The **JS reviewer** role is most critical during Phase 2 (packaging) and Phase 5
  (governance/VSIX distribution) — the two phases with the most external surface area.
