# Bounding Box Overlay

Bounding Box Overlay [#bounding-box-overlay]

The **Bounding Box Overlay** renders detection results (`{ label, score, box }`) as absolutely-positioned, color-coded boxes with label chips over a parent image. Pixel-coordinate boxes are converted to **percentage offsets** using the image's natural width/height, so placement is correct at any display size — resize the container and the boxes stay aligned. A companion **`DetectionLabelLegend`** renders a wrapping strip of color → class pills that match the overlay colors.

The overlay consumes one shape — `{ label, score, box }`, where `box` is `{ x, y, width, height }` in natural image pixels. [`useDetectObjects`](https://localmode.dev/docs/react) returns this directly (`DetectObjectsResult.objects`, each a `DetectedObject`). `useDetectFace` returns boxes too (`FaceDetectionResultItem` has `box` + `score`, but no `label`), and `useDetectHands` / `useDetectPose` return landmark sets (`{ landmarks, worldLandmarks, … }`, no `box`) — for those, derive a `label` (and, for hands/pose, a `box` from the landmark extents) before passing them in.

**When to use it:** object/face detection viewers, pose/hand annotation, anything that draws boxes over a still image. (For real-time webcam landmark drawing, use the [Video Canvas](/docs/media-vision/video-canvas) instead.)

Preview [#preview]

```tsx
'use client';

import { BoundingBoxOverlay, DetectionLabelLegend } from '@/components/bounding-box-overlay';
import type { Detection } from '@/components/bounding-box-overlay';

/**
 * Demo for the BoundingBoxOverlay component, used by the docs live preview.
 * Uses a static placeholder image with fixed detections so the percentage
 * placement is visible at any container size — no model download.
 */
const NATURAL_WIDTH = 640;
const NATURAL_HEIGHT = 400;

const DETECTIONS: Detection[] = [
  { label: 'person', score: 0.98, box: { x: 64, y: 60, width: 200, height: 300 } },
  { label: 'dog', score: 0.91, box: { x: 320, y: 200, width: 240, height: 170 } },
  { label: 'frisbee', score: 0.74, box: { x: 430, y: 70, width: 110, height: 90 } },
];

export default function BoundingBoxOverlayDemo() {
  return (
    <div className="w-full max-w-lg space-y-3">
      <div className="relative overflow-hidden rounded-lg border border-border bg-muted">
        {/* A placeholder "image" surface at the natural aspect ratio. */}
        <div
          className="w-full bg-gradient-to-br from-muted to-muted-foreground/10"
          style={{ aspectRatio: `${NATURAL_WIDTH} / ${NATURAL_HEIGHT}` }}
        />
        <BoundingBoxOverlay
          detections={DETECTIONS}
          naturalWidth={NATURAL_WIDTH}
          naturalHeight={NATURAL_HEIGHT}
        />
      </div>
      <DetectionLabelLegend labels={DETECTIONS.map((d) => d.label)} />
    </div>
  );
}
```

Installation [#installation]

```bash
npx shadcn@latest add @localmode/ui/media-vision/bounding-box-overlay
```

Dependencies [#dependencies]

* **Data source:** renders the `{ label, score, box }` detections you pass — works with any backend that returns boxes. Recommended LocalMode producer: `useDetectObjects` from `@localmode/react` (its `DetectedObject` matches this shape directly). `useDetectFace` boxes (no `label`) and `useDetectHands` / `useDetectPose` landmark output map in with a small adapter (optional).

* `clsx` + `tailwind-merge` — via the shared `cn()` util (installed automatically as a registry dependency)

Files installed [#files-installed]

* `bounding-box-overlay.tsx` — `BoundingBoxOverlay` + `DetectionLabelLegend`
* `lib/utils.ts` — the `cn()` helper (if not already present)

Props [#props]

**BoundingBoxOverlay**

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `detections` | `array` | — | **Required.** Detections to render. Boxes are in natural-pixel coordinates. |
| `naturalWidth` | `number` | — | **Required.** The image's NATURAL width in pixels (e.g. `imgEl.naturalWidth`). Required to convert pixel boxes to display-independent percentage offsets. |
| `naturalHeight` | `number` | — | **Required.** The image's NATURAL height in pixels (e.g. `imgEl.naturalHeight`). |
| `hideLabels` | `boolean` | `false` | Hide the per-box label chip (border-only boxes). Useful for dense landmark/pose output. |

**DetectionLabelLegend**

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `labels` | `array` | — | **Required.** Detected class labels (duplicates allowed — they are de-duplicated). Colors match {@link BoundingBoxOverlay} for the same label set/order. |

Backing hooks [#backing-hooks]

Feed it `{ label, score, box }` detections plus the image's `naturalWidth` / `naturalHeight` (read off the rendered `<img>`). `useDetectObjects` from [`@localmode/react`](https://localmode.dev/docs/react) returns `objects` in this shape directly; `useDetectFace` returns boxes without a `label`, and `useDetectHands` / `useDetectPose` return landmark sets (no `box`) — map those to `{ label, score, box }` before passing them in.

Examples [#examples]

Over a detection result [#over-a-detection-result]

```tsx
import { BoundingBoxOverlay, DetectionLabelLegend } from '@/components/bounding-box-overlay';
import { useDetectObjects } from '@localmode/react';

export function Example({ src }: { src: string }) {
  const imgRef = useRef<HTMLImageElement>(null);
  const { data } = useDetectObjects({ model });

  return (
    <>
      <div className="relative">
        <img ref={imgRef} src={src} alt="" className="w-full" />
        {data && (
          <BoundingBoxOverlay
            detections={data.objects}
            naturalWidth={imgRef.current?.naturalWidth ?? 0}
            naturalHeight={imgRef.current?.naturalHeight ?? 0}
          />
        )}
      </div>
      {data && <DetectionLabelLegend labels={data.objects.map((o) => o.label)} />}
    </>
  );
}
```

Boxes without label chips (dense output) [#boxes-without-label-chips-dense-output]

```tsx
// `detections` is already shaped as { label, score, box }[]
<BoundingBoxOverlay detections={detections} naturalWidth={w} naturalHeight={h} hideLabels />
```

Customization [#customization]

Colors come from Tailwind palette classes in the internal `COLOR_SLOTS` array — edit them in the copied `bounding-box-overlay.tsx` to match your design system; the legend reads the same map so the two stay in sync. The overlay is non-interactive (`pointer-events-none`) and fills its `relative` parent.