Layout Analysis Plugin

The Layout Analysis Plugin brings AI-powered document understanding directly into your PDF viewer. It uses ONNX Runtime to run deep learning models locally in the browser, detecting layout elements like text blocks, tables, images, headings, formulas, and more.

Because all inference happens on-device, no data ever leaves the browser. This makes it ideal for applications that handle sensitive or confidential documents. Models are loaded on demand from HuggingFace CDN (or a custom server) and cached locally using the browser’s Cache API.

The plugin optionally supports table structure analysis, which identifies rows, columns, and cells within detected tables. All results are mapped to PDF page coordinates for precise overlay rendering.

Installation

This plugin depends on the AI Manager and Render plugins. You also need to install onnxruntime-web, which provides the ONNX Runtime that powers inference in the browser.

npm install @embedpdf/plugin-layout-analysis @embedpdf/plugin-ai-manager @embedpdf/ai @embedpdf/plugin-render onnxruntime-web

pnpm add @embedpdf/plugin-layout-analysis @embedpdf/plugin-ai-manager @embedpdf/ai @embedpdf/plugin-render onnxruntime-web

yarn add @embedpdf/plugin-layout-analysis @embedpdf/plugin-ai-manager @embedpdf/ai @embedpdf/plugin-render onnxruntime-web

bun add @embedpdf/plugin-layout-analysis @embedpdf/plugin-ai-manager @embedpdf/ai @embedpdf/plugin-render onnxruntime-web

Registration

First create an AI runtime, then register the AI Manager and Layout Analysis plugins.

import { createPluginRegistration } from '@embedpdf/core'
// ... other imports
import { RenderPluginPackage } from '@embedpdf/plugin-render/vue'
import { createAiRuntime } from '@embedpdf/ai/web'
import { AiManagerPluginPackage } from '@embedpdf/plugin-ai-manager/vue'
import { LayoutAnalysisPluginPackage } from '@embedpdf/plugin-layout-analysis/vue'
 
// Create the AI runtime (loads ONNX Runtime on demand)
const aiRuntime = createAiRuntime({
  backend: 'auto', // tries WebGPU first, falls back to WASM
  cache: true,      // caches models in the browser Cache API
})
 
const plugins = [
  // ... other essential plugins
  createPluginRegistration(DocumentManagerPluginPackage, { /* ... */ }),
  createPluginRegistration(ViewportPluginPackage),
  createPluginRegistration(RenderPluginPackage), // Required dependency
 
  // Register the AI manager with the runtime
  createPluginRegistration(AiManagerPluginPackage, {
    runtime: aiRuntime,
  }),
 
  // Register and configure layout analysis
  createPluginRegistration(LayoutAnalysisPluginPackage, {
    layoutThreshold: 0.35,          // confidence threshold for layout detections
    tableStructureThreshold: 0.8,   // confidence threshold for table elements
    tableStructure: false,          // enable table structure analysis
  }),
]

Usage

The plugin provides the <LayoutAnalysisLayer /> component for displaying results, the useLayoutAnalysis composable for document-scoped reactive state, and the useLayoutAnalysisCapability composable for direct API access.

The useLayoutAnalysis Composable

The useLayoutAnalysis(documentId) composable is the primary way to interact with the plugin from Vue. It returns reactive refs and scoped methods for a specific document.

<script setup lang="ts">
import { useLayoutAnalysis } from '@embedpdf/plugin-layout-analysis/vue'
 
const props = defineProps<{ documentId: string }>()
const la = useLayoutAnalysis(() => props.documentId)
</script>
 
<template>
  <div>
    <button @click="la.provides.value?.setLayoutOverlayVisible(!la.layoutOverlayVisible.value)">
      Toggle Layout Overlay
    </button>
    <button @click="la.provides.value?.setTableStructureEnabled(!la.tableStructureEnabled.value)">
      Toggle Table Analysis
    </button>
  </div>
</template>

Analyzing Pages

Use the scope to trigger analysis. The returned task supports progress tracking (including model download progress) and cancellation.

The plugin implements smart caching — pages that have already been analyzed are skipped automatically. Pass { force: true } to re-analyze regardless.

<script setup lang="ts">
import { ref } from 'vue'
import { useLayoutAnalysis } from '@embedpdf/plugin-layout-analysis/vue'
 
const props = defineProps<{ documentId: string }>()
const la = useLayoutAnalysis(() => props.documentId)
const status = ref('idle')
 
function handleAnalyze() {
  if (!la.scope.value) return
 
  const task = la.scope.value.analyzeAllPages()
 
  task.onProgress((p) => {
    if (p.stage === 'downloading-model') {
      status.value = `Downloading model: ${((p.loaded / p.total) * 100).toFixed(0)}%`
    } else if (p.stage === 'creating-session') {
      status.value = 'Initializing model...'
    } else if (p.stage === 'page-complete') {
      status.value = `Page ${p.completed}/${p.total} complete`
    } else {
      status.value = p.stage
    }
  })
 
  task.wait(
    (result) => {
      const totalBlocks = result.pages.reduce((sum, p) => sum + p.blocks.length, 0)
      status.value = `Done — ${totalBlocks} blocks detected`
    },
    (error) => {
      status.value = error.type === 'abort' ? 'Cancelled' : error.reason.message
    },
  )
}
</script>
 
<template>
  <button @click="handleAnalyze">{{ status }}</button>
</template>

Cancellation

The task returned by analyzePage and analyzeAllPages can be cancelled at any time by calling abort():

const task = la.scope.value.analyzeAllPages()
 
// Later, to cancel:
task.abort({ type: 'no-document', message: 'Cancelled by user' })

When aborted, the error callback receives error.type === 'abort' and the page status resets to idle.

Displaying Results

Place the <LayoutAnalysisLayer /> component on top of the <RenderLayer /> inside your <Scroller />’s default slot. It draws color-coded bounding boxes for each detected layout element with labels and confidence scores. When table structure analysis is enabled, table rows, columns, and cells are rendered as dashed overlays.

The layer reads thresholds, overlay visibility, and selected block state directly from the plugin state — no props needed beyond documentId and pageIndex.

<script setup lang="ts">
import { LayoutAnalysisLayer } from '@embedpdf/plugin-layout-analysis/vue'
import { RenderLayer } from '@embedpdf/plugin-render/vue'
</script>
 
<template>
  <Scroller :document-id="activeDocumentId">
    <template #default="{ page }">
      <RenderLayer :document-id="activeDocumentId" :page-index="page.pageIndex" />
      <LayoutAnalysisLayer :document-id="activeDocumentId" :page-index="page.pageIndex" />
    </template>
  </Scroller>
</template>

Live Example

This example shows a full-featured layout analysis viewer with controls for analyzing pages, toggling layout and table structure overlays independently, adjusting confidence thresholds with sliders, and enabling/disabling table structure analysis at runtime. The model (~130 MB) is downloaded on first use and cached for subsequent runs.

API Reference

Configuration (LayoutAnalysisPluginConfig)

Component: <LayoutAnalysisLayer />

Renders bounding boxes over a PDF page for detected layout elements and table structure. Thresholds, overlay visibility, and selected block state are read directly from the plugin state.

Composable: useLayoutAnalysis(documentId)

Document-scoped composable that returns reactive refs for plugin state and methods.

Parameters

Returns

Composable: useLayoutAnalysisCapability()

Low-level composable that returns the full plugin capability directly.

LayoutAnalysisCapability Methods

Progress Stages (PageAnalysisProgress)

The onProgress callback receives one of these stages during analyzePage:

When using analyzeAllPages, an additional page-complete stage is emitted with pageIndex, completed, and total fields.

Result: LayoutBlock

Each detected element in the PageLayout.blocks array has these fields:

Result: TableStructureElement

When table structure analysis is enabled, each table block in PageLayout.tableStructures maps to an array of these elements: