Skip to Content

chunkText

chunkText(text: string, options?: ChunkingOptions): TextChunk[]

Defined in: src/lib/memoryRetrieval/chunking.ts:68 

Split text into overlapping chunks using sentence boundaries.

Algorithm:

  1. Split text into sentences
  2. Accumulate sentences until chunk size is reached
  3. Create chunk with overlap from previous chunk
  4. Handle edge cases (very long sentences, short texts)

Parameters

ParameterType

text

string

options?

ChunkingOptions

Returns

TextChunk[]

Last updated on