Telerik.Documents.AI.RAG

Default implementation of that splits text into token sized fragments respecting a maximum token limit per fragment.

Provides a default implementation for retrieving relevant context fragments based on semantic similarity to a given question.

Gets the manager responsible for handling context fragments within the current instance.

Initializes a new instance of the DefaultContextRetriever class with the specified context fragments manager, embedder, similarity calculator, tokens counter, and embedding settings.

The manager responsible for handling context fragments used in retrieval operations. Cannot be null. The component used to generate embeddings for context fragments. Cannot be null. The calculator used to determine similarity between embeddings. Cannot be null. The service used to count tokens in context fragments and embeddings. Cannot be null. The settings that configure embedding parameters, such as the maximum number of embeddings, embedding token size, and model input token limit. Cannot be null.

Gets the text representation of the relevant embeddings for the provided question.

The question text to find relevant context for. Cannot be null. The text representation of the relevant embeddings.

Asynchronously splits the specified text into fragments based on the provided embedding token size.

The text to be divided into fragments. Cannot be null. The maximum number of tokens allowed in each fragment. Must be a positive integer. A task that represents the asynchronous operation of fragmenting the text.

Provides a default implementation of the interface that stores a collection of fragments.

Initializes a new instance of the DefaultFragments class with the specified fragments.

An array of fragments to be used. Cannot be null.

Gets the collection of fragments that compose the complete content.

Provides a default implementation of the ISimilarityCalculator interface for finding the most similar text fragments to a given question using embeddings.

This class uses an IEmbedder to generate embeddings for input text and compares them to provided fragments to determine similarity.

Initializes a new instance of the DefaultSimilarityCalculator class using the specified embedder.

The embedder used to generate vector representations for similarity calculations. Cannot be null.

Finds the most similar text fragments to the specified question and returns their similarity scores.

The method uses the current embedding model to compute similarity between the question and each fragment. If the specified limit exceeds the number of fragments, all fragments are considered. The results are ordered by descending similarity score. The input question to compare against the provided fragments. Cannot be null, empty, or whitespace. An array of embeddings representing the text fragments to search. Cannot be null or empty. An array of similarity scores, each containing a fragment and its similarity to the question. Returns an empty array on error conditions.

Provides functionality to count tokens in a given input, such as words or other units of text.

This class implements the interface, allowing for token counting operations. The specific behavior and definition of a "token" depend on the implementation of the interface.

Initializes a new instance of the class with the specified embedding settings.

The parameter is used to configure the tokenizer encoding based on the specified model ID and tokenization encoding. Ensure that the object is properly configured before passing it to this constructor. The embedding settings that define the model ID and tokenization encoding to be used.

Counts the number of tokens in the specified input string.

The input string to analyze. Cannot be null. The total number of tokens found in the input string.

Provides helper methods for calculating similarity scores between an already embedded question vector and a collection of fragment instances.

This helper is intentionally static and independent from any concrete implementation so that external components (e.g. test applications using different embedding providers such as Ollama) can reuse the core similarity computation logic without duplicating code.

Calculates cosine-similarity scores between a question embedding vector and fragment embeddings, returning the top results ordered descending by similarity.

The embedding vector of the question. The fragment embeddings to compare against. Array of where T is IFragment.