Skip to content

LangTorch Utility Functions

LangTorch implements many of the same utility functions one can use in Torch. To streamline usage, these are also automatically invoked if one passes TextTensor inputs to the analogous torch function. The exception are creation operations, which have to be imported from LangTorch.

Creation Ops

Operation Description
zeros Returns a text tensor with empty entries, with the shape defined by the variable argument size.
zeros_like Returns a text tensor with empty entries, with the same size as input.
full Creates a text tensor of the argument size filled with the given Text.
full_like Returns a text tensor with the same size as input filled the given Text (fill_value).

Indexing, Slicing, Joining, Mutating Ops

Functions that take in TextTensors and output TextTensors (note, functions like reshape are also TextTensor methods):

Operation Description
concatenate Concatenates the given sequence of seq tensors in the given dimension.
permute Returns a view of the original tensor input with its dimensions permuted.
reshape Returns a tensor with the same data and number of elements as input, but with the specified shape.
squeeze Returns a tensor with all specified dimensions of input of size 1 removed.
stack Concatenates a sequence of tensors along a new dimension.
swapaxes Alias for transpose.
transpose Returns a tensor that is a transposed version of input.
split Splits input, a tensor with three or more dimensions, into multiple tensors depthwise according to indices_or_sections.
cat Concatenates the given sequence of seq TextTensors in the given dimension.
unsqueeze Returns a TextTensor with a dimension of size one inserted at the specified position.

Semantic Algebra vs Tensor Algebra

One of the benefits of representing information geometrically in tensors is that we can have meaningful dimensions that contain, for example, different features or, in LangTorch, different answers to the same query. In such cases, we often wish to perform some action along this dimension, such as averaging the features. As such operations make sense for both numerical and text tensors, we keep both functionalities by making these operations act differently depending on which package they are imported from. Passing a TextTensor to these operations imported from torch will automatically substitute text tensors with the tensors of its embeddings and invoke the regular PyTorch operation

Operation Description
torch.mean Returns the mean of the embeddings tensor in the given dimension for a given TextTensor
torch.max Returns the maximum value of the embeddings tensor in the given dimension for a given TextTensor
torch.min Analogous

What would then be a mean or max on text entries? We often want to take the "average" of different answers or pick out the best one. What LangTorch proposes is that we may think of such operations as taking the (semantic) mean and max value along that dimension. These "semantic" functions send entries along a dimension to an LLM with task instructions (to leverage them or pick the best one). These functions have additional parameters, but by default:

Operation Description
langtorch.mean Returns a TextTensor with "average texts", that is entries that synthesize the important (and correct) information from the input TextTensor along dimension dim.
langtorch.max Returns a TextTensor with "the best texts", that is an entry selected from the input TextTensor along dimension dim that stands out as the best and most relevant.
langtorch.min Analogous

As these functions call a language model with a custom prompt, they take additional arguments that allow us to modify the model or instructions for this task. A fast modification is to just supply the definition, but we can also modify the whole prompt:

  • definition: Text that defines what is meant by the "average" or "best" text in the specific context. This is inserted into the prompt template and can be used e.g. to pick the most verbose text by setting definition in max to be "".
  • prompt: The full prompt template sent to the language model. Should include the {definition} placeholder.
  • model: The name of the language model to use for the operation. Defaults to the model set in the current context.
  • T, max_tokens, etc.: Additional keyword arguments passed to the Activation constructor to control the model call, such as temperature, maximum tokens, etc.

For example: 

import langtorch as lt

texts = lt.TextTensor(["The movie was great!", 
                       "The film was okay, but too long.",
                       "I didn't enjoy the movie much."])

avg_review = lt.mean(texts, 
                     definition="a concise review that captures the overall sentiment",
                     model="gpt-3.5-turbo", 
                     T=0.5, 
                     max_tokens=20)

This allows performing semantic algebra operations with fine-grained control over the language model call. The definition and prompt arguments make the intended operation clear to the model, while the remaining arguments configure the API call itself.