lollms

LLMBinding

The LLMBinding class is an interface class that serves as a template for implementing bindings with different large language models (LLMs) frameworks. It provides a set of methods that need to be implemented by specific bindings to enable interaction with LLMs.

Here’s a breakdown of the important components and methods in the LLMBinding class:

1. Constructor (__init__):
   • Parameters:
     • binding_dir (Path): The path to the binding directory.
     • lollms_paths (LollmsPaths): An instance of the LollmsPaths class that provides paths to different directories.
     • config (LOLLMSConfig): The global configuration object for LOLLMS.
     • binding_config (TypedConfig): The configuration object specific to the binding.
     • installation_option (InstallOption, optional): The installation option for the binding. Defaults to InstallOption.INSTALL_IF_NECESSARY.
     • supported_file_extensions (str, optional): The file extension for models supported by the binding. Defaults to "*.bin".
   • Description: The constructor initializes the LLMBinding object and sets up various properties based on the provided parameters. It also handles the installation of the binding if necessary.
2. Installation Methods:
   • install(): This method handles the installation procedure for the binding and should be implemented by each specific binding. It can perform tasks like downloading and installing the necessary dependencies or models. This method is called during object initialization if the binding is not already installed or if the installation is forced.
3. Model Methods:
   • build_model(): This method is responsible for constructing the model specific to the binding. It needs to be implemented by each binding and should return the constructed model object.
   • get_model_path(): This method retrieves the path of the model based on the configuration. It handles different scenarios, such as reading the model path from a file or constructing it based on the configuration.
4. Text Generation Methods:
   • generate(prompt, n_predict, callback, verbose, **gpt_params): This method generates text based on the provided prompt using the LLM model. It should be implemented by each binding and handle the generation process specific to the underlying LLM framework.
   • tokenize(prompt): This method tokenizes the given prompt using the model’s tokenizer. It needs to be implemented to provide tokenization functionality.
   • detokenize(tokens_list): This method detokenizes the given list of tokens using the model’s tokenizer. It should be implemented to provide detokenization functionality.
5. Utility Methods:
   • load_binding_config(): This method loads the content of the binding’s local configuration file. It reads the configuration data and syncs it with the binding_config object.
   • save_config_file(path): This method saves the binding’s configuration file to the specified path.
   • list_models(config): This method lists the available models for the binding. It should return a list of model names that match the specified file extension.

The LLMBinding class also includes a few other utility methods, such as get_current_seed() and reinstall_pytorch_with_cuda(), which provide additional functionality specific to the LOLLMS tool.

To create a new binding for a different LLM framework, you need to subclass the LLMBinding class and implement the required methods based on the framework’s API and functionality. Each method should be customized to work with the specific LLM framework you’re integrating.

Additionally, there are two builder classes, BindingBuilder and ModelBuilder, provided for convenience. These classes assist in building LLMBinding objects and LLM models, respectively.

This site is open source. Improve this page.