Merge pull request #336 from wenwei-lin/master

add config LLamaSharp tutorial in docs

.gitignore

@@ -292,4 +292,5 @@ XMLs
 logs
 wwwroot
 appsettings.Production.json
-*.csproj.user
+*.csproj.user
+env/

docs/index.rst

@@ -96,6 +96,15 @@ The main documentation for the site is organized into the following sections:
    llm/few-shot-learning
    llm/provider
 
+.. _llamasharp:
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Use Local LLM Models
+
+   llama-sharp/config-llamasharp
+   llama-sharp/use-llamasharp-in-ui
+
 .. _architecture-docs:
 
 .. toctree::

docs/llama-sharp/config-llamasharp.md

@@ -0,0 +1,60 @@
+# Config LLamaSharp
+
+BotSharp contains LLamaSharp plugin that allows you to run local llm models. To use the LLamaSharp, you need to config the BotSharp project with few steps.
+
+## Install LLamaSharp Backend
+
+Before use LLamaSharp plugin, you need to install one of the LLamaSharp backend services that suits your environment.
+
+- [`LLamaSharp.Backend.Cpu`](https://www.nuget.org/packages/LLamaSharp.Backend.Cpu): Pure CPU for Windows & Linux. Metal for Mac.
+- [`LLamaSharp.Backend.Cuda11`](https://www.nuget.org/packages/LLamaSharp.Backend.Cuda11): CUDA 11 for Windows and Linux
+- [`LLamaSharp.Backend.Cuda12`](https://www.nuget.org/packages/LLamaSharp.Backend.Cuda12): CUDA 12 for Windows and Linux
+
+**Please install the same version of LLamaSharp Backend with the LLamaSharp in BotSharp.Plugin.LLamaSharp.csproj.**
+
+![Check LLamaSharp Version](assets/check-llamasharp-version.png)
+
+```shell
+# move to the LLamaSharp Plugin Project
+$ cd src/Plugins/BotSharp.Plugin.LLamaSharp
+# Install the LLamaSharp Backend
+$ dotnet add package LLamaSharp.Backend.Cpu --version 0.9.1
+```
+
+## Download and Config Local LLM Models
+
+LLamaSharp supports many LLM Models like LLaMA and Alpaca. Download the `gguf` format models and save them in your machine.
+
+We will use a [Llama 2](https://huggingface.co/TheBloke/llama-2-7B-Guanaco-QLoRA-GGUF) model in this tutorial.
+
+After downloading the model, open the `src/WebStarter/appsettings.json` file to config the LLamaSharp models. Set the `LlmProviders` and `LlamaSharp` fields to correct settings as your computer. For example:
+
+```json
+{
+    ...,
+    "LlmProviders": [
+        ...,
+        {
+            "Provider": "llama-sharp",
+            "Models": [
+                {
+                "Name": "llama-2-7b.Q2_K.gguf",
+                "Type": "chat"
+                }
+            ]
+        },
+        ...
+    ],
+    ...,
+    "LlamaSharp": {
+        "Interactive": true,
+        "ModelDir": "/Users/wenwei/Desktop/LLM",
+        "DefaultModel": "llama-2-7b.Q2_K.gguf",
+        "MaxContextLength": 1024,
+        "NumberOfGpuLayer": 20
+    },
+    ...
+}
+```
+
+For more details about LLamaSharp, visit [LLamaSharp - GitHub](https://github.com/SciSharp/LLamaSharp).

docs/llama-sharp/use-llamasharp-in-ui.md

@@ -0,0 +1,29 @@
+# Use LLamaSharp in BotSharp
+
+Start the BotSharp backend and frontend services, and follow this tutorial.
+
+## Install LLamaSharp Plugin in UI.
+
+Go to the Plugin page and install LLamaSharp Plugin.
+
+![Install LlamaSharp Plugin](assets/install-llamasharp-plugin.png)
+
+## Config LLamaSharp as LLM Providers for Agents
+
+Edit or create an agent in Agents page, and config the agent.
+
+![Edit Agent](assets/edit-agent.png)
+
+In the edit page, config the provider as llama-sharp.
+
+![Choose LLamaSharp as Provider](assets/choose-llamasharp-as-provider.png)
+
+Then test the agent.
+
+![Click Test Agent Button](assets/click-test-button.png)
+
+![Test Agent Example](assets/converstaion-examples.png)
+
+If run successfully, you will see log like this in BotSharp service's console.
+
+![Console Output](assets/console-output-in-botsharp.png)