Pantella is a fork of the popular Skyrim/FO4 mod Mantella, which allows you to naturally speak to NPCs using Speech-to-Text or Text Input, various LLM inference backends/OpenAI API Compatible (text generation), and half a dozen Text to Speech options.
- Interact with 1,000+ NPCs, all with their own unique backgrounds (or add your own)
- Support for local (eg Llama 2, 3, 3.1, Gemma, etc.), OpenAI (GPT-4, 4o, etc.), and OpenRouter (Toppy-M 7B, various other LLMs from dozens of providers, including Anthropic, Meta, OpenAI, etc.) language models
- Compatibility with 20+ languages and support for changing the whole prompt sent to the LLM with the prompt_style settings to the desired language. This should improve the quality of the responses from the LLM when using a language other than English.
- NPCs remember past conversations with you (better than before!)
- NPCs are aware of in-game events, remember them long term, and can take action based on them during conversations
- Vision support for vision based LLMs, both local and cloud based
- Fully playable in Skyrim VR / SE / AE and Fallout New Vegas (plus Tale of Two Wastelands!)
Documentation here: Pantella Documentation
Note: Pantella is not yet bug free or ready for all users. Please read the entire README before installing and reach out on Discord or GitHub if you have any questions or issues. It may have some issues that need to be ironed out, but I'm still working on it and will try to help anyone who needs it.
- Preface - "What's different from Mantella?"
- Things to Consider Before Installing...
- Installation
- Troubleshooting
- Development
- Attributions
Pantella has been forked from Mantella v0.11 and more or less redesigned from the ground up. A lot of the systems have been broken out into generic modules to make it easier for developers to add new LLM backends, change how conversations are stepped through, etc., without having to redo any of the work for the parts they don't want to replace. It also gives end users much finer control over the exact shape of the prompt sent to the LLM with the prompt_style settings. xTTS and PiperTTS are both included with Pantella by default, and the launcher has been pre-setup to cut out any preparation steps. xVASynth is currently the only TTS that requires extra steps to get working because I can't include it by default.
Pantella has a completely new memory system that depreciates the old summary memory. The new memory uses chromadb to more or less sort everything you've ever said to an NPC into the most relevant chunks of memories. No more lossy summaries that forget details because the LLM doesn't know to include them/can't shove everything into one paragraph.
Pantella also has a completely revamped NPC behaviors system that has been made as easy to program for as possible to let the NPC choose to do more complex actions than the pre built "Follow" and "Attack" behaviors from Mantella. Lemme know if you want details on how to make your own behaviors. Pantella Spell also implements an Actor API to let the python behaviors control the Actor in game, though this only runs when the NPC next speaks.
Pantella additionally has completely reworked generation processing to basically read the output text and figure out which chunks should be spoken by the NPC and which chunks should be read by the new Narrator, which reads roleplayed actions in asterisks by default but can be set to other roleplaying styles in settings. The Narrator is completely customizable, and can be disabled if you prefer the old Narrator-less experience. But I recommend giving it a shot, it's very Baldur's Gate 3-y.
And Pantella also just got a new add-on system to let other mod authors create self contained patches for Pantella to add voice models, character entries, support for game events for mods and new behaviors, new components like inference engines and TTSes, without having to tinker around in the source much.
Almost every part of Pantella, including the inference engines(LLM backends), the TTS backends, the memory system, the behavior system and even the game_interface which is used to interact with the game, is broken out into its own module that can easily be replaced with a new module. This makes it easy to add new features to Pantella without having to redo any of the work for the parts you don't want to replace and is the main reason I'm able to add so many new features so quickly. Want to add support for another game? Replace the game_inferface and get the right characters in your characters directory and there you go. New inference engine just dropped and I haven't had time to add it? Adding it yourself from documentation isn't that hard and I'd love to help walk you through how I'd do it if you get stuck(plus if it works, you should PR it <3).
Note: There is still some work to be done to make the game_interface more generic, but it's a start. The character_manager class is also a bit of a mess and could use some work to make it more generic for example, this is definitely still an in progress change still due to the scope of the project.
No more just OpenAI API compatible servers, if you have the computer to run it, you can run the LLM directly in Pantella itself. No 3rd party software or setup required, it works out of the box. llama-cpp-python is built in and allows running GGUF models natively in Pantella. There's also support for running transformers models directly through Pantella. If someone wanted to add vLLM support, or exllama, or any other LLM backend, it would be as easy as adding one new file containing all the inference code, updating the requirements.txt with any new packages that are required and adjusting the config_loader.py file to add any new config settings. Feel free to ask in the Discord if you have a backend you want to add and I can help you get started and help troubleshoot any issues you may have. Additionally, there is support for vision based LLMs now as well.
Pantella comes with added support for several new TTSes compared to Mantella. Additionally, multiple TTSes can be loaded at once and used in a fallback order. Including the ones in Mantella v0.12, here is a list of all available TTSes in Pantella with their pros and cons:
- PiperTTS(Also Available for Mantella) is a fast and really easy to run on most computers. It doesn't require special hardware like a CUDA enabled GPU, runs on CPU, and I've ever managed to run it on my phone via Winlator. It's not the highest quality TTS in my opinion, but it's enabled by default because everything needed to run it is included with Pantella and it's the easiest to run in general.
- xVASynth(Also Available for Mantella) is a TTS that uses the xVASynth engine to generate voices. It's a bit more complicated to set up than the other TTSes, but it's a very good TTS that can run on CPU or GPU fast enough to actually use, and it's the most powerful and flexible TTS available for Pantella. Training new voices does required a really decent GPU, but it's possible to train new voices with xVASynth and use them in Pantella pretty easily if you have the hardware. Good for Skyrim, but doesn't have all of the voices for Fallout 3 or New Vegas to properly support Fallout TTW. If you're planning to use Pantella for Fallout TTW, xVASynth is pretty much incompatible with it at this time due to the lack of voice models, less than half of the voice models for Fallout TTW are available for xVASynth at this time. But if you're using Pantella for Skyrim, xVASynth is a very good option for a TTS, especially if you don't have much extra VRAM to spare for TTSes, as it can run on CPU pretty well.
- xTTS 2(Also Available for Mantella) is a really good TTS for human sounding voices, but it struggles for dragons, robots, etc. It also is basically unusable on CPU and requires an additional 4GB of VRAM usage on top of your game to run it. And it's slower than xVASynth and PiperTTS. But for the voices it's better for, it's a lot better than xVASynth or PiperTTS. It can be sampled on a short sample between 5-10 seconds with decent quality. It also has an English accent by default, but you can actually change the accent to any of the other supported languages in the settings and override specific characters/races to use specific accents with the prompt_style/character JSON files.
- StyleTTS2 is very similar to xTTS 2, but it's a bit more robotic sounding and has pronounciation issues sometimes. But higher pitch voices, like some female voices, sound a bit better with StyleTTS2 than xTTS 2. It's also a bit faster than xTTS 2, and doesn't require as much VRAM, hovering closer to 3GB of usage. It can be sampled on a short sample between 15-25 seconds with decent quality.
- F5-TTS is a new diffusion based TTS that was released alongside E2 TTS. It only requires like 1-1.5GB of VRAM, and sounds as good or better than xTTS for a lot of voices. It's about the same speed, maybe a little slower, but I could see a few characters getting this one over xTTS or StyleTTS2. It can be sampled on a short sample between 10-15 seconds with decent quality. F5 sounds less robotic than E2.
- E2-TTS is a new Unet based TTS that was released alongside F5 TTS. It only requires like 1-1.5GB of VRAM, and sounds as good or better than xTTS for a lot of voices. It's about the same speed, maybe a little slower than F5-TTS, but I could see a few characters getting this one over xTTS or StyleTTS2. It can be sampled on a short sample between 10-15 seconds with decent quality. E2 sounds more robotic than F5.
- GPT-SoVITS is a Chinese based TTS/voice conversion model that's very good for the compute required for it. It only requires 1GB of VRAM, and sounds fantastic on a lot of voices. Anything with a heavy accent seems to be a struggle for it, like Nords and some of the Elves. And it has some pronunciation issues at times. But for the voices and words it's good at, it's genuinely unparalleled in the local space in my opinion and easily surpasses xTTS for those voices.
- Chatterbox(new) is a "production-grade open source TTS model. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations." Very good emotional tinting in speaking, can do a lot of different voices very well, and isn't unusably slow, though it is a bit slower than xTTS 2 on the same hardware. It needs 6GB of VRAM to run on GPU, but it can run on CPU as well, albiet, very slowly, about five times slower than the GPU(2080 ti) on a Ryzen 9 5950X with DDR4. It's more stable than xTTS and I personally prefer it over most of the other TTSes so far. But it can add strange accents to some voices, likely due to the way it was trained. It can be sampled on a short sample between 5-10 seconds with very good quality, and I haven't had a bad time giving it longer samples either but they rarely seemed to do significantly different and sometimes trended to be worse overall. Consistency is better than xTTS, less modem noises coming out of NPCs. Overall a very good TTS if you can run it.
In summary xTTS 2 and StyleTTS2 are both very good TTSes for a lot of voices, but they require a decent amount of VRAM and are slower than the other options. F5-TTS and E2-TTS are both very good diffusion based TTSes that require much less VRAM and are about the same speed as xTTS, but they can be a bit inconsistent with some voices. GPT-SoVITS is a Chinese based TTS that is very good for the compute required for it, but it struggles with heavy accents and has some pronunciation issues. Chatterbox is a new TTS that is very good for emotional tinting and consistency, but it can add strange accents to some voices and is slower than xTTS on the same hardware. PiperTTS is the easiest to run and included with Pantella by default, but it's not the highest quality TTS. xVASynth is a very good TTS that can run on CPU or GPU, but it doesn't have all the voice models for Fallout TTW so it's not recommended for use with Fallout TTW at this time.
Pantella also has support for other TTS options that aren't included with Pantella by default, like Qwen3 TTS and IndexTTS2, which can be added with the add-on system. Read more about those down here.
Pantella supports a special type of response output that does Chain-of-Thought(CoT) in a single prompt. Locally using GBNF grammars, and remotely using response formats on OpenRouter. Only some OpenRouter models are supported by this feature, and it's recommended to use the local models for this feature. This feature is still in development and may not always work as expected. It will also add an inconsistent delay before the character speaks, as it has to go through the thought process before speaking. It's also customizable in the settings, allowing you to architect how NPCs think before they respond to you, and can also be disabled if you don't like it. Operates like reasoning models (GPT o1, DeepSeek r1, etc.) but doesn't require special finetuning to work. I'm not really sure if it's actually improving performance for roleplay, would want to benchmark it, but I find it personally interesting. It has a lot more potential for making tool calling(Behaviors) more powerful, so I'm excited to play with this more in the future.
When a character is first encountered, if a character entry is not found in the characters directory, Pantella can automatically generate a character entry for the character. This is done by using the character's name, race, sex, location, etc. to generate a character entry. This feature is still in development and may not always work as expected. This feature also requires your model to support CoT as it relies on using response formats/GBNF grammars to generate the character entry.
Pantella currently supports Skyrim Special Edition, Skyrim Anniversary Edition, Skyrim VR, Fallout New Vegas and (technically) Fallout 3 via Tale of Two Wastelands. Support for other games can be added by adding another game_interface module and adding the appropriate character entries to the characters directory. If you want to add support for another game and don't know where to start or what help, feel free to reach out to me on Discord or open an issue on GitHub and we can discuss how to do it. Enderal is also somewhat supported, but there are no character entries for Enderal included with Pantella at this time.
There are no discovered minimum requirements at the time of writing for Pantella. Pantella needs a certain amount of hardware allocation to run successfully for specific setups though, and if this is being soaked up by other hardware intensive mods, it may crash.
The minimum requirements for xVASynth can be found on its Steam page. It runs in CPU mode by default, using a single CPU core/thread. Only supports GPU acceleration on NVIDIA cards that have CUDA. Using the same GPU as the game will produce stutter, especially if it can't allocate ~2 GB of VRAM. You may try using an older NVIDIA card that has CUDA on another free PCI-Express slot of your PC and run any CUDA enabled services on that.
If you're trying to minimize the amount of compute needed to run Pantella, I'd advise using xVASynth or PiperTTS and using Open Router. This should provide the easiest to run experience.
Pantella requires around 4-6 GB of storage space and includes the voice models/latents for PiperTTS, and xTTS_api out of the box. xVASynth requires around 4 GB of storage space for the software itself and after downloading all the voice models for Skyrim and Fallout New Vegas I have used around 38 GB of storage space. If you're using a local language model, you'll have to factor in the size of the model you're using yourself. But suffice to say, you'll need a fair amount of storage space to run Pantella with all the bells and whistles.
For the most basic setup, you won't need most of this software. However to get the most out of Pantella, you will need to install the following software:
Pantella optionally requires git to be installed to perform all updates correctly. If you don't intend to update your installation, you can skip this step. This is required even if using the launcher. If you try to run the ./install_pantella_requirements.bat script without git installed, it will fail immediately without an error message(unless you run the bat file in the terminal).
- Download the latest version of Git for Windows from here.
- Run the installer and follow the instructions. The default settings should be fine.
Hah! You're probably already done with this part! Git is installed by default on most Linux distributions. If you don't have it, you can install it with sudo apt install git or sudo yum install git or sudo pacman -S git depending on your distribution.
The easiest way to install FFmpeg on Windows is to install it via Chocolatey. You can certainly download the compiled build from ffmpegs site and add it to the system path, but I feel like that's not exactly approachable to everyone.
- To install Chocolatey, follow their Individual Use installation guide over here: Install Chocolatey You should just have to run the command they recommend.
- Then to install ffmpeg with chocolatey, run
choco install ffmpegin the same terminal you just used to install Chocolatey. - Restart your computer to ensure the changes take effect and are usable by Pantella.
This is super simple. It might already come installed on your distro of choice, but if it doesn't, simply run sudo apt install ffmpeg.
Pantella requires the Microsoft C++ Build Tools to be installed to build some of the required Python packages to install/reinstall them. If you don't have this installed, you can install it by following the instructions below. However if the requirements change over time, and you continue to receive updates for Pantella, you may need to install the Build Tools to keep up with the changes. It will try to automatically update your requirements whenever new requirements are added, but if it fails to do so, you may see error messages and no longer be able to run Pantella until you install the Build Tools. If you try to run the ./install_pantella_requirements.bat script without the Build Tools installed, it will fail immediately with an error message about the Build Tools not being installed.
- Download the latest version of the Microsoft C++ Build Tools from here.
- Follow the instructions to install the Build Tools from the top answer over here: Install Build Tools
- Run the installer and follow the instructions. The default settings should be fine.
You're probably already done with this part too. Microsoft C++ Build Tools are not required on Linux.
If you want to use F5-TTS, E2 TTS, StyleTTS2, OuteTTS, ParlerTTS, ChatTTS, or xTTS 2 with GPU acceleration, you will need to have CUDA installed on your system. If you don't have CUDA installed, you can skip this step and use PiperTTS or xVASynth instead.
- Download CUDA 12.8 from here.
- Run the installer and follow the instructions. The default settings should be fine.
Newer versions of CUDA may work, but you'll have to test them yourself and install torch for your CUDA version. Newer versions of torch may not work with Pantella, so it's recommended to stick with CUDA 12.8 for now as it's the most tested version of CUDA with Pantella at this time. Long term I'd love to support newer versions of CUDA, but for now, if you want to use CUDA acceleration with Pantella, it's recommended to use CUDA 12.8. Make sure to select the correct version for your operating system and architecture. If you are trying to use a newer version of CUDA, you can try it out and see if it works with Pantella, and if you have any issues, feel free to reach out to me on Discord or GitHub and I can try to help you troubleshoot any issues you may have with using a newer version of CUDA with Pantella.
- Pantella works on Windows 10, Windows 11 and Debian Linux (it is yet unconfirmed whether it works on Windows 7)
As Pantella accesses and writes to files within your game if choice's folder, it is unlikely to work correctly if your game is installedin Program Files(as is typical with normal Steam installations). Please ensure that you have your game stored outside of this folder (Such as C:\Games\Steam for example).
If you use the Steam version of Skyrim or Fallout New Vegas, then note that Steam does not allow to move the old or create a new Steam Game Library on the same disk partition by simply ignoring the attempt to do so. You either move the whole Steam client outside as described on this Steam Support page or use LostDragonist/steam-library-setup-tool to create a Steam Game Library besides another.
xVASynth is a very good Text-To-Speech software that can be used by Pantella. It is available on Steam and Nexus. It is recommended to download it via Steam, as it is easier to update and manage. If you are using the Nexus version, do not store xVASynth in your Skyrim folder.
-
Download xVASynth via Steam or Nexus. Do not store xVASynth in your Skyrim folder.
-
Download xVASynth trained voice models of Skyrim for all or any characters that you are likely to encounter. If downloading all models sounds a bit daunting, you can start with the "Male Nord" and "Male Soldier" voice models to at least allow talking to Skyrim guards. You can either download all models via a torrent, via the xVASynth UI if you have Nexus Premium, or manually via the Nexus Mods page.
xVASynth Model Installation Options
If you are subscribed to Nexus Premium, open the xVASynth UI and select "Get More Voices" from the bottom left corner. Unselect all games except for Skyrim and download all models. Note that this may require restarting a few times for the downloads to properly complete.
Voice models can be downloaded via a single torrent. Torrents can be downloaded via Bittorent clients such as qBittorent. Note that download speeds vary depending on the time of day. Paste the below magnet link in your browser to receive a popup to open it via your Bittorent client, and set the download location to your_xVASynth_folder/resources/app/models/skyrim:
magnet:?xt=urn:btih:798BB3190E776BFDCF590910C0805656420F45BC&dn=skyrim&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337&tr=udp%3a%2f%2fexplodie.org%3a6969&tr=wss%3a%2f%2ftracker.btorrent.xyz&tr=wss%3a%2f%2ftracker.openwebtorrent.comNote that this magnet URI may be removed from this page if any voice model becomes unavailable on Nexus Mods.
If you do not have Nexus Premium, or if the torrent is not available, you can also download the voice models directly from Nexus here (under "Optional", not "Old). Once you have manually downloaded each voice model into a folder, open xVASynth and drag all zipped voice model files from this folder into the voices panel. Wait for the installation to complete (this may take some time; a popup will display when finished saying "x models installed successfully"). If this method doesn't work for you, you can also unzip the models manually into the correct xVASynth folder (xVASynth\resources\app\models\skyrim). Once the extraction is complete, you can delete the zipped voice model files.
-
Download the .lip plugin for xVASynth here and download FaceFXWrapper from here (you do not need to download CK64Fixes). Instructions on how to install these are on the .lip plugin Nexus page. Make sure to place FaceFXWrapper in the plugins folder as stated on the .lip plugin page.
-
Download the Elder Scrolls pronunciation dictionary from here, and follow the instructions to install.
-
In the xVASynth UI, if "Skyrim" is not already selected, please do so by clicking the arrows symbol in the top left corner. On the navigation bar on the top right of the xVASynth UI, click on the "ae" icon. Once opened, click on the CMUDict dictionary and select "Enable All" in the bottom left corner. Do the same for "xVADict - Elder Scrolls" received from the above step.
Make absolutely sure xVASynth isn't already running when Pantella starts unless it's running in headless mode. It will error out if it's already running when Pantella tries to start it in headless mode. If you want to use the xVASynth UI, you can start it after Pantella has started and it should work fine.
Depending on which game you want to get Pantella working for, you will need to install the required mods for the in-game plugin to function. You can see the required mods for each plugin on the plugin github page:
- PantellaNV for Fallout New Vegas
- Pantella Spell for Skyrim SE/AE/VR
There are a number of different LLMs to choose from, ranging from small local models to expensive externally hosted models. Note that some smaller models may struggle to handle long term conversations or may say the wrong thing at the wrong time. If you just want to get started without thinking too much about it / explore alternative options later and are new to Pantella, skip to the Installtion section.
Copy your OpenAI secret API key (see here if you need help finding it (you will need to set up an account if you haven't already) and paste into PantellaSoftware/GPT_SECRET_KEY.txt. Do not share this key with anyone. While there is a free trial, you will need to set up your payment details for the API to work. OpenAI is NOT recommended, their API is very limited, and their content policy is utterly draconian. Please use any other option, literally any other option.
Create an account with OpenRouter. Go to the "Keys" tab and generate a new key, saving its value to PantellaSoftware/GPT_SECRET_KEY.txt. Do not share this secret key with anyone. In config.json, set model to a model from the list here (eg undi95/toppy-m-7b). Set alternative_openai_api_base to "https://openrouter.ai/api/v1" (without quotes).
Install text-generation-webui from here. Place a local model into the text-generation-webui\models folder (to get started, you can download toppy-m-7b.Q4_K_S.gguf from here). Paste the text "--extensions openai --auto-launch" (as well as "--cpu" for CPU users) into the installed folder's CMD_FLAGS.txt file. Start text-generation-webui and wait for the UI to open in your web browser. Navigate to the "Model" tab, select your model from the drop-down list, and click "Load". In your config.json file, set alternative_openai_api_base to "http://127.0.0.1:5000/v1" (without quotes). Just to note, you need to make sure text-generation-webui is running when running Pantella!
Install koboldcpp's latest release from here: https://github.com/LostRuins/koboldcpp/releases. If you have a nvidia gpu with cuda support, download the koboldcpp.exe file. If you do not or do not want to use cuda support, download the koboldcpp_nocuda.exe. Download it outside of your Skyrim, Fallout New Vegas, xVASynth or pantella folders. Download a local large language model, such as toppy-m-7b.Q4_K_S.gguf from here). Save that somewhere you can easily find it, again outside of Skyrim, Fallout New Vegas, xVASynth or pantella. Run the koboldcpp.exe. When presented with the launch window, drag the "Context Size" slider to 4096. Click the "Browse" button next to the "Model:" field and select the model you downloaded. Under the presets drop down at the top, choose either Use CLBlas, or Use CuBlas (if using Cuda). You will then see a field for GPU Layers. If you want to use CPU only leave it at 0. If you want to use your GPU, you can experiement with how many "layers" to offload to your GPU based on your system. Then click "Launch" in the bottom right corner. In your config.json file, set alternative_openai_api_base to "http://localhost:5001/v1" (without quotes). Just to note, you need to make sure koboldcpp is running when running Pantella!
This option does not require a powerful computer to run a large language model, because it runs in the google cloud. It is free and easy to use, and can handle most .gguf models that are up to 13B parameters with Q4_K_M quantization all on the free T4 GPU you get with google colab. The downside is Google controls dynamically when the GPUs are available and could throttle your access at any time, so it may not always work / be available. To use this method, go to this web page: https://colab.research.google.com/github/LostRuins/koboldcpp/blob/concedo/colab.ipynb. Click the play button that appears below the text "Enter your model below and then click this to start Koboldcpp." Wait until text stops generating (probably will take a minute or two). You should see a URL link near the end of the text after a statement like "Connect to the link below," with a silly name, in a format like https://its-taking-time-indeed.trycloudflare.com. You may want to click on the link just to ensure koboldcpp pops up to ensure its ready before proceeding. Select that link and copy it with CTRL+C. In your config.json file, set alternative_openai_api_base to that URL by pasting it, and then add /v1 at the end. So it will look something like alternative_openai_api_base = https://its-taking-time-indeed.trycloudflare.com/v1. Make sure to keep your browser open to the koboldcpp colab notebook while using Pantella so it does not turn off. If you want to choose a different llm model to use with this method, make sure it is a .gguf model and follow the instructions on the colab to do so. Be sure to close your browser tab once you've finished your Pantella session, to free up the GPU and help avoid hitting Google's usage limits.
If you have a powerful computer, you can run a local LLM. This is the most powerful and flexible option, but it requires a lot of resources. For example, you can run a local LLM using the llama-cpp-python backend. To do this, you will need to install the llama-cpp-python backend by running the script included with the launcher for your specific setup/hardware. You will also need to download a local model, such as toppy-m-7b.Q4_K_S.gguf from here. Once you have the model, set model_path to the path to the model in your config.json file, and it should be good to go.
First things first, you'll need the latest launcher. You can find that here: https://github.com/Pathos14489/Pantella-Launcher/releases
First click on the latest release under Releases. And follow the installation instructions for the launcher on the release. The instructions will likely be something like this:
- Download
PL-v0.0.6.7zand extract it where you want Pantella to be installed. - Download the Python environment necessary for your repo of choice. You can download only one or both, but at least one of them is necessary for the launcher to uh, do anything.
- Pantella requires
python-3.10.11-embed.7z. - Mantella requires
python-3.11.6-embed.7z. I recommend somewhere on the same drive as your game of choice, this will reduce latency caused by moving the voicelines from Pantella into the game directory, but somewhere like your desktop is also probably fine if you only use SSDs. Once you've unpackged the release, go open the launcher.
- Pantella requires
When you first open the launcher, you'll see a bunch of repositories available to download. Don't touch them yet. First, click up here on the settings button:
Enable a game and configure the settings for that game. Here's an example of how my Mod Organizer 2 config looks in the launcher:
Close settings and download the repository of Mantella/Pantella you want to use(we're going to assume you want to use Pantella). For support using other forks, please go ask for it in their server(s) unless the problem is definitely an issue with the launcher. Note that the repositories are all subject to live updates, and may not run on a given update if the committer wasn't careful/they're not maintaining a stable main branch(guilty as charged). This is not the launcher's fault, this is not the other developers fault, this might be my fault, but this is just the pain of using the live updated source code. Please bear with us, change hurts(especially if you like throwing your spare change at people) and takes time. ❤️
After it's done downloading, your front menu should look like this:
If your plugins haven't appeared with a deploy button, please restart the launcher. If they still don't appear, reach out for support, that's a bug with the launcher and we want to help fix it. If they have appeared, please click deploy on the plugin of your choice. If you're using Mod Organizer 2, you might have to click this button to refresh your modslist:
If you're using Vortex, tough luck, restart your mod manager and get yelled at for not using it the way they wanted you to. If your plugin doesn't appear in your mod manager, again this is a bug, please reach out. you can test this by either checking the folder directly, or check if the plugin deploy button is available to click on a launcher restart. if it is, it's failing to deploy the plugin and again we want to help you.
Next I'd like to explain the basic structure of the launcher directory. Your final launcher directory should now look like this:
This is your launcher's root directory. If you're using llama-cpp-python, or LLaVA, you'll want to install the correct version using the included bat files now. If you have Show Debug Console enabled in settings, it will open a console that lets you read alon with Pantella as it processes your requests. Useful for debugging. Then click start for Pantella in the launcher. If you have Automatic Crash Recovery enabled in the launcher settings page. the console window that opens will always restart if it closes for any reason. This is an intentional feature, not a flaw. If you don't like it, turn it off. The first launcher of Pantella will error out, but it has to generate some config files for you, first. So please let it error out once.
Now navigate from your launcher directory, to ./repositories/Pathos14489_Pantella/ where we can finally start configuring Pantella itself for how you intend to use it!
First, another look to make sure our Pantella repository folders look the same. If you notice any differences, you might have either a newer version than is depicted here, an older version, or something went horribly wrong. Try updating just incase, and failing that, feel free to reach out for support on the Discord if anything looks too different from here:
First thing you need to do is go to launcher_dir/repositories/Pathos14489_Pantella/interface_configs and set up the interface_config json for the game you wish to play.
skyrim - Skyrim Special Edition/Enderal
skyrimvr - Skyrim VR
falloutnv - Fallout New Vegas
fallout4 - Fallout 4
fallout4vr - Fallout 4 VR
Make a note of the game IDs you see here, you'll need these in just a moment. My skyrim.json looks like this for example after being set up:
{
"game_path": "Z:\\SteamLibrary\\steamapps\\common\\Skyrim Special Edition",
"mod_path": "Z:\\Modded\\SkyData\\mods\\Pathos14489_Pantella-Spell",
"conversation_manager_type": "creation_engine_file_buffers",
"interface_type": "creation_engine_file_buffers",
"behavior_manager": "default_behavior_manager"
"chat_manager": "creation_engine"
}After that, you'll want to open your config.json and set the game_id under "Game" to the ID of the game you want to play Pantella with.
Now just click start for Pantella and let it perform first time start up. It should error out at first, this is expected. It has to generate some config files for you so please let it error out once. After that, you'll have a brand new config.json file in your Pantella directory. This is where you'll be configuring most of Pantella's core settings from now on.
After you decide on type of LLM you want to use, we're going to assume you left it on the default, OpenRouter with Toppy-M 7B. If you want to use a different LLM, you can change the model and alternative_openai_api_base settings in your config.json file. If you want to change the inference engine, you can set inference_engine in your config.json file to either koboldcpp or text-generation-webui. If you're using a local model, you'll need to set the path to the model in your config.json file. If you're using OpenRouter, you'll need to set the model to a model from the list here (eg undi95/toppy-m-7b). Set alternative_openai_api_base to "https://openrouter.ai/api/v1/" (it's already set by default, so if you haven't changed it, you shouldn't have to do anything here.)
If you're using OpenRouter, you'll need to create an account with OpenRouter. Go to the "Keys" tab when you hover over your user icon at the top right while logged in and generate a new key, saving its value to ./GPT_SECRET_KEY.txt. Do not share this secret key with anyone as it's effectively your login information for OpenRouter.
Do you intend to use Speech-To-Text? If so, you'll need to set up your speech recognition settings as it's disabled by default. Set stt_enabled to true to enable the speech to text. Then in game in the MCM menu, if this is your first time running Pantella on a save, you will have to toggle the microphone off/on again. I'm not sure why, will fix later. It's somehow bugged halfway on and off such that neither will work. Toggling it will reset it to the correct state and then you may use the microphone as intended or use text input if you prefer.
By default PiperTTS is set to be used, and it can cover 90% of the voices in Skyrim. If you want to use a different TTS, you can change the tts_engine setting in your config.json file to either xtts_api, style_tts_2, xvasynth, etc. Check the available TTS options. If you're using xVASynth, you'll need to set the path to the xVASynth executable in your config.json file. If you're using PiperTTS, you don't need to do anything here. Read about the pros and cons of the various TTS options here.
Normally you won't be using this. Most cloud based models are not very good at vision. OpenAI compatible chat completion with vision is supported, but the LLaVA support via llama-cpp-python is far better and I recommend using it instead if you want to use vision. If you're trying to use it, set vision_enabled to true in your config.json file. If you're using LLaVA, you'll need to set the inference engine to llama-cpp-python
Start the launcher, and click start for Pantella. Have fun! If you have any issues, please reach out on the Discord. We're here to help. Note: Vision support has to be started after the game Window is already open, and might depend on windowed/windowed borderless mode to work correctly. If you're having issues with vision, try changing your game to windowed mode. On the first time you load Pantella into your save, even on a new game, load in, save, and reload the save you just made. If you don't for some reason NPCs won't be able to talk to you.
PantellaSL adds (basic) SexLab integration for Skyrim SE/VR/AE and operates as a demonstration of how to add support for another mod as an addon to Pantella.
Pantella-qwen_3_tts adds Qwen3TTS as a TTS option to Pantella in the form of an addon. Made to demonstrate how to add a TTS as an addon, and because of the License possibly being in conflict with the Pantella license? Wasn't sure, erred on the side of caution. Works pretty good, about on par with Chatterbox.
Pantella-indextts2 adds IndexTTS2 as a TTS option to Pantella in the form of an addon. Made to demonstrate how to add a TTS as an addon and because it's kinda mid for a number of reasons. The emotion of this TTS is very expressive, but the flow of english dialogue can be rather stilted, and the dependencies required are kinda hard to fit with the rest of Pantella in a way that they can all work together properly. (It requires a really new version of transformers that breaks a lot of the older TTSes.) I mainly separeted this one because of the dependency issues, but also because of the somewhat experimental nature of the TTS itself. It doesn't sound bad per se, and I would like to hear it in game before passing judgement fully, but the somewhat stilted english dialogue is a bit of a red flag for me and the built in qwen emotion support is a bit of a double edged sword. It's nice that it can sound more emotional, but the emotion doesn't always fit the context of the dialogue, and it can make some lines sound pretty distorted. Still, it's an interesting TTS and I'm curious to see how it performs in game. I'm thinking I could maybe tie it into the torchmoji emojis generated for chromadb(finally a reason to use those besides just thinking they're neat!) to try to help it pick more fitting emotions, but we'll see.
Pantella-depreciated_ttses adds a number of older TTSes that have been removed from the main Pantella repository for various reasons, but are still available to use as addons if you want to use them.
Pantella-player2 (By Gerik Uylerk) adds support for the Player2 inference API and STT API.
Something you might have noticed coming from summarizing_memory like Mantella's, is that ChromaDB doesn't store everything in plain text. It's actually kind of a pain in the ass to edit it. As such, I've made a basic memory editor for the ChromaDB memories and included it as a web UI with Pantella. When Pantella is open, and the setting is enabled(it is by default) go to http://localhost:8022 in your browser to access the memory editor. You can use this to edit memories, delete memories, and add new memories. It's a bit basic, but it should be enough to let you fix any major issues that come up.
Click to expand
Conversation ends as soon as spell is cast / [Errno 2] No such file or directory: 'path\to\Skyrim Special Edition/some_text_file.txt'
This is either an issue with the path set for skyrim_folder in config.json, an issue with your Skyrim folder being in Program Files, an issue with the installation of PapyrusUtil, or you are not running Skyrim via SKSE (please see the included readme.txt file in SKSE's downloaded folder for instructions on how to use it).
Some VR users miss that there is a separate VR version of PapyrusUtil, double check that you have downloaded this version of the mod if you are a VR user (it should be under the Miscallaneous Files section of the Nexus download page). To put it another way, if you have PapyrusUtil AE SE - Scripting Utility Function in your modlist, you have the wrong version.
If you are an SE user, please double check your Skyrim version by right-clicking its exe file in your Skyrim folder and going to Properties -> Details. The "File version" should be listed here. If it is 1.6 or above, you actually have Skyrim AE, not SE (its confusing I know), so please download the AE versions of the required mods. You can tell if PapyrusUtil is working by checking if you have a file called _pantella_skyrim_folder.txt in your skyrim_folder path.
If you have the required mods installed, then this issue might instead be caused by the skyrim_folder being set incorrectly. This only seems to be an issue for Mod Organizer 2 / Wabbajack modlist users. Some Mod Organizer 2 setups move the text files created by the Pantella spell to another folder. Try searching for a folder called overwrite/root or "Stock Game" in your Mod Organizer 2 / Wabbajack installation path to try to find these Pantella text files, specifically a file called _pantella_skyrim_folder.txt. If you find this file, then please set its folder as your skyrim_folder path.
Try using the button labeled Fix multiple NPC repeating lines bug in the MCM to fix this. Basically if your game or Pantella crashes, it will leave the Pantella Spell effect on the NPC and stop tracking them, and thus, won't remove the effect when you end the conversation. This button will remove the effect from all NPCs nearby(or in the game, not sure) and should fix the issue.
This is an issue with the way the spell mod itself has been installed. Please check your Skyrim version by right-clicking its exe file in your Skyrim folder and going to Properties -> Details. The "File version" should be listed here. If it is 1.6 or above, you have Skyrim AE. If it is below 1.6, you have Skyrim SE. If you are using VR, there are separate versions of the required mods for VR (PapyrusUtil tends to catch out a lot of VR users, the VR version of this file is under "Miscellaneous Files" on the download page). If you are running the mod via the GOG version of Skyrim, there are slight differences in setting up a mod manager as discussed in this tutorial.
If you're using a Skyrim version older than 1.6.1130, please use either one of these mods to add ESL support to your game: Backported Extended ESL Support or Skyrim VR ESL Support
Try creating a save and then reloading that save. This ensures that the Pantella voice files get registered correctly by the game engine.
If the above fails, a more unlikely reason for voicelines not playing is if you have updated the Pantella spell with a more recent version by replacing files in the mod's folder. If this is the case, open Skyrim, end all Pantella conversations and unequip the Pantella spell, and create a save. In your mod organizer, disable the Pantella spell plugin. Open your newly created save and create another save (now with no Pantella mod). Finally, in your mod organizer re-enable the Pantella spell plugin. This should effectively "reset" the mod. When you next open your recent save, you should see a notification that the Pantella spell has been added to your inventory.
If you're playing Skyrim, make sure Skyrim Script Extender (SKSE) is started before Skyrim itself. SKSE ReadME
If you'e playnig Fallout New Vegas, check that the requirements are installed. If they are, please reach out for support on the Discord.
This means there's an error with the language model. Please check your logging.log file for more information on what the error is. If you are still unsure, please share your logging.log file to the Discord and ask for help!
Make sure that your mic is picking up correctly on other software and that it is set as your default. For example, you can go to User Settings -> Voice & Video on Discord to test your mic. Otherwise, try adjusting the audio_threshold setting in config.json. If all else fails, make sure that no other microphones are plugged in except the one you want to use. There may be a rogue microphone such as a webcam picking up as your default!
This error means that Whisper is unable to find a connected microphone. Please ensure that you have a working microphone plugged in and enabled.
xVASynth related warning when started by Pantella. Thus far has not impacted Pantella so it can be safely ignored.
Click to expand
If an xVASynth voice model is corrupted, this error will display in logging.log. Please re-download the voice model in this case. You may alternatively need to redownload xVASynth.
A way to check for other corrupted voice models, is to compare the file sizes within /models/skyrim/ folder of xVASynth. If they diverge from the norms, redownload just those. The norms for voice model sizes are ~54 MB and/or ~90 MB (v2 voice models) & ~220 MB or ~260 MB (v3 voice models)
Loading voice model... xVASynth Error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
If this xVASynth Error occurs after the "Loading voice model..." message (as can be seen in your logging.log file), this is likely an issue with a corrupted voice model. Please try redownloading the model from here. If you have use_cleanup enabled, try setting this value to 0 in config.json.
If this does not resolve your issue, please share the text found in your xVASynth/server.log file on the Discord's #i-need-help forum channel for further support.
The source code for Pantella is included in this repo. Here are the quick steps to get set up:
Pantella allows you to talk to any NPC, but expects you to make sure they're set up(Or use a model that supports character generation!). If you start a Pantella conversation with an unknown/unsupported NPC (eg a modded NPC), Pantella will typically error and you will be asked to fill out the character information for this character.
Of course, if you are unhappy with Pantella's assumptions, you can add full support for modded NPCs to ./characters/ by adding a new JSON file containing the NPC's name (name), base_id, ref_id(the IDs are optional but recommended!) background description (bio), and voice model (voice_model). The rest of the column entries can be left blank. You can also add this as an addon pack released seperate from Pantella, and users can download it and place it in the ./addons/ folder to add support for the modded NPC.
Note that if the modded NPC is custom voiced there may not be a model available, and you will either need to assign the NPC a vanilla voice, or use a voice sampling TTS and provide a voice sample in ./data/voice_samples/.
For further support and examples of how other users have added modded NPCs, ask in the Discord.
- Clone the repo to your machine
- Create a virtual environment via
py -3.11 -m venv PantellaEnvin your console (Pantella requires Python 3.10.11, I've not had a chance to test it with 3.11 yet) - Start the environment in your console (
.\PantellaEnv\Scripts\Activate) - Install the required packages via
pip install -r requirements.txt - Set up your paths / any other required settings in the
config.jsonand your choseninterface_configsfile - Run Pantella via
main.pyin the parent directory
- Clone the repo to your machine
- Install the required packages via
pip install -r requirements.txt - Set up your paths / any other required settings in the
config.jsonand your choseninterface_configsfile - Run Pantella via
main.pyin the parent directory
- Clone the repo to your launcher's repositories folder under the name
Pathos14489_Pantella - Set up your paths / any other required settings in the
config.jsonand your choseninterface_configsfile - That should be it! You can now run the source code via the launcher using the same start button as the main Pantella repe. I do not advise using the built in update feature though, use
git pullinstead.
If you have any trouble in getting the repo set up, please reach out on Discord!
The source code/latest version for the Pantella Spell plugin can be found here. Updates made on one repo can be intertwined with the other, so it is best to ensure you have the latest versions of each. Typically the launcher will handle this, but if you're running the source code directly, you'll need to do this manually.
Pantella uses material from the "Skyrim: Characters" articles on the Elder Scrolls wiki, Fallout New Vegas: Characters and Fallout 3: Characters articles on the Fallout wiki at Fandom and is licensed under the Creative Commons Attribution-Share Alike License.
Huge thanks to the original developers of Mantella for the excellent foundation they provided. The original Mantella repo can be found here. And thanks to MrHaurrus for his work on xtts-api-server-mantella that the fork I included with Pantella is based on. The original repo can be found here. Massive thanks to the authors of the GECK Wiki for their very well written documentation for NVSE, JIP LN, ShowOff and JohnnyGuitar.