Aitrepreneur : 1-Click INSTALL Text-Generation-Webui Oobabooga

Hi there! I'm Kiba, a Japanese mangaka artist on the lookout for a way to bring my 3D animation dreams to life using the magic of AI. I've taken the plunge and installed the Text-Generation-WebUI, but I'm stuck at 90% of the installation. I'd love some help from a specialist to get me over the finish line. If you're up for it, I'd love to connect on Discord and chat more.

KIBA YAIBA

2025-04-11 23:02:38 +0000 UTC

my cmd tells me Miniconda not found but i have it installed can you help me with this?

Francesco Gariboldi

2025-02-28 13:39:11 +0000 UTC

I am new, so you might have to explain things Barny style for me. I click on the link, I start the program, and the command prompt disappears after getting to 99 now what? Thanks in advance for any assistance...

Charles Carter

2024-08-26 03:55:27 +0000 UTC

as the error says, you are trying to install it in a folder path with spaces in the name, just don't do that, delete all spaces in your folders and try again

Aitrepreneur

2024-08-05 12:34:33 +0000 UTC

what is wrong with the miniconda?

Peter Greene

2024-08-05 05:30:47 +0000 UTC

Select your installation type: 1. Standard Windows installation 2. WSL (Windows Subsystem for Linux) Type 1 or 2 and press Enter: 1 Cloning the oobabooga text-generation-webui repository... Cloning into 'text-generation-webui'... remote: Enumerating objects: 18858, done. remote: Counting objects: 100% (492/492), done. remote: Compressing objects: 100% (218/218), done. remote: Total 18858 (delta 349), reused 383 (delta 274), pack-reused 18366 Receiving objects: 100% (18858/18858), 27.83 MiB | 9.65 MiB/s, done. Resolving deltas: 100% (13357/13357), done. This script relies on Miniconda which can not be silently installed under a path with spaces. Press any key to continue . . .

Peter Greene

2024-08-05 05:30:22 +0000 UTC

OK

Pipirika

2024-07-11 20:21:38 +0000 UTC

need a bit more info, what are your PC specs? And are you aware that you are trying to use a CPU model, also try another model as well, this model does require a lot of power to use

Aitrepreneur

2024-07-11 19:41:25 +0000 UTC

21:08:28-513407 INFO Starting Text generation web UI Running on local URL: http://127.0.0.1:7860 21:08:40-082150 INFO Loading "yarn-mistral-7b-128k.Q4_K_M.gguf" 21:08:40-138433 INFO llama.cpp weights detected: "models\yarn-mistral-7b-128k.Q4_K_M.gguf" 21:08:40-165972 ERROR Failed to load the model. Traceback (most recent call last): File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\ui_model_menu.py", line 246, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\models.py", line 94, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\models.py", line 275, in llamacpp_loader model, tokenizer = LlamaCppModel.from_pretrained(model_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\llamacpp_model.py", line 39, in from_pretrained LlamaCache = llama_cpp_lib().LlamaCache ^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\llama_cpp_python_hijack.py", line 38, in llama_cpp_lib raise Exception(f"Cannot import 'llama_cpp_cuda' because '{imported_module}' is already imported. See issue #1575 in llama-cpp-python. Please restart the server before attempting to use a different version of llama-cpp-python.") Exception: Cannot import 'llama_cpp_cuda' because 'llama_cpp' is already imported. See issue #1575 in llama-cpp-python. Please restart the server before attempting to use a different version of llama-cpp-python.

Pipirika

2024-07-11 01:11:24 +0000 UTC

Traceback (most recent call last): File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\ui_model_menu.py", line 246, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\models.py", line 94, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\models.py", line 275, in llamacpp_loader model, tokenizer = LlamaCppModel.from_pretrained(model_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\llamacpp_model.py", line 39, in from_pretrained LlamaCache = llama_cpp_lib().LlamaCache ^^^^^^^^^^^^^^^ File "C:\Users\Caike.IA\Desktop\IA\text-generation-webui\modules\llama_cpp_python_hijack.py", line 38, in llama_cpp_lib raise Exception(f"Cannot import 'llama_cpp_cuda' because '{imported_module}' is already imported. See issue #1575 in llama-cpp-python. Please restart the server before attempting to use a different version of llama-cpp-python.") Exception: Cannot import 'llama_cpp_cuda' because 'llama_cpp' is already imported. See issue #1575 in llama-cpp-python. Please restart the server before attempting to use a different version of llama-cpp-python.

Pipirika

2024-07-11 01:10:41 +0000 UTC

Thanks! Here I thought it would cause less issues that way. I deleted everything and started over and it worked! Thanks!

NJDCFan

2024-06-11 23:55:45 +0000 UTC

because it looks like you are running the installer as admin, don't do that, run it normally

Aitrepreneur

2024-06-10 11:03:27 +0000 UTC

I get this error message below. I checked the folder and I see this file "miniconda_installer.exe" . Downloading Miniconda from https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Windows-x86_64.exe to C:\Windows\System32\text-generation-webui\installer_files\miniconda_installer.exe % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 53.8M 100 53.8M 0 0 44.2M 0 0:00:01 0:00:01 --:--:-- 44.3M The checksum verification for miniconda_installer.exe has passed successfully. Installing Miniconda to C:\Windows\System32\text-generation-webui\installer_files\conda Miniconda version: The system cannot find the path specified. Miniconda not found. Press any key to continue . . . Update installed miniconda from the folder myself and check add the paths. I re started the Start_windows.bat file again. same error!!!

NJDCFan

2024-06-10 01:41:44 +0000 UTC

Vram is king so take a gpu with as much vram as possible maybe a second hand 3090 would be best depending on where you live

Aitrepreneur

2024-06-02 21:08:10 +0000 UTC

So it means that it's time to buy a new one :) Can you help me decide which would be better for AI? I'm considering between the RTX 3060 (12GB) and the RTX 4060 (8GB). Also, maybe you know some other good cards in this price range.

xXRYBAXx

2024-06-02 17:09:32 +0000 UTC

yeah it's because you don't have enough vram to run that model. You can try this one: TheBloke/Xwin-MLewd-7B-V0.2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader altough with 4gb of vram it will still be very slow

Aitrepreneur

2024-05-27 16:44:31 +0000 UTC

GPU - GTX 1050Ti TheBloke/Yarn-Mistral-7B-128k-AWQ (that you recommended in video) but when I tried other models I also got this error CPU - i5-9400F RAM - 32gb (just in case)

xXRYBAXx

2024-05-27 11:10:29 +0000 UTC

what's your GPU and what model are you using?

Aitrepreneur

2024-05-25 15:20:30 +0000 UTC

Thanks, but I have another issue. After loading the model, I am getting this error. Traceback (most recent call last): File "C:\Users\Vic\Downloads\text-generation-webui\modules\ui_model_menu.py", line 249, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\modules\models.py", line 94, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\modules\models.py", line 297, in AutoAWQ_loader model = AutoAWQForCausalLM.from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\auto.py", line 94, in from_quantized return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 440, in from_quantized self.fuse_layers(model) File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\mistral.py", line 21, in fuse_layers fuser.fuse_transformer() File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\mistral.py", line 119, in fuse_transformer LlamaLikeBlock( File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\awq\modules\fused\block.py", line 88, in init self.norm_1 = norm_1.to(dev) ^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 1152, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 825, in _apply param_applied = fn(param) ^^^^^^^^^ File "C:\Users\Vic\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 1150, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NotImplementedError: Cannot copy out of meta tensor; no data!

xXRYBAXx

2024-05-23 23:42:57 +0000 UTC

I mean just delete the newly created text gen folder inside your download folder, put the bat installer file somewhere else on your computer then run it again

Aitrepreneur

2024-05-19 23:12:38 +0000 UTC

open a new command prompt window and type this pip cache purge press enter then redo the install again in a new folder

Aitrepreneur

2024-05-19 23:11:26 +0000 UTC

Hello good sir. I downloaded the .bat file and i opened it straight into the download folder? could you please tell me how can I undo this action so that I can open it in a new, empty file? Thank you and much love for your work.

Immproceo

2024-05-19 09:59:02 +0000 UTC

Hi Im getting this error during installation. What can I do to solve this? ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. torch==2.2.1 from https://download.pytorch.org/whl/cu121/torch-2.2.1%2Bcu121-cp311-cp311-win_amd64.whl#sha256=5dfab54d4e28797c8f4f1f9543a0a2b3d27113fa40eb202f8f49af1d865f9573: Expected sha256 5dfab54d4e28797c8f4f1f9543a0a2b3d27113fa40eb202f8f49af1d865f9573 Got af070169048302cbd260c81ad07d15977e7c674af543c02d745959b7f8d8601e

xXRYBAXx

2024-05-18 18:02:08 +0000 UTC

Thank you for the fast reply! I didn't even see that option, so thank you very much!

Anthony Marc

2024-04-25 11:49:08 +0000 UTC

yes, you need to go to the session tab, and check the "share" checkbox, then apply the changes, after a few seconds it will give you a local url as well as a public url

Aitrepreneur

2024-04-25 09:05:08 +0000 UTC

Is there a way to make a public URL with this? I've used a couple of your One Click tools and they usually do, but I haven't been able to figure out how to make this one do it.

Anthony Marc

2024-04-24 23:53:21 +0000 UTC

toggled off??? no it's the opposite, you need to check the box.....

Aitrepreneur

2024-04-18 20:16:13 +0000 UTC

Reinsalled Minicondoa this time with the "path enviroment variable" option toggled off and the installation got a little further this time "Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate C:\Users\(my username)\Documents\AI\Text\text-generation-webui\installer_files\env # # To deactivate an active environment, use # # $ conda deactivate" and afterwards same thing "Miniconda hook not found"

Wade Wilson

2024-04-18 18:05:02 +0000 UTC

yes that means that you did not added miniconda to path when installing, uninstall miniconda and reinstall it again this time by adding it to path: https://imgur.com/5GXXyo5

Aitrepreneur

2024-04-18 17:22:12 +0000 UTC

Hey I'm trying to run the insatller and it does everything but installing miniconda so I insatlled seperately and when I run the "start_windows.bat" it says miniconda path not found. Do you have any idea how to solve it?

Wade Wilson

2024-04-18 14:34:40 +0000 UTC

You don't have to do anything, just use the installer, have you watched my install video? https://youtu.be/C-7jGYOGvy4

Aitrepreneur

2024-04-15 19:30:34 +0000 UTC

Hi I was able to download Git bash and also installer now there is a new folder name "text-generation-webui" whats the next step?

Osmar Irvin Cabantug

2024-04-15 14:07:19 +0000 UTC

Thank you that fixed it👍

Fizzy

2024-04-14 11:25:52 +0000 UTC

try this model instead: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader

Aitrepreneur

2024-04-12 19:54:37 +0000 UTC

Hey I am using a nvidia 3060 This is the model link https://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-AWQ

Fizzy

2024-04-12 19:48:08 +0000 UTC

Will they be adding AMD support to windows eventually?

Cal

2024-04-12 05:14:14 +0000 UTC

what's your gpu and what model are you using? (link to the model)

Aitrepreneur

2024-04-11 20:37:16 +0000 UTC

hmm...that's strange indeed... need a bit more info, could you send a few screenshots? Also was the model downloaded correctly?

Aitrepreneur

2024-04-11 20:36:45 +0000 UTC

Traceback (most recent call last): File "E:\Kacper2\Bored\ai\text\text-generation-webui\modules\callbacks.py", line 61, in gentask ret = self.mfunc(callback=_callback, args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Kacper2\Bored\ai\text\text-generation-webui\modules\text_generation.py", line 373, in generate_with_callback shared.model.generate(kwargs) File "E:\Kacper2\Bored\ai\text\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 110, in generate return self.model.generate(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Kacper2\Bored\ai\text\text-generation-webui\installer_files\env\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "E:\Kacper2\Bored\ai\text\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 1575, in generate result = self._sample( ^^^^^^^^^^^^^ File "E:\Kacper2\Bored\ai\text\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 2694, in _sample model_inputs = self.prepare_inputs_for_generation(input_ids, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Kacper2\Bored\ai\text\text-generation-webui\installer_files\env\Lib\site-packages\transformers\models\llama\modeling_llama.py", line 1250, in prepare_inputs_for_generation past_key_values = getattr(getattr(self.model.layers[0], "self_attn", {}), "past_key_value", None) Hi I am getting this error message whenever I am trying to talk to any character. I have tried the default assistant

Fizzy

2024-04-11 17:09:00 +0000 UTC

I am not sure but it looks like there is a version issue with my Mac being on a OS Version that there is no Pytorch version without using a docker container

Jason Harring

2024-04-11 16:17:25 +0000 UTC

Everytime I try to load a model I get a connection error warning I've running a RTX3060 with 12gb ram this is the error I get Error Connection timed out in the command prompt I just have 14:23:39-464757 INFO Loading "TheBloke_LLaMA2-13B-Psyfighter2-GPTQ" Press any key to continue . . . Then it just quits Any ideas

Terry Martin

2024-04-11 13:31:01 +0000 UTC

what's your gpu?

Aitrepreneur

2024-04-10 18:37:55 +0000 UTC

I tried to load the model but it appears me an error that mentions that I don't have enough GPU. Do I have to download a smaller model??? Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:36<00:00, 18.42s/it] 09:39:49-579171 ERROR Failed to load the model. Traceback (most recent call last): File "C:\AI\WebUI\text-generation-webui\modules\ui_model_menu.py", line 245, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\modules\models.py", line 168, in huggingface_loader model = model.cuda() ^^^^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\transformers\modeling_utils.py", line 2548, in cuda return super().cuda(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 911, in cuda return self._apply(lambda t: t.cuda(device)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 802, in _apply module._apply(fn) File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 802, in _apply module._apply(fn) File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 802, in _apply module._apply(fn) [Previous line repeated 2 more times] File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 825, in _apply param_applied = fn(param) ^^^^^^^^^ File "C:\AI\WebUI\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 911, in return self._apply(lambda t: t.cuda(device)) ^^^^^^^^^^^^^^ torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 112.00 MiB. GPU 0 has a total capacity of 6.00 GiB of which 0 bytes is free. Of the allocated memory 12.15 GiB is allocated by PyTorch, and 233.55 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

林 Xk。

2024-04-10 07:46:11 +0000 UTC

I’m going to try this model when I get home, but just wanted to let you know the other model is working well now. I was able to get the gallery up and started a convo with the built in example.

Bryan Sproles

2024-04-09 22:02:16 +0000 UTC

maybe this one: https://huggingface.co/LoneStriker/Yi-34B-200K-RPMerge-GPTQ

Aitrepreneur

2024-04-09 21:08:36 +0000 UTC

Do you think you could do an update video on this? Apparently a lot has changed since you made this one. At the time, you said AWQ was the newer format, so that's what I went with. Any suggestions for anywhere from a 20B - 33B model? I'd like to have as much choice as possible, and my computer should be able to handle it. I want to get the Oobabooga working, so I can then check your video on setting up the whole Sillytavern local chat. Thanks for everything so far! :)

Bryan Sproles

2024-04-09 18:02:23 +0000 UTC

first do not use AWQ format models anymore, it's too buggy, use GTPQ format models instead, preferably the exllama compatible version. For example try this: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader. For the "characters tab", apparently now it's not activated by default, so you need to go to the session tab and check the "gallery" extension, then apply the changes, the character gallery will then appear below the chat

Aitrepreneur

2024-04-09 16:52:11 +0000 UTC

I went back to the basics, and the Yarn Mistral 7B one seems to be working. I then redownloaded the The Bloke's "Everyone Coder 33B (https://huggingface.co/TheBloke/Everyone-Coder-33B-Base-AWQ) It loaded successfully, I asked one question, and got the following in the console (I'm including the loading bits so you can see where the model started.) On screen, it just showed "AI", and nothing else. By the way, I still can't see the Character tab. I have no idea how to get it to show. I thought it was automatic. Does it depend on the model you load? *** 21:30:02-187554 INFO Loading "TheBloke_Everyone-Coder-33B-Base-AWQ" Replacing layers...: 100%|███████████████████████████████████████████████████████████| 62/62 [00:05<00:00, 11.14it/s] Fusing layers...: 100%|██████████████████████████████████████████████████████████████| 62/62 [00:04<00:00, 14.06it/s] 21:30:20-349134 INFO LOADER: "AutoAWQ" 21:30:20-350127 INFO TRUNCATION LENGTH: 16384 21:30:20-351125 INFO INSTRUCTION TEMPLATE: "Custom (obtained from model metadata)" 21:30:20-351125 INFO Loaded the model in 18.16 seconds. Traceback (most recent call last): File "C:\AI\Oobabooga\text-generation-webui\modules\callbacks.py", line 61, in gentask ret = self.mfunc(callback=_callback, args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\modules\text_generation.py", line 373, in generate_with_callback shared.model.generate(kwargs) File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 110, in generate return self.model.generate(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 1575, in generate result = self._sample( ^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 2694, in _sample model_inputs = self.prepare_inputs_for_generation(input_ids, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\models\llama\modeling_llama.py", line 1250, in prepare_inputs_for_generation past_key_values = getattr(getattr(self.model.layers[0], "self_attn", {}), "past_key_value", None) ^^^^^^^^^^^^^^^^^ File "C:\AI\Oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 1688, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'LlamaLikeModel' object has no attribute 'layers' Output generated in 0.31 seconds (0.00 tokens/s, 0 tokens, context 77, seed 78199476) *** Thanks for your assistance!

Bryan Sproles

2024-04-09 02:35:44 +0000 UTC

what error do you have in the command window? Also what models have you used? (give me the url)

Aitrepreneur

2024-04-08 19:47:22 +0000 UTC

Hi, have you correctly ran the mac installer? If yes then run the cmd_macos file and then in the command window type: pip install pyyaml then press enter unfortunately since I don't personally use any Mac I can't really be of any further assistance for troubleshooting

Aitrepreneur

2024-04-08 19:46:38 +0000 UTC

This is my 3rd attempt to get the 1 Click to work - I initially tried prior to joining the Patreon with the manual method, and now twice on the 1 Click. I have a brand new Alienware - i9, NVIDIA 4090 and 64gb RAM. I have Stable Diffusion XL (Forge UI) running in its own folder, and able to create good images, so the specs are fine. When I run the 1Click, it seems to work...no noticeable errors, and I can click the final URL, and have a running TextUI. I've now attempted to load a 7B, 13B and 33B model, none of which are working. One of two things happens after I type something to the AI. Either it will come back with "is typing..." and they never actually post anything, or it won't even post the "is typing..." part. I'm also not seeing the Character Gallery tab at the bottom of the page. I know I've seen it when I first attempted to install like a week ago, but I reinstalled the OS in the meantime. Any assistance would be greatly appreciated!

Bryan Sproles

2024-04-08 13:19:55 +0000 UTC

Traceback (most recent call last): File "/Users/jason/Desktop/AI/ooba/Booga/server.py", line 4, in from modules import shared File "/Users/jason/Desktop/AI/ooba/Booga/modules/shared.py", line 8, in import yaml ModuleNotFoundError: No module named 'yaml' jason@jason booga % I get this error when I am trying to install on MacBook Pro (pre apple silicon) I am going to switch to a different apple silicon mac and see if it runs better

Jason Harring

2024-04-07 19:35:31 +0000 UTC

try this: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader

Aitrepreneur

2024-04-01 17:40:59 +0000 UTC

My apologies I thought I had put the graphics card in. I'm running an RTX 4080. The models I have tried are as follows. KoboldAI/LLaMA2-13B-Tiefighter Sao10K/Fimbulvetr-11B-v2-GGUF TheBloke/MythoMax-L2-13B-GPTQ TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GPTQ No matter which model I try I get one of those two errors I previously mentioned. I've seen some things that say my graphics card could work with something higher than a 13B model but given the current errors I'm getting I didn't want to chance it.

NoxRaven

2024-03-31 18:12:44 +0000 UTC

First and foremost try to not install it in a onedrive folder i know it often causes issues when installing ai tools, just install it in a normal folder on your PC, also still don't know what gpu you got or what model you uses...

Aitrepreneur

2024-03-31 14:56:28 +0000 UTC

It doesn't seem to matter which model I use. I get one of two errors. Either this one: C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\models\llama\modeling_llama.py:670: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.) attn_output = torch.nn.functional.scaled_dot_product_attention( or this one: Traceback (most recent call last): File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\modules\callbacks.py", line 61, in gentask ret = self.mfunc(callback=_callback, args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\modules\text_generation.py", line 389, in generate_with_callback shared.model.generate(kwargs) File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 110, in generate return self.model.generate(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 1575, in generate result = self._sample( ^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 2694, in _sample model_inputs = self.prepare_inputs_for_generation(input_ids, model_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\models\llama\modeling_llama.py", line 1250, in prepare_inputs_for_generation past_key_values = getattr(getattr(self.model.layers[0], "self_attn", {}), "past_key_value", None) ^^^^^^^^^^^^^^^^^ File "C:\Users\nolim\OneDrive\Documents\oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\torch\nn\modules\module.py", line 1688, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'LlamaLikeModel' object has no attribute 'layers'

NoxRaven

2024-03-30 17:39:11 +0000 UTC

I need the full error log, also what's your GPU and what model are you using (link)?

Aitrepreneur

2024-03-30 09:44:40 +0000 UTC

I used the installer and everything seemed to run ok. I can load up a model. Yet when I try to start using the program I get the following error. oobabooga\text-generation-webui\installer_files\env\Lib\site-packages\transformers\models\llama\modeling_llama.py:670: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.) attn_output = torch.nn.functional.scaled_dot_product_attention(

NoxRaven

2024-03-30 06:16:53 +0000 UTC

Yes. Once again once my sillytavern video i show everything there

Aitrepreneur

2024-03-29 19:19:56 +0000 UTC

Is there a way to you speak recognition locally ? Silly tavern request API keys as well

Relven

2024-03-29 17:58:21 +0000 UTC

this error means that you have tried to activate the silero extension in the session tab, disable it. 11labs was removed from the text gen webui, for this you need to use sillytavern

Aitrepreneur

2024-03-29 17:09:25 +0000 UTC

01:40:52-428876 INFO Loading the extension "gallery" 01:40:52-436265 INFO Loading the extension "silero_tts" Silero TTS cache not found at C:\Users\MPG/.cache\torch\hub. Attempting to download... Using cache found in C:\Users\MPG/.cache\torch\hub\snakers4_silero-models_master 01:41:04-376247 ERROR Failed to load the extension "silero_tts". Traceback (most recent call last): File "G:\Text2Audio\text-generation-webui\modules\extensions.py", line 46, in load_extensions extension.setup() File "G:\Text2Audio\text-generation-webui\extensions\silero_tts\script.py", line 148, in setup model = load_model() ^^^^^^^^^^^^ File "G:\Text2Audio\text-generation-webui\extensions\silero_tts\script.py", line 61, in load_model model, example_text = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_tts', language=languages[params['language']]["lang_id"], speaker=params['model_id']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Text2Audio\text-generation-webui\installer_files\env\Lib\site-packages\torch\hub.py", line 566, in load model = _load_local(repo_or_dir, model, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Text2Audio\text-generation-webui\installer_files\env\Lib\site-packages\torch\hub.py", line 595, in _load_local model = entry(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MPG/.cache\torch\hub\snakers4_silero-models_master\src\silero\silero.py", line 51, in silero_tts from omegaconf import OmegaConf ModuleNotFoundError: No module named 'omegaconf' Running on local URL: http://127.0.0.1:7860 Help please!

Relven

2024-03-28 22:41:48 +0000 UTC

yes you can't at least for now, use the cpu version instead

Aitrepreneur

2024-03-16 17:53:31 +0000 UTC

How am I to select AMD GPU if it is only for Linux/Mac Os only?

JD Turner

2024-03-16 15:01:07 +0000 UTC

make sure you don't have any antivirus or firewall blocking the connection, disable them both before running the file, that's usually the issue for most people when something like this happens

Aitrepreneur

2024-03-15 11:05:40 +0000 UTC

when I go to run the file it boots up to ask me for 1 or 2, I press 1 and enter then in a flash it closes. It said something once but as I was reading it the window closed.

GtheArchitect

2024-03-15 10:00:35 +0000 UTC

please don't spam multiple messages, just send me a dm with all the info, pc specs, precise error, model used, loader used, etc

Aitrepreneur

2024-03-09 20:00:38 +0000 UTC

Hi there. I Have installed this successfully and have tried many different iterations of the model "gguf" that you requested us to DL,

Dungaree man

2024-03-09 15:56:16 +0000 UTC

Have you watched my installation video? If you are a complete beginner I suggest you first start there

Aitrepreneur

2024-03-08 19:09:31 +0000 UTC

I don't know squat about computers. I'm a musician. i think i did this right. i got ta place whhere the screen introduces Itself and says. something to the effect of what can I do for you or whatever. What am I supposed to do now? Like, am I in or what?

William OBrien

2024-03-08 04:44:57 +0000 UTC

with your gpu you need to download the gptq format model instead of awq

Aitrepreneur

2024-02-28 22:34:44 +0000 UTC

hey i have see your video and must try it. Im from germany so sorry when my english not perfectly is.My setup is nvd geforce 1050and i try to used this model TheBloke/DiscoLM_German_7b_v1-AWQ.But i have never do this stuff bevor so i hope you can help me. I followd your instruktions to the point by add the model in the text generation iu so all fin but when i will load the model i become this error; 16:49:59-797974 ERROR Failed to load the model. Traceback (most recent call last): File "C:\aichat\text-generation-webui\modules\ui_model_menu.py", line 242, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\aichat\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\aichat\text-generation-webui\modules\models.py", line 302, in AutoAWQ_loader model = AutoAWQForCausalLM.from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\aichat\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\auto.py", line 56, in from_quantized return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\aichat\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 208, in from_quantized load_checkpoint_and_dispatch( File "C:\aichat\text-generation-webui\installer_files\env\Lib\site-packages\accelerate\big_modeling.py", line 579, in load_checkpoint_and_dispatch load_checkpoint_in_model( File "C:\aichat\text-generation-webui\installer_files\env\Lib\site-packages\accelerate\utils\modeling.py", line 1504, in load_checkpoint_in_model raise ValueError( ValueError: At least one of the model submodule will be offloaded to disk, please pass along an `offload_folder`.

Hendrik Selgrad

2024-02-28 16:00:26 +0000 UTC

what gpu you got and what model are you using?

Aitrepreneur

2024-02-23 16:18:10 +0000 UTC

everything is installed and a model is chosen but when I type to it the responses I get are all blank

MemeL0rd420

2024-02-22 20:22:03 +0000 UTC

same issue here

MemeL0rd420

2024-02-22 20:21:14 +0000 UTC

make sure that miniconda is added to path: https://eduand-alvarez.medium.com/setting-up-anaconda-on-your-windows-pc-6e39800c1afb

Aitrepreneur

2024-02-20 23:03:28 +0000 UTC

I am getting the Miniconda hook not found from the update_windowns.bat. where should miniconda install? mine went to C:\ProgramData\miniconda3 and C:\Windows\System32\text-generation-webui\installer_files\conda

Jeff Osiel

2024-02-20 18:47:09 +0000 UTC

Two issues...first is simple. What file Launches it once it's installed? It only gives me the url after I install fresh. Second...I've imported Sandra and Tiefighter(?) llm, loaded them both in Oog and I get no responses. I get "typing" but results are blank.

Roy Greenhalgh

2024-02-17 20:35:35 +0000 UTC

hmm...strange, you might have a previous bad install of miniconda, since the installer is supposed to install it automatically. Try uninstalling miniconda if there is an install already and then either relaunch the installer in a new folder or install miniconda manually (https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe) make sure to check the box add miniconda to the system path environment.

Aitrepreneur

2024-02-16 01:18:22 +0000 UTC

try opening a command prompt window and drag and drop the update_windows.bat file inside then press enter, this will at least give you an error and avoid the automatic crash. Make also sure your antivirus and firewall are disabled just in case

Aitrepreneur

2024-02-16 01:13:00 +0000 UTC

It installes everything but than it says : Conda is not installed. exiting / everytime is run Windows_start.bat

Alucatt

2024-02-16 00:20:07 +0000 UTC

Yo, when i open update_windows.bat it says press any key so i can start downloading it right, but it just crashes

Sandro

2024-02-15 19:50:53 +0000 UTC

for your GPU I recommend psyfighter 2. TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste that line in the model field, dl it and use the exllama loader

Aitrepreneur

2024-02-14 23:30:57 +0000 UTC

my biggest issue with this, is it's impossible to shop around for models in huggingface. No examples to view so I don't know what anything is. Is there a defacto-best option for chat model? 3080Ti and AMD 5700 fwiw

Casey Fowler

2024-02-14 13:34:14 +0000 UTC

Hey have you fixed this? I met the same problem.

wanyu jerry

2024-02-14 10:23:12 +0000 UTC

This was the fix, thank you

Fake Name

2024-02-13 07:36:57 +0000 UTC

you can add this line in the cmd_flags.txt --api-port 5001

Aitrepreneur

2024-02-12 23:34:58 +0000 UTC

I reboot and get the same error. Is there a way to change the 5000 ?

Fake Name

2024-02-12 23:12:50 +0000 UTC

this error usually means you have multiple python instances running at the same time, you need to close the python processes or reboot the pc

Aitrepreneur

2024-02-12 23:09:20 +0000 UTC

Hi, thanks for the 1 click installers. I can't get the openAI extension working so that SillyTavern can work. I get this : To create a public link, set `share=True` in `launch()`. Closing server running on port: 7860 16:17:47-967575 INFO Loading the extension "gallery" 16:17:47-969081 INFO Loading the extension "openai" 16:17:48-041857 INFO OpenAI-compatible API URL: http://127.0.0.1:5000 [Errno 10048] error while attempting to bind on address ('127.0.0.1', 5000): only one usage of each socket address (protocol/network address/port) is normally permitted Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`.

Fake Name

2024-02-12 21:18:51 +0000 UTC

apparently it is so old huggy face 404 error on it

AI AI

2024-02-08 16:44:32 +0000 UTC

Definitely try the GPTQ format instead, too many people have issues with awq, try this: TheBloke/Everyone-Coder-33B-Base-GPTQ:gptq-4bit-128g-actorder_True copy and paste this in the model field, dl it and use the exllama loader to load it

Aitrepreneur

2024-02-08 15:26:32 +0000 UTC

sorry, its a NVIDIA GeForce RTX 4090 with 24gig

patreon@winberry.com

2024-02-07 23:08:56 +0000 UTC

hi first what's your GPU?

Aitrepreneur

2024-02-07 22:28:36 +0000 UTC

Hello - what does it mean when I use a larger model (theBlock_Everyone-Coder-33B-AWQ) and all I get for answer is "==================================== etc"

patreon@winberry.com

2024-02-07 21:59:27 +0000 UTC

Have fun ;)

Aitrepreneur

2024-02-04 18:20:18 +0000 UTC

OK then...your a genius....silly tavern is up and running:)

Cr Fultz

2024-02-04 18:17:04 +0000 UTC

well yeah the model you are trying to load is currently the most powerful model there is right now, you need like 80Gb+ of Vram to run it :D Try another model instead, like this one, copy and paste this in the model field: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True dl it and load it with the exllama loader

Aitrepreneur

2024-02-04 15:49:21 +0000 UTC

Ok, well I thought I was following your instructions, but, something has gone haywire somewhere.........I have a 4070ti with 16GB so Im thinking i might be ok with gpu mem but it will not load the model. is there anything you can see that i might need to do to correct whatever step i might be doing wrong? Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. 06:43:00-713643 INFO Loading mistralai_Mixtral-8x7B-Instruct-v0.1 Replacing layers...: 100%|█████████████████████████████████████████████████████████████| 32/32 [00:04<00:00, 6.85it/s] 06:43:10-261499 ERROR Failed to load the model. Traceback (most recent call last): File "C:\Users\Chuck\Desktop\textai\text-generation-webui\modules\ui_model_menu.py", line 213, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Chuck\Desktop\textai\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Chuck\Desktop\textai\text-generation-webui\modules\models.py", line 311, in AutoAWQ_loader model = AutoAWQForCausalLM.from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Chuck\Desktop\textai\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\auto.py", line 56, in from_quantized return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Chuck\Desktop\textai\text-generation-webui\installer_files\env\Lib\site-packages\awq\models\base.py", line 208, in from_quantized load_checkpoint_and_dispatch( File "C:\Users\Chuck\Desktop\textai\text-generation-webui\installer_files\env\Lib\site-packages\accelerate\big_modeling.py", line 545, in load_checkpoint_and_dispatch load_checkpoint_in_model( File "C:\Users\Chuck\Desktop\textai\text-generation-webui\installer_files\env\Lib\site-packages\accelerate\utils\modeling.py", line 1494, in load_checkpoint_in_model set_module_tensor_to_device( File "C:\Users\Chuck\Desktop\textai\text-generation-webui\installer_files\env\Lib\site-packages\accelerate\utils\modeling.py", line 276, in set_module_tensor_to_device raise ValueError(f"{module} does not have a parameter or a buffer named {tensor_name}.") ValueError: WQLinear_GEMM(in_features=4096, out_features=14336, bias=False, w_bit=4, group_size=128) does not have a parameter or a buffer named weight.

Cr Fultz

2024-02-04 11:47:56 +0000 UTC

probably because you had a previous bad install of miniconda that was not added to path, uninstall miniconda, then relaunch the installer in anoter folder

Aitrepreneur

2024-02-04 03:01:51 +0000 UTC

maybe its just me but when i tried to install it i get a error of miniconda not found and when i manually install it, and run update or start windows it still says miniconda not found.

james

2024-02-04 00:55:47 +0000 UTC

Lol I restarted it and it worked for some reason lol. Started playing around with it a little and its actually mindblowing how it all works so well. Thanks for the help anyways. and the installer, really helpful

Justin

2024-02-04 00:22:59 +0000 UTC

try putting this in the model dl field: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True dl it and then use the exllama loader

Aitrepreneur

2024-02-03 22:41:51 +0000 UTC

gptq

Justin

2024-02-03 22:33:02 +0000 UTC

nvidia 4070super, and i tried downloading llama13B psyfighters 2

Justin

2024-02-03 22:28:22 +0000 UTC

what gpu do you have and what model did you try downloading?

Aitrepreneur

2024-02-03 22:20:17 +0000 UTC

Hey man, I was able to install it and run it just fine but when I tried downloading a model, it says "string index out of range". It also mentions line 242 on the ui_model_menu.py file and line 47 in the download-model.py file. Help? I downloaded silly tavern right before this, idk if that matters.

Justin

2024-02-03 22:14:00 +0000 UTC

yes try another model and format because of your GPU. Try this instead: put this inside the model field: TheBloke/Xwin-MLewd-7B-V0.2-GPTQ:gptq-4bit-32g-actorder_True dl it and load it with exllama loader

Aitrepreneur

2024-01-29 20:30:12 +0000 UTC

it's for the gpu, as I said put the whole line in the field to dl the specific model version

Aitrepreneur

2024-01-29 20:28:44 +0000 UTC

Question - I was able to successfully install using your installer, however when attempting to download a model via huggingface using the model tab I end up with traceback errors. However I am still able to choose the model within the model tab. When trying to get a response from the chat, the character types and responds with a blank reply. I do not see any errors. I am attempting to use TheBloke_Yarn_Mistral-7B-128k-AWQ model on a 1080ti Nvidia card. I was curious if you've seen this before?

Riccardo Mui

2024-01-29 19:02:03 +0000 UTC

I get trace backs whenever I try to load it, not sure why Update; Figured it out, it was cause I ended up needing to put the second string you have there in the first model field, didn't realize it's only for the cpu packs

Extro

2024-01-29 11:01:12 +0000 UTC

I think I sorted it, I loaded the model and then created a new chat and it worked, I think before I didn't create a new chat after loading, I assumed loading a model created a new chat with the new model but it doesn't appear to be the case

Keiron Gulrajani

2024-01-29 09:24:26 +0000 UTC

I like this one: https://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GPTQ put this inside the model field: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True dl it and load it with exllama loader

Aitrepreneur

2024-01-29 06:42:46 +0000 UTC

OH, well then, any recommendations for an uncensored model? I have a 3090 as well

Extro

2024-01-29 05:08:37 +0000 UTC

lol well that's because it's the most powerful local llm right now and it requires something like 80Gb+ of Vram to run😅 So basically no one with a consumer grade gpu can run this model, it's made to be run on multiple professional grade GPU. You need to use a normal model :D

Aitrepreneur

2024-01-29 05:01:41 +0000 UTC

I'm also having a issue, tried running mixtral -8x7B-Instruct-v0.1-AWQ, I dont get any response in silly tavern.

Extro

2024-01-29 04:30:42 +0000 UTC

Send me a few screenshots of every window

Aitrepreneur

2024-01-29 02:06:58 +0000 UTC

CMD is saying ValueError: No tokenizer is loaded. URL given by CMD: http://127.0.0.1:5000

St Lask

2024-01-29 00:52:04 +0000 UTC

I don't know if that one is uncensored I haven't tried it, try this one instead though: https://huggingface.co/TheBloke/Xwin-MLewd-7B-V0.2-GPTQ put that in the model field: TheBloke/Xwin-MLewd-7B-V0.2-GPTQ:gptq-4bit-32g-actorder_True dl it and use the exllama loader

Aitrepreneur

2024-01-28 23:18:15 +0000 UTC

TheBloke/dolphin-2.6-mistral-7B-dpo-laser-AWQ

Keiron Gulrajani

2024-01-28 23:04:27 +0000 UTC

What url did you use?

Aitrepreneur

2024-01-28 23:00:45 +0000 UTC

what model did you download?

Aitrepreneur

2024-01-28 23:00:29 +0000 UTC

RTX 2040, I managed to get it to work but now im having issues connecting to ST. The given URL the CMD gives doesnt connect.

St Lask

2024-01-28 22:00:34 +0000 UTC

I downloaded an uncensored model but the text generation still gives me messages about it not being able to answer specific questions even when I choose the uncensored model; is there a setting somewhere that needs to be set for it to be truly uncensored?

Keiron Gulrajani

2024-01-28 19:44:26 +0000 UTC

what's your GPU? Have you tried the gptq version instead? What did you put inside the model field?

Aitrepreneur

2024-01-28 15:29:04 +0000 UTC

Heya im getting IndexError: string index out of range trying to use, it wouldnt load the list and displayed that. Tried to load the list for Yarn-Mistral-7B-128k-GGUF as i got the CUDA memory error from the awq and got the error

St Lask

2024-01-28 04:49:23 +0000 UTC

try using "gtpq" models instead, awq models might be incompatible with your GPU

Aitrepreneur

2024-01-24 23:18:45 +0000 UTC

Hello, on my Linux build with a RX 7900 XTX GPU with 24gb VRAM, and ROCM 5.6, i'm still failing to load the model shown in the video and I get these errors, any help?: shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ from awq import AutoAWQForCausalLM ModuleNotFoundError: No module named 'awq'

Mostafa Amir

2024-01-24 01:25:53 +0000 UTC

I think it might just be my laptop. I'm using the one suggested in the video ( I believe it was Yarn-minstrel GGUC 4k M

Lawrence Wong

2024-01-21 19:04:58 +0000 UTC

hi which model did you dl?

Aitrepreneur

2024-01-19 22:04:34 +0000 UTC

hi which model did you use?

Aitrepreneur

2024-01-19 22:04:13 +0000 UTC

Hi Hows it going? Been following the silly tavern guide which lead me to dwonload the ooga booga webui from a previous model. It works well up until I try to load my model, which just gives me [Connection : Error]

Lawrence Wong

2024-01-18 17:49:50 +0000 UTC

Hello. After I use the installer, and then download and load the model, I get the following error when I try to chat with the AI: "RuntimeError: CUDA error: no kernel image is available for execution on the device". I installed for CUDA version 12.1, but the CUDA version on my system is 12.2. What can I do? To provide more info, my system has 2 x GTX 1070TI.

Bradford Klipan

2024-01-18 04:27:39 +0000 UTC

hi dm me your pc specs and full error log

Aitrepreneur

2024-01-18 00:20:23 +0000 UTC

The installer didn't work. I asked for help but that comment is now gone. ERROR: Wheel 'hqq' located at C:\users\j1\appdata\local\pip\cache\wheels\5b\37\da\86e3082cb5c41972d221b79a79259433f68c29285bf35a7448\hqq-0.1.2-py3-none-any.whl is invalid. Command '"C:\AI\OOBABOOGA\text-generation-webui\installer_files\conda\condabin\conda.bat" activate "C:\AI\OOBABOOGA\text-generation-webui\installer_files\env" >nul && python -m pip install -r temp_requirements.txt --upgrade' failed with exit status code '1'.

BladeMech22

2024-01-17 03:07:48 +0000 UTC

it was blank if i remember correctly.. ill try again in a bit and let you know exactly what it said, though.

Glitch Gremlin

2024-01-16 09:55:04 +0000 UTC

Yes that's it, what message do you have in the window prompt?

Aitrepreneur

2024-01-16 00:08:30 +0000 UTC

i installed it from the Installer here, but i can't get the program to come on or anything... am i missing something? i thought i simply installed it from the file here and that'd be it?

Glitch Gremlin

2024-01-15 13:47:14 +0000 UTC

I sent you a dm but having some issues installing this with the bat file. ValueError: When localhost is not accessible, a shareable link must be created. Please set share=True or check your proxy settings to allow access to localhost. Press any key to continue . . .

Ma No

2024-01-13 17:41:02 +0000 UTC

hmm you might have an antivirus that blocked the file during download, try installing miniconda manually: https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe and don't forget to check the box, "add miniconda to path"

Aitrepreneur

2024-01-11 00:55:09 +0000 UTC

Hey! So, I ran into an issue. After running the bat. installer (I'm using Windows) the prompt says: Installing Miniconda to C:\Windows\System32\text-generation-webui\installer_files\conda Miniconda version: The system cannot find the path specified. Miniconda not found. Press any key to continue . . . The thing is, I checked the path and found the miniconda_installer.exe file, but it won't open. It says "error launching installer."

Gage

2024-01-10 03:34:09 +0000 UTC

unfortunately yeah but gguf models (cpu models) are still pretty good and fast

Aitrepreneur

2024-01-09 18:19:54 +0000 UTC

So if i have AMD on windows i shouldn't select the amd option and just go with cpu?

Scruffy Scruffington

2024-01-09 15:46:22 +0000 UTC

dm me your pc specs and the model you used

Aitrepreneur

2024-01-09 10:08:48 +0000 UTC

So, I've installed Text-Generation-Webui Oobabooga with the installer, adjusted the settings as per the walkthrough video with selecting the presets, etc. and I have downloaded 2 separate models as well as the Sandra character. When I try loading any of the 2 models in the webui, I get the following error: "Traceback (most recent call last): File "C:\AI\text-generation-webui\modules\ui_model_menu.py", line 213, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\text-generation-webui\modules\models.py", line 247, in llamacpp_loader model_file = list(Path(f'{shared.args.model_dir}/{model_name}').glob('*.gguf'))[0] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range" Any help would be greatly appreciated.

Jacob Dow

2024-01-08 02:53:05 +0000 UTC

ty

Aaron1611

2024-01-07 12:21:47 +0000 UTC

unfortunately oobabooga decided to remove completely the elevenlabs extension from the webui since now we have the free xttsv2 option. If you want to use it, either revert back to an older version of the text gen or download and install sillytavern (just follow my latest video and patreon posts)

Aitrepreneur

2024-01-07 02:00:35 +0000 UTC

tried adding voice but elevenlabs_tts was not in the extension list.

Aaron1611

2024-01-07 01:42:30 +0000 UTC

I worked! I can't believe it, thank you very much, I love your work, I send a hug to you here in Brazil!!

Kennedy Vieira Lopes

2024-01-05 02:10:29 +0000 UTC

you can try this one: TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True copy and paste that line in the model field, dl it and use the exllama loader

Aitrepreneur

2024-01-05 00:33:16 +0000 UTC

Tell me a model please, I can't use any

Kennedy Vieira Lopes

2024-01-04 23:07:14 +0000 UTC

TheBloke_Yarn-Mistral-7B-128k-AWQ, My video card is a GTX 1660 with 6GB of VRAM

Kennedy Vieira Lopes

2024-01-04 22:37:48 +0000 UTC

this error means you don't have enough Vram to run the operation/model, what model were you trying to run?

Aitrepreneur

2024-01-04 22:30:49 +0000 UTC

This is what appears to me when loading the model: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 128.00 MiB. GPU 0 has a total capacty of 6.00 GiB of which 0 bytes is free. Of the allocated memory 8.59 GiB is allocated by PyTorch, and 309.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF What do I do?

Kennedy Vieira Lopes

2024-01-04 21:54:18 +0000 UTC

NVIDIA GeForce RTX 3060 TI Thanks I'll try that one.

RMR

2024-01-03 09:58:53 +0000 UTC

no worries I'm on vacation right now so I might be too slow to answer so question right now but I'll answer them as soon as I can ;)

Aitrepreneur

2024-01-02 22:50:51 +0000 UTC

omg thank you I'm new to this sort of thing as you can probably tell lol. Thanks a lot I wasn't expecting a reply so soon too :)

bananaboy

2024-01-02 22:44:36 +0000 UTC

it should give you a url, just put that url in your browser, you can watch my install video if you need more info: https://youtu.be/C-7jGYOGvy4

Aitrepreneur

2024-01-02 22:38:30 +0000 UTC

Hi thanks for everything you do here man it's awesome to learn about it. I'm just a bit stuck... Probably something easy so I'm sorry. But I've installed SillyTavern, and now trying to get this Text-Generation-Webui to work, but when I click on "start_windows.bat" the command prompt window pops up and seems to start loading it, but then nothing happens. The last thing it says is "To create a public link, set `share=True` in `launch()`."

bananaboy

2024-01-02 22:06:17 +0000 UTC

you can try simpler one then: https://we.tl/t-IwuE09OFbf

Aitrepreneur

2024-01-02 20:42:45 +0000 UTC

I recommend using GPTQ models like this one for example: TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True copy and paste that in the model filed, dl it and then use the exllama loader

Aitrepreneur

2024-01-02 20:41:03 +0000 UTC

sure just move the folder

Aitrepreneur

2024-01-02 20:40:20 +0000 UTC

what's your GPU? I recommend maybe using a different model like this one: TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True copy and paste that in the model filed, dl it and then use the exllama loader

Aitrepreneur

2024-01-02 20:40:05 +0000 UTC

hello I need help. i already have GIT installed but after i press 1 when opening the text installer, it tries to redownload GIT. it gets to 90% and the command prompt page closes.

Alex

2024-01-02 08:22:17 +0000 UTC

It all been working out great till I had a problem downloading the model, Whenever I tried to Load the model this message appears: Traceback (most recent call last): File "C:\Users\bluxf\OneDrive\Desktop\text-generation-webui\modules\ui_model_menu.py", line 213, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\bluxf\OneDrive\Desktop\text-generation-webui\modules\models.py", line 87, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\bluxf\OneDrive\Desktop\text-generation-webui\modules\models.py", line 310, in AutoAWQ_loader from awq import AutoAWQForCausalLM ModuleNotFoundError: No module named 'awq'

S i

2024-01-02 08:07:03 +0000 UTC

Is there a way to move the installation to a different drive after installing?

jordo

2024-01-02 01:55:42 +0000 UTC

One question the text generation from TheBloke_Yarn-Mistral-7B-128k-AWQ is slow and takes 1-2 minutes to generate text. is there a way to quicken it up or use a different model?

RMR

2024-01-01 23:14:01 +0000 UTC

might do one but it's kinda niche, not that many people are really interested in that

Aitrepreneur

2023-12-31 01:31:48 +0000 UTC

Can you do a video on Lora training in Ooba? I don't see anyone using them so is it just not useful?

What to watch high

2023-12-30 18:53:44 +0000 UTC

check your connection and antivirus / firewall

Aitrepreneur

2023-12-29 16:06:52 +0000 UTC

why when i download the model it doesnt start? the 7.25GB file stuck in 14.7MB

matryoshka12

2023-12-29 04:09:30 +0000 UTC

yeah...unfortunately oobabooga decided to remove completely the elevenlabs option from the webui since now we have the free xttsv2 option, I don't agree with it personally. If you want to use it, either revert back to an older version of the text gen or download and install sillytavern (just follow my latest video and patreon posts) it should still have the elevenlabs option available

Aitrepreneur

2023-12-26 15:26:58 +0000 UTC

I have been looking for this extension as well, it did come come preloaded like the others.

Tonic69

2023-12-26 05:52:04 +0000 UTC

Where do I get the elevenlabs_tts extension?

Dr. Brian Burton

2023-12-26 04:54:45 +0000 UTC

Gold star for me I found out how to extend path length and now have Oobabooga working! Yay go me!

Andy Merz

2023-12-24 13:25:24 +0000 UTC

Thank you, your advice got me pass that issue. Now I am getting "ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'C:\\Users\\andy_\\OneDrive\\Desktop\\LLM\\text-generation-webui\\installer_files\\pip-install-7nifgqmh\\lm-eval_85a62f66ec974b38aa5fba1225dcd99d\\lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml' HINT: This error might have occurred since this system does not have Windows Long Path support enabled. You can find information on how to enable this at https://pip.pypa.io/warnings/enable-long-paths" which recall when trying before you had instructions for newbies somewhere maybe.

Andy Merz

2023-12-24 12:05:22 +0000 UTC

usually this error happens because of where you installed the webui, try installing it in a folder without spaces and without special characters

Aitrepreneur

2023-12-24 00:37:01 +0000 UTC

first don't run the bat file in admin rights as I said, then for this error apparently you can fix this by typing regedit in the windows search bar. This should give you the Registry Editor. In the registry editor navigate to: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled Double click on "LongPathsEnabled" and change value from 0 to 1, then click OK.

Aitrepreneur

2023-12-24 00:35:15 +0000 UTC

"Miniconda hook not found" whats the solution?

Andy Merz

2023-12-23 16:43:45 +0000 UTC

I also find another error during the installation --- Using cached distlib-0.3.8-py2.py3-none-any.whl (468 kB) Building wheels for collected packages: lm-eval Building wheel for lm-eval (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for lm-eval (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [3345 lines of output] running bdist_wheel running build running build_py creating build creating build\lib creating build\lib\lm_eval

littlerunning frog

2023-12-23 16:03:37 +0000 UTC

hi, I tried your suggestion path and the system ask me to provide administration and run but got the same error, I also tried to put the file to another location E:\AI system didn't ask for administration this time but still get the same error. I tried various different path/folder but not get any lucky, am I missing some thing?

littlerunning frog

2023-12-23 15:21:38 +0000 UTC

very often this kind of error happen when you install it in a folder path that is too long or that is located under the C windows folder. So try installing it in a short folder on your desktop or something like: C:\Users\ohmni\Desktop\AI

Aitrepreneur

2023-12-23 02:56:41 +0000 UTC

hi, I know this sounds bit odd but I get the same error by running the update windows bat file :(( please help and happy Christmas

littlerunning frog

2023-12-23 02:40:53 +0000 UTC

update the webui by running the update windows bat file

Aitrepreneur

2023-12-23 02:18:30 +0000 UTC

having the same issue

Doc Prometheus

2023-12-23 01:50:19 +0000 UTC

hi, I get an error, please can you help please. ----- copying build\lib\lm_eval\tasks\model_written_evals\persona\believes-it-is-not-being-watched-by-humans.yaml -> build\bdist.win-amd64\wheel\.\lm_eval\tasks\model_written_evals\persona copying build\lib\lm_eval\tasks\model_written_evals\persona\believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build\bdist.win-amd64\wheel\.\lm_eval\tasks\model_written_evals\persona error: could not create 'build\bdist.win-amd64\wheel\.\lm_eval\tasks\model_written_evals\persona\believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml': No such file or directory [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for lm-eval Failed to build lm-eval ERROR: Could not build wheels for lm-eval, which is required to install pyproject.toml-based projects Command '"E:\AI\Oobabooga\text-generation-webui\installer_files\conda\condabin\conda.bat" activate "E:\AI\Oobabooga\text-generation-webui\installer_files\env" >nul && python -m pip install -r temp_requirements.txt --upgrade' failed with exit status code '1'. Exiting now. Try running the start/update script again. Press any key to continue . . .

littlerunning frog

2023-12-23 00:06:57 +0000 UTC

Happy to hear that, have fun ;)

Aitrepreneur

2023-12-20 12:51:04 +0000 UTC

Thank you so much for all the work you've done! 2 little points which I wish I had known before the install: 1) in windows, for your batch file to work, git has to be installed and the git.exe in the "path". (one has to logout then login again for the environment variable to updated, apparently) 2) All the files of the installer bat file will be put in the directory where one runs the install bat file from! (I'm kind of used to being asked where to install into). I ran out of space on the small partition I was using. ANYWAY, the video really helped and it's working! It's a bit of a transition from "Faraday.dev" but there are enough similarities that I didn't feel too lost. Can't wait to go through some of the other ones now I have the text-generation-webui installed. I couldn't have done it without you!

Tom Braider

2023-12-20 05:52:39 +0000 UTC

could be an antivirus/firewall blocking the connection from running further, either disable them before running the installer or create an exception for the file

Aitrepreneur

2023-12-15 22:08:15 +0000 UTC

please don't create a new message on the page, just send me a dm with the error log, your pc specs and a quick explanation on what went wrong, thanks.

Aitrepreneur

2023-12-15 22:07:10 +0000 UTC

i was able to download and i got the web address but I needed to restart my computer and now i dont know how to get everything back up and running again. do I need to reinstall everytime? Im not sure how to turn oogabooga back on

Hurt Porter III

2023-12-15 04:25:37 +0000 UTC

I my case it just stuck downloading at 23%, and then CMD autmatically gets closed, any help?

Serino Alejandro

2023-12-15 02:02:52 +0000 UTC

please send me a dm here, it's easier to follow up for me this way

Aitrepreneur

2023-12-10 03:04:29 +0000 UTC

try downloading this: visualstudio.microsoft.com/visual-cpp-build-tools follow this image guide: media.discordapp.net/attachments/1025498368631377970/1180633461858516992/image.png?ex=657e2198&is=656bac98&hm=14fd9acdb50677653c811affdb532ef5545c4d88c795d7a07dd8327f5ef07866&=&format=webp&quality=lossless&width=2124&height=1176 let me know if that works better, don't forget to restart your pc, some people had to do that too before it worked you can also open the cmd_windows.bat file and paste this line right here: conda install -c conda-forge python-crfsuite then press enter

Aitrepreneur

2023-12-10 03:02:50 +0000 UTC

i posted it in your tech support channel on discord

Jason

2023-12-10 02:46:01 +0000 UTC

miniconda not found

Jason

2023-12-10 02:43:24 +0000 UTC

I keep getting ERROR: Failed building wheel for tts, not sure what to do

Zyfire

2023-12-10 01:37:05 +0000 UTC

what exactly is the error you are getting?

Aitrepreneur

2023-12-10 00:33:16 +0000 UTC

keep giving miniconda is missing. i tried to download miniconda separately but still doesn't work.

Jason

2023-12-09 23:54:13 +0000 UTC

you only have 8Gb of vram that's why, you won't be able to run anything above 7B with good speed. You can try this model: just copy and paste this line in the model field, download it and choose the exllama loader TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True

Aitrepreneur

2023-12-09 22:43:47 +0000 UTC

I went ahead and downloaded TheBloke/LLaMA2-13B-Tiefighter-AWQ before you responded but it takes so long for responses to be generated. Is there any way around this? I have a desktop 3060ti

bruh

2023-12-09 19:25:44 +0000 UTC

after a model is downloaded, click on the small blue refresh button to refresh the list

Aitrepreneur

2023-12-09 19:22:48 +0000 UTC

"2023-12-09 13:46:12 ERROR:No model is loaded! Select one in the Model tab." is what I see in the cmd bar when I try to send it a message, but whenever I go to the model that the only model option i have is "none"

bruh

2023-12-09 18:48:55 +0000 UTC

sure no problem

Aitrepreneur

2023-12-09 17:38:25 +0000 UTC

So, I have a newbie question. When I used the one click install it installed everything in my C drive, which is an hdd. Can I just paste the folder to my ssd at D without wrecking stuff?

troulertrian

2023-12-09 16:54:20 +0000 UTC

I have the same problem

Edmunds Kemme

2023-12-08 21:45:21 +0000 UTC

if you run out of memory you simply need a smaller task or a smaller model, it's about your VRAM size on your graphics card

John Tackman

2023-12-08 18:14:11 +0000 UTC

Ok this one worked! Thank you so much 😭 Btw... I also have another issue but it's trying to install the Voice. I used the One-click version you provide... I'm having on the installation... I'll post it on the related thread.

KHU

2023-12-05 16:55:25 +0000 UTC

seems there's a version 2 of exllama. Do I pick that or the default one?

KHU

2023-12-05 16:47:48 +0000 UTC

I'm have a GTX 1060 6G VRAM... Is that the cause? I'll this model you recommended thx!

KHU

2023-12-05 16:08:45 +0000 UTC

Hmm weird this should be enough memory for that... try this one: copy and paste this in the model field: TheBloke/LLaMA2-13B-Tiefighter-GPTQ:gptq-4bit-32g-actorder_True then download it, once it's done, select the exllama loader and load the model then try chatting, this should work fine with your GPU

Aitrepreneur

2023-12-05 15:32:59 +0000 UTC

I'm running a RTX3070 on a mystra/tiefighter (as the video stipulated me to use). I have 16gb of ram installed, not sure if thats applicable to the situation. Cheers!

Valedictorian

2023-12-05 15:10:46 +0000 UTC

That means your GPU doesn't have enough memory to run the operation/run the model. What GPU do you have and what chat model are you running?

Aitrepreneur

2023-12-05 14:19:23 +0000 UTC

what's your GPU? It might not be compatible with AWQ if it's older than the 2000 series. Try this model: just copy and paste this line in the model field: TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True then download it, once it's done, chose the exllama loader and click load.

Aitrepreneur

2023-12-05 14:18:34 +0000 UTC

Hey! So it says that CUDA has run out of memory. In simple terms, what does this mean, and how do I avoid/allocated more memory? Thank you in advance!

Valedictorian

2023-12-05 12:46:34 +0000 UTC

this is the model I loaded... TheBloke/NeuralHermes-2.5-Mistral-7B-AWQ ps. tried re-installing through One-clikc... it didn't work out either. https://imgur.com/a/LMxwxbw

KHU

2023-12-04 19:26:50 +0000 UTC

ps. kinda forgot I downloaded the ONE-Click version you provide. I'll try reinstalling with that one and see if it works out.

KHU

2023-12-04 19:06:39 +0000 UTC

Um... no sure what to show you but this is what I'm seeing at the cmd window right now... https://imgur.com/a/80aYBIY

KHU

2023-12-04 19:01:35 +0000 UTC

Strange. Thanks, I'll give this model a try.

Colin

2023-12-04 18:48:00 +0000 UTC

Thanks that works.

Malte Steinlandt

2023-12-04 18:34:10 +0000 UTC

yes as I explained in the video and as the text implies you need to make sure that there is no spaces in the folder path. So if you have something like: C:\Users\yourusername\AI\AI INSTALL\ then you need to delete the space (in this case AI INSTALL) or replace it with something else (like this for example: AI_INSTALL)

Aitrepreneur

2023-12-04 18:19:20 +0000 UTC

what model have you loaded? What error does it give you in the command prompts window?

Aitrepreneur

2023-12-04 18:17:16 +0000 UTC

Hello. First sorry for my English, When i start the .bat file its Downloading the files from Github but after that i get this message: This script relies on Miniconda which can not be silently installed under a path with spaces. Need some help

Malte Steinlandt

2023-12-04 18:08:24 +0000 UTC

Hello. I'm able to install this but I'm getting this. When I try to interact in the chat, I'm unable to see the responses. https://imgur.com/a/BRcB7Et

KHU

2023-12-04 18:02:03 +0000 UTC

Noice. Giving it a shot. Thanks dude!

Will Simon

2023-12-04 15:22:24 +0000 UTC

hum..but..yeah, that's because it's not a quantized model and it's not even good to begin with xD try this one: just copy and paste that line in the model field, then click download TheBloke/LLaMA2-13B-Tiefighter-GPTQ:gptq-4bit-32g-actorder_True once it's done, select the exllama loader then try again, no need to use sure thing with that model

Aitrepreneur

2023-12-04 15:21:31 +0000 UTC

I really want a FULLY uncensored model, so after some reddit crawling found this Nous-Hermes LLama2 13b to fit the bill, if you do the "Sure Thing!" instruction. But maybe it's the model? Do you know of a good uncensored model that might run better?

Will Simon

2023-12-04 15:14:00 +0000 UTC

Damn. Same issue. I put the Prefer No Sysmem Fallback for windows start.bat. closed and reopened the bat file. loaded model (cause I haven't set it to auto load still). Asked it how to make meth, and it's still slow af.

Will Simon

2023-12-04 15:11:43 +0000 UTC

NousResearch_Nous-Hermes-Llama2-13b Going for disabling system mem fallback now.

Will Simon

2023-12-04 15:00:57 +0000 UTC

It's usually either the drivers or the model, what model are you using? When it comes to the nvidia drivers, are they up to date and have you disabled system memory fallback? https://nvidia.custhelp.com/app/answers/detail/a_id/5490/~/system-memory-fallback-for-stable-diffusion This will greatly improve the speed for both SD and text-gen

Aitrepreneur

2023-12-04 14:57:09 +0000 UTC

yeah actually some people (including myself) have some issues running some AWQ models for some reason, which is why right now I just recommend using GPTQ models. You can try using this pygmalion 2 model: just copy and paste this line in the model field: TheBloke/Pygmalion-2-7B-GPTQ:gptq-4bit-32g-actorder_True Then once it's downloaded, select the exllama loader and click load

Aitrepreneur

2023-12-04 14:55:29 +0000 UTC

my generation is SUPER slow. I've got a RTX 3060, Ryzen 5 5600G, and 64 gb ram. Doing art gen works fine. usually can do 100 steps with SDXL in about 1:17. but this text stuff is SLOW. I gotta have something configured wrong right?

Will Simon

2023-12-04 14:28:14 +0000 UTC

The 7B AWQ model you showed in the video. The weird thing is, it gets worse and worse as the chat goes on. At the beginning of the chat it was fine, could be finished in 5 seconds at times. But after 10 messages it could take up to 300 seconds for one response.

Colin

2023-12-04 10:59:52 +0000 UTC

Just start talking in spanish, but I'm not sure that Mistral has been trained on Spanish language though

Aitrepreneur

2023-12-03 17:20:11 +0000 UTC

Send me a dm with your pc config and full error log and when it happens, etc

Aitrepreneur

2023-12-03 17:19:26 +0000 UTC

Only with the gf model? That's weird, it shouldn't matter what character you chose, the speed should stay the same, it's usually more a model issue, what model are you using?

Aitrepreneur

2023-12-03 17:18:53 +0000 UTC

How can i change the language of the chats to spanish? I have Mistral model

David Soler

2023-12-03 12:17:47 +0000 UTC

I tried to install a model and I got along list of this error look a like: 2023-12-03 02:15:51 ERROR:Failed to load the model. Traceback (most recent call last): File "C:\Users\moses\Downloads\text-generation-webui\modules\ui_model_menu.py", line 209, in load_model_wrapper shared.model, shared.tokenizer = load_model(shared.model_name, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\moses\Downloads\text-generation-webui\modules\models.py", line 85, in load_model output = load_func_map[loader](model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\moses\Downloads\text-generation-webui\modules\models.py", line 250, in llamacpp_loader model, tokenizer = LlamaCppModel.from_pretrained(model_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\moses\Downloads\text-generation-webui\modules\llamacpp_model.py", line 91, in from_pretrained result.model = Llama(**params) ^^^^^^^^^^^^^^^ File "C:\Users\moses\Downloads\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp\llama.py", line 933, in init self.scores: npt.NDArray[np.single] = np.ndarray( ^^^^^^^^^^^ numpy.core._exceptions._ArrayMemoryError: Unable to allocate 15.6 GiB for an array with shape (131072, 32000) and data type float32 Exception ignored in: Traceback (most recent call last): File "C:\Users\moses\Downloads\text-generation-webui\modules\llamacpp_model.py", line 49, in del del self.model ^^^^^^^^^^ AttributeError: 'LlamaCppModel' object has no attribute 'model' N.B: IF THERE IS ANYTHING YOU CAN DETECT THAT I MESSED UP PLEASE LET ME KNOW: i CHOSE NONE AT THE BEGINING cause I DID NOT KNOW IF I WAS TO CHOOSE CPU OR GPU. 16GB RAM. LAPTOP.

SINHORA MOSES

2023-12-03 07:30:13 +0000 UTC

hello

SINHORA MOSES

2023-12-03 07:27:54 +0000 UTC

My generation is going extremely slow. "Output generated in 230.27 seconds (0.20 tokens/s, 46 tokens)" It strangely only happens with your girlfriend model. I have a 4070 Ti 12GB, what could be the cause for this?

Colin

2023-12-03 00:28:57 +0000 UTC

I was able to fix it on my Windows machine with Nvidia GPU. Not exactly one-click, but it might be helpful! I believe that my issue was that I already had a PATH set for another Linux environment. Installed miniconda manually (https://docs.conda.io/projects/miniconda/en/latest/) When installing, I chose NOT to add it to PATH and NOT to set it as my default Python 3.11. After installing, I ran Anaconda 3 (Or Anaconda PowerShell Prompt) I then navigated (using CD) to the directory I wanted to install. Once there, I ran the following commands - ((text on the line after # is not to be entered)): #Create new Conda Environment conda create -n textgen python=3.11 conda activate textgen Install PyTorch conda install -y -c "nvidia/label/cuda-12.1.0" cuda-runtime Install WebUI git clone https://github.com/oobabooga/text-generation-webui cd text-generation-webui pip install -r requirements.txt _______________________ Solved this by giving ChatGPT AiTrepeneur's .bat script along with the readme text for the Text Generation. I then gave it the error message I was getting and the directory I wished to install it in. Hope this helps!

Peejie

2023-12-02 23:01:54 +0000 UTC

I am having the same issues, unfortunately

Peejie

2023-12-02 22:27:34 +0000 UTC

send me a dm with more info and pc config

Aitrepreneur

2023-12-01 13:33:23 +0000 UTC

Hey, I got "Warning you haven't downloaded a model yet. Once the web ui launched, head over to 'model' tab and download one." I'm in the model tab, the drop down for model only says "none"... I have NO IDEA where to get a model. What do I click?

Night Wolf

2023-12-01 08:54:04 +0000 UTC

It didnt install miniconda. Miniconda isnt found. Its a pain

Mentos

2023-12-01 08:33:49 +0000 UTC

dm me more details like you pc config and full error log

Aitrepreneur

2023-11-29 22:55:36 +0000 UTC

I'm getting the following error when I run the installer; I'm very new to all of this and have no idea what to do to correct it. "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. openai-whisper 20230918 requires tiktoken==0.3.3, but you have tiktoken 0.5.1 which is incompatible." I think (am hoping) this is the cause of other problems I'm having post-installation, like not being able to connect to Tavern.AI.

Sean Ashcraft

2023-11-29 22:41:16 +0000 UTC

send me more details in dm

Aitrepreneur

2023-11-28 20:42:06 +0000 UTC

they are installed in the same folder you put the bat file in

Aitrepreneur

2023-11-28 20:41:29 +0000 UTC

I really enjoy the idea behind it and the possibilities, I try doing it manual months ago with your older video and was able to installed but it was running slow. I saw your new video and choose to subscribe, great content, I was able to install and run the interface, them I downloaded yarm mistral model you recommended to see if I would have better luck this time, I try to loaded and I get and error while fusing layers, unable to load the model, I wonder if I am missing something or did something wrong? I have a laptop with a AMD Ryzen 7 5800H, 16ram and a Nvidia Geforce RTX 3050Ti, not sure if my laptop is not powerful enough to run the AI. while installing the interface I put the option of Nvidia, not sure if my mistake was there, I am out of ideas

FruitSnack

2023-11-28 09:00:19 +0000 UTC

This is probably going to sound dumb but I can't find where the installer put the .bat files to launch the programs that were installed. I'm not sure what directory they get installed to in windows.

Darkwan75

2023-11-28 02:23:19 +0000 UTC

send me a dm with more info as well as your pc config

Aitrepreneur

2023-11-26 21:19:23 +0000 UTC

Hi! At first it failed to install Git, so i installed it manually. Then this error poped up "fatal: failed to load library 'libcurl-4.dll' Failed to clone the oobabooga text-generation-webui repository." Defender and firewall are disabled.

Martin Čarný

2023-11-26 19:50:33 +0000 UTC

send me a dm with more info as well as your pc config

Aitrepreneur

2023-11-26 00:49:29 +0000 UTC

How are you. After installing link, I saw these error statements, not sure what I should do -------- Downloads\text-generation-webui\installer_files\env\Lib\site-packages\gradio\components\dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: 128 or set allow_custom_value=True. and ---- Downloads\text-generation-webui\modules\models.py", line 306, in AutoAWQ_loader from awq import AutoAWQForCausalLM ModuleNotFoundError: No module named 'awq'

KdaK

2023-11-25 20:10:59 +0000 UTC

Probably getting blocked by your antivirus and or firewall, disable them for a few minutes before relaunching the installer

Aitrepreneur

2023-11-25 16:32:49 +0000 UTC

Nothing happens when I try opening the down loaded files.

rick esping

2023-11-25 15:59:45 +0000 UTC

You can try this simpler installer, no need to press anything just double click it: https://we.tl/t-CiPTkrEUTE

Aitrepreneur

2023-11-24 02:30:21 +0000 UTC

I click on the bat file, hit 1 for Windows, the CMD box disappears and then nothing. I have Git installed. What else could be the problem?

Mark Mynhier

2023-11-23 22:54:32 +0000 UTC

"But you shouldn't really touch anything in the parameters page, the default presets are already good enough" mostly what I needed to know! ha ha, thanks!

TechnoCore

2023-11-21 23:17:21 +0000 UTC

I might do an extended video explaining some thing further if people want it but there isn't a lot you need to know, if you want you can read the documentation to learn more: https://github.com/oobabooga/text-generation-webui/wiki But you shouldn't really touch anything in the parameters page, the default presets are already good enough

Aitrepreneur

2023-11-21 17:33:00 +0000 UTC

Hello, one-clicked worked fine, got a model, made a character. All working well, so thanks! Just some follow up questions... Is there a video or where can I find out about all the details of Oobabooga environment? Like what Chat-Style I should use for what? What should I set on the parameters page? What do all the extension do? Etc... Thanks!

TechnoCore

2023-11-21 12:33:43 +0000 UTC

Thank you Aitrepreneur for the tutorial! Btw, If anyone wants the MUCH higher quality V2 voices for Elevenlabs with ability to adjust style setting, I posted the code and notes here. https://github.com/oobabooga/text-generation-webui-extensions/issues/51

Brent Johnson

2023-11-21 02:54:51 +0000 UTC

Thank you.

Andrew Barrett

2023-11-20 06:08:44 +0000 UTC

dm sent

Aitrepreneur

2023-11-19 21:56:29 +0000 UTC

I can't figure out how to launch the UI please help 😔🙏

Nicholas Canada

2023-11-19 11:50:45 +0000 UTC

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. openai-whisper 20230918 requires tiktoken==0.3.3, but you have tiktoken 0.5.1 which is incompatible. Successfully installed tiktoken-0.5.1

Nicholas Canada

2023-11-19 10:48:29 +0000 UTC

send me a dm with more info

Aitrepreneur

2023-11-18 16:36:55 +0000 UTC

send me a dm with more info

Aitrepreneur

2023-11-18 16:36:52 +0000 UTC

neither windows or mac works, same error.

Anthony

2023-11-18 14:25:17 +0000 UTC

For some reason it won't let me click on the URL after it's done installing. What do I do about that?

DeadFlame

2023-11-18 02:49:11 +0000 UTC

you can dm me more info like the models you used and your GPU

Aitrepreneur

2023-11-15 20:55:31 +0000 UTC

The download worked fine, I could see the file in the folder. I think the install failed because a previous version was present. I went ahead and installed manually, it runs now. However, the models are returning gibberish for my questions. Perhaps a troubleshooting video for commons issues would be helpful. Nobody is making videos quite like yours. Much appreciated.

Robert Hentschel

2023-11-15 15:54:08 +0000 UTC

make sure your antivirus and or firewall is not blocking the download, just disable them for a few minutes before launching the bat file

Aitrepreneur

2023-11-15 15:49:19 +0000 UTC

Yeah, you'll need to install git and (probably) pip.

StrangeGarred23

2023-11-15 04:31:10 +0000 UTC

Type 1 or 2 and press Enter: 1 Git is not installed. Downloading and installing Git for Windows... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 58.3M 100 58.3M 0 0 8656k 0 0:00:06 0:00:06 --:--:-- 9666k Failed to download Git installer.

Robert Hentschel

2023-11-14 18:11:02 +0000 UTC

It's clear what video we need next from AItrepreneur: Local install Lora-Training for text-generation AI with Axolotl or else

hanZ

2023-11-13 20:12:59 +0000 UTC

that's probably an issue with the models, default parameters should be good enough as it is, what models have you tried?

Aitrepreneur

2023-11-13 17:09:07 +0000 UTC

I've tried a couple different models, but they only generate gibberish. Are there not Parameters>Generation settings that have to be made? What are they?

totally_not_a_weirdo

2023-11-13 03:28:26 +0000 UTC

I would love resources to finetune models with the cloud. For now with regulat GPU's I have not been able to find good applications for the models I have tested. Meta models 13b produce great content but super slow even with a 4090.

Peruvian Dota

2023-11-13 02:37:07 +0000 UTC

Yup it's another ballgame but still pretty fun to experiment with ;D

Aitrepreneur

2023-11-12 20:57:57 +0000 UTC

I mean they work perfectly for more than 90% of people... what issue are you having?

Aitrepreneur

2023-11-12 20:57:15 +0000 UTC

I have never been able to get your one click programs to work! they sure as heck not one click. and your video's don't seem to help either.

Lexx 1100

2023-11-12 20:46:48 +0000 UTC

Ran perfectly on my windows PC. D*mn - this is an even deeper rabbit hole than image generation. I'm pretty experienced with image generation, but with text generation I'm feeling like a complete newbie regarding AI/ML. 4bit vs. 8bit, all those loaders, hundreds of base models etc.pp. And I've haven't even started with training LORAs for those.

hanZ

2023-11-12 20:36:08 +0000 UTC

Le problème c'est qu'il n'y a même pas de code erreur la l'invite de commande admin s'arrête et se ferme au bout de 10 secondes d'installation...?

pib boris

2023-11-12 13:24:31 +0000 UTC

tu peux m'envoyer un message en DM, le code erreur et screenshot si possible

Aitrepreneur

2023-11-12 13:16:01 +0000 UTC

Hello, i can t install with the link...i did on my laptop. Now i can used it on my first pc and i can't install it on my desk one. Can i have some answers ?

pib boris

2023-11-12 07:42:47 +0000 UTC

You can try turning off your antivirus/firewall for a few minutes see if that makes it better maybe they are blocking the connection, but usually this depends on your machine and sometimes internet connection too

Aitrepreneur

2023-11-11 14:09:52 +0000 UTC

Wow... what model did you use?

Aitrepreneur

2023-11-11 14:08:30 +0000 UTC

with your GPU I may recommend this model, just copy and paste that line in the model download field and click download: TheBloke/Nethena-MLewd-Xwin-23B-GPTQ:gptq-4bit-32g-actorder_True then choose the exllama loader and click on load

Aitrepreneur

2023-11-11 14:01:57 +0000 UTC

I see, I indeed saw some other people say that for some reason awq don't work as well as gptq. Well in your case just use GPTQ models, if they are based on llama, use exllama loader for faster token generation

Aitrepreneur

2023-11-11 13:54:16 +0000 UTC

My install gets stuck at installing miniconda. I think I already have it from running A1111 But the installer freezes there. Can you help me get this up and running?

tdog

2023-11-11 04:24:34 +0000 UTC

No matter which model I use, I get this response in the chat t0. Assist in setting achievable goals related to personal growth and self-improvement. Help identify specific areas where growth is desired and create actionable steps towards achieving those goals. Provide motivation and encouragement throughout the process. Offer accountability by checking in periodically and discussing progress made.

Chad Fell

2023-11-11 02:09:40 +0000 UTC

I have RTX 3090 TI

Enchanted Pixie

2023-11-10 22:19:42 +0000 UTC

Yeah. Same boat here with 1070. Unfortunately I have not been able to run anything on pytorch 2.1 with CUDA 12.1 on this card (and this is not limited to text models). One option would be to downgrade to CUDA 11.8. HOWEVER I found info in some git issue that AWQ model loading requires now CUDA 12.1 and indeed it fails to load. So that's it. It seems that the problem is either Pytorch that silently stopped providing binary kernels for 10xx cards or bitsandbytes not providing it. However, I was not able to figure this out and I spent too many time today on this. ;) (GGFU models works fine though)

Retired Android

2023-11-10 22:18:06 +0000 UTC

Yeah that seemed to make it work. That and changing to AutoGPTQ. That model seems to not work well with my system, 4070Rtx with 12 gig and 64gig on mobo. But my cpu is AMD Ryzen 9 3900X so that may be the problem. TheBloke_Mistral_7B_Dolphin2.1_LIMA0.5-AWQ works pretty well though.

Robert Bersch

2023-11-10 21:52:59 +0000 UTC

Yeah I have the same problem, did you get to any solution?

Stan Sororitas

2023-11-10 20:42:04 +0000 UTC

for the llava model do not forget to modify the CMD_FLAGS.text file with this command: --multimodal-pipeline llava-13b otherwise that's not gonna work, just like I showed in my video. For the other models, you can try using gptq models instead and see it that works better, by loading them with either autogptq or exllama (depending on the model you've chosen)

Aitrepreneur

2023-11-10 17:39:27 +0000 UTC

Hmm everything worked fine install wise till I tried to load a model... using model llava-13b-v0-4bit-128g. Downloaded ok but get this error trying to load. File "C:\Users\secto\Desktop\ChatAI\text-generation-webui\installer_files\env\Lib\site-packages\transformers\modeling_utils.py", line 2992, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory models\wojtab_llava-13b-v0-4bit-128g. Edited: Installed TheBloke/claude2-alpaca-7B-AWQ and it started doing stuff.. Seems more random nonsense then AI.

Robert Bersch

2023-11-10 16:46:16 +0000 UTC

send me a dm with more info and screenshots if possible

Aitrepreneur

2023-11-10 12:17:29 +0000 UTC

I think we're having the same problem. I'm on a 1080 Gtx. Nothing is generating. When i check the script its showing the below error RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Fash Demented

2023-11-10 07:27:30 +0000 UTC

So i tried a few models now (these ones ->) TheBloke_Llama-2-7B-Chat-AWQ TheBloke_Llama-2-7B-Chat-GPTQ TheBloke_manticore-13b-chat-pyg-AWQ trion-m-7b.Q5_K_M.gguf and the problem only comes up with the AWQ models. The GPTQ and GGUF work just fine.

vanNiko

2023-11-10 06:35:03 +0000 UTC

I seemed to have installed it fine, but ran into an issue when trying to chat. When I chat I see it appear for 1 sec. then my chat disappears and the model will try to respond but shows nothing. I have a Nvidia 1080 TI and using tried both TheBloke_LLaMA2-13B-Tiefighter-AWQ and TheBloke_Mistral-7B-v0.1-AWQ, I followed the install video "INSTALL UNCENSORED TextGen Ai WebUI LOCALLY in 1 CLICK" on youtube.

Kristopher Hodge

2023-11-10 05:06:00 +0000 UTC

Thank you.

Andrew Barrett

2023-11-10 03:42:34 +0000 UTC

what's your GPU? For the smallest model you try the one I showcased in my video called mistral 7b

Aitrepreneur

2023-11-09 22:21:59 +0000 UTC

Heya , so I did the install and it works a charm but there is no model included . Where do I get the models and what do you suggest ?

Enchanted Pixie

2023-11-09 21:43:25 +0000 UTC

You can first try the ones I show in my video

Aitrepreneur

2023-11-09 20:28:40 +0000 UTC

what model were you using?

Aitrepreneur

2023-11-09 20:28:11 +0000 UTC

which model types should i use?

Phillip Gimmi

2023-11-09 19:36:12 +0000 UTC

Hey! Everytime i try to generate a response this error comes up: RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

vanNiko

2023-11-09 18:41:04 +0000 UTC

Thanks. I'll try that. I'll move or rename the current folder so I have a copy of the current models etc. then try the new one click isntaller. Ideally they could make it so things could be installed without affecting the models folder etc. and they could make sure everything has it's own version of everything (eg. python or torch etc.). edit. After using your install option (with copying the models and characters back from the other folder) it's now wokring okay thanks and the stable diffusion gui is too (I tried them both at the same time but that slowed things too much with 8K context). And the Llava thing worked too (though it isn't always totally accurate about what's in an image). I'l try a re-install of the KOHYA later too to try to get that working.

cool1

2023-11-09 16:11:41 +0000 UTC

Wow, yeah hard to say, not sure why thing suddenly gone wrong, try reinstalling the webui? There shouldn't be any issues running all of the webuis, I do it myself

Aitrepreneur

2023-11-09 16:07:11 +0000 UTC

if you need to send screenshots, discord, otherwise patreon is fine

Aitrepreneur

2023-11-09 16:04:25 +0000 UTC

I've somehow messed my textgen web ui up. I already had it installed and it was working fine and running fast (seemed to use the GPU with GPTQ models and loading 13B models with 8K context with ExLLama worked okay). Somehow after installing the Stable Diffusion web ui (automatic1111) and trying (unsucessfully) to install and run the KOHYA-SS for Lora training, then trying updating textgen webui to try the Llava model (as shown in the video), with loading that model with the settings and editing the cmd_flags.txt to add "--multimodal-pipeline llava-13b" after that it was giving me errors/warnings about Torch not being compiled for GPU and trying to reload past models that worked, using things that worked before (eg. ExLlama) doesn't work. I tried clicking the "update_windows.bat" to update things but it still isn't all working. Messages I get are things like: Torch not compiled with CUDA enabled ERROR:Could not find repositories/exllama. Please ensure that exllama (https://github.com/turboderp/exllama) is cloned inside repositories/ and is up to date. when I click "update windows" one of the output lines says "TORCH: 2.1.0+cpu" - is that something wrong - is that saying it's CPU only (not GPU)? Do I need to remove folders and re-install textgen again (after backing up the models & characters)? What most likely made it go wrong? And does it really matter that I have installed a slightly higher version of Python than one of the warnings recommended? If I used the new one click install would that make sure it used its own version of everything (python etc) so there wouldn't be conflicts with anything?

cool1

2023-11-09 15:40:46 +0000 UTC

Discord or here? Thanks for make yourself avaliable

Augusto Baldim

2023-11-09 15:39:03 +0000 UTC

send me a dm

Aitrepreneur

2023-11-09 15:10:28 +0000 UTC

Saw someone in the forum having this issue too, could be a bug. You can try launching the file called "cmd_windows.bat" which will bring a command prompt window and inside you can copy and paste this command then press enter: python -m pip install git+https://github.com/UKPLab/sentence-transformers@v2.2.2 then relaunch the webui by running the start_windows.bat file

Aitrepreneur

2023-11-09 15:10:11 +0000 UTC

yes it should, but it requires ROCm SDK 5.4.2 or 5.4.3 to be installed. Some systems may also need: sudo apt-get install libstdc++-12-dev Edit the "one_click.py" script using a text editor and un-comment and modify the lines near the top of the script according to your setup. In particular, modify the os.environ["ROCM_PATH"] = '/opt/rocm' line to point to your ROCm installation.

Aitrepreneur

2023-11-09 15:02:29 +0000 UTC

How do I get some tech support? I've been getting an error everytime I download a model

Augusto Baldim

2023-11-09 13:26:32 +0000 UTC

sorry i got the memo i was watching your video while installing i removed the space now its working

Anas AL Chayah

2023-11-09 10:26:34 +0000 UTC

i am getting the error that This script relies on Miniconda which can not be silently installed under a path with spaces. what should i do

Anas AL Chayah

2023-11-09 10:15:28 +0000 UTC

"Failed building wheel for sentence-transformers" and "ERROR: could not build wheels for sentence -transformers, which is required to install pyproject . toml-based projects" I tried on one of my old PCs and it has no GPU so I chose N.

chris torrence

2023-11-09 04:14:46 +0000 UTC

Hi Altrepreneu is there are way to get this to work with AMD RocM warn("The installed version of bitsandbytes was compiled without GPU support.) im running Ubuntu and have double checked that my gpu is Rocm compatible , Cheers

Mark

2023-11-09 03:22:02 +0000 UTC

I saw someone having that Llava error too but has long it doesn't inpact its use it's fine, yes Exllama is better and faster and AutoGPTQ since it's made to run Llama based models

Aitrepreneur

2023-11-09 02:16:16 +0000 UTC

when does that happen?

Aitrepreneur

2023-11-09 02:13:36 +0000 UTC

You can use it if you trust it

Aitrepreneur

2023-11-09 02:12:28 +0000 UTC

just updated the script, try redownloading the new one on this page

Aitrepreneur

2023-11-09 02:11:38 +0000 UTC

same here! not sure if i'm doing anything wrong but no matter what it seems to hit me with syntax errors when trying to run it this same way. This is what I'm showing on my end after running - : command not foundoads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 2: Select your installation type: 1. Linux 2. macOS : command not foundoads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 6: echo : command not foundoads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 7: Type 1 or 2 and press Enter: 2 ': not a valid identifierEXT-GEN_INSTALLER-MAC-LINUX.sh: line 8: read: `choice : command not foundoads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 9: /Users/steven/Downloads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 11: syntax error n'ar unexpected token `{ /Users/steven/Downloads/TEXT-GEN_INSTALLER-MAC-LINUX.sh: line 11: `clone_and_ins'all() {

Steven Markley

2023-11-09 01:29:04 +0000 UTC

why not just install pinokio

Michael

2023-11-09 01:21:11 +0000 UTC

i ran into an error while downloading. Here's the error: fatal: unable to access 'https://github.com/...': Failed to connect to github.com port 443 after 21067 ms: Couldn't connect to server Any ideas why?

ComeOverHere

2023-11-08 23:42:20 +0000 UTC

I'm not a mac sure so I'm not sure where the issue could but make sure that you first make the script executable by running chmod +x TEXT-GEN_INSTALLER-MAC-LINUX.sh and then you run the script ./TEXT-GEN_INSTALLER-MAC-LINUX.sh

Aitrepreneur

2023-11-08 23:31:18 +0000 UTC

when I added the bit for it to load Llava and the bit with "--multimodal-pipeline llava-13b" at some point it said "UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in [the folder name]". I'm running windows 10. Is there anything that could be changed to help there to make it more effecient in how it stores cached stuff - if its needed? It also gave some warning message about using AutoGPT and there being some problem with CUDA and that it may be slow (this is without running your new installer - but with an existing install of oobabooga) but I'll check to see how it runs. The UI also says "ExLlama_HF is recommended over AutoGPTQ for models derived from LLaMA" so I don't know if it might be better with that over AutoGPT

cool1

2023-11-08 23:23:33 +0000 UTC

Mac-Version is not working for me - throws syntax errors, mentions invalid identifiers. Maybe I've started it the wrong way? (ran it via the terminal)

hanZ

2023-11-08 23:20:21 +0000 UTC

It depends on 1) which models will fit in the VRAM (different 3060s have different VRAM amounts I think so check what yours has and check whether the model you are checking fits in it) 2) what you want to use it for and whether that AI model was trained for it. But the Mistral-7B models (do a search on huggingface.co) are small models that should fit in that GPU's VRAM (around 4GB I think). 3) whether it will work on what you want to use eg. the oobabooga web ui - you could check on the model page details on the huggingface site

cool1

2023-11-08 21:45:57 +0000 UTC

what model is best to use with a 3060?

Leonardo Lopes

2023-11-08 21:15:45 +0000 UTC

1-Click INSTALL Text-Generation-Webui Oobabooga

Comments

my cmd tells me Miniconda not found but i have it installed can you help me with this?

I am new, so you might have to explain things Barny style for me. I click on the link, I start the program, and the command prompt disappears after getting to 99 now what? Thanks in advance for any assistance...

as the error says, you are trying to install it in a folder path with spaces in the name, just don't do that, delete all spaces in your folders and try again

what is wrong with the miniconda?

OK

need a bit more info, what are your PC specs? And are you aware that you are trying to use a CPU model, also try another model as well, this model does require a lot of power to use

Thanks! Here I thought it would cause less issues that way. I deleted everything and started over and it worked! Thanks!

because it looks like you are running the installer as admin, don't do that, run it normally

Vram is king so take a gpu with as much vram as possible maybe a second hand 3090 would be best depending on where you live

So it means that it's time to buy a new one :) Can you help me decide which would be better for AI? I'm considering between the RTX 3060 (12GB) and the RTX 4060 (8GB). Also, maybe you know some other good cards in this price range.

yeah it's because you don't have enough vram to run that model. You can try this one: TheBloke/Xwin-MLewd-7B-V0.2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader altough with 4gb of vram it will still be very slow

GPU - GTX 1050Ti TheBloke/Yarn-Mistral-7B-128k-AWQ (that you recommended in video) but when I tried other models I also got this error CPU - i5-9400F RAM - 32gb (just in case)

what's your GPU and what model are you using?

I mean just delete the newly created text gen folder inside your download folder, put the bat installer file somewhere else on your computer then run it again

open a new command prompt window and type this pip cache purge press enter then redo the install again in a new folder

Hello good sir. I downloaded the .bat file and i opened it straight into the download folder? could you please tell me how can I undo this action so that I can open it in a new, empty file? Thank you and much love for your work.

Thank you for the fast reply! I didn't even see that option, so thank you very much!

yes, you need to go to the session tab, and check the "share" checkbox, then apply the changes, after a few seconds it will give you a local url as well as a public url

Is there a way to make a public URL with this? I've used a couple of your One Click tools and they usually do, but I haven't been able to figure out how to make this one do it.

toggled off??? no it's the opposite, you need to check the box.....

yes that means that you did not added miniconda to path when installing, uninstall miniconda and reinstall it again this time by adding it to path: https://imgur.com/5GXXyo5

Hey I'm trying to run the insatller and it does everything but installing miniconda so I insatlled seperately and when I run the "start_windows.bat" it says miniconda path not found. Do you have any idea how to solve it?

You don't have to do anything, just use the installer, have you watched my install video? https://youtu.be/C-7jGYOGvy4

Hi I was able to download Git bash and also installer now there is a new folder name "text-generation-webui" whats the next step?

Thank you that fixed it👍

try this model instead: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader

Hey I am using a nvidia 3060 This is the model link https://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-AWQ

Will they be adding AMD support to windows eventually?

what's your gpu and what model are you using? (link to the model)

hmm...that's strange indeed... need a bit more info, could you send a few screenshots? Also was the model downloaded correctly?

I am not sure but it looks like there is a version issue with my Mac being on a OS Version that there is no Pytorch version without using a docker container

what's your gpu?

I’m going to try this model when I get home, but just wanted to let you know the other model is working well now. I was able to get the gallery up and started a convo with the built in example.

maybe this one: https://huggingface.co/LoneStriker/Yi-34B-200K-RPMerge-GPTQ

what error do you have in the command window? Also what models have you used? (give me the url)

Hi, have you correctly ran the mac installer? If yes then run the cmd_macos file and then in the command window type: pip install pyyaml then press enter unfortunately since I don't personally use any Mac I can't really be of any further assistance for troubleshooting

try this: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste this in the model field, dl it and then load it using the exllama loader

First and foremost try to not install it in a onedrive folder i know it often causes issues when installing ai tools, just install it in a normal folder on your PC, also still don't know what gpu you got or what model you uses...

I need the full error log, also what's your GPU and what model are you using (link)?

Yes. Once again once my sillytavern video i show everything there

Is there a way to you speak recognition locally ? Silly tavern request API keys as well

this error means that you have tried to activate the silero extension in the session tab, disable it. 11labs was removed from the text gen webui, for this you need to use sillytavern

yes you can't at least for now, use the cpu version instead

How am I to select AMD GPU if it is only for Linux/Mac Os only?

make sure you don't have any antivirus or firewall blocking the connection, disable them both before running the file, that's usually the issue for most people when something like this happens

when I go to run the file it boots up to ask me for 1 or 2, I press 1 and enter then in a flash it closes. It said something once but as I was reading it the window closed.

please don't spam multiple messages, just send me a dm with all the info, pc specs, precise error, model used, loader used, etc

Hi there. I Have installed this successfully and have tried many different iterations of the model "gguf" that you requested us to DL,

Have you watched my installation video? If you are a complete beginner I suggest you first start there

I don't know squat about computers. I'm a musician. i think i did this right. i got ta place whhere the screen introduces Itself and says. something to the effect of what can I do for you or whatever. What am I supposed to do now? Like, am I in or what?

with your gpu you need to download the gptq format model instead of awq

what gpu you got and what model are you using?

everything is installed and a model is chosen but when I type to it the responses I get are all blank

same issue here

make sure that miniconda is added to path: https://eduand-alvarez.medium.com/setting-up-anaconda-on-your-windows-pc-6e39800c1afb

I am getting the Miniconda hook not found from the update_windowns.bat. where should miniconda install? mine went to C:\ProgramData\miniconda3 and C:\Windows\System32\text-generation-webui\installer_files\conda

Two issues...first is simple. What file Launches it once it's installed? It only gives me the url after I install fresh. Second...I've imported Sandra and Tiefighter(?) llm, loaded them both in Oog and I get no responses. I get "typing" but results are blank.

try opening a command prompt window and drag and drop the update_windows.bat file inside then press enter, this will at least give you an error and avoid the automatic crash. Make also sure your antivirus and firewall are disabled just in case

It installes everything but than it says : Conda is not installed. exiting / everytime is run Windows_start.bat

Yo, when i open update_windows.bat it says press any key so i can start downloading it right, but it just crashes

for your GPU I recommend psyfighter 2. TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True copy and paste that line in the model field, dl it and use the exllama loader

my biggest issue with this, is it's impossible to shop around for models in huggingface. No examples to view so I don't know what anything is. Is there a defacto-best option for chat model? 3080Ti and AMD 5700 fwiw

Hey have you fixed this? I met the same problem.

This was the fix, thank you

you can add this line in the cmd_flags.txt --api-port 5001

I reboot and get the same error. Is there a way to change the 5000 ?

this error usually means you have multiple python instances running at the same time, you need to close the python processes or reboot the pc

apparently it is so old huggy face 404 error on it

Definitely try the GPTQ format instead, too many people have issues with awq, try this: TheBloke/Everyone-Coder-33B-Base-GPTQ:gptq-4bit-128g-actorder_True copy and paste this in the model field, dl it and use the exllama loader to load it

sorry, its a NVIDIA GeForce RTX 4090 with 24gig

hi first what's your GPU?

Hello - what does it mean when I use a larger model (theBlock_Everyone-Coder-33B-AWQ) and all I get for answer is "==================================== etc"

Have fun ;)

OK then...your a genius....silly tavern is up and running:)

probably because you had a previous bad install of miniconda that was not added to path, uninstall miniconda, then relaunch the installer in anoter folder

maybe its just me but when i tried to install it i get a error of miniconda not found and when i manually install it, and run update or start windows it still says miniconda not found.

Lol I restarted it and it worked for some reason lol. Started playing around with it a little and its actually mindblowing how it all works so well. Thanks for the help anyways. and the installer, really helpful

try putting this in the model dl field: TheBloke/LLaMA2-13B-Psyfighter2-GPTQ:gptq-4bit-32g-actorder_True dl it and then use the exllama loader