Detailed Comparison of JoyCaption Alpha One vs JoyCaption Pre-Alpha - 10 Different Style Amazing Images - I think JoyCaption Alpha One is the very best image captioning model at the moment for model training - Works very fast and requires as low as 8.5 GB VRAM
Added 2024-09-22 22:51:54 +0000 UTCWhere To Download And Install
You can download our APP from here : https://www.patreon.com/posts/110613301
1-Click to install on Windows, RunPod and Massed Compute
Official APP is here where you can try : https://huggingface.co/spaces/fancyfeast/joy-caption-alpha-one
Have The Following Features
Auto downloads meta-llama/Meta-Llama-3.1-8B into your Hugging Face cache folder and other necessary models into the installation folder
Use 4-bit quantization - Uses 8.5 GB VRAM Total
Overwrite existing caption file
Append new caption to existing caption
Remove newlines from generated captions
Cut off at last complete sentence
Discard repeating sentences
Don't save processed image
Caption Prefix
Caption Suffix
Custom System Prompt (Optional)
Input Folder for Batch Processing
Output Folder for Batch Processing (Optional)
Fully supported Multi GPU captioning - GPU IDs (comma-separated, e.g., 0,1,2)
Batch Size - Batch captioning
Comments
Okay thanks. I wasn't sure exactly what it was but it seemed more accurate according to their stats/tests and the demo and more checkboxes for customising. But if it's not a proper version then I agree it's probably best waiting.
cool1
2024-10-12 22:42:36 +0000 UTCthat version is not a general caption model i will skip it. waiting his next proper release
Furkan Gözükara
2024-10-12 22:38:44 +0000 UTCThere seems to have been a JoyCaption Alpha 2 released recently that's supposed to be more accurate I think than the last 2. Will there be a 1 click install for that at some point please (or is it best waiting for a fuller one)? In the demo it's got more checkboxes for different prompt instructions but the full instruction for those could also be entered into the "custom prompt" of the current one. eg. it has checkboxes for it to caption about image quality, though I'm not sure how accurate that would be if it's only seeing a lower res version of the image, not the full (eg. 1024px) version.
cool1
2024-10-12 16:35:37 +0000 UTCAlright, thanks! 😀
Igor Matheus Bassitt Turco
2024-10-07 23:14:00 +0000 UTCas far as i know yes best
Furkan Gözükara
2024-09-28 10:45:00 +0000 UTCWorks - Alpha One is much more faster than Pre, is it really the best model now ?
Bolli Hotshots
2024-09-28 08:03:09 +0000 UTCyou can change your hugging face folder to another disk. but if you do move your existing cache files to there to avoid re-download
Furkan Gözükara
2024-09-26 16:23:02 +0000 UTCThanks. I don't see what could have though. I used Koyha after the previous time I used Joycaption Alpha. I don't know if it was anything to do with me have the Pre Alpha one in another folder (that couldn't find gradio later). If everything got stored within the drive and folder I put Joycaption Alpha in then surely not thing should affect the models in there. If it's storing things (such as cache or other files) in the C: drive somewhere rather than the drive I installed it in, then maybe it might get affected by other things (if some other things use the same temporary folders/cache) if it's just a temporary cache instead of permanent. Ideally it could keep things permanently in the specified drive and folder (and there's less space on my C drive that's why I put it in one of my other drives that has my room) so that nothing else will affect it.
cool1
2024-09-26 16:16:52 +0000 UTCnot possible. something else must have deleted your hugging face cache also with newest version we switched to a new repo
Furkan Gözükara
2024-09-26 16:04:11 +0000 UTCAbout 4 hours ago I used JoyCaption Alpha 1 to caption images in a folder, then after it finished closed the app. Then 4 hours later I restarted the app and asked it to caption a single image. It then starts downloading a 3.51 GB model.safetensors file and now it's downloading the multi-GB shards again. Have some model files been changed (or did it think there was some problem with the existing files, or is it caching the files somewhere instead of storing them more permanently?)? Couldn't it use the model files that were already installed (that it had been used before, including 4 hours ago) unless prompted to update to newer versions if possible? This would save having to wait ages while it downloads files again (where some version of those files had already been downloaded and used before). At one point it did say "Loading VLM's custom text model Error loading custom text model: We couldn't connect to 'https://huggingface.co' to load this file,". But the model files were already installed I'd been using them before it started downloading them again.
cool1
2024-09-26 11:09:06 +0000 UTCawesome. please tell SECourses to your friends :)
Furkan Gözükara
2024-09-25 10:26:18 +0000 UTCgood job, now batch processing can work well, the efficiency has increased by 8 times
楠 陈
2024-09-25 10:03:25 +0000 UTCfixed with v22 Batch size fixed for newer Alpha One app but i felt like it slowed down for RTX 3090 on 4-bit mode
Furkan Gözükara
2024-09-25 09:09:22 +0000 UTCPrompt: Write a very long descriptive caption for this image in a formal tone. Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation. Prompt: Write a very long descriptive caption for this image in a formal tone. Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation. cuda:0 status: 1824 processed, 0 skipped, 2541 left, average speed 8.77 s/image, last batch speed 11.88 s/image, estimated time 371 min 33 seconds
楠 陈
2024-09-25 08:37:11 +0000 UTCMy graphics card is A6000, I tried the maximum batch size of 32, but it still processes one by one, and the video memory uses 18.5GB. The previous version can process 32 batches at the same time.
楠 陈
2024-09-25 08:36:15 +0000 UTCbatch processing does i tested yesterday. can you give me details? you try batch size?
Furkan Gözükara
2024-09-25 08:02:51 +0000 UTCBatch processing does not work。 (Alpha One)
楠 陈
2024-09-25 03:22:50 +0000 UTCthanks, look like my account got hacked. It was sending out steamgift
Ngoc Ngo
2024-09-24 22:02:54 +0000 UTCban revoked
Furkan Gözükara
2024-09-24 21:55:32 +0000 UTCmy account is ahyeahoo
Ngoc Ngo
2024-09-24 21:50:04 +0000 UTCi only banned accounts that were hacked. can you tell me your discord username?
Furkan Gözükara
2024-09-24 20:36:03 +0000 UTCI think I got banned. I was a member of your discord since 2023. I have not posted anything in your discord for a long time.
Ngoc Ngo
2024-09-24 20:22:08 +0000 UTCi think the developer used clip model is not very high resolution but give it a try
Furkan Gözükara
2024-09-24 20:08:17 +0000 UTCThank you. I'll download that and try that. I assume it may now see things a bit more detailed, maybe with better text reading.
cool1
2024-09-24 20:05:08 +0000 UTCjust removed forced resize and now clip_processor is used to resize / process - use v21
Furkan Gözükara
2024-09-24 19:35:01 +0000 UTCThanks a lot. I wasn't sure if that was just because the model was trained at that resolution and aspect ratio or if it could handle higher ones and if so what it could handle (or whether it was done for processing speed/lower memory usage reasons).
cool1
2024-09-24 18:33:05 +0000 UTCwow good catch i hadnt looked it. now i will fix it
Furkan Gözükara
2024-09-24 18:12:41 +0000 UTCit is here : https://discord.com/servers/software-engineering-courses-secourses-772774097734074388
Furkan Gözükara
2024-09-24 18:11:42 +0000 UTCI can't find your discord. What happened?
Ngoc Ngo
2024-09-24 15:19:29 +0000 UTCThough one thing I meant is there's a line in the code in what seems to be a function for generating captions that says "image = Image.fromarray(input_image).resize((384, 384), Image.LANCZOS)" It seems like that's resizing the image to 384x384 pixels before it captions them. If it is that might make some things harder for it to see/interpret (such as text, or maybe the alligator's eyes being visible etc).
cool1
2024-09-24 14:58:43 +0000 UTCi find that 1536 x 1536 works best. smaller resolutions gave me worse results
Furkan Gözükara
2024-09-24 14:53:57 +0000 UTCI installed the new one and renamed my pre alpha folder. Though when I tried using the one in the pre alpha folder later it said it was missing "gradio". I don't know what did wrong there but I can use the pre alpha one for testing (against the Alpha One version) that's in the new Alpha One folder, so I should be able to remove the old folder. Does the captioning need to be done at certain lowish size and certain aspect ratio? In one place in the app when trying to generate the caption it seems to resize the image to 384x384 pixels? Does it do that for all images to caption them? When I tried captioning a picture frame (1028x1280 so taller than wide, but not that much) it said "a photograph of a black square set within a wooden frame". So it thought it was square. But I tried editing it to make it a lot taller and it correctly knew it was rectangular and gave rough dimensions (maybe it was how it was edited). Also if it resizes all images to 384x384 won't that effect the text it can read within images (wouldn't OCR be easier at higher image sizes if it was trained at that)? Is their model trained at 384x384 and no higher? Or is 384x384 resize only in some circumstances?
cool1
2024-09-24 12:15:34 +0000 UTCyes some people said same. so we have both apps :)
Furkan Gözükara
2024-09-24 01:42:04 +0000 UTCI think the new version has too many illusions and is not much better than the old version
楠 陈
2024-09-24 01:34:53 +0000 UTCgood strategy and test both
Furkan Gözükara
2024-09-23 15:28:34 +0000 UTCThanks. Yes. I'll install the newer one in a different folder so I can try the new one but also keep trying the old one too with the same images and see which seems mostly best. Hopefully it will get better over time. Maybe there will be more ways to direct it to tell it how you want the captions, or more ways to tell it about what the images contain. It's also in US English (or the pre alpha was) but I type captions/prompts in UK English (though maybe doesn't make much difference since similar words are probably related in the neural net).
cool1
2024-09-23 15:24:36 +0000 UTCi agree with you. i think over time it will get better but currently not a definitive winner we still have both older and newer version so you can use either
Furkan Gözükara
2024-09-23 15:17:54 +0000 UTCthank you. not that i know at the moment. we already load some parts onto cpu. did you try the app? maybe this will work with 4bit quantization
Furkan Gözükara
2024-09-23 15:16:52 +0000 UTCHi, teacher!! First of all, thanks for all of your tutorials on youtube and for all your patreon posts, they are definitely eyes-opening, amazing content 👏🏻👏🏻👏🏻 Is there any way I can reduce the vram usage requirement? I have an 8GB GPU, therefore, I've been testing llama 8B through an app that lets the model run on VRAM + CPU + RAM. Is there anything similar to this process that I can use with JoyCaptioning? Thank you so much for your attention, have a nice day!!! 😉
Igor Matheus Bassitt Turco
2024-09-23 14:09:12 +0000 UTCThanks. Hopefully it's better than the Pre Alpha versions, though I'm not totaly sure. On Reddit one person said it hallucinates more then the previous one. Hopefully with their increased training it's it will be able to better describe stuff, and there are more options for how it outputs the text. Though a few things I noticed: Max new tokens - in the new one (Alpha One) there's a selection box where it gives the option of 260 and it says "you can force up to 260" but the old one allowed a slider - where your captions examples on the old one show "300". It seems like they've reduce the choice now. They allowed more tokens before (you could select 400 if you wanted, now it's up to 260 it seems), and having a slider allowed you to be more specific but a selection box will only give a limited choice. I preferred the more specific slider (if there's other things in the selection box then maybe there could be a separate thing for them and have the more accurate slider too). In the example with the rose, the Alpha One version doesn't say the rose is CGI, but the Pre-Alpha one does, yet it does seem CGI, so the Pre-Alpha one seems more correct. In the alligator/crocodile one (not sure which is correct), the Alpha One (new) version says "The alligator's eyes are not visible in this close-up view" but that's incorrect. They are visible. The Pre-Alpha versions doesn't mention its eyes but also doesn't incorrectly say they're not visible. For the dragon, maybe the Alpha One is better as it mentions the teeth when the Pre Alpha one doesn't. though Alpha One says the tail is coiled, when it isn't really. It's slightly curved. The first images, with the cats, Pre Alpha says "sherlock holmes inspired". I'm not sure either of their clothes are really Sherlock Holmes inspired. Pre Alpha doesn't say it's CGI but the Pre Alpha does (and the cats are CGI). The image of "The Rock". The Alpha One says his name when the Pre Alpha doesn't, so they've added some famous people to the captions, but Alpha One doesn't specifically mention that it's a man, whereas pre-alpha one does (and saying that might help the training). The Pre Alpha says "dark blue suit" (which seems correct), the Alpha One says "dark grey, tailored suit". The RGB colours show dark blue is more accurate. So I'm not totally sure which is really more accurate, whether it's true that the Alpha One does hallucinate more, and I prefer the more specific slider of the old version (but don't know the other options there). But the more outpost options for text format might be good but I don't know if I'd actually use them yet (whether they'd help training).
cool1
2024-09-23 08:39:42 +0000 UTC
