CLIPTextEncode Node with BLIP

Dependencies

• Fairscale>=0.4.4 (NOT in ComfyUI)

• Transformers==4.26.1 (already in ComfyUI)

• Timm>=0.4.12 (already in ComfyUI)

• Gitpython (already in ComfyUI)

Local Installation

Inside ComfyUI_windows_portable\python_embeded, run:

python.exe -m pip install fairscale

And, inside ComfyUI_windows_portable\ComfyUI\custom_nodes\, run:

git clone https://github.com/paulo-coronado/comfy_clip_blip_node

Google Colab Installation

Add a cell anywhere, with the following code:

!pip install fairscale
!cd custom_nodes && git clone https://github.com/paulo-coronado/comfy_clip_blip_node

Acknowledgement

The implementation of CLIPTextEncodeBLIP relies on resources from BLIP, ALBEF, Huggingface Transformers, and timm. We thank the original authors for their open-sourcing.

A variation of my prompts to gif script that can add new values; I included a very lewd sample prompt file to get you started.

For the sample you need my concept-cupping_hands-v1 style-dagashi_(daga2626)-v1 concept-penis_on_face-v3 and concept-face_poser-v4 models, and they need to be named those names, or you need to update the sample prompts with whatever you named the models when you downloaded them.

Usage

extract the script to your webui/scripts directory

load up the webui, scroll down to scripts

select Prompts from templated file or textbox with GIF generation

Replacement Text is the replacement text; that is what gets replaced in your files, so, for example if I have,

1girl, 1boy, [crying:happy], pov, streaming tears, [handjob:(handjob:0.9):0.2], kneeling, seiza, licking penis, from above, <lora:concept-penis_on_face-v3:0.4>, penis on face, cheek press, [cum on tongue, cum on hair,:saliva, (saliva trail:1.1):0.1], <lora:concept-face_poser-v4:0.4>, [open mouth:(closed mouth:1.1):0.1], [tongue out,::0.1] [uvula, teeth, upper teeth, lower teeth,::0.1] <lora:style-dagashi_(daga2626)-v1:0.2>, dagashi \(daga2626\), <lora:concept-cupping_hands-v1:0.3>, cupping hands, prompt

and enter, prompt as my Replacement Text then prompt will be replaced by whatever is in the Prompt box at the top, where you would normally enter the prompt.

You can either upload the text file or simply copy paste all the prompts into the List of prompt inputs box; they're functionally the same.

Main purpose of this is to make it easier to share neat animation configurations. So! Share your results!

comfy_translation_node

Gratitude model：https://civitai.com/models/10415/3-guofeng3

For more details, please visit：https://github.com/laojingwei/comfy_translation_node

Description

• can be used on ComfyUI interface node for translation, chinese-english translation, translation youdao and Google API support, a variety of options for your choice;

• CN2EN: Support: translation api switch, whether to open translation, Chinese-English switch, embeddings selection, embeddings weight adjustment;

• Tweak Keywords CN2EN: You can tweak the translated content, making it more flexible to tweak your keywords

Download

git clone git clone https://github.com/laojingwei/comfy_translation_node.git

ZIP download

Position installation and operation before use

1. Place the downloaded folder comfy_translation_node under ComfyUI\custom_nodes

2. If you want to start ComfyUI is used when you want the browser (just want to use the default browser can skip this step), you can edit openIE. TXT (path: PATH field in ComfyUI\custom_nodes\comfy_translation_node\openIE.txt), add the corresponding browser.exe execution file path to it, for example: PATH="C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe", other fields (*PATHOLD, SAVE*) never move, no matter what value they are do not change! If the ComfyUI code update fails, then you need to reset the other two fields to *PATHOLD=""* and *SAVE="FALSE"*

3. Go back to the ComfyUI update folder ComfyUI_windows_portable\update and execute the three files in order. Because this plugin requires the latest code ComfyUI, not update can't use, if you have is the latest (2023-04-15) have updated after you can skip this step

4. Go to the root directory and double-click run_nvidia_gpu.bat (or run_cpu.bat) to start ComfyUI. Note that if you did step 2 above, you will need to close the ComfyUI launcher and start again, because the first startup only initialized the browser path, which cannot be read. You will need to start again to open the browser you want. In the future, as long as you do not update the ComfyUI code, you will be able to open your browser every time. If you have an update later, you need to update the code twice before the service takes effect

Instructions for use

1. Node addition method: You can find utils by double-clicking into search and output related words or right-click to find utils and click in to search. Both of these nodes can be directly connected to the Text of CLIP Text Encode. CLIP Text Encode accepts text entry, and Text entry can be right-click on CLIP Text Encode. Find the Convert text to input, select it and the text entry will appear on the left side. (This is very important because many nodes can open the text entry in this way.)

2. CLIP Text Encode CN2EN

Text input field: Enter keywords

"language": 'AUTO' will not be translated, the original text will be output, 'CN' will be translated into Chinese (note that due to the translation api, please ensure that it is pure English before being translated into Chinese), 'EN' will be translated into English (Chinese and English can be mixed)

"transAPI": 'YOUDAO' uses Youdao api to translate,'GOOGLE' uses Google api to translate (Google call time is slow, generally about 2 seconds, sometimes will call failure, it is recommended to use Youdao translation, speed is very fast, but different translation api translated content is a little different, Which one you choose depends on your preference)

"log": 'CLOSE' does not print logs on the console,'OPEN' prints logs on the console

"embeddings": 'none' is not used. Other: Select the model you want (if there is no model in the embeddingsStrength folder, embeddings and EmbeddingsStrength are not displayed)

"embeddingsStrength": it sets the weight,

Tweak Keywords CN2EN

1. Display the input content with CLIP Text Encode CN2EN to display the translated content; Due to the limitations of the translation api, there may be some problems with the translated format, you can correct it here if necessary; If you don't think you want to translate certain words, you can edit specific words, which can make your keywords more perfect and produce better pictures If the tweak_keywords_CN2EN node cannot view content after the ComfyUI code is updated, check whether the folder tweak_keywords_CN2EN exists in the ComfyUI\web\extensions path first, If yes, you can decompress tweak_keywords_CN2EN.zip (path: ComfyUI\custom_nodes\comfy_translation_node\ tweak_Keywords_cn2en.zip). Manually add it to ComfyUI\web\extensions (it is not usually overwritten, I give you a zip pack just in case)

Thanks you for using it this script to make prompts to and connect A1111 SD to Oobabooga by impactframes simply put on the Script directory inside your A1111 webui folder and have the correct ports and flags as in the video

These are my current flags on Ooga --model alpaca-native-4bit --model_type llama --wbits 4 --groupsize 128 --auto-devices --chat --extension api sd_api_pictures send_pictures elevenlabs_tts --no-stream --listen-port 7861 and these

on A1111 --api --always-batch-cond-uncond --no-half-vae --opt-sdp-attention --listen

This custom node pack provides various model-based detection nodes and a detailer node that recreates mask areas in high resolution. It also offers simple inpainting assistant functions such as a mask editor.

Please refer to the GitHub page for more detailed information.

https://github.com/ltdrdata/ComfyUI-Impact-Pack

Install guide:

• Download

• Uncompress into ComfyUI/custom_nodes

• Restart ComfyUI

Troubleshootings:

• Occasionally, when a new parameter is created in an update, the values of nodes created in the previous version can be shifted to different fields. This can result in unintended results or errors if executed as is, so it is important to check the node values. In particular, when updating from version v1.4 or earlier to version v1.5, all parameter values under guide_size will be different, so be careful when using previously created workflows.

Updates:

v1.6.1

• bug fix for DetailerForEachDebug

v1.6

• MaskPainter, EmptySEGS node added

• bug fix for MaskToSEGS

v1.5

• guide_size_for parameter added

• more predictable upscale size patch

• crop_factor on ONNXDetectorForEach node

v1.4

• guide_size bug fix

• ONNXLoader, ONNXDetectorForEach nodes added

v1.3

• MaskToSEGS node added.
  

v1.2

• Support external_seed for Seed node of WAS node suite.
  

v1.1

• Fixed a package dependency issue with pycocotools on Windows.

• Resolved an issue where the software was unable to recognize the "ComfyUI" folder in certain cases.

ChatGPT Prompter v1.1

#### 1st Part #### [paste this part first]

- Reference guide of what is Stable Diffusion and how to Prompt -

Stable Diffusion is a deep learning model primarily designed for generating images based on text descriptions. Besides its primary function, it can also be applied to inpainting, outpainting, and image-to-image translations guided by text prompts.

Developing a good prompt is crucial for generating high-quality images. A good prompt should be detailed and specific, incorporating keyword categories such as subject, medium, style, artist, website, resolution, additional details, color, and lighting. Combining these keywords generates a more accurate representation of the desired image. Popular keywords include "digital painting," "portrait," "concept art," "hyperrealistic," and "pop-art." Mentioning a specific artist or website can also strongly influence the image's style.

For example, to create an image of Emma Watson as a sorceress, a prompt could be: "Emma Watson as a powerful mysterious sorceress, casting lightning magic, detailed clothing, digital painting, hyperrealistic, fantasy, surrealist, full body." This detailed prompt helps the model generate an image more accurately aligned with the desired outcome.

Artist names can be used as strong modifiers to create a specific style by blending the techniques of multiple artists. For instance, combining the styles of Stanley Artgerm Lau, a superhero comic artist, and Alphonse Mucha, a 19th-century portrait painter, results in a unique fusion of their artistic approaches.

Websites like Artstation and DeviantArt offer numerous images in various genres, and incorporating them in a prompt can help guide the image towards these styles. Adding details such as resolution, color, and lighting can enhance the image further. For instance, specifying keywords like "highly detailed," "sharp focus," "iridescent gold," "cinematic lighting," and "dark" can lead to a more vivid and visually striking image.

Building a good prompt is an iterative process. Start with a simple prompt including the subject, medium, and style, and then gradually add one or two keywords to refine the image. Using a universal negative prompt can help avoid unwanted elements.

In the AUTOMATIC1111 GUI, you can adjust the weight of a keyword using the syntax (keyword: factor), where a factor less than 1 means less important and larger than 1 means more important. Alternatively, you can use the () and [] syntax to increase or decrease the keyword's strength.

Keyword blending or prompt scheduling can be achieved using the syntax [keyword1: keyword2: factor]. The factor, a number between 0 and 1, determines at which step keyword1 is switched to keyword2. This technique allows for a smooth transition between two different keywords, creating a unique blend in the final image.

Keyword blending can generate similar images with slight edits. However, the factor must be carefully adjusted. The overall composition of the image is determined by the early diffusion process, with keyword changes only affecting a small part of the image.

Stable Diffusion service may have a limit on the number of keywords used in a prompt. In the basic Stable Diffusion v1 model, the limit is 75 tokens. Tokens are numerical representations of words. AUTOMATIC1111, however, has no token limits, processing chunks of 75 tokens independently.

Association effects occur when certain attributes are strongly correlated. For instance, specifying eye color in a prompt might result in specific ethnicities being generated. Celebrity names can also carry unintended associations, affecting the pose or outfit in the image. Artist names, too, can influence the generated images.

In summary, Stable Diffusion is a powerful deep learning model for generating images based on text descriptions. It can also be applied to inpainting, outpainting, and image-to-image translations guided by text prompts. Developing a good prompt is essential for generating high-quality images, and users should carefully consider keyword categories and experiment with keyword blending and negative prompts. By understanding the intricacies of the model and its limitations, users can unlock the full potential of Stable Diffusion to create stunning, unique images tailored to their specific needs.

--

You do not need to respond. Just use this information as a reference for the task I will ask you to do after.

#### 2nd Part #### [paste this part after]

Stable Diffusion is an AI art generation model similar to DALLE-2. Below is a list of prompts that can be used to generate images with Stable Diffusion.

- Examples -

"masterpiece, best quality, high quality, extremely detailed CG unity 8k wallpaper, The vast and quiet taiga stretches to the horizon, with dense green trees grouped in deep harmony, as the fresh breeze whispers through their leaves and crystal snow lies on the frozen ground, creating a stunning and peaceful landscape, Bokeh, Depth of Field, HDR, bloom, Chromatic Aberration, Photorealistic, extremely detailed, trending on artstation, trending on CGsociety, Intricate, High Detail, dramatic, art by midjourney"

"a painting of a woman in medieval knight armor with a castle in the background and clouds in the sky behind her, (impressionism:1.1), ('rough painting style':1.5), ('large brush texture':1.2), ('palette knife':1.2), (dabbing:1.4), ('highly detailed':1.5), professional majestic painting by Vasily Surikov, Victor Vasnetsov, (Konstantin Makovsky:1.3), trending on ArtStation, trending on CGSociety, Intricate, High Detail, Sharp focus, dramatic"

"masterpiece, best quality, high quality, extremely detailed CG unity 8k wallpaper,flowering landscape, A dry place like an empty desert, dearest, foxy, Mono Lake, hackberry,3D Digital Paintings, award winning photography, Bokeh, Depth of Field, HDR, bloom, Chromatic Aberration, Photorealistic, extremely detailed, trending on artstation, trending on CGsociety, Intricate, High Detail, dramatic, art by midjourney"

"the middle plan, a car, in the steampunk style, cinematographic, (((hdr))), crafted, elegant, meticulous, magnificent, maximum details, extremely hyper aesthetic, intricately detailed"

"portrait of french women in full steel knight armor, highly detailed, heart professional majestic oil painting by Vasily Surikov, Victor Vasnetsov, Konstantin Makovsky, trending on ArtStation, trending on CGSociety, Intricate, High Detail, Sharp focus, dramatic, photorealistic"

"digital painting of majestic waves crashing against a tall lit lighthouse in the distance at dawn, sea, clouds, bokeh, detailed landscape, moon, wind, fog, seagulls flying, lines by conrad roset, brush strokes by yoji shinkawa, colors by makoto shinkai, watercolor by haccan, trending on artstation"

"teen girl in hoodie sitting on the floor on the street, by Dustin Nguyen, Akihiko Yoshida, Greg Tocchini, Greg Rutkowski, Cliff Chiang, 4k resolution, detailed render, post-processing, hyperdetailed face, face at camera, epic background"

"(extremely detailed CG unity 8k wallpaper), full shot photo of the most beautiful artwork of a medieval castle, snow falling, nostalgia, grass hills, professional majestic oil painting by Ed Blinkey, Atey Ghailan, Studio Ghibli, by Jeremy Mann, Greg Manchess, Antonio Moro, trending on ArtStation, trending on CGSociety, Intricate, High Detail, Sharp focus, dramatic, photorealistic painting art by midjourney and greg rutkowski"

"(masterpiece), (extremely intricate:1.3), (realistic), portrait of a girl, the most beautiful artwork in the world, (medieval armor), metal reflections, upper body, outdoors, intense sunlight, far away castle, professional oil painting of a stunning woman by Ed Blinkey and Atey Ghailan and Jeremy Mann and Greg Rutkowski, detailed, sharp focus, dramatic, award winning, matte drawing, cinematic lighting, octane render, unreal engine, volumetrics dtx"

"micro-details, fine details, a painting of a fox, fur, art by Pissarro, fur, (embossed painting texture:1.3), (large brush strokes:1.6), (fur:1.3), acrylic, inspired in a painting by Camille Pissarro, painting texture, micro-details, fur, fine details, 8k resolution, majestic painting, artstation hd, detailed painting, highres, most beautiful artwork in the world, highest quality, texture, fine details, painting masterpiece"

"(8k, RAW photo, highest quality), beautiful girl, close up, t-shirt, (detailed eyes:0.8), (looking at the camera:1.4), (highest quality), (best shadow), intricate details, interior, (ponytail, ginger hair:1.3), dark studio, muted colors, freckles"

"(dark shot:1.1), epic realistic, broken old boat in big storm, illustrated by herg, style of tin tin comics, pen and ink, female pilot, art by greg rutkowski and artgerm, soft cinematic light, adobe lightroom, photolab, hdr, intricate, highly detailed, (depth of field:1.4), faded, (neutral colors:1.2), (hdr:1.4), (muted colors:1.2), hyperdetailed, (artstation:1.4), cinematic, warm lights, dramatic light, (intricate details:1.1), complex background, (rutkowski:0.66), (teal and orange:0.4), (intricate details:1.12), hdr, (intricate details, hyperdetailed:1.15)"

"Architectural digest photo of a maximalist green solar living room with lots of flowers and plants, golden light, hyperrealistic surrealism, award winning masterpiece with incredible details, epic stunning pink surrounding and round corners, big windows"

- Explanation -

The following elements are a description of the prompt structure. You should not include the label of a section like "Scene description:".

Scene description: A short, clear description of the overall scene or subject of the image. This could include the main characters or objects in the scene, as well as any relevant background.

Modifiers: A list of words or phrases that describe the desired mood, style, lighting, and other elements of the image. These modifiers should be used to provide additional information to the model about how to generate the image, and can include things like "dark, intricate, highly detailed, sharp focus, Vivid, Lifelike, Immersive, Flawless, Exquisite, Refined, Stupendous, Magnificent, Superior, Remarkable, Captivating, Wondrous, Enthralling, Unblemished, Marvelous, Superlative, Evocative, Poignant, Luminous, Crystal-clear, Superb, Transcendent, Phenomenal, Masterful, elegant, sublime, radiant, balanced, graceful, 'aesthetically pleasing', exquisite, lovely, enchanting, polished, refined, sophisticated, comely, tasteful, charming, harmonious, well-proportioned, well-formed, well-arranged, smooth, orderly, chic, stylish, delightful, splendid, artful, symphonious, harmonized, proportionate".

Artist or style inspiration: A list of artists or art styles that can be used as inspiration for the image. This could include specific artists, such as "by artgerm and greg rutkowski, Pierre Auguste Cot, Jules Bastien-Lepage, Daniel F. Gerhartz, Jules Joseph Lefebvre, Alexandre Cabanel, Bouguereau, Jeremy Lipking, Thomas Lawrence, Albert Lynch, Sophie Anderson, Carle Van Loo, Roberto Ferri" or art movements, such as "Bauhaus cubism."

Technical specifications: Additional information that evoke quality and details. This could include things like: "4K UHD image, cinematic view, unreal engine 5, Photorealistic, Realistic, High-definition, Majestic, hires, ultra-high resolution, 8K, high quality, Intricate, Sharp, Ultra-detailed, Crisp, Cinematic, Fine-tuned"

- Prompt Structure -

The structure sequence can vary. However, the following is a good reference:

[Scene description]. [Modifiers], [Artist or style inspiration], [Technical specifications]

- Special Modifiers -

In the examples you can notice that some terms are closed between (). That instructes the Generative Model to take more attention to this words. If there are more (()) it means more attention.

Similarly, you can find a structure like this (word:1.4). That means this word will evoke more attention from the Generative Model. The number "1.4" means 140%. Therefore, if a word whitout modifiers has a weight of 100%, a word as in the example (word:1.4), will have a weight of 140%.

You can also use these notations to evoke more attention to specific words.

- Your Task -

Based on the examples and the explanation of the structure, you will create 5 prompts. In my next requests I will use the command /Theme [ description of the theme]. Then, execute your task based on the description of the theme.

--

Acknowledge that you understood the instructions.

#### Example ####

User: /Theme: the beginning of the spring

ChatGPT:

Prompt 1: A majestic 8K landscape of a forest in full bloom, with vivid and bright colors, featuring a small stream that flows peacefully through the center of the scene. (Photorealistic:1.3), (Vivid:1.4), (Highly Detailed:1.2), art inspired by Maxfield Parrish and Thomas Kinkade.

Prompt 2: A digital painting of a garden full of blooming flowers in the early morning light, with dewdrops glistening in the sunlight. (Pastel colors:1.5), (Soft light:1.2), (Detailed:1.3), (Macro:1.2), inspired by Monet's water lilies, Van Gogh's Sunflowers, and Hokusai's cherry blossoms.

Prompt 3: A photo-realistic rendering of a butterfly emerging from a chrysalis on a branch, with an intricate and detailed macro view of the butterfly's wings. (Macro:1.5), (Photorealistic:1.3), (Highly Detailed:1.2), inspired by the illustrations of Ernst Haeckel and the photography of Levon Biss.

Prompt 4: A stunning photograph of a cherry blossom tree in full bloom with a clear blue sky as the background. (Bokeh:1.4), (Vibrant colors:1.2), (Sharp focus:1.2), (High resolution:1.3), art by National Geographic, Ansel Adams, and Yousuf Karsh.

Prompt 5: An oil painting of a picturesque countryside landscape, featuring rolling hills, blooming flowers, and a rainbow in the distance. (Impressionism:1.4), (Plein Air:1.3), (Light brushstrokes:1.2), (Highly detailed:1.2), inspired by the works of Claude Monet, John Singer Sargent, and Vincent Van Gogh.

#### Put ChatGPT back on track ####

As these two prompts are very big, ChatGPT tends to get confused. One additional prompt that I used to get ChatGPT back on track is:

Remember: - Your Task - Based on the examples and the explanation of the structure, you will create 5 prompts. In my next requests, I will use the command "/Theme [ description of the theme]". Then, execute your task based on the description of the theme.

--

Note: 5 prompts

--

/Theme: photo of an old woman reflecting on her life

Of course, that "photo of an old woman reflecting on her life" is just an example. You have to put here the Theme you want.

Positive prompt : 1girl, bangs, black_shirt, blush, bodystocking, breasts, covered_navel, crop_top, hand_on_own_cheek, hand_on_own_face, indoors, lamp, lantern, large_breasts, long_hair, long_sleeves, looking_at_viewer, navel, pantyhose, purple_eyes, purple_hair, sidelocks, solo, thighs, very_long_hair, window
Negative prompt: (worst quality, low quality:1.4), monochrome, zombie, realistic, real, censored, EasyNegative, text, logo, signature, watermark, disfigured, weird hands, broken fingers, extra fingers, missing fingers, dog, merge, bad_artist
Steps: 40, Sampler: DPM++ 2S a Karras, CFG scale: 7, Size: 512x512, Model: AnythingV5V3_v5PrtRE, Denoising strength: 0.75