[ComfyUI]Flux：Molmo基于Qwen2开源视觉大模型，性能媲美GPT4V

大家好！今天我要向大家介绍一个超级酷炫的视觉大模型——[ComfyUI]Flux的Molmo模型！这个模型基于Qwen2开源视觉大模型，性能媲美GPT4V，为你的创意视觉作品带来前所未有的震撼效果！

想象一下，你有一个强大的视觉创意团队，他们能够理解你的创意想法，然后帮你把它们变成现实。这就是[ComfyUI]Flux的Molmo模型在视觉创作中的作用。无论你是想创作一幅画…它们都能够轻松实现。

在这里插入图片描述

而且，[ComfyUI]Flux的Molmo模型的性能非常出色，能够为你提供媲美GPT4V的视觉创作体验。这意味着你可以更加自由地发挥创意，创作出令人惊叹的视觉作品。

所以，如果你对视觉创作充满热情，或者想要让你的创意更加独特和吸引人，那就赶紧试试[ComfyUI]Flux的Molmo模型吧！它将会给你带来无尽的惊喜和乐趣！
)

Molmo LLM大模型简介

Molmo大模型是由艾伦人工智能研究所（Ai2）基于国产大模型Qwen2大模型开发的一系列开放的多模态LLM模型。模型是在一个包含100万张精心策划的图文对的PixMo数据集上进行训练微调的。Molmo是一款完全开源且业界领先的LLM多模态视觉模型，具有最先进的性能。其中Molmo 7B-D模型是基于Qwen2-7B构建的，并使用OpenAI CLIP作为视觉骨干网络。在LLM基准测试和人类评估中的表现在GPT-4V和GPT-4o之间。

• 模型权重：https://huggingface.co/allenai/Molmo-7B-D-0924
• 4位量化模型权重：https://huggingface.co/cyan2k/molmo-7B-D-bnb-4bit
• 在线体验：https://molmo.allenai.org/blog

Molmo LLM大模型性能评估

Molmo系列不仅在开放性和数据质量上超越其他模型，性能也与GPT-4o、GPT-4V、Gemini 1.5 Pro、Claude 3.5等闭源模型相媲美。具体性能评估如下所示：

Molmo大模型ComfyUI体验

首先需要在ComfyUI中通过管理器中搜索ComfyUI-Molmo插件并安装。模型和依赖都会在首次运行节点时自动下载和安装所需的依赖项。注意，初始下载权重文件较大，需要等待一段时间。

• ComfyUI插件地址：https://github.com/CY-CHENYUE/ComfyUI-Molmo
• 模型如无法下载，则可以通过网盘下载并将模型放置ComfyUI/models目录。

（需要的同学可自行扫描获取）

在这里插入图片描述

Flux 文生图工作流

Molmo反推工作流

注意：

• 插件核心参数：image: 需要描述或分析的输入图像；prompt_type: 选择"Describe"进行一般描述或"Detailed Analysis"进行更全面的分析；custom_prompt: 可选。如果提供则将会覆盖选定的prompt_type；seed: 用于可重现性的种子；max_new_tokens: 生成的最大标记数；temperature: 控制生成的随机性；top_k: 限制下一个词选择的词汇表；top_p: 采样参数；unload_model_after_generation: 选择是否在生成后自动卸载模型以释放GPU内存。
• Molmo不擅长处理透明图像，目前建议在将图像传递给模型之前，为图像添加白色或深色背景。

01. 洗漱台

This image is a creative and surreal illustration that combines elements of a bathroom sink with a beach scene. The style is artistic and imaginative, blending reality with fantasy. The theme appears to be about the contrast between the mundane and the extraordinary, or perhaps the idea of " cleanly " washing away one’s problems. The scene depicts a bathroom sink that has been transformed into a miniature beach paradise. Compositionally, the image is centered around the sink and faucet, which serve as the focal point. The beach scene is contained within the sink basin, creating a striking juxtaposition. The lighting is well-executed, with shadows cast by the people on the beach and the hands at the bottom of the image, adding depth and realism to the scene. Additional notable information: 1. The image is tagged with “Yeo_John_Miller” and an Instagram logo, suggesting it’s likely a piece of digital art shared on social media. 2. The level of detail is impressive, from the texture of the sand to the reflections in the water. 3. The image plays with perspective, showing a top-down view of the sink while also depicting a beach scene within it. 4. There are subtle details like the soap bottles and towel in the background, which add to the bathroom setting. Overall, this image is a clever and visually striking piece of art that challenges our perceptions and invites the viewer to imagine stepping into this fantastical scene.

在这里插入图片描述

02.凤凰

此处反推描述相对精确，只因Flux底模缺少对应凤凰训练数据集。

This image is a digital illustration in a fantasy style, depicting two phoenixes in a circular composition. The theme is clearly related to fire and rebirth, as phoenixes are mythical birds known for rising from their own ashes. The scene shows two phoenixes facing each other in a yin-yang-like arrangement, their wings outstretched. The phoenix on the left is predominantly yellow with orange accents, while the one on the right is mostly orange with yellow highlights. This color variation creates visual interest and balance in the composition. The background transitions from dark brown in the corners to a lighter brown in the center, with a hint of yellow in the upper right. This gradient effect enhances the fiery atmosphere and draws the viewer’s attention to the central figures. The phoenixes’ feathers are depicted with numerous triangular shapes, giving them a detailed and textured appearance. Their heads feature long, feathered mohawks, adding to their mythical and dramatic look. The overall composition is symmetrical and well-balanced, with the phoenixes’ bodies forming a circle and their tails curling around to meet at the bottom. This creates a sense of unity and continuity in the image. Additional notable information: 1. The illustration is entirely digital, showcasing the artist’s skill in creating a fantasy scene using digital tools. 2. The level of detail in the feathers and the use of color gradients demonstrate the artist’s attention to detail and ability to create depth and atmosphere. 3. The image’s style and subject matter would be well-suited for use as a book cover, game art, or as part of fantasy-themed artwork. This illustration effectively combines elements of mythology, nature, and digital artistry to create a visually striking and thematically rich representation of phoenixes.

03.莲花

This image is a highly detailed, photorealistic illustration of a flower, likely a lotus, rendered in a style reminiscent of 1980s computer graphics. The flower is depicted with multiple layers of petals, creating a sense of depth and complexity. The petals exhibit a gradient of colors, transitioning from green at the base to blue and white towards the tips, with some gold accents visible in the center. The flower is set against a gradient background that shifts from light gray at the bottom to darker gray at the top, with a slight blue tint in the upper left corner. This background creates a subtle shadow effect beneath the flower, enhancing the overall depth of the image. The composition is centered, with the flower dominating the frame and occupying nearly the entire height and about 70% of the width. This arrangement draws the viewer’s focus to the intricate details of the flower itself. Lighting plays a crucial role in the image, with a light source appearing to come from the top right, casting shadows to the left. This lighting setup accentuates the three-dimensional nature of the flower and adds to its visual appeal. Additional notable information includes:1. The image has a square format, being approximately as tall as it is wide. 2. There are no people, animals, buildings, or mechanical objects present in the image. 3. The overall style is a blend of photographic realism and digital artistry, with a slightly cartoonish quality reminiscent of 1980s computer graphics. 4. The image is well-lit and in sharp focus, contributing to its photorealistic appearance. This illustration combines elements of botanical art with digital techniques to create a visually striking and detailed representation of a lotus flower, evoking a