Introduction

Image synthesis has seen remarkable advancements in recent years, with FLUX leading the charge. Developed by Black Forest Labs, this model builds on the foundations of Stability AI’s work, pushing the boundaries of prompt accuracy and image detail. Unlike earlier models like Stable Diffusion and MidJourney, FLUX introduces a hybrid architecture and enhanced training techniques that significantly improve performance, especially in complex scenes. In this article, we dive into how FLUX revolutionizes image synthesis and why it’s a game-changer for both commercial and personal projects.

Introduction to FLUX

We’ve talked a lot about the potential of Deep Learning Image Generation on the Caasify Blog. These tools aren’t just fun to use, they’re also super intuitive and have become one of the most widely accessible AI models out there for the public. In fact, they’re probably the second most socially impactful deep learning tech, right after Large Language Models.

For the past couple of years, Stable Diffusion—the first publicly available, fully functional image synthesis model—has totally taken over the AI image generation space. We’ve looked into competitors like PixArt Alpha/Sigma, and even researched models like AuraFlow. But honestly, none of these have really made the same impact as Stable Diffusion has. Stable Diffusion 3 is still one of the best open-source models around, and many in the AI world are still trying to match its success.

But then, everything changed just last week with the release of FLUX from Black Forest Labs. FLUX is a huge leap forward in image synthesis technology, offering some serious upgrades in areas like prompt understanding, object recognition, vocabulary expansion, writing capabilities, and a ton of other factors that help boost its performance.

In this guide, we’ll break down what little information we know about the two open-source FLUX models, FLUX.1 schnell and FLUX.1-dev, before the FLUX team releases their official research paper. We’ll also walk you through how to set up and run FLUX on a Cloud Server with an NVIDIA H100 GPU, so you can get hands-on with its advanced capabilities.

Read more about image generation models and their capabilities FLUX Image Synthesis: A Comprehensive Guide.

FLUX Model Overview

FLUX was created by the Black Forest Labs team, which mainly consists of engineers who used to work at Stability AI. These engineers were directly involved in the creation of some groundbreaking models, including VQGAN, Latent Diffusion, and the Stable Diffusion model suite. Although not all the details about FLUX’s development are available, the team has shared some important insights into its model architecture and training process.

All public FLUX.1 models are based on a “hybrid architecture of multimodal and parallel diffusion transformer blocks, scaled to 12B parameters.” This sophisticated design was created to enhance the model’s ability to generate high-quality images from text prompts. FLUX was trained using a method called flow matching, which is different from traditional diffusion methods. It uses something called Continuous Normalizing Flows, and this approach has been shown to produce “consistently better performance than alternative diffusion-based methods, in terms of both likelihood and sample quality.” This means FLUX can generate more accurate and higher-quality images.

In addition to this unique training method, FLUX includes rotary positional embeddings and parallel attention layers. These features help improve the model’s hardware efficiency and overall performance, especially when handling complex inputs or large datasets.

This is the extent of the available information about how FLUX improves on traditional Latent Diffusion models. Luckily, the team has announced that they will soon release an official technical report that will dive deeper into FLUX’s architecture and functionality. In the meantime, we can get more qualitative and comparative insights through the team’s official statements, which also shed light on how FLUX compares to other leading models.

The main goal of releasing FLUX is to “define a new state-of-the-art in image detail, prompt adherence, style diversity, and scene complexity for text-to-image synthesis.” To reach this goal, the FLUX team has released three versions of the model: Pro, Dev, and Schnell. Each version has a different level of accessibility and performance. The FLUX.1 Pro model is available only through an API, while FLUX.1 Dev and FLUX.1 Schnell are open-sourced to different extents, offering users more flexibility when using the model.

A comparison of the performance of these versions, based on their ELO (ranking) scores, shows that each of the FLUX models is on par with some of the best-performing models available, both open-source and closed-source, in terms of output quality. This means that FLUX doesn’t just excel at understanding text prompts, but it also handles complex scenes and creates highly detailed images.

Let’s take a closer look at the differences between these versions:

• FLUX.1 Pro: This is the highest-performing version of FLUX. It offers top-tier image synthesis capabilities that beat even Stable Diffusion 3 Ultra and Ideogram in key areas like prompt following, image detail, quality, and output diversity. As the flagship model, FLUX.1 Pro is ideal for users who need the best possible results and are okay with accessing it through an API.
• FLUX.1 Dev: FLUX.1 Dev is a more efficient, open-weight, guidance-distilled model designed for non-commercial use. It was distilled directly from FLUX.1 Pro and offers nearly the same level of performance, but in a more optimized form. It’s the most powerful open-source model for image synthesis, and while it’s available for free on platforms like HuggingFace, its license restricts use to non-commercial purposes.
• FLUX.1 Schnell: Schnell is FLUX’s fastest model, built for local development and personal use. Unlike the other versions, Schnell can generate high-quality images in just four steps, making it one of the quickest image generation models out there. This makes it perfect for users who want speed without compromising image quality. Like FLUX.1 Dev, Schnell is available on HuggingFace, and you can find its inference code on GitHub if you want to try it out directly.

The Black Forest Labs team has identified five key traits for evaluating image generation models: Visual Quality, Prompt Following, Size/Aspect Variability, Typography, and Output Diversity. According to their ELO ranking, both the FLUX Pro and Dev models outperform other major image generation models, including Ideogram, Stable Diffusion 3 Ultra, and MidJourney V6, in every category. Additionally, FLUX models are designed to handle a wide range of resolutions and aspect ratios, making them some of the most versatile image synthesis tools available.

All in all, the release of FLUX represents a big leap forward in text-to-image synthesis, offering models that shine in both performance and flexibility.

Read more about the advancements in image synthesis models like FLUX FLUX Model Advancements and Comparison to Other Top Image Synthesis Tools.

FLUX Versions: Pro, Dev, Schnell

The release of FLUX is a big deal in the world of image generation, and it’s all about “defining a new state-of-the-art in image detail, prompt adherence, style diversity, and scene complexity for text-to-image synthesis.” Black Forest Labs is really aiming high with this one! To hit this bold target, they’ve released three different versions of FLUX: Pro, Dev, and Schnell. Each version is crafted to meet different user needs, offering varying levels of performance and accessibility. The FLUX.1 Pro model is available only through an API, while the FLUX.1 Dev and FLUX.1 Schnell versions are open-sourced to different degrees, giving more flexibility to users who want to get their hands dirty and work directly with the models.

Looking at the performance data—check out the plot if you can—it’s clear that each version of FLUX holds its own, often matching or even outdoing the top models available, whether they’re closed or open-source. In other words, FLUX is great at understanding what you type in and can create really complex scenes, giving users high-quality image synthesis for a wide range of use cases.

Now, let’s break down the key differences between these FLUX versions:

• FLUX.1 Pro: This is the powerhouse of the bunch. The FLUX.1 Pro version is the top-tier model in the FLUX family, representing the cutting edge of image synthesis. It goes above and beyond even other high-performance models like Stable Diffusion 3 Ultra and Ideogram when it comes to things like following prompts, detail accuracy, image quality, and the diversity of output. If you’re someone who needs the absolute best for image generation—whether for professional or enterprise-level applications—this model is your go-to.
• FLUX.1 Dev: Next, we have FLUX.1 Dev, which is an open-weight, guidance-distilled version of FLUX.1 Pro, specifically designed for non-commercial use. It was distilled directly from FLUX.1 Pro, so it still offers almost identical performance in terms of image generation, but it’s much more efficient and optimized. If you’re a developer or researcher looking for high-quality outputs but you’re working within non-commercial constraints, FLUX.1 Dev is your best friend. You can find the model’s weights on HuggingFace, but just remember, its license restricts its use to non-commercial projects.
• FLUX.1 Schnell: Schnell is the speedster of the FLUX family. This model is made for local development and personal use, and here’s the kicker: it can generate high-quality images in just four steps. Yeah, you read that right—four steps! That makes it one of the fastest image generation models out there, perfect for users who need quick results without losing quality. Like FLUX.1 Dev, Schnell is available on HuggingFace, and its inference code is available on GitHub for anyone who wants to dive in and use it directly.

To measure how well these image generation models are performing, Black Forest Labs has come up with five key traits: Visual Quality, Prompt Following, Size/Aspect Variability, Typography, and Output Diversity. According to the ELO ranking, both FLUX.1 Pro and FLUX.1 Dev outperform other popular models like Ideogram, Stable Diffusion 3 Ultra, and MidJourney V6 in every one of these categories. That’s pretty impressive! It shows that FLUX is great at generating high-quality images that stick closely to your prompts while also offering lots of visual variety.

Plus, FLUX models are designed to handle a wide range of resolutions and aspect ratios. That means they’re super versatile and can create images that work for various formats—from the usual 1024×1024 images to more specialized ones for print or digital media.

In a nutshell, the FLUX family represents an incredibly powerful set of tools for image generation, pushing the limits of what we can do in text-to-image synthesis. Whether you need the high-performance FLUX.1 Pro or the fast and efficient FLUX.1 Schnell, FLUX gives you plenty of options to match your needs. It’s a win for anyone who’s serious about image synthesis.

Read more about the different FLUX model versions and their performance differences in this detailed comparison FLUX Versions: Pro, Dev, Schnell.

FLUX Demo Setup

To run the FLUX demos for the schnell and dev models, the first thing you need to do is set up a GPU-powered cloud server, either from Caasify or any other cloud service provider you prefer. For the best performance, you’ll want to go with a server that has either an H100 or A100-80G GPU. These GPUs are more than capable of handling the heavy load that FLUX requires. If you don’t have access to those, the A6000 GPU should work just fine as well. If you’re new to setting up cloud servers, no worries—just check out your cloud provider’s documentation for all the steps on how to get started with provisioning a GPU server and setting up SSH access.

Setup Process

Once your cloud server is up and running, and you’ve successfully configured SSH access, you’re going to need to log into your server. After that, head over to the directory where you want to set up the FLUX demo. The Downloads folder is a common choice, but really, you can use any folder you want.

Now, from within your chosen directory, go ahead and clone the official FLUX GitHub repository onto your server. You can do this by running the following command:

$ cd Downloads
$ git clone https://github.com/black-forest-labs/flux
$ cd flux

Once the repository is cloned and you’re in the flux directory, it’s time to set up the demo environment. You’ll start by creating a new virtual environment. This will help keep all the dependencies isolated and won’t mess with any other Python environments you’ve got running on your system. Just run these commands to set it up:

$ python3.10 -m venv .venv
$ source .venv/bin/activate

After that, you’ll need to install the dependencies for FLUX. To do that, run this:

$ pip install -e ‘.[all]’

The installation might take a few minutes depending on your server speed and internet connection, but once it’s done, you’ll be almost there.

HuggingFace Login

Before you can actually run the demo, you need to log into HuggingFace to access the FLUX models. This step is super important because the models are hosted on HuggingFace, and you’ll need proper authentication to use them. If you haven’t done it already, head over to the FLUX development page on HuggingFace. There, you’ll need to agree to their licensing terms in order to use the models. If you’re just planning to use the schnell model, you can skip this step.

Once you’ve agreed to the licensing terms, go to the HuggingFace tokens page and create or refresh a new “Read” token. Then, with that token, run this command:

$ huggingface-cli login

You’ll be prompted to enter the token, and once you do, it will authenticate your session. That’ll allow the FLUX models to be downloaded to your server’s HuggingFace cache, so they’re ready for the demo.

Starting the Demo

With everything set up, it’s time to get the demo started! To begin, you’ll need to run the appropriate Python script for the model you want to use. You have two options: the schnell model and the dev model. Here are the commands to start each demo:

schnell demo

$ python demo_gr.py –name flux-schnell –device cuda

dev demo

$ python demo_gr.py –name flux-dev –device cuda

We recommend starting with the schnell demo. This version is much faster and more efficient right out of the gate, so you’ll get quicker results. In our experience, the dev model might need a bit more fine-tuning and tweaking before it works perfectly. Schnell, on the other hand, can take full advantage of FLUX’s capabilities from the start.

Once you execute the script, the demo will start running. During this time, the models will be downloaded onto your machine’s HuggingFace cache. It’ll take around five minutes for each model (schnell and dev) to download. After that, you’ll get a public Gradio link to interact with the demo in real time. If you prefer, you can open the demo locally in your browser using the Core Machine desktop view.

And that’s it! With everything set up, you’re ready to start experimenting with FLUX’s amazing ability to generate high-quality images. Enjoy!

Read more about setting up and running the FLUX demo with detailed instructions and setup steps FLUX Demo Setup Guide.

Running the FLUX Demo

The FLUX demo is super easy to use, all thanks to Gradio’s simple and user-friendly interface. When you open the demo, you’ll notice a prompt entry field right at the top left. This is where you’ll type in the description of the image you want the model to generate. Both FLUX models (schnell and dev) are pretty solid at processing text prompts, so feel free to get creative and try out all sorts of fun and wild combinations of terms to see how the model handles them.

For the dev model, there’s also an “image-to-image” feature that lets you give it an image along with your description. But, here’s the thing—it doesn’t work as smoothly as you might hope. From our testing, the model had a hard time mapping the objects from the input image onto the new prompt, so the connection between the image elements and the generated output wasn’t super strong. Hopefully, this will improve with future updates, but for now, it’s really best used for simpler image-to-image tasks.

The demo interface also has an optional toggle for “Advanced Options.” These options let you take the reins and have more control over the image generation process. You can tweak the height, width, and number of inference steps, which will affect both the quality of the image and how long it takes to generate. For the schnell model, the guidance value is set to 3.5, which helps ensure a balanced level of detail and coherence in the generated images. On the other hand, the dev model lets you adjust this value, so you’ve got more flexibility if you want to fine-tune the output.

Another cool feature in the demo is the ability to control the “seed” value. What’s the seed, you ask? Well, it’s a parameter that lets you reproduce previously generated images. By changing the seed, you can get the same image again and again, keeping the results consistent. This is really handy if you want to compare different versions of an image or fine-tune your prompt for better results.

Once you’ve filled in all the fields and adjusted the parameters to your liking, you’re ready to generate an image. Here’s an example of a prompt you might use:

Prompt: “robot fish swimming in a digital ocean robotic aquarium coral microchips patterns logo spells ‘Flux Image Generation with Caasify'”

After you enter that prompt and adjust everything to your liking, the model will produce an image based on the description. You can play around with variations in the prompt, adjust things like the number of inference steps, and change the seed to see how it affects the final image. The process is pretty straightforward, and the Gradio interface makes it easy to experiment and fine-tune the generated images for whatever you’re working on.

For a comprehensive guide on running the FLUX demo and optimizing your image synthesis results, check out this detailed FLUX Demo Setup Guide.

First Impressions with FLUX

We’ve spent about a week testing out FLUX, and let me tell you, the results have been pretty impressive! It’s easy to see why this model has picked up steam so quickly after its release. The utility and progress it brings to image generation are pretty significant. We’ve experimented with a lot of different artistic tasks, focusing mainly on the schnell model. Let me walk you through some of the examples we worked with:

Prompt: “Travel poster depicting a group of archaeologists studying the white bones of a giant monster in a blue sandy desert on an alien planet with pink plants and an orange sky, 3 suns. Bordered caption spells ‘Discover the hidden past! Come to Rigel-4!'”

The model did an amazing job capturing the majority of the details from the prompt. The landscape, with its alien desert and cool color palette, turned out stunning. However, the people and dog in the scene seemed a bit out of place, with an uncanny valley vibe, especially when it comes to how they were blended into the image. Oh, and the word “Rigel” in the caption ended up being misspelled as “Rigler” in the bottom corner. Still, despite those small quirks, the overall result was a fantastic representation of the prompt.

Prompt: “Advertisement ad in a magazine, hand-painted by Norman Rockwell, featuring a 1950s style family home living room, a small boy playing with a humanoid robot on the floor, a floating television set, and retrofuturistic decor. The caption reads ‘Skeltox Robotics: For The Whole Family!'”

In this case, the goal was to capture Norman Rockwell’s iconic style. The model did a decent job with the scene, but we noticed that the text in the ad was just nonsense – not readable at all. And the absence of a subtitle in the ad made it feel a little incomplete. But the composition of the scene was spot on, especially the lighting and those retrofuturistic elements, which looked great.

Prompt: “Lego figurines and lego animation, featuring a lego next to a toybox. The box logo spells ‘James’ (plastic). The figurine has short auburn red hair, a thin frame, a mustache, wearing a t-shirt, shorts, athletic shoes, and holding an acoustic guitar, a Coca-Cola bottle, and a soccer ball. There are also stacks of books, with the figurine holding a book and reading.”

Now this one was a bit trickier, with multiple objects and lots of detail. The model captured most of the key things, but there were a few hiccups. For example, the figurine didn’t have its shorts or Coca-Cola bottle, and instead of holding the book as described, it was holding the guitar. It seems like the model had a hard time juggling multiple objects in one image, which led to these small mistakes. But honestly, it still did a pretty great job of representing the prompt, and the accuracy of the description made it a desirable final output.

Prompt: “3D Pixar-style animation, featuring a cute and adorable cartoon cactus ninja.”

Finally, we decided to go with a simple prompt, and boy, did it deliver! The image of the cute cactus ninja turned out fantastic, and it was exactly what we were hoping for. Interestingly, since the prompt was pretty straightforward, there were fewer artifacts in the image. This makes me think that FLUX might actually perform better with simpler prompts – the less complex the request, the clearer the results.

So, after this round of testing, it’s clear that FLUX can handle a wide variety of creative prompts with impressive detail and accuracy. However, there’s still room for improvement, especially when it comes to handling complex compositions with multiple objects. But overall, FLUX – especially the schnell version – has proven itself to be a powerful tool for generating high-quality and creative images across a broad range of prompts.

To dive deeper into the impressive capabilities and first impressions of FLUX, check out this insightful FLUX First Impressions Review.

Tips for Using FLUX

Prompting for Text Prompt: “Coral forest underwater sea. The word ‘Caasify’ is painted over it in big, blue bubble letters.”

Getting text to appear in an image generated by FLUX can be a bit tricky. There isn’t a special word or symbol that FLUX automatically recognizes to generate text. But don’t worry, there are ways to improve your chances of getting that perfect text into your image. One of the easiest tricks is to put the text you want in quotation marks in your prompt. It also helps to be super clear about the text you want to appear. So, instead of just saying “text,” try something like “the word ‘Caasify’ is painted over it in big, blue bubble letters.” This simple change will definitely help FLUX get the text right.

General Prompt Engineering

FLUX is seriously intuitive compared to older versions of diffusion models. If you’ve ever used other models like Ideogram or MidJourney, you’ll quickly notice that FLUX understands your prompts without needing much tweaking or extra effort. But hey, there are a few things you can do to make sure you get the best results possible.

Here’s the thing: the order of the words in your prompt really matters. Putting the main subject at the start helps FLUX understand what to focus on. And using commas to separate different parts of the prompt is a huge help. FLUX, like a human, needs punctuation to understand where one idea ends and another begins. Fun fact: commas actually carry more weight in FLUX than they did in Stable Diffusion. This means that using them well can lead to better accuracy in the generated image.

Now, a little heads up: there’s a bit of a trade-off between the level of detail you put in and the final image. The more words you use, the more accurate the prompt will be, but the model might struggle to add extra details or objects into the image. For example, if you just want to change someone’s hair color, you can do that with just one word. But if you want to change their whole outfit, you’ll need to add more detail. And be careful with that – too many specifics might mess with FLUX’s ability to get the scene exactly right. So, it’s all about finding the right balance between detail and simplicity.

Aspect Ratios

FLUX has been trained on a wide variety of image sizes, ranging from 0.2 to 2 MegaPixels. But, based on our experience, it really shines when you use specific resolutions. For example, FLUX does an awesome job with images at 1024 x 1024 or higher. But when you use 512 x 512 images, they might look a little flat, even though the pixel size is smaller.

We also found that some resolutions just work better than others. For instance, these specific ones tend to give you great results:

• 674 x 1462 (this one matches the iPhone’s typical screen ratio of 9:19.5)
• 768 x 1360 (a standard default resolution)
• 896 x 1152
• 1024 x 1280
• 1080 x 1920 (this is a popular one for wallpapers)

These resolutions tend to give you cleaner, more detailed images with fewer weird glitches, so they’re solid choices when you’re generating images with FLUX.

For more practical tips on optimizing your FLUX experience, check out this helpful guide to mastering FLUX prompt engineering.

Conclusion

In conclusion, FLUX represents a groundbreaking advancement in image synthesis, offering improved prompt accuracy and the ability to generate highly detailed images. By leveraging hybrid architecture and innovative training techniques, FLUX outperforms previous models like Stable Diffusion and MidJourney in both prompt adherence and scene complexity. With its open-sourced Dev and Schnell versions, FLUX provides a versatile solution for both personal and non-commercial use. As image generation technology continues to evolve, FLUX is poised to set new standards, helping creators and developers unlock even more powerful possibilities in visual content creation. Keep an eye on FLUX as it paves the way for the future of text-to-image synthesis.

Unlock High-Fidelity Image Synthesis with Fooocus and Stable Diffusion