javafx - Controller Stage Freezing Display Stage - Stack Overflow

admin管理员组

文章数量:1122846

I have a project that uses a controller stage to manipulate and color a display stage (which has an ImageView with a WritableImage). The problem I have been running into is that the display stage will freeze up before it has finished the task of coloring the entire image. It appears as though the problem is with the second stage popping up, since whenever it is removed and the display stage is filled automatically, it loads without a problem - except for the problem that there is no way to control the display stage.

Here are some results from playing around with this:

1. Although clicking the play button once hardly ever works, clicking it multiple times usually gets the display to load.

2. If the button is clicked a few times and the display still hasn't loaded, selecting the display stage will sometimes load it in.

3. Showing the display or control stage first and using one or the other as the given primaryStage makes no difference.

4. If either stage is put into another thread or task, it doesn't reach the show statement, crashing whenever it hits the new stage decleration.

   public void start(Stage display_stage) { // Given "primaryStage"
   // Create a basic ImageView to be colored
   WritableImage image = new WritableImage(1920, 1080);
   PixelWriter writer = image.getPixelWriter();
   ImageView view = new ImageView(image);
   
   // Set up display stage
   display_stage.setScene(new Scene(new Pane(view), 1920, 1080));
   
   // Set up play button with coloring action
   Button play_button = new Button("PLAY");
   play_button.setOnAction(e -> {
       new Thread( new Task<Void>() {
           @Override
           protected Void call() {
               // Iterate through every pixel and color it arbitrarily
               for (int y = 0; y < 1080; y++)
                   for (int x = 0; x < 1920; x++)
                       writer.setColor(x, y, Color.rgb(0, 0, 255));
   
               return null;
           }
       }).start();
   });
   
   // Set up control stage
   Stage control_stage = new Stage();
   control_stage.setScene(new Scene(play_button, 100, 100));
   
   // Show stages
   control_stage.show();
   display_stage.show();
   

   }

This is the reduced code - I am using IntelliJ IDEA if that matters. Any pointers are welcome - thanks in advance.

EDIT: I should mention that this is not simply about performing and completing a short task for an image. This for-loop should be expected to continue for a long period of time or indefinitely (unless stopped by the controller). This is why the control stage is needed and why the display must be updating repeatedly.

I have a project that uses a controller stage to manipulate and color a display stage (which has an ImageView with a WritableImage). The problem I have been running into is that the display stage will freeze up before it has finished the task of coloring the entire image. It appears as though the problem is with the second stage popping up, since whenever it is removed and the display stage is filled automatically, it loads without a problem - except for the problem that there is no way to control the display stage.

Here are some results from playing around with this:

1. Although clicking the play button once hardly ever works, clicking it multiple times usually gets the display to load.

2. If the button is clicked a few times and the display still hasn't loaded, selecting the display stage will sometimes load it in.

3. Showing the display or control stage first and using one or the other as the given primaryStage makes no difference.

4. If either stage is put into another thread or task, it doesn't reach the show statement, crashing whenever it hits the new stage decleration.

   public void start(Stage display_stage) { // Given "primaryStage"
   // Create a basic ImageView to be colored
   WritableImage image = new WritableImage(1920, 1080);
   PixelWriter writer = image.getPixelWriter();
   ImageView view = new ImageView(image);
   
   // Set up display stage
   display_stage.setScene(new Scene(new Pane(view), 1920, 1080));
   
   // Set up play button with coloring action
   Button play_button = new Button("PLAY");
   play_button.setOnAction(e -> {
       new Thread( new Task<Void>() {
           @Override
           protected Void call() {
               // Iterate through every pixel and color it arbitrarily
               for (int y = 0; y < 1080; y++)
                   for (int x = 0; x < 1920; x++)
                       writer.setColor(x, y, Color.rgb(0, 0, 255));
   
               return null;
           }
       }).start();
   });
   
   // Set up control stage
   Stage control_stage = new Stage();
   control_stage.setScene(new Scene(play_button, 100, 100));
   
   // Show stages
   control_stage.show();
   display_stage.show();
   

   }

This is the reduced code - I am using IntelliJ IDEA if that matters. Any pointers are welcome - thanks in advance.

EDIT: I should mention that this is not simply about performing and completing a short task for an image. This for-loop should be expected to continue for a long period of time or indefinitely (unless stopped by the controller). This is why the control stage is needed and why the display must be updating repeatedly.

• javafx
• imageview
• task
• controls
• stage

Share Improve this question Follow edited Nov 24, 2024 at 21:29 ResidentSojourn asked Nov 22, 2024 at 2:28 ResidentSojournResidentSojourn 7755 bronze badges 1

• Just looking at your code, I would guess that the simplest solution is to wrap writer.setColor(x, y, Color.rgb(0, 0, 255)); in Platform.runLater. I think this idea is not a good coding practice. What I would do is create a POJO with x, y, and color. I would then use updateValue to return partial results. See this. – SedJ601 Commented Nov 22, 2024 at 14:51

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 4

Issue: Race Condition

Don't read or write stuff in the active scene graph from another thread.

You are creating a race condition.

A race condition can be difficult to reproduce and debug because the end result is nondeterministic and depends on the relative timing between interfering threads. Problems of this nature can therefore disappear when running in debug mode, adding extra logging, or attaching a debugger. A bug that disappears like this during debugging attempts is often referred to as a "Heisenbug". It is therefore better to avoid race conditions by careful software design.

Specifically, in your case, don't call writer.setColor on a PixelWriter associated with a displayed Image in an ImageView in a Scene when you aren't on the JavaFX thread.

Potential Solutions

A: Don't use concurrency

Don't use concurrency unless the problem is naturally asynchronous or you need to do stuff async in a task for performance reasons (e.g. to perform blocking I/O or very long calculation).

Running processing in parallel for performance reasons may be a premature optimization. You might be able to run your logic on the JavaFX thread. Using a single thread might be easier to write and maintain. The majority of the logic in JavaFX deliberately uses a single thread.

If you know that the logic will complete in < 1/60th of a second, it will run fine on the JavaFX thread without slowing everything down.

For animations, the platform offers tools to support processing on the JavaFX thread. Examples are Animation, Transition, Timeline, and AnimationTimer. All can run your logic at specified intervals, so you don't need another thread to perform most animations.

B: Double buffering

Sometimes concurrency is necessary. If working with pixel data, one technique is to use double buffering.

Fill a buffer in the task and return it from the task result. When it is ready, set the pixels on the image to the value in the buffer using the PixelWriter. That is a kind of double buffer where you are managing the off-screen buffer and the JavaFX system is controlling the on-screen buffer (the displayed Image in the ImageView).

C: Page flipping

Or, you could keep two WritableImages, one active on the scene graph and one written to by your async task. Flip them once the task is completed. Set the image in the displayed ImageView to the one just written to by the task, see the "Page Flipping" technique from the same wiki article as double buffering.

Caveat: Be careful not to issue too many Runnables

As noted in comments by SedJ601

Just looking at your code, I would guess that the simplest solution is to wrap writer.setColor(x, y, Color.rgb(0, 0, 255)); in Platform.runLater.

While this will work, you have to be careful about how you use Platform.runLater. Your code is looping and calling setColor two million times. Flooding the JavaFX processing queue with millions of runnables can have a severely negative impact on performance. This is one of the reasons this answer suggested using filling a buffer or flipping an image instead, even though those are more complex solutions.

---

If the above theoretical discussion answers your questions and you don't need a concrete example or performance analysis, then skip the rest of this answer.

Sample App

Example for buffer flipping and page flipping.

In this instance the buffer flip is quicker, and the image flip is slower.

FlipperApp.java

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;

import java.time.Duration;

public class FlipperApp extends Application {
    private static final int W = 1920, H = 1080;
    public static final int FRAMES_PER_SECOND = 24;

    public void start(Stage stage) {
        WritableImage image = new WritableImage(W, H);
        PixelWriter writer = image.getPixelWriter();
        ImageView view = new ImageView(image);

        BufferFlipperTask flipperTask = new BufferFlipperTask(writer, W, H, Duration.ofSeconds(1).dividedBy(FRAMES_PER_SECOND));
// to try an image flipper comment out the buffer flipper and replace with the image flipper.        
//        ImageFlipperTask flipperTask = new ImageFlipperTask(view, W, H, Duration.ofSeconds(1).dividedBy(FRAMES_PER_SECOND));
        Thread bufferThread = new Thread(
                flipperTask,
                "buffer-processor"
        );
        bufferThread.setDaemon(true);

        stage.setScene(new Scene(new Pane(view), W, H));
        stage.show();

        bufferThread.start();
    }

    public static void main(String[] args) {
        launch(args);
    }
}

BufferFlipperTask.java

import javafx.application.Platform;
import javafx.concurrent.Task;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.paint.Color;

import java.time.Duration;
import java.util.concurrent.FutureTask;

public class BufferFlipperTask extends Task<Void> {
    private static final int BYTES_PER_PIXEL = 4;

    private final ColorCycler colorCycler = new ColorCycler();
    private final PixelWriter writer;
    private final byte[] byteBuffer;
    private final int width, height, scanlineStride;
    private final Duration cycleDuration;

    public BufferFlipperTask(PixelWriter writer, int width, int height, Duration cycleDuration) {
        this.writer = writer;
        this.width = width;
        this.height = height;
        this.scanlineStride = this.width * BYTES_PER_PIXEL;
        this.byteBuffer = new byte[width * height * BYTES_PER_PIXEL];
        this.cycleDuration = cycleDuration;
    }

    @Override
    protected Void call() throws Exception {
        int nCycles = 0;

        while (!isCancelled()) {
            long startCycle = System.nanoTime();

            byte[] colorBytes = toBgra(colorCycler.nextColor());
            System.out.println("Processing color (bgra): " + colorBytesToString(colorBytes));

            long startFill = System.nanoTime();
            for (int y = 0; y < height; y++) {
                int yoff = y * scanlineStride;

                for (int x = 0; x < width; x++) {
                    int off = yoff + x * BYTES_PER_PIXEL;

                    byteBuffer[off + 0] = colorBytes[0];
                    byteBuffer[off + 1] = colorBytes[1];
                    byteBuffer[off + 2] = colorBytes[2];
                    byteBuffer[off + 3] = colorBytes[3];
                }
            }

            long endFill = System.nanoTime();
            System.out.println("Time to fill buffer " + (endFill - startFill) + " (ns)");

            FutureTask<Void> writeBufferTask = new FutureTask<>(() -> {
                long startEmpty = System.nanoTime();
                writer.setPixels(
                        0,0, width, height,
                        PixelFormat.getByteBgraInstance(),
                        byteBuffer,
                        0,
                        scanlineStride
                );
                long endEmpty = System.nanoTime();
                System.out.println("Time to empty buffer " + (endEmpty - startEmpty) + " (ns)");

                return null;
            });

            // write the buffer to the active scene using the JavaFX thread.
            Platform.runLater(writeBufferTask);

            // wait for the buffer to be written using the JavaFX thread.
            writeBufferTask.get();

            long endCycle = System.nanoTime();

            Duration sleepTime = cycleDuration.minusNanos(endCycle - startCycle);
            if (sleepTime.isNegative()) {
                sleepTime = Duration.ZERO;
            }

            System.out.println("Cycle: " + nCycles + ". Time to run cycle " + (endCycle - startCycle) + " (ns). Sleeping " + sleepTime.toMillis() + " (ms)\n");

            Thread.sleep(sleepTime);
            nCycles++;
        }

        return null;
    }

    private static String colorBytesToString(byte[] colorBytes) {
        return "%d:%d:%d:%d".formatted(
                colorBytes[0] & 0xff,
                colorBytes[1] & 0xff,
                colorBytes[2] & 0xff,
                colorBytes[3] & 0xff
        );
    }

    private static byte[] toBgra(Color c) {
        int r = (int) Math.round(c.getRed() * 255.0);
        int g = (int) Math.round(c.getGreen() * 255.0);
        int b = (int) Math.round(c.getBlue() * 255.0);
        int a = (int) Math.round(c.getOpacity() * 255.0);

        return new byte[] { (byte) (b & 0xff), (byte) (g & 0xff), (byte) (r & 0xff), (byte) (a & 0xff) };
    }
}

ImageFlipperTask.java

import javafx.application.Platform;
import javafx.concurrent.Task;
import javafx.scene.image.ImageView;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.paint.Color;

import java.time.Duration;
import java.util.concurrent.FutureTask;

public class ImageFlipperTask extends Task<Void> {
    private final ColorCycler colorCycler = new ColorCycler();
    private final int width, height;
    private final Duration cycleDuration;
    private final ImageView imageView;
    private WritableImage onScreenImage;
    private WritableImage offScreenImage;

    public ImageFlipperTask(ImageView imageView, int width, int height, Duration cycleDuration) {
        this.imageView = imageView;

        onScreenImage = new WritableImage(width, height);
        offScreenImage = new WritableImage(width, height);

        this.width = width;
        this.height = height;
        this.cycleDuration = cycleDuration;
    }

    @Override
    protected Void call() throws Exception {
        int nCycles = 0;

        while (!isCancelled()) {
            long startCycle = System.nanoTime();

            Color nextColor = colorCycler.nextColor();
            System.out.println("Processing color: " + nextColor);

            long startFill = System.nanoTime();
            PixelWriter pixelWriter = offScreenImage.getPixelWriter();
            for (int y = 0; y < height; y++) {
                for (int x = 0; x < width; x++) {
                    pixelWriter.setColor(x, y, nextColor);
                }
            }

            long endFill = System.nanoTime();
            System.out.println("Time to fill buffer " + (endFill - startFill) + " (ns)");

            FutureTask<Void> writeBufferTask = new FutureTask<>(() -> {
                long startEmpty = System.nanoTime();

                imageView.setImage(offScreenImage);

                WritableImage tmp = onScreenImage;
                onScreenImage = offScreenImage;
                offScreenImage = tmp;

                long endEmpty = System.nanoTime();
                System.out.println("Time to empty buffer " + (endEmpty - startEmpty) + " (ns)");

                return null;
            });

            // write the buffer to the active scene using the JavaFX thread.
            Platform.runLater(writeBufferTask);

            // wait for the buffer to be written using the JavaFX thread.
            writeBufferTask.get();

            long endCycle = System.nanoTime();

            Duration sleepTime = cycleDuration.minusNanos(endCycle - startCycle);
            if (sleepTime.isNegative()) {
                sleepTime = Duration.ZERO;
            }

            System.out.println("Cycle: " + nCycles + ". Time to run cycle " + (endCycle - startCycle) + " (ns). Sleeping " + sleepTime.toMillis() + " (ms)\n");

            Thread.sleep(sleepTime);
            nCycles++;
        }

        return null;
    }
}

ColorCycler.java

A utility class to provide a cycling color palette¹.

import javafx.scene.paint.Color;

public class ColorCycler {
    private static final Color[] palette = {
            Color.RED,
            Color.ORANGE,
            Color.YELLOW,
            Color.GREEN,
            Color.BLUE,
            Color.INDIGO,
            Color.VIOLET
    };

    private int nextColorIndex = 0;

    public Color nextColor() {
        Color nextPaletteColor = palette[nextColorIndex];
        updateColorIndex();

        return nextPaletteColor;
    }

    private void updateColorIndex() {
        nextColorIndex = (nextColorIndex + 1) % palette.length;
    }
}

Performance analysis of techniques from the sample app

On my machine:

• The image flip takes about 100ms to run, so it cannot be run at 24 frames per second (setting the color of each pixel on the image takes a lot of time relative to manipulating raw bytes in a buffer).
• The buffer flip takes ~10ms to run so it can be run at 24 frames per second with no problem.

Using a Premultiplied BGRA buffer

The buffer I used for the buffer flip is a Bgra buffer.

The native buffer used on my machine (the type retrieved from the pixelwriter for an image) is a BgraPre buffer that pre-multiplies alpha values.

If you use a BgraPre buffer, then the time reading from the buffer on the JavaFX thread will be lessened from ~5ms to ~1ms. This is because the JavaFX system does not need to convert the pixel data in the buffer to use it. Another system might use a different native pixel format, so this optimization might not apply to all systems.

An example is:

writer.setPixels(
        0, 0, width, height,
        PixelFormat.getByteBgraPreInstance(),
        byteBuffer,
        0,
        scanlineStride
);

If opacity is 1 (i.e. there is no transparency) then this will work fine with the solution provided above. But if the image has pixels with opacity < 1, then you need to apply a premultiplication algorithm when setting the pixels in the buffer. An implementation for premultiplied non-opaque data is beyond the scope of what is covered here.

NIO buffers, arrays and PixelBuffers

The PixelWriter can work with NIO buffers or arrays. In addition, the NIO buffers can be allocated or direct.

Simple arrays as in the examples above performed best. So there is no need to use NIO, as it is slower for this task and has a more complex API.

However, (not used in this example), JavaFX has a different API called a PixelBuffer which can be used when a Writable Image is constructed. A writable image constructed with a pixel buffer has these properties:

The Buffer provided by the PixelBuffer will be used directly as the pixel data for this image. The PixelBuffer can be shared by multiple WritableImages. Images constructed this way are readable using Image.getPixelReader(), but are not writable using WritableImage.getPixelWriter().

So a PixelBuffer is an alternative to using a PixelWriter which is used in the question and this answer. The PixelBuffer likely will offer better performance than using a PixelWriter. The high performance video rendering in vlcj-javafx uses a Writeable Image with a PixelBuffer:

A video surface that uses a PixelBuffer is the recommended way to use vlcj in a JavaFX application.

The PixelBuffer, which uses a native memory buffer, gives the best possible performance. PixelBuffer was introduced with JavaFX 13.

Multi-thread writes to the pixel buffer

If the operation that writes to the buffer can be multi-threaded, you can use multiple threads to do that, which could also improve performance. Often pixel manipulation tasks can be multi-threaded, for example by using a parallel stream:

IntStream.range(0, height).parallel().forEach(y -> {
    int yoff = y * scanlineStride;

    for (int x = 0; x < width; x++) {
        int off = yoff + x * BYTES_PER_PIXEL;

        byteBuffer[off + 0] = colorBytes[0];
        byteBuffer[off + 1] = colorBytes[1];
        byteBuffer[off + 2] = colorBytes[2];
        byteBuffer[off + 3] = colorBytes[3];
    }
});

On my machine, this drops the time to write the pixels from ~5ms to ~1ms.

So, by applying both a BGRApre pixel format and writing to the buffer using parallel streams, the overall update time per frame for a 1920x1080 image is ~2ms and the app (UI interaction) is very responsive.

In comparison, on my machine, to perform the same task on the JavaFX thread using the original code in the question:

for (int y = 0; y < 1080; y++)
    for (int x = 0; x < 1920; x++)
        writer.setColor(x, y, Color.rgb(0, 0, 255));

This code took ~1/2 second per frame (~250 times slower than the optimized version).

---

1. The ColorCycler could be used as part of a color cycling animation implementation by applying the cycled colors to a byte indexed pixel format (the full solution for that is not in this answer).

Task actually contains a very clever mechanism to avoid flooding the FXAT with jobs. It accumulates the changes in an AtomicReference which it updates in the background thread and then clears it out from the FXAT - atomically, so there are no concurrency issues.

Here's the code for it:

    protected void updateValue(V var1) {
        if (this.isFxApplicationThread()) {
            this.value.set(var1);
        } else if (this.valueUpdate.getAndSet(var1) == null) {
            this.runLater(() -> {
                this.value.set(this.valueUpdate.getAndSet((Object)null));
            });
        }
    }

Here valueUpdate is the AtomicReference, and this.value is the ObservableValue which can be seen from outside Task. This is a short method, but it takes a little pondering to see how it works. Essentially, Platform.runLater() only gets called once each time the AtomicReference has been cleared out.

For this use case, you need to have a list of pending PixelWriter updates that are processed on the FXAT. So I implemented it as an ObservableList and then put a Subscription on it to do the calls to PixelWriter on the FXAT.

The AtomicReference is a bit funky because it's a List which means you have to keep replacing it as it grows which potentially might have some performance issues, although I didn't see any. The methodology is pretty much the same as the internal Task.updateValue() logic.

This is Kotlin, but the ideas are exactly the same as Java:

class Example0 : Application() {
    override fun start(stage: Stage) {
        stage.scene = Scene(createContent(), 1800.0, 960.0).apply {
            Example0::class.java.getResource("abc.css")?.toString()?.let { stylesheets += it }
        }
        stage.show()
    }

    private fun createContent(): Region = BorderPane().apply {
        val image = WritableImage(1920, 1080)
        val writer = image.pixelWriter
        val view: ImageView = ImageView(image)
        val pixelList: ObservableList<PixelValue> = FXCollections.observableArrayList<PixelValue>()
        pixelList.subscribe {
            pixelList.forEach { writer.setColor(it.x, it.y, it.color) }
            pixelList.clear()
        }
        top = TextField()
        center = Pane(view)
        bottom = Button("Go").apply {
            onAction = EventHandler {
                isDisable = true
                val task = object : Task<Unit>() {
                    val partialUpdate = AtomicReference<MutableList<PixelValue>>(mutableListOf())
                    override fun call() {
                        for (y in 0..879) for (x in 0..1780) {
                            doUpdate(PixelValue(x, y, Color.rgb(Random.nextInt(255), 0, 255)))
                        }
                    }

                    fun doUpdate(newVal: PixelValue) {
                        if (partialUpdate.getAndUpdate {
                                mutableListOf<PixelValue>().apply {
                                    addAll(it)
                                    add(newVal)
                                }
                            }.isEmpty()) {
                            Platform.runLater {
                                pixelList.addAll(partialUpdate.getAndUpdate { mutableListOf() })
                            }
                        }
                    }
                }
                task.onSucceeded = EventHandler { this.isDisable = false }
                Thread(task).apply { isDaemon = true }.start()
            }
        }
    }
}

data class PixelValue(val x: Int, val y: Int, val color: Color)

fun main() = Application.launch(Example0::class.java)

Note that you don't need multiple Stages or Scenes or any of that. It just works. I added a TextField at the top so that you could type away while the Task is running and see that it doesn't interfere with the UI.

There's no need for buffer flipping or any of that either. PixelWriter has to be call a gazillion times on the FXAT, but it doesn't seem to affect the performance of the GUI to any degree. It could run forever and it doesn't seem to be a practical issue.

本文标签： javafxController Stage Freezing Display StageStack Overflow