FX Playground

21 07 2014

 

 

 

Update: The FX Playground project can be found at: https://bitbucket.org/cdea/fxplayground

Logo5

Introduction

FX Playground is a JavaFX-based prototyping tool or live editor that eliminates the step of compiling Java code. This concept isn’t new, for instance the Web world there are many HTML5 playgrounds that offer online editors that enable developers to quickly prototype or experiment with various JavaScript libraries. This allows the developer to focus on visualizations or UI details without needing to set-up an IDE project or mess with files. Even older (pre-dating) than playgrounds are REPLs (Read Eval Print Loop) where dynamic languages such as Groovy, Python, Ruby, etc. provide an interactive interpreter command line tool to allow developers to quickly script code to be executed. Scala is a compiled language, but also provides a REPL tool.

After finishing the book JavaFX 8 Introduction by Example I noticed each example was created as separate NetBeans projects which seemed a little overkill for small examples. Because the book is based on Java the language each program needed to be compiled (via javac) prior to execution. Larger projects will typically need to be set-up with a proper classpath and resources in the appropriate directory locations. Even larger projects will also need dependencies which typically reside on Maven repositories.

JavaOne 2014

Based on timing I was able to submit a talk regarding JavaFX based playgrounds just in time. After awhile I was pleasantly surprised that my proposal (talk) was accepted. You can check out the session here. Also, I will be presenting with my good friend Gerrit Grunwald (@hansolo_). So, be prepared to see awe-inspiring demos. Since the talk is a BoF (birds of a feather) the atmosphere will be low-key and very casual. I hope to see you there!

The JavaOne talk is titled “JavaFX Coding Playground (JavaFX-Based Live Editor Tool) [BOF2730]“.  Based on the description you’ll find that the tool will be using the NEW! Nashorn (JavaScript) engine to interact with JavaFX primitives. The figure below depicts the FX Playground tool’s editor windows and a JavaFX Display area. Starting clockwise at the lower left is the code editor window allowing the user to use JavaScript (Nashorn) to interact with nodes. Next, is the JavaFX FXML editor window allowing the user to use FXML (upper left). The FXML window is an optional.  In the upper right, you will notice the JavaFX CSS editor window allowing you to style nodes on the display surface. Lastly, to the bottom right is the output area or better known as the DISPLAY_SURFACE.

FXPlayground's editor windows

FXPlayground’s editor windows

FX Playground in Action

Because FX Playground is still in development I will give you a glimpse of some demos that I’ve created on Youtube. The following are examples with links to videos.

 

Roadmap

There are plans to opensource the code, but for now there is much needed functionality before public consumption.

The following features are a work in progress:

  • Make use of FXML editor window.
  • Pop out the display panel into its own window
  • Save, SaveAs, and Load Playgrounds
  • Build software to be an executable for tool users. (90% done)
  • Make the tool capable of using other languages (JSR 223)

I want to thank Oracle corp. especially the following engineers who helped me (some of the engineers below are not Oracle employees):

References

The FX Playground project can be found at: https://bitbucket.org/cdea/fxplayground

CarlFX’s Channel – https://www.youtube.com/channel/UCNBYRHaYk9mlTmn9oAPp1VA

7 of the Best Code Playgrounds – http://www.sitepoint.com/7-code-playgrounds

NetBeans – https://www.netbeans.org

JavaFX 8 Introduction by Example – http://www.apress.com/9781430264606

Nashorn – https://wiki.openjdk.java.net/display/Nashorn/Main

Enzo – https://bitbucket.org/hansolo/enzo/wiki/Home

Harmonic Code – http://harmoniccode.blogspot.com/





JavaFX 8u20 Days of Future Past (Always On Top)

19 05 2014

It’s been a long time since I’ve posted topics relating to JavaFX. So, if you are still following along, awesome!

Introduction

In this blog post I want to blog about a very cool feature starting with JavaFX 8 update 20 that allows your application to always be on top of other applications. What this means is that on your desktop your JavaFX based application can be a floating widget that will remain above all other applications (z-order). An example would be a weather widget in the upper right corner never to be obscured by other applications. So, I’m very excited to share with you this amazing feature ‘Always On Top’.

History

In the past Java Swing developers would rely on the method Window.setAlwaysOnTop(boolean).  This feature allowed Swing developers to build native looking and native behaving desktop applications. Ever since JavaFX 1.x this very feature was highly requested (originally requested by Stephen Chin @steveonjava for the WidgetFX framework). This feature request is JIRA ticket RT-153. Figure 1 is the feature request shown with a status of ‘Resolved’.

Jira ticket RT-153

figure 1: Jira ticket RT-153

 

Although this feature didn’t get into JavaFX versions 1.x, 2.x and the version prior to Java 8u20, it is finally here now. For those who don’t know how to report bugs or file new features requests I encourage you to head over to the JavaFX JIRA system.

Example: Weather Widget

Assuming you know the basics of JavaFX since version 2.0 a typical application would consist of extending from the javafx.application.Application class. When developing JavaFX desktop applications the platform API would provide you with a (javafx.stage.Stage) window. The Stage object will have the following methods to access the always on top property.

  • alwaysOnTopProperty()
  • setAlwaysOnTop(boolean)
  • isAlwaysOnTop()

The following code snippet sets the Stage to be always on top using the method setAlwaysOnTop(boolean).

public class KeyholeDemo extends Application {

   @Override public void start(Stage primaryStage) {
      primaryStage.initStyle(StageStyle.TRANSPARENT);
      primaryStage.setAlwaysOnTop(true);
      // code omitted...
   }

   public static void main(String[] args) {
     launch(args);
   }
}

How it works

Shown in the listing above the start() method sets the stage to be transparent to be a JavaFX window without a title bar. This allows applications to have irregular shaped windows on the desktop. If a web person trolls you again about JavaFX ask them the following question: “Can  HTML5 create irregular shaped windows on the desktop? (always on top)”. Next, the stage is setup to be always on top via the setAlwaysOnTop() method.

Demo

After realizing that this highly requested feature was available I basically updated my old ‘KeyholeWidget‘ project at the Github and created a video demonstrating the widget on my desktop. Here is the video:

To see the full code listing visit the Github project ‘KeyholeWidget‘.

 

I hope you’ve enjoyed this cool feature (I know I did). As usual please leave comments below.

Happy coding,

Carl





JavaOne 2013 BOF2605 JavaFX, Widgets, and Apps, Oh My! Launching Frameworks for Platforms Large and Small

9 09 2013

Image

Multipurpose widget called KeyHole displaying maps and the weather.

Introduction

This is probably the longest title I’d ever create for a blog entry.

In my spare time I’ve been working with Mark Heckler(The Java Jungle), Hendrik Ebbers (GuiGarage), and Gerrit Grunwald (Harmonic Code) on the idea of application launching frameworks. We will be presenting two approaches to running widgets and applications on the desktop and embedded device (Raspberry Pi) environments. The talk will be a BoF (Birds of a feather) session at JavaOne 2013 (BOF2605). This is where the Java/JavaFX community will have lots of fun demoing, discussing and sharing ideas on topics such as:

  • Application Containers / Dashboards
  • Widgets/Apps/Games/Gadgets
  • Widget/App Development workflow
  • Widget/App life cycle
  • Deployment / App stores

To see a sneak preview of Hendrik Ebbers’ implementation of a Roku-AppleTV-like application container check out his blog entry herehttp://www.guigarage.com/2013/08/boxfx-javaone-preview-1/ The application container is called BoxFX.

Mark and I have been working on an implementation of a desktop-centric application container called TrayFX. To see a sneak preview of TrayFX visit an interview with Mark Heckler hosted by Stephen Chin. Another sneak preview of TrayFX in action is on my youtube channel herehttp://www.youtube.com/watch?v=x_KLIzVSiu8 and http://www.youtube.com/watch?v=WwzwsGxd-Jk

In this blog entry I will discuss some high level concepts relating to the framework eWidgetFX and the application container TrayFX. Since there is a lot to digest I would like to at least (briefly) show you each layer in the architecture.

Disclaimer: BoxFX and TrayFX are currently private APIs. Both approaches have similarities, however they are independent projects. It’s important to hear from the audience for potential use cases. They eventually will wind up as open source projects. Also, leading up to the JavaOne talk some diagrams or APIs are subject to change in this blog entry.

eWidgetFX

eWidgetFX is a core library (API) that widgets and app containers will depend on.  Shown below depicts the layers:

High Level Architecture of the core widget framework

Core widget framework API that widgets and widget containers depend on.

The APIs provide services for loading widgets, widget life cycles, and utilities for the widget developer. The main idea was to provide a cross platform application API for application container developers and widget developers to coexist. In fact the goal was to have cross platform widgets & apps capable of running on any app container (based on the eWidgetFX lib).

Application Containers (TrayFX)

TrayFX is an App Container instance that depends on the core widget framework (eWidgetFX). Shown below is the middle layer which is basically different custom application containers based on the core.

High Level Architecture depicting App Containers layer.

Application Containers depend on the eWidgetFX API (core). This allows developers to build their own custom application containers.

TrayFX is an App Container instance that depends on the core widget framework (eWidgetFX). This means that you to can develop your own JavaFX widget & app container of your liking if you don’t like TrayFX :-) For instance you could create one for a tablet device which may have different UI metaphors and varied screen sizes (real estate). Shown below is TrayFX launching a clock widget (on the Desktop).

TrayFX launching a clock widget.

A clock widget launched from TrayFX

Widgets

Widgets or Apps are either standalone applications or eWidgetFX based widgets that can be run inside of a application container. Shown below is the widgets layer:

High Level Architecture of the Widgets layer

Widgets are mini-applications that can launch standalone or launch inside a widget container such as TrayFX.

When creating widgets you can implement a Widget interface or you can extend from the abstract DefaultWidget adapter class overriding optional life cycle methods as a convenience to the developer. Shown below is a high level class diagram:

A hello world widget class extending from the eWidgetFX default widget class.

Widget’s Class Diagram using the eWidgetFX core library.

In the JavaOne BoF  (BOF2605) since the framework is still very alpha during the talk we will present a possible developer workflow to create, build, deploy, and launch widgets.  Below I would like to share a small bit of code to give you an idea of how easy it is to implement a widget on the eWidgetFX platform.

Usually, application launching frameworks have life cycles for the application containers to manage widgets & apps. Widget developers will not need to implement a Stage nor a Scene and basically focus on what they know best (their JavaFX widget as a Node). The framework APIs will allow you to interact with the JavaFX parent containers such as Stage and Scene. The eWidgetFX Widget interface is shown below which has methods that a widget developer would need to implement. For brevity please look at the comments on the methods to understand the basic widget life cycle.

package org.ewidgetfx.core;

import javafx.beans.property.ObjectProperty;
import javafx.beans.property.StringProperty;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;

/**
 * User: cdea
 * Date: 4/7/13
 */
public interface Widget {
    public static enum DECORATION {
        STAGED_OS_TITLE_BAR,
        STAGED_CLOSE,
        STAGED_CONFIG_CLOSE,
        STAGED_CONFIG,
        STAGED_UNDECORATED,
        NON_STAGED_CLOSE,
        NON_STAGED_CONFIG_CLOSE,
        NON_STAGED_CONFIG,
        NON_STAGED_UNDECORATED
    };

    DECORATION getDecoration();
    void setDecoration(DECORATION decoration);

    String getName();
    void setName(String name);
    StringProperty nameProperty();

    String getVersion();
    void setVersion(String version);
    StringProperty  versionProperty();

    String getDescription();
    void setDescription(String descr);

    String getVendor();
    void setVendor(String vendor);

    String getVendorUrl();
    void setVendorUrl(String vendorUrl);

    String getVendorEmail();
    void setVendorEmail(String vendorEmail);

    LaunchInfo getLaunchInfo();
    void setLaunchInfo(LaunchInfo launchInfo);

    WidgetIcon getWidgetIcon();
    void setWidgetIcon(WidgetIcon widgetIcon);
    ObjectProperty widgetIconProperty();

    Pane getAsNode();
    Stage getParentStage();
    void setParentStage(Stage stage);

    WidgetState getWidgetState();

    /**
     * Returns a created WidgetIcon for the app container to use. Called 1st.
     * @return WidgetIcon containing a raw node representing the icon. App containers can resize.
     */
    WidgetIcon buildWidgetIcon();

    /**
     * Called after buildWidgetIcon() method to allow background processes to occur. Called 2nd.
     * Typically to collect data to update WidgetIcon's Icon overlay. An example would be
     * an email widget periodically checking email to update Icon overlay the number of emails received.
     */
    void startBackground();

    /**
     * Initialize the widget. If the developer calls this method the framework will not call it. Called 3rd.
     */
    void init();

    /**
     * Start is meant to be called when the user clicks the icon to launch widget. Called 4th.
     * Typically to start animations or position on the desktop. This should not be confused with startBackground()
 */
 void start();

 /**
 * Pause is typically used to pause animations. Or other developer defined resources.
 */
 void pause();

 /**
 * Resume is typically used to resume a animations. Or other developer defined resources.
 */
 void resume();

 /**
 * Stop is called when the widget is closed and not visible. Typically this is to stop animations, and minor cleanup.
 * The framework will call stop and stopBackground() method when exiting the app container.
 */
 void stop();

 /**
 * Stops any background processes. Called when widget is being closed by the framework when exiting app contain. Called last.
 *
 */
 void stopBackground();

}

Conclusion

While building this framework we kept coming up with more features and ideas, however with very little time left we still were able to provide common capabilities that you’ve grown to love on your device. Hopefully, this will whet your appetite before you attend the BoF at JavaOne.

All are welcome! Enjoy!

Carl





Introduction by Example: JavaFX 8 Printing

15 07 2013

I‘ve not blogged in awhile, and I miss sharing with others about all things JavaFX (My day job and family are likely excuses). For those who are new to this blog, I am the author of JavaFX 2 Introduction by Example (JIBE), co-author of Java 7 Recipes, and technical reviewer of the Pro JavaFX 2 books from Apress publishing. For those who already know me, I’d like to thank you for supporting me and the other authors by purchasing these books. More importantly, my hope is to reach out to  Java enthusiasts and share ideas.

The book JavaFX 2 Introduction by Example, was published in Nov. 2011 and many more APIs were added since then. During the writing of the book, I was working on the early editions of JavaFX 2.0 up until the announcement at JavaOne Oct. 2011. It was pretty crazy trying to update the book based on API changes as things were almost set in stone. I thought it was amazing how it even got out the door. However, I was pretty pleased. Some of you who have read the beginning of the book (JIBE) understand that the chapters of JIBE are also found in the book Java 7 Recipes (actually it is originally taken from Java 7 recipes). This little fact explains why the book, JavaFX 2 Introduction by Example, is reminiscent of recipe or cookbook style technical books. My intent was to help the reader get introduced quickly without a lot of tech blather. Instead of trying to convince people about the JavaFX platform, I’d rather demonstrate things with useful examples. I find it counter productive discussing  deep philosophical debates regarding why one particular technology is superior to the other (cheesy 80’s Highlander reference).

After the release of JavaFX 2.0, there came subsequent versions such as JavaFX 2.1, 2.2 and the upcoming release of JavaFX 8 (January 2014). In this blog entry, I will provide a recipe for the JavaFX 8’s Printing API. Similar to my book (JIBE), I will follow the same pattern as before where I present a problem, solution, code, and a “How it Works” section.

Declaimer: In this blog you will encounter Java Functional Interfaces using Lambda expressions. I will not be discussing them here, but will refer you to Oracle’s tutorials on Project Lambda .

Prerequisite software:

JDK 8 – https://jdk8.java.net/download.html

 

Problem

You want to create a JavaFX application that prints out a visited website.

Solution

Use the JavaFX 8 PrintJob and Printer APIs to print any JavaFX scene graph node. Also, use the WebView and WebEngine APIs to display a Website or Web page.

Instructions

Assuming you’ve compiled and have run the application, follow the instruction below:

  1. Enter website address or url into the text field.
  2. Hit the enter key
  3. After the page is loaded, click on the “Print” button
  4. Go to the printer to get the printed web page

Code

package org.carlfx;

import javafx.application.Application;
import javafx.beans.property.BooleanProperty;
import javafx.beans.property.SimpleBooleanProperty;
import javafx.beans.value.ChangeListener;
import javafx.concurrent.Worker.State;
import javafx.print.*;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.TextField;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.HBox;
import javafx.scene.transform.Scale;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

/**
 * Demo to use JavaFX 8 Printer API.
 *
 * @author cdea
 */
public class PrintDemo extends Application {
    @Override
    public void start(Stage primaryStage) {

        final TextField urlTextField = new TextField();
        final Button printButton = new Button("Print");
        final WebView webPage = new WebView();
        final WebEngine webEngine = webPage.getEngine();

        HBox hbox = new HBox();
        hbox.getChildren().addAll(urlTextField, printButton);
        BorderPane borderPane = new BorderPane();
        borderPane.setTop(hbox);
        borderPane.setCenter(webPage);
        Scene scene = new Scene(borderPane, 300, 250);
        primaryStage.setTitle("Print Demo");
        primaryStage.setScene(scene);

        // print button pressed, page loaded
        final BooleanProperty printButtonClickedProperty = new SimpleBooleanProperty(false);
        final BooleanProperty pageLoadedProperty = new SimpleBooleanProperty(false);

        // when the a page is loaded and the button was pressed call the print() method.
        final BooleanProperty printActionProperty = new SimpleBooleanProperty(false);
        printActionProperty.bind(pageLoadedProperty.and(printButtonClickedProperty));

        // WebEngine updates flag when finished loading web page.
        webEngine.getLoadWorker()
                 .stateProperty()
                 .addListener( (ChangeListener) (obsValue, oldState, newState) -> {
                    if (newState == State.SUCCEEDED) {
                        pageLoadedProperty.set(true);
                    }
                 });

        // When user enters a url and hits the enter key.
        urlTextField.setOnAction( aEvent ->  {
            pageLoadedProperty.set(false);
            printButtonClickedProperty.set(false);
            webEngine.load(urlTextField.getText());
        });

        // When the user clicks the print button the webview node is printed
        printButton.setOnAction( aEvent -> {
            printButtonClickedProperty.set(true);
        });

        // Once the print action hears a true go print the WebView node.
        printActionProperty.addListener( (ChangeListener) (obsValue, oldState, newState) -> {
            if (newState) {
                print(webPage);
            }
        });

        primaryStage.show();

    }

    /** Scales the node based on the standard letter, portrait paper to be printed.
     * @param node The scene node to be printed.
     */
    public void print(final Node node) {
        Printer printer = Printer.getDefaultPrinter();
        PageLayout pageLayout = printer.createPageLayout(Paper.NA_LETTER, PageOrientation.PORTRAIT, Printer.MarginType.DEFAULT);
        double scaleX = pageLayout.getPrintableWidth() / node.getBoundsInParent().getWidth();
        double scaleY = pageLayout.getPrintableHeight() / node.getBoundsInParent().getHeight();
        node.getTransforms().add(new Scale(scaleX, scaleY));

        PrinterJob job = PrinterJob.createPrinterJob();
        if (job != null) {
            boolean success = job.printPage(node);
            if (success) {
                job.endJob();
            }
        }
    }

    /**
     * The main() method is ignored in correctly deployed JavaFX application.
     * main() serves only as fallback in case the application can not be
     * launched through deployment artifacts, e.g., in IDEs with limited FX
     * support. NetBeans ignores main().
     *
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        launch(args);
    }
}
Print Demo using JavaFX 8

Print Demo using JavaFX 8

How it Works

The code begins by creating a TextField, a Button, and a WebView control to be placed into a BorderPane. When using the BorderPane layout, you will be able to place controls into the following regions: Top, Right, Left, Bottom, and Center.

Similar to a web browser, the text field allows the user to enter a website url. Once the url is entered, the user will hit the enter key to load the web page into the WebView node. When placing controls on any of the side regions, the BorderPane layout will take on the preferred height of any controls that are added. The center region will allow a node to take up the available space minus the remaining space taken up by the width and height of the bordering side regions. In other words, if the side regions contain no nodes (empty), a node in the center region has the opportunity to take all the available width and height space provided by its parent (Scene). Since the WebView node will occupy the center region, it will take all the available width and height (minus top region) when the web page is fully loaded. You’ll also notice scroll bars allowing the user to view pages larger than the current view port.

After laying out all the components for the UI, you will need to wire things up. Here you will simply create three boolean property
(javafx.beans.property.SimpleBooleanProperty) instances. The first property variable printButtonClickedProperty is a flag indicating when the print button was clicked. The second property pageLoadedProperty is a flag indicating that the web page was finished loading. Lastly, you will want to note the printActionProperty which binds the printButtonClickedProperty and the pageLoadedProperty by using the fluent API. As they evaluate, the printActionProperty will be true if both the printLoadedProperty and the printLoadedProperty are true values.

// print button pressed, page loaded
final BooleanProperty printButtonClickedProperty = new SimpleBooleanProperty(false);
final BooleanProperty pageLoadedProperty = new SimpleBooleanProperty(false);

// when the a page is loaded and the button was pressed call the print() method.
final BooleanProperty printActionProperty = new SimpleBooleanProperty(false);
printActionProperty.bind(pageLoadedProperty.and(printButtonClickedProperty));

Continuing the wiring up of the UI, I took an event driven approach where the handler code will respond to events and property changes. Starting with the WebView node, I attached handler code to the statePropery instance (ChangeListener) in order that the pageLoadedProperty will be set to true once the web page is loaded successfully.

  // WebEngine updates flag when finished loading web page.
  webEngine.getLoadWorker()
           .stateProperty()
           .addListener( (ChangeListener) (obsValue, oldState, newState) -> {
               if (newState == State.SUCCEEDED) {
                    pageLoadedProperty.set(true);
               }
           });

Next, you will see the text field’s ‘setOnAction‘ method containing handler code that resets the pageLoadedProperty and printButtonClickedProperty objects. Also, the code will initiate the loading of the page via the WebView‘s WebEngine load() method.


  // When user enters a url and hits the enter key.
  urlTextField.setOnAction( aEvent ->  {
     pageLoadedProperty.set(false);
     printButtonClickedProperty.set(false);
     webEngine.load(urlTextField.getText());
  });

After the TextField control’s action code is wired up, the print button will also need handler code to set the printButtonClickedProperty flag to true. Lastly, the printActionProperty property will need a ChangeListener to respond when its state evaluates to true. When this evaluates to true, my print() method is invoked.

        // When the user clicks the print button the webview node is printed
        printButton.setOnAction( aEvent -> {
            printButtonClickedProperty.set(true);
        });

        // Once the print action hears a true go print the WebView node.
        printActionProperty.addListener( (ChangeListener) (obsValue, oldState, newState) -> {
            if (newState) {
                print(webPage);
            }
        });

Finally, the print() method takes a JavaFX Node object to be printed. The Printer object has a method which returns the default printer your computer is set to. Before actually printing, we can derive a default page layout to scale the node before printing the node. If you don’t do this, only part of the web page will be printed. With the default printer obtained, the createPrinterJob() method is invoked to return a PrinterJob instance that does the actual printing. To print a JavaFX displayable type node, you simply invoke the PrinterJob object’s printPage() method by passing in the Node instance as a parameter.

    /** Scales the node based on the standard letter, portrait paper to be printed.
     * @param node The scene node to be printed.
     */
    public void print(final Node node) {
        Printer printer = Printer.getDefaultPrinter();
        PageLayout pageLayout = printer.createPageLayout(Paper.NA_LETTER, PageOrientation.PORTRAIT, Printer.MarginType.DEFAULT);
        double scaleX = pageLayout.getPrintableWidth() / node.getBoundsInParent().getWidth();
        double scaleY = pageLayout.getPrintableHeight() / node.getBoundsInParent().getHeight();
        node.getTransforms().add(new Scale(scaleX, scaleY));

        PrinterJob job = PrinterJob.createPrinterJob();
        if (job != null) {
            boolean success = job.printPage(node);
            if (success) {
                job.endJob();
            }
        }
    }

In conclusion, I find that the APIs are simpler to use compared to Java Swing/AWT APIs. I would like to mention that there are many features you can play around with since this blog entry only scratches the surface on the APIs currently available.

NOTE: JavaFX 8 printer API is still in the early stages and there are still outstanding issues (Jira issues).

Please visit to vote and help test functionality at https://javafx-jira.kenai.com

Outstanding Tickets: http://bit.ly/1dw7Vc8





A Case Study: A JavaFX Widget Framework API

20 04 2013

A prototype of a widget container

A prototype of a widget container


Building a better mousetrap

In short I want to revisit the idea of a cross platform widget framework using JavaFX. In this article I’d like to share some ideas, concepts and prototypes that might invigorate this old but ingenious idea for the desktop and embedded world. An old idea you say? Does anyone remember WidgetFX?

In the early days of JavaFX there were a few Java Champions and community members who decided to create a cross platform JavaFX 1.x widget framework for the desktop (year 2008). I was fortunate enough to know many of those contributors involved.  I grew to love the capabilities the framework and widgets were able to provide.  However, as we all know JavaFX is now redone as Java APIs as opposed to the JavaFX script language. This eventually became apparent regarding to the future of most frameworks back then. Before I begin I would like to give credit to the folks who contributed to the WidgetFX framework project.  I want to thank them for their enthusiasm and their tireless efforts on building the community to what it is today. To see some of their cool widgets please visit the site http://widgetfx.org (catch them before they’re gone). Two of the original founders of the project are Stephen Chin and Keith Combs. Even though I did not partake in the fun during the height of the project I was quite inspired.

Inception phase

To fast forward just last year at the JavaOne 2012 conference I was able to catch up with Keith Combs (Silicon Valley JavaFX User Group organizer) and Stephen Chin (Java Technology Evangelist at Oracle) after his keynote speaking to inquire about the WidgetFX project now that JavaFX 1.x is no more. Steve kindly asked if anyone would be interested in giving it a reboot on the JavaFX 2.x platform (As he peered over towards Keith and me). I believe I said, “I don’t think, so… “. I thought it might be too much to bite off than I could possibly chew. So I politely declined the offer. During the conference I was later enamored by all things Java embedded which later fueled to some of my home automation project ideas using JavaFX with (Arduino, Raspberry Pi, and Beagle Board).

After returning home I was not only excited about the things I’ve learned at the conference but, I also was in the middle of transitioning to a new job and company. Working as a UI and Visualization developer for cloud based monitoring software I often encountered stakeholders/SMEs mentioning the term “SPOG” or single pane of glass.

What does the term “SPOG” mean?

According to Floyd Strimling a cloud technical evangelist at Zenoss explains how so many organizations will create desperate (silos) systems and he goes on to say, “The Holy Grail is a single pane of glass that provides IT managers with a single access point to make sense of their entire underlying infrastructure in real time”. (Searching for the Mythical Single Pane of Glass)

Isn’t “SPOG” an interesting concept for large operation centers? So, I continued to rethink of the old idea of a flexible widget framework that would provide richer interfaces that would allow better desktop integration, decoupled services, higher performance, and capable of empowering users with many of the modern UI metaphors that they’d expect on their favorite OS platform. I believe such a framework should also work on embedded devices for home automation.  Actually I got the idea of a widget framework for embedded devices from my good friend Gerrit Grunwald (Java Champion and author of the popular Steel Series API) while exchanging ideas about graphics performance and launching JavaFX applications on embedded devices.

Wow, with so many different problem domains and so many perspectives could such a widget framework truly exist? Could such a widget framework be able to make everyone happy? Is it possible to create a simple widget framework to rule them all? I believe it is possible.

eWidgetFX is born

In the back of my mind I noticed similar use cases with many customers wanting widget frameworks having this concept of a “SPOG”, but also provide a framework that I could use myself on my own desktop or touch device. So, I kept going back and forth on the possibility of rebooting the old project WidgetFX. Instead I wanted to throw a monkey wrench into the whole business of widget frameworks. Since there are so many kinds of ways to launch, display, and manage widgets it would also be a good idea to build a single widget framework SPI (service provider interface API) which would enable developers to build their own widget containers thus allowing widget developers to run their widgets in any widget container. There would basically be two developer perspectives: Container Developers and Widget Developers. So, I’d like to propose a new API called eWidgetFX (core) that would allow the developers to create widget containers and widgets.

For now I will defer the work of designing the core APIs and make it a future post. So, in this article I’ve decided to generate some prototypes of widget containers that would basically depend on the future eWidgetFX framework API.

Prototyping Widget Containers

Prototyping is probably the most fun part during any software development activity. Shown below are videos I recently uploaded to Youtube.com to kick some ideas around for different kinds of widget containers (Icon App tray) on a desktop environment.

Conclusion

One of the main issue I found in UI development is that customers will often know what they want, but aren’t able to communicate what they ultimately want. It’s almost like playing a game of Charades or a game of Pictionary where guessing is a sought after skill in software design.  We are often faced with providing prototypes and end up settling on one particular view (one view, one way, one silo). The problem became pretty apparent where everyone had an opinion or philosophical stance on a particular GUI interface, usability aspect or UI metaphor. Sometimes it’s based on subjective personal preferences that can go into long discussions. No one person in the room was completely happy, but at the end of the day we compromised and we settled.

I believe that by creating a widget API it would allow organizations to build different single pane of glass instances (widget containers) and capable of sharing all widgets across other panes.

So, imagine seeing different types of widget containers in the wild. Some future widget containers could look like interfaces similar to the following movies: Avengers, Tron, or Minority Report

A final note: Many of of the ideas expressed and proposed are a collective effort from some of my friends (team members) of the JFXtras.org project (Mark Heckler, Hendrik Ebbers, and Gerrit Grunwald).

Would you like to see a talk regarding widget frameworks at JavaOne?

As always comments are welcome.

Carl ;-)

Resources

JavaFX: http://www.oracle.com/technetwork/java/javafx/overview/index.html
Java Champions: https://java.net/website/java-champions/bios.html
WidgetFX: http://widgetfx.org
Keith Combs from the Silicon Valley JavaFX user group: http://www.svjugfx.org/member/10394895/?op=&memberId=10394895
Steve on Java by Stephen Chin: http://steveonjava.com
Arduino : http://www.arduino.cc
Raspberry Pi: http://www.raspberrypi.org
Beagle Board: http://beagleboard.org
Guest Post: Searching for the Mythical Single Pane of Glass: http://siliconangle.com/blog/2012/02/29/guest-post-searching-for-the-mythical-single-pane-of-glass
Anti-patterns: http://en.wikibooks.org/wiki/Introduction_to_Software_Engineering/Architecture/Anti-Patterns
Google Images of Operation centers: http://bit.ly/106p3h0
Avengers UI design: http://cargocollective.com/jayse/Avengers
Tron UI design: http://dlew.me/Tron-Legacy-Boardroom
Minority Report UI designer demos his tech at TED (video): http://www.engadget.com/2010/06/03/minority-report-ui-designer-demos-his-tech-at-ted
The Java Jungle by Mark Heckler: https://blogs.oracle.com/javajungle/entry/welcome_to_the_jungle
Gui Garage by Hendrik Ebbers: http://www.guigarage.com
Harmonic Code by Gerrit Grunwald: http://harmoniccode.blogspot.de
JFXtras.org : http://jfxtras.orgLeap Motion: https://www.leapmotion.comLeap Motion: https://www.leapmotion.com





JavaFX 2 GameTutorial Part 5

26 08 2012
JavaFX sound effects

Figure 1: JavaFX Sound FX

Introduction

This is part five of a six part series related to a JavaFX 2 Game Tutorial. I know it’s been a long time since I blogged about gaming, but hopefully you’re still with me. If you would like a recap, please read Part 1Part 2Part 3, and Part 4 to find out where we left off. If you are up to date, then let’s get started!  In this blog entry we will be incorporating sounds into our game.

There are many elements which make games incredibly fun such as animated effects, collisions, AI, and input. However, one of the most important ingredients to game play is sound. When games incorporate sound effects and music, the gamer will become highly immersed (ugh… like not realizing you are about to see the sun rise). Before we get into the details, let me give you some background history on sound used in PC games for the home computer. If you want to skip the history and get down to business, jump to the section called the ‘Sound Manager Service.’ The Sound Manager Service is responsible for maintaining sound assets used during the game. If you are really impatient and don’t care about the implementation details, jump down to ‘JavaFX Sound Demo.’ Important note: Remember to read the requirements before launching the demo.

History

If you want to understand today, you have to search yesterday.  ~Pearl Buck 

Back in the day, when I was growing up, I learned that the Apple ][ computer was capable of playing sounds. The Apple ][ had one speaker that was only able to produce simple tones (8 bit mono sound). When I first generated tones (Mary had a little lamb), I was totally amazed. If you are interested in machine code using Applesoft Basic’s peek and poke commands to compose music, visit 8 bit Sound and Fury.  Even though 8 bits seemed very simple (because there were so few values), it was not. When creating sound effects for games, one of the most difficult things to manage was the timing or duration of the tones in conjunction with the sprites flying across the screen in a (near) simultaneous fashion. In the 90s during  the reign of the Intel x86 architecture (the PC), the most popular sound card was called the Sound Blaster 16 made by Creative Technologies. In its prime, this sound card was quite amazing when playing games because it was a separate card having a chip set with the ability to play midi sounds and music in stereo (two channels). The sound card was bundled with a CD rom player allowing one to pop in a music CD. Another cool feature of the Sound Blaster was its 15-pin MIDI/Joystick multiport enabling game input devices to be connected. Today (the future), sound cards are able to support surround sound (3D audio effects), various sound formats, record, various music formats, midi, and mixing.  Multitasking enables modern computers to  play sounds/music on parallel tracks (simultaneously).

Next, we will be creating a sound manager service that will be added to the game engine framework library (JFXGen).

Sound Manager Service

The GameWorld class contains services such as the sprite manager and (more recently) a reference to an instance of a SoundManager (singleton). The sound manager service is responsible for managing all of the game’s sound effects. This service allows the developer to load sound clips (AudioClip) using the loadSoundEffects() method. After loading sound effects each audio clip can be retrieved using a unique id (String) mapped to the sound. The last method is the shutdown() method. When the application is exited, the stop method will invoke the GameWorld‘s shutdown() method which, in turn, calls the SoundManager object’s shutdown to clean up any resources. The SoundManager instance has a thread pool that gets gracefully shutdown.

Note:  For brevity, I designed the SoundManager class to play simple audio clips, though not music, during the game. If you want to add music, please refer to the JavaDoc on the Media and MediaPlayer APIs.

Shown below is the SoundManager class diagram:

SoundManager Class Diagram

Figure 2: Class Diagram of the Sound Manager

The following is source code for the SoundManager class:

package carlfx.gameengine;

import javafx.scene.media.AudioClip;

import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * Responsible for loading sound media to be played using an id or key.
 * Contains all sounds for use later.
*</pre>
<pre> * User: cdea
 */
public class SoundManager {
    ExecutorService soundPool = Executors.newFixedThreadPool(2);
    Map<String, AudioClip> soundEffectsMap = new HashMap<>();

    /**
     * Constructor to create a simple thread pool.
     *
     * @param numberOfThreads - number of threads to use media players in the map.
     */
    public SoundManager(int numberOfThreads) {
        soundPool = Executors.newFixedThreadPool(numberOfThreads);
    }

    /**
     * Load a sound into a map to later be played based on the id.
     *
     * @param id  - The identifier for a sound.
     * @param url - The url location of the media or audio resource. Usually in src/main/resources directory.
     */
    public void loadSoundEffects(String id, URL url) {
        AudioClip sound = new AudioClip(url.toExternalForm());
        soundEffectsMap.put(id, sound);
    }

    /**
     * Lookup a name resource to play sound based on the id.
     *
     * @param id identifier for a sound to be played.
     */
    public void playSound(final String id) {
        Runnable soundPlay = new Runnable() {
            @Override
            public void run() {
                soundEffectsMap.get(id).play();
            }
        };
        soundPool.execute(soundPlay);
    }

    /**
     * Stop all threads and media players.
     */
    public void shutdown() {
        soundPool.shutdown();
    }

}

How do I play sound effects in JavaFX?

In JavaFX 2, you can play small sound files efficiently with less overhead by using the AudioClip API. This API allows a sound to be played repeatably. An example would be a gamer firing the weapon (left mouse press) which makes a laser sound “pew pew!” Speaking of lasers in the demo game, I used a free sound file from the website FreeSound.org having the Creative Commons license. Since the file was a wav file format, it was larger than it needed to be. So, I decided to convert the file to an mp3 sound format. I felt it was important to reduce the size of the file (smaller footprint) for faster loading. When converting the file to an mp3 sound format, I used Sony’s Sound Forge software. Shown below is a code snippet to play small sound files:

   AudioClip sound = new AudioClip("laser.mp3");
   sound.play();

How do I play music in JavaFX?

Although the SoundManager(my implementation) doesn’t play music, it is easy to add the capability. The following code snippet shows how to load an MP3 file to be played using the Media and MediaPlayer API:

   Media media = new Media("hymetojoy.mp3");
   MediaPlayer player = MediaPlayerBuilder.create()
                         .media(media)
                         .onReady( new Runnable() {
                             @Override
                             public void run() {
                                player.play();
                             })
                         .build();

JavaFX Sound Demo

Requirements:

  • Java 7 or later
  • JavaFX 2.1 or later
  • Windows XP or later (Should be available soon for Linux/MacOS)

A simple Asteroid type game called ‘The Expanse’.

Instructions:

  • Right mouse click (on Windows) to fly ship.
  • Left mouse click (left click on Windows mouse) to fire weapon.
  • Key press ’2′ to change to large missiles. (blue circular projectiles)
  • Other key press defaults to smaller missiles. (red circular projectiles)
  • Space bar key press will toggle a force field to protect the ship from enemies and asteroids.

Click on the Launch button below to start the demo:

Tutorial demo

Part 5 ‘The Expanse’ Sound

References

Apple ][ specs: http://apple2history.org/history/ah03/
8 bit on the Apple ][: http://eightbitsoundandfury.ld8.org/programming.html
Sound Blaster : http://en.wikipedia.org/wiki/Sound_Blaster
JFXGen: https://github.com/carldea/JFXGen
JavaFX’s AudioClip API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/AudioClip.html
Sony Sound Forge: http://www.sonycreativesoftware.com/soundforgesoftware
Freesound.org: http://www.freesound.org
Laser sound from Freesound.org: http://www.freesound.org/people/THE_bizniss/sounds/39459/
Creative commons license: http://creativecommons.org/licenses/sampling+/1.0/
Media API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/Media.html
MediaPlayer API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/MediaPlayer.html





I’ll be back…

8 07 2012

Hey everyone!

I’m sorry to inform you that I will be taking a break from the JavaFX Game Tutorials to prepare for my upcoming presentation at the Montgomery county (Maryland) Java user group (http://www.mcjug.org) on July 18, 2012. If any of you Java fans are in town stop by between 6:00-8:00pm EST. My talk will be about an introduction to building JavaFX 2 enterprise applications using Grails.

I look forward to resuming where we left off in the JavaFX Game Tutorial series. ;-)

Code well my friends!

Carl








Follow

Get every new post delivered to your Inbox.

Join 78 other followers