Haxe NME Follow-up

A few weeks ago, I began working on a Haxe NME version of the NEO Scavenger encounter editor. I endeavored to share my first impressions of NME, but I had two more weeks of experimenting with it, so I wanted to share what I learned. What follows, then, is a brief discussion of each of the highlights from that experiment.

Encounter Editor Screenshot

Speed

The main reason I chose to rewrite the NEO Scavenger editor in Haxe NME was because the Flash version had performance issues. The recent increase in data caused loading to timeout and crash. And what's more, even before this loading failure, I was reluctant to use the tool due to its sluggishness. Faced with rewriting lots of code no matter what, I decided to try Haxe NME.

I'm happy to say that Haxe NME showed its strength as a high performance framework. I haven't done any proper comparisons between NME and Flash, but the editor is quite responsive at 2560x1600 resolution. This was while drawing hundreds of boxes, hundreds of bitmaps, and thousands of triangles simultaneously, all with scaling (no bitmap rotations, though). 

The Windows target also loaded (SQL/URL-based) data much faster than the old Flash version. Though, the NME Flash target still needed optimization to complete data loading before the 15 second timeout.

UI

Form fields are still a weakness with NME on Windows targets. I was able to hack together some quick and dirty UI elements as stopgaps for buttons, toggle buttons, and drop-down boxes. But not having Flash's built-in UI elements was an obstacle.

TextFields

TextFields work on Windows targets, but are extremely limited. They lack support for getting the current caret position, selected text, and copy/paste. Flash targets work fine, however.

Performance can be an issue, though. I was able to get about 2-3 TextField elements per encounter node (about 2200 total) without any noticeable slow-down on my machine. However, much beyond that, and the framerate seemed to drop. I ultimately opted to hide TextFields on all nodes, and reveal them as-needed to keep framerates high.

A closer look at some of the editable fields I have in each node/connector.

Copy/Paste

Copy/paste, itself, turns out to be a general issue in Windows targets. I wasn't able to get access to the clipboard with vanilla NME, and the systools library that I tried wouldn't compile for NME Windows targets. Copy/paste on Flash targets appeared to work fine, though.

Object/Dynamic Weirdness

Haxe uses a type called "Dynamic" to handle generic objects (instead of Flash's Object class). On the Windows target, I was able to use .get() to access dynamically-named properties in a URLVariables object. But when compiling on Flash, the compiler didn't like that method. What's more, the traditional Flash method of Object["propertyName"] threw compiler errors. In the end, I had to use Reflect.getProperty(). Not too much of a pain, just unexpected.

URLRequest and 413 Error

Using URLRequest on Flash targets worked as expected, but caused errors when tried on Windows targets. I kept receiving 413 "request entity too large" errors when trying to execute the request. As it turned out, I needed to initialize the .data field of the URLRequest object to quell the 413 error. I'm not sure why a null data field would cause a "request entity too large" error, which is why it took me a while to figure out. I ended up finding the solution by looking up libcurl issues of a similar nature, since that's the library used by NME.

Key Handling

Flash and Windows targets for NME seem to trigger different key codes for ascii characters. I ended up writing special handling code to convert all key codes to uppercase so each platform interpreted results consistently.

Mouse Handling

I ran into issues getting any mouse scroll wheel events to fire on Windows. I didn't pursue this very far, though, as I could work around it, and had bigger fish to fry.

Extensibility

There were one or two instances where I started digging into NME source code itself, to see if I could work around problems I was having (such as enabling unexposed TextField properties). However, I was unable to navigate the matryoshka-doll-like layers of code to the source I needed.

Specifically, I was trying to expose a field in TextField, like caretIndex, so I could hack together a copy/paste stopgap on Windows. I decided to look at how existing fields were exposed, like .numLines. When I dug into the TextField.hx class, I found that Windows targets used the neash.text.TextField implementation.

However, the neash.text.TextField code redirected .numLines to:

Loader.load("nme_text_field_get_num_lines", 1);

Loader.load uses cpp.Lib on Windows targets, so looking into that, I found that cpp.Lib.load used:

__global__.__loadprim(lib,prim,nargs);

which was loading a primitive from a DLL. I wasn't clear where to go from here, though. It seemed like there should be some sort of table or Rosetta stone for mapping strings like "nme_text_field_get_num_lines" to a block of code somewhere, but no amount of searching seemed to reveal it.

I was pretty frustrated by my inability to get to the bottom of that question. And ultimately, had to switch targets to Flash to get the TextField behavior I needed. Though, that in itself is a pretty strong endorsement for Haxe NME.

Cross-Platform

Being able to change a drop-down from "Windows" to "Flash" was almost all that I needed to do to get around the aforementioned TextField issues. That, and some tweaking in the way I accessed dynamically-named fields of a Dynamic object, had me well on my way to continuing work with almost no interruption. Literally, a couple weeks' worth of development on the Windows target was ported to the Flash target within about two hours. Pretty impressive.

Updates

It's also important to point out that NME is updating fairly frequently. Even in the couple months between the version of NME I first downloaded, and one I recently downloaded, there was significant improvement. In fact, several bugs I was encountering were solved by an update to NME 3,4,4.

Conclusion

I'm still pretty impressed with what I'm finding in NME. It has gaps here and there, and it can be tricky to find support. But I was surprised how far I could take it, given it's relative infancy compared to platforms like Flash. I suspect that if I were making a purely game-like app, without the need for text editing, it would do all that I need without issue. I think that the only feature NEO Scavenger uses which I've not yet tested is the audio playback.
Hopefully, some of the info I've shared above helps other intrepid developers in overcoming similar obstacles. And it'd be great if this info was enough to embolden a few new developers to put Haxe NME through its paces. I truly feel that it's a powerful tool, and just a few more concurrent users could be all that's needed to sustain further growth.

By the way, if you're interested in a more chronological account of my explorations, you can find out more about the editor overhaul in these Blue Bottle Games posts:

• Encounter Editor Overhaul
• New Editor Progress
• New Editor: UI Widgets
• Encounter Editor: UI Almost Ready
• It's Alive! (Almost)
• Encounter Editor Running, Plot Work Resumed

Thanks for reading, and see you next time!

Haxe NME: First Impressions

I've been talking a lot of business the past few months, so I thought I'd mix things up a bit this time. Today, we'll talk about Haxe NME.

What is Haxe NME?

Haxe NME is two things, really, so let's start with the first part: Haxe.

Haxe (officially pronounced "hex," though I tend to pronounce it differently), is a multi-platform, open source, programming language. Using a syntax similar to Java or Actionscript 3 (AS3), one can compile applications to a wide range of platforms, including:

• C++
• C#
• JavaScript
• Flash
• PHP

This, in turn, means that one set of source code can ostensibly be made to work on PCs, browser plug-ins, devices running iOS, Android, Windows Mobile, and webOS.

You might be thinking, "big deal. Java and Flash/Air already make that possible." For the most part, that's true. One difference, however, is that Haxe compiles into other languages, including ones required to make native applications. Java and Flash applications, on the other hand, are interpreted through a virtual machine (VM) on their target platforms.

With Haxe, one can write a single set of code for multiple platforms. And in theory, applications developed in Haxe have the ability to run much faster than VM-based applications like Flash, Java, and Air. It also means that Haxe apps can access features unavailable to some VMs, such as local file access and hardware acceleration.

And NME? What's that?

NME stands for Neko Media Engine, and is a library/framework for use in Haxe. It provides an API that looks a lot like Flash's (literally, many API calls are just a matter of replacing import flash.display.* with import nme.display.*), which can be used to deploy native apps to Windows, Linux, Mac, iOS, Android, webOS, BlackBerry, and Flash Player.

Haxe generates C++, which NME then compiles into the desired target platform's native code. The output applications use native components, such as OpenGL, libjpeg, curl, etc, to provide performance and features like what one would expect from writing platform-specific applications.

In a nutshell, it purports to generate blazing fast native apps on nearly every platform, using a familiar, AS3-like code and API. Even the compiling is simplified, requiring only that the user specify "flash," "windows," "ios," or any of the other target keywords to generate the desired native application.

Wait, are you porting NEO Scavenger to Haxe NME?

No, not yet. It's true that there would be performance benefits, and it would open up some handy local file access, modding, and other neat features. But switching technology mid-stride would be a pretty big risk. It would be a large undertaking, cause lots of down time, and there's no guarantee it would work.

However, NEO Scavenger's encounter editor is in need of an overhaul. Currently, it's built into the game, so that it can piggyback on various data-loading features already in place. That worked fine for the first couple hundred encounter nodes. It was slow to move things around, but I could mostly edit what I needed to.

With the recent influx of random encounters, though, there are over 750 nodes being displayed. And with each of those having UI elements to display, Flash just can't handle it. In fact, Flash can't even finish loading the data before it times out. I could rewrite the data loader to segment across frames (as I recently did for the save game load method), but I'm pretty sure that even after loading, the editor will be unusably slow.

So the editor is going to require an overhaul, one way or the other. I could do that in Flash, but I'll definitely be limited by rendering speed. Haxe NME should be faster, provided I can port things over. And it'll give me a better idea of whether it's worth building anything else in Haxe NME.

I've given myself a time box of a few days, in order to determine whether it's worth proceeding, or backing off and returning to Flash for the editor changes. So far, I've made enough progress over two days to encourage further work in the Haxe NME direction. Though, it wasn't without it's obstacles.

Transitioning from AS3 to Haxe

I'll start with an unqualified success: syntax familiarity. Transitioning from AS3 to Haxe has been trivial. There are some differences, but by far, the syntax feels very familiar. A large amount of the differences can be done with a search and replace, and the ones that can't weren't hard to figure out.

In fact, if you're using Haxe to target the Flash platform, that's about all you have to worry about. All of the API calls one used in AS3 are pretty much the same in Haxe. I was able to get a project compiling and debugging in FlashDevelop with about the same level of effort as a regular AS3 project.

Write Once, Run Anywhere?

Unfortunately, compiling code for non-Flash targets wasn't as easy as advertised.

For really basic applications (ones that use very few API calls outside of flash.display), worked well-enough. My editor used some drawing functions to make arrows and boxes, and these were well-supported. In fact, I was so emboldened by my first test app that I copied the editor's source over from AS3 wholesale, and started porting code. It was when I started delving into non-display libraries that things got ugly.

NME lacks a UI library like Flash's, so that was my first hang-up. There are some UI elements built-in, such as simple buttons. But complex items, such as drop-down boxes, radio buttons, and scrollbars were missing. Some effort has gone into UI libraries for Haxe and NME, but they're not as smooth to use nor as complete as what AS3 users will be used to.

Waxe looked like a pretty good place to start, but I quickly became stuck. For one thing, it seemed to be at odds with the NME way of displaying things, so it wasn't clear how to get them to play nice. The documentation was pretty sparse, too.

Joshua Granick's NME GUI library turned out to be a better fit, and covered much of what I'd need. However, it was still (apparently) missing a drop-down box, which was the UI item that I was currently stuck on. I decided I could probably make something work using basic UI building blocks, so put UI on hold while I vetted some other features.

NEO Scavenger uses some Flash URL libraries to request data from a server, and populates some tables in memory for running the game. The editor piggybacks on this code, in order to display encounters, their images, game items that connect them, and some other bits.

Unfortunately, compiling a C++/Windows target didn't seem to work, even though the Flash target did. And this is where we encounter a potential deal-breaker.

IDE Debugger Support

While the debugging process was identical to Flash development when deploying a Flash build, this was not so when trying the Windows target. There is no support yet for FlashDevelop IDE debugging a Haxe C++ target. There is a C++ debugger, but that is a command-line tool. One can set breakpoints in code, and it will pause, but then the user must interact with the debugger through a console.

One could also use on-screen text or trace, but as someone who suffered Maxscript's lack of IDE debugging for nearly seven years, I wasn't happy about the prospect of tracing my way out of bugs.

One thing I haven't yet tried is compiling the C++ output in Visual Studio, and debugging from there. Theoretically, that seems like it should work. The down side, however, is that I'm debugging C++, not the more user-friendly Haxe syntax. And worse, I'm debugging a target, not the source.

I looked around for quite a while, trying to figure out if I was missing something. Maybe another IDE was a better fit? Maybe I just didn't install hxcpp correctly?

In the end, I turned up little information to guide me through this problem. And that may be the other major flaw in Haxe and NME.

Documentation

Haxe and NME have documentation, first of all. I don't want to make it sound like you walk away from the Haxe and NME websites with a box of tools and a noose.

In fact, Haxe has pretty extensive documentation, if we're being fair. Quite a few of my questions, particularly about object types, syntax, and language features were all answered there. And there is a growing amount NME documentation out there, too.

However, if you're a Flash developer, the current state of Haxe NME support is going to be a cold splash of water. Forget about typing a few keywords into Google and getting reams of documentation and examples to choose from. Worse, if you have a really specific question in mind, there's nowhere near the same chances of finding the answer posted someplace, unlike one does with AS3.

As I struggled with getting the nme.net API to work on the Windows target, I had to turn to a mix of old and new skills to find my answer. 

The "old" part involved brushing the dust off my Maxscript debugging skills: tracing output, and displaying debug variables on-screen. It's clunky, and trace often doesn't output anything until the Windows target exits, but at least it's possible to tweak and get some feedback without having to jump between IDEs.

The "new" skill was learning to seek out answers to library and API issues in non-AS3 (non-Haxe, even) sources. When a URL request failed, I could dig up useful help around the web on libcurl, the library NME wraps in a Flash-like API for URL tools. It gave me more insight into why my specific case was failing. I could see, for example, what C++ devs would do to fix their issues, and then adapt those solutions to NME.

Based on the state of C++ debugging and documentation, I was almost ready to give up on Haxe NME, relegating it to the realm of "would be nice if it worked." However, pushing through those issues has given me some new-found courage. Enough to consider continuing my investigation, rather than retreating to AS3.

Haxe NME: The Verdict

Is it worth it? Is porting to or building in Haxe NME worth doing? That'll depend a lot on your aversity to risk. If your already using AS3, can handle some uncertainty, and have wiggle room for learning a new system, the benefits of speed, portability, and more features are quite tempting.

If, on the other hand, you need a predictable timeline, or reliable support, you may want to wait. Haxe NME sidesteps much of the cost of learning C++, but its relative youth means you'll be on your own from time-to-time.

For me, I've seen enough potential to warrant further investigation with the editor subset of NEO Scavenger. I was able to overcome the major obstacles so far, so I feel that I will be able to continue progress in the face of future issues. And in the worst case, I only stand to lose whatever time I am willing to devote to it. I can always return to AS3 if all else fails.

I'm interested to see how Haxe NME pans out. Currently, it stands in a sort of middle ground between Flash and Unity3D. On the Flash end of the scale, one has accessibility of development, wide consumer deployment, and cheap-to-free costs, but more limited output paths and performance. On the Unity3D end, one has fast and powerful capabilities on a very wide range of target platforms, but significant costs in terms of software and revenue share.

Haxe NME seems to offer the best of both at no cost, but does not yet have the community support and tools of the other two packages. Fortunately, that situation seems poised only to improve over time. If more people decide, like me, to give it a shot, we may see that community support reach a tipping point.

Interesting times may be ahead!