MadComponents3D – part1: The road less travelled
“Two roads diverged in a wood, and I—
I took the one less travelled by,
And that has made all the difference.” (Robert Frost)
I started working on open-source MadComponents in response to Flash Builder’s mobile components. Flex was bloated and developing an app was much more convoluted than it needed to be. So I created MadComponents. A lightweight and easy to use UI framework with server communication capabilities. It became popular with developers. The MadComponents library has been downloaded over 4000 times, and the blog has had well over 100,000 visitors since its inception.
MadComponents3D Conception: The Vision
With the advent of Stage3D, and GPU-accelerated graphics, developers started to achieve impressive hardware-accelerated graphics performance. Josh Tynjala created the fledgling Foxhole library, (now Feathers) and demonstrated the potential of a hardware-accelerated UI framework.
MadComponents has always worked quite smoothly in GPU rendermode. Even on the New iPad. But I could see the tide of change from Adobe as they focussed more on developing Direct rendermode capabilities in AIR – so I started to think about the next phase in the evolution of MadComponents.
I didn’t want to lose the existing advantages that MadComponents offers via the display list:
- UI automatically adapts for different devices, screen sizes, and pixel densities.
- Style-able components. Easily change colours, gradients, and appearance without bitmap skins. (Although MadComponents allows skinning too).
- Easily extensible. A developer can make their own custom component – and extend MadComponents to include it.
- Pixel snapped images. <imageX> component that chooses the best resolution image for the dpi of the device.
I experimented with geometric shaders. Initially I had the idea to replace the entire graphics rendering aspect of MadComponents with Stage3D shaders. But in the end I decided on a scheme where I could seamlessly switch between the classic display-list, and Stage3D rendering (or have both on the screen simultaneously). Because the only advantage that Stage3D offers over the display list is when things move quickly. Like scrolling a list. But if you’re interacting with a UI form – it’s not usually in motion – so use the display-list here.
I decided that Stage3D acceleration would be optional. Developers may use the new Stage3D features if they wish – or they can omit Stage3D from their project if they wish – and just use the display list.
There are a lot of Blackberry developers using MadComponents – and they’re currently stuck with a pre-Stage3D incarnation of AIR. I didn’t want to leave them in the lurch.
I wanted to accomplish more than just Stage3D rendered components and improved list scrolling. I was getting inspired by the way UIs move and flow with user interaction. Especially on tablet devices. Cover Flow on a Mac, Grid Metro View on Windows 8, Fluid Page Flipping, Transitions, Graphics Menus, etc.
And now the bad news: Stage3D Bottlenecks
It’s not until you delve deeply enough into Stage3D development that you realise how naive your initial perceptions are about its speed and capabilities. While we’ve all seem the demos with things whizzing around screens at an impressive 60fps – but it’s worth noting that these demos rarely involve a lot of image data. Stage3D does not scale-up well. If you’re playing around with a lot of visual information (as is usually the case with a UI) – then you’re going to get caught in the restrictive space between two fundamental Stage3D limitations. Texture Upload Speed, and Memory.
Stage3D was clearly designed for games. That’s where Adobe are focussed currently. While a great many Flash Platform developers are still interested in form-based apps, or even enterprise apps – Stage3D development is a lot less versatile than it should be.
For example, smooth scrolling of full-size, full resolution, content is virtually impossible if the size of the scrollable content is too big (Or many regions that change dynamically).
There are two possible approaches – and they are both flawed:-
1. Preload all of the textures at the start. However, if there is too much image content, then you’ll run out of memory. (Adobe AIR on Mobile tends to fail silently when this happens – no diagnostics when debugging – just crash).
2. Sub-divide the area into smaller Texture regions, and recycle these regions as the content scrolls. Dynamically dispose of textures that scroll off the screen, recycle the Texture objects, and dynamically upload image textures that are about to scroll onto the screen.
The problem is that texture upload is SLOW! – unbelievably slow. It is a huge bottleneck. And if you scroll the content too quickly, then UI performance will choke under the overhead of uploading textures.
In MadComponents3D, I employed devious tricks to make recycled-row list scrolling seem smoother – but texture.uploadFromBitmapData(), is so slow that it’s difficult to make this feel natural to the user.
Let’s hope that Adobe can optimise the performance of texture.uploadFromBitmapData(). Or even look beyond their games use-case so that they comprehend a necessity for optimising it.
These limitations don’t affect games. Games have less image data, and tend to preload them all at the start when the game first starts up.
Some developers still advocate advantages of GPU mode, and fast BitMapdata bliting, as opposed to Stage3D. Shawn Blias has some conducted some great performance showdowns. His conclusions might surprise a lot of developers who equate Stage3D with fast graphics performance :-
“Another frustrating thing is that GPU mode continues to heavily outperform ND2D and Starling on modern Android Devices. For example, on the cutting edge Galaxy Nexus, GPU mode can push an extra +179 sprites, where Starling can only do +32. This is a huge difference!”
My own conclusions are that the advantages of Stage3D, aren’t as clear-cut as they initially seemed. Let’s hope that Adobe invest more development into resolving the bottlenecks.
A road less travelled
MadComponents has been an all-absorbing project. I struggle to do it justice in my spare time – but other projects, both paid, and unpaid complete for my attention. The popularity of the framework hasn’t been entirely fortunate for me. I have other ideas, and other projects beyond MadComponents. Other things I want to do.
Many developers have suggested that this is the kind of project that should be supported by Adobe. Yes, it should be. But it’s not. MadComponents is not funded in any way. Increasingly, I’m asking the community to get more involved – as I can’t invest as much of my own free time anymore.
I can see where Adobe is focussed – and I considered aligning MadComponents to fall more within their area of interest. I briefly considered building MadComponents3D on top of Starling – the way that Foxhole/Feathers is. I even considered building MadComponents3D ON TOP OF FOXHOLE itself. While there was absolutely no technical merit in either of these ideas – they may have gained MadComponents3D some favour .
In the end, I chose to avoid Starling – for both technical, and ethical reasons. I took the road less travelled – but I’m in good company. I think it’s a pity that Adobe’s money and marketing around Starling is drawing developer attention away from much more interesting Stage3D initiatives.
Lars Gerckens’ ND2D was always a popular pioneering 2D engine. But Lars regrettably doesn’t have the time to compete with professional 2D engines anymore – so he is ceasing development.
Peter “Shtif” Štefček’s Genome 2D is an immensely powerful 2D engine incorporating some very clever capabilities that out-performs Starling.
MadComponents – the new Flex?
Ages ago, I demonstrated how to wrap MadComponents inside Flex wrappers and how this might be the basis for a new lightweight Flex Mobile component set. As far as the developer is concerned, this approach could maintain many of the conventions that they’re used to – as well as interoperability with existing components.
Nobody seemed interested in the idea before.
Now with my foray into Stage3D acceleration – I’ve been talking about this idea again. Admittedly, I’ve been pretty scathing about Mobile Flex in the past – and in it’s current bloated form, I still think it’s horrendous. But I’d be willing to work with the Open Flex community (free time permitting) – if they wanted to move towards a much more lightweight and Stage3D-capable Mobile Flex framework based on MadComponents.
So what does MadComponents3D do?
UPDATE: New MC3D Components added since I first wrote this
9. Folding Pages (Inspired by the FlipBoard App).
10. Really Cool Wheel Menus!