Skip to content
October 20, 2012 / Daniel Freeman

MadComponents3D – part1: The road less travelled

“Two roads diverged in a wood, and I—
I took the one less travelled by,
And that has made all the difference.” (Robert Frost)

I started working on open-source MadComponents in response to Flash Builder’s mobile components.  Flex was bloated and developing an app was much more convoluted than it needed to be.  So I created MadComponents. A lightweight and easy to use UI framework with server communication capabilities. It became popular with developers. The MadComponents library has been downloaded over 4000 times, and the blog has had well over 100,000 visitors since its inception.

MadComponents3D Conception: The Vision

With the advent of Stage3D, and GPU-accelerated graphics, developers started to achieve impressive hardware-accelerated graphics performance. Josh Tynjala created the fledgling Foxhole library, (now Feathers) and demonstrated the potential of a hardware-accelerated UI framework.

MadComponents has always worked quite smoothly in GPU rendermode.  Even on the New iPad.  But I could see the tide of change from Adobe as they focussed more on developing Direct rendermode capabilities in AIR – so I started to think about the next phase in the evolution of MadComponents.

I didn’t want to lose the existing advantages that MadComponents offers via the display list:

  • UI automatically adapts for different devices, screen sizes, and pixel densities.
  • Style-able components.  Easily change colours, gradients, and appearance without bitmap skins.  (Although MadComponents allows skinning too).
  • Easily extensible.  A developer can make their own custom component – and extend MadComponents to include it.
  • Pixel snapped images.  <imageX> component that chooses the best resolution image for the dpi of the device.

I experimented with geometric shaders.  Initially I had the idea to replace the entire graphics rendering aspect of MadComponents with Stage3D shaders.  But in the end I decided on a scheme where I could seamlessly switch between the classic display-list, and Stage3D rendering (or have both on the screen simultaneously).  Because the only advantage that Stage3D offers over the display list is when things move quickly.  Like scrolling a list.  But if you’re interacting with a UI form – it’s not usually in motion – so use the display-list here.

I decided that Stage3D acceleration would be optional.  Developers may use the new Stage3D features if they wish – or they can omit Stage3D from their project if they wish – and just use the display list.

There are a lot of Blackberry developers using MadComponents – and they’re currently stuck with a pre-Stage3D incarnation of AIR.  I didn’t want to leave them in the lurch.

I wanted to accomplish more than just Stage3D rendered components and improved list scrolling.  I was getting inspired by the way UIs move and flow with user interaction.  Especially on tablet devices.  Cover Flow on a Mac, Grid Metro View on Windows 8, Fluid Page Flipping, Transitions, Graphics Menus, etc.

And now the bad news: Stage3D Bottlenecks

It’s not until you delve deeply enough into Stage3D development that you realise how naive your initial perceptions are about its speed and capabilities.  While we’ve all seem the demos with things whizzing around screens at an impressive 60fps – but it’s worth noting that these demos rarely involve a lot of image data.  Stage3D does not scale-up well.  If you’re playing around with a lot of visual information (as is usually the case with a UI) – then you’re going to get caught in the restrictive space between two fundamental Stage3D limitations.  Texture Upload Speed, and Memory.

Stage3D was clearly designed for games.  That’s where Adobe are focussed currently.  While a great many Flash Platform developers are still interested in form-based apps, or even enterprise apps – Stage3D development is a lot less versatile than it should be.

For example, smooth scrolling of full-size, full resolution, content is virtually impossible if the size of the scrollable content is too big (Or many regions that change dynamically).

There are two possible approaches – and they are both flawed:-

1. Preload all of the textures at the start.  However, if there is too much image content, then you’ll run out of memory.  (Adobe AIR on Mobile tends to fail silently when this happens – no diagnostics when debugging – just crash).

2. Sub-divide the area into smaller Texture regions, and recycle these regions as the content scrolls.  Dynamically dispose of textures that scroll off the screen, recycle the Texture objects, and dynamically upload image textures that are about to scroll onto the screen.

The problem is that texture upload is SLOW! – unbelievably slow.  It is a huge bottleneck.  And if you scroll the content too quickly, then UI performance will choke under the overhead of uploading textures.

In MadComponents3D, I employed devious tricks to make recycled-row list scrolling seem smoother – but texture.uploadFromBitmapData(), is so slow that it’s difficult to make this feel natural to the user.

Let’s hope that Adobe can optimise the performance of texture.uploadFromBitmapData().  Or even look beyond their games use-case so that they comprehend a necessity for optimising it.

These limitations don’t affect games.  Games have less image data, and tend to preload them all at the start when the game first starts up.

Some developers still advocate advantages of GPU mode, and fast BitMapdata bliting, as opposed to Stage3D.  Shawn Blias has some conducted some great performance showdowns.  His conclusions might surprise a lot of developers who equate Stage3D with fast graphics performance :-

“Another frustrating thing is that GPU mode continues to heavily outperform ND2D and Starling on modern Android Devices. For example, on the cutting edge Galaxy Nexus, GPU mode can push an extra +179 sprites, where Starling can only do +32. This is a huge difference!”

My own conclusions are that the advantages of Stage3D, aren’t as clear-cut as they initially seemed.  Let’s hope that Adobe invest more development into resolving the bottlenecks.

A road less travelled

MadComponents has been an all-absorbing project.  I struggle to do it justice in my spare time – but other projects, both paid, and unpaid complete for my attention.  The popularity of the framework hasn’t been entirely fortunate for me.  I have other ideas, and other projects beyond MadComponents.  Other things I want to do.

Many developers have suggested that this is the kind of project that should be supported by Adobe.  Yes, it should be.  But it’s not.  MadComponents is not funded in any way.  Increasingly, I’m asking the community to get more involved – as I can’t invest as much of my own free time anymore.

I can see where Adobe is focussed – and I considered aligning MadComponents to fall more within their area of interest.  I briefly considered building MadComponents3D on top of Starling – the way that Foxhole/Feathers is.  I even considered building MadComponents3D ON TOP OF FOXHOLE itself.  While there was absolutely no technical merit in either of these ideas – they may have gained MadComponents3D some favour .

In the end, I chose to avoid Starling – for both technical, and ethical reasons.  I took the road less travelled – but I’m in good company.   I think it’s a pity that Adobe’s money and marketing around Starling is drawing developer attention away from much more interesting Stage3D initiatives.

Lars Gerckens’ ND2D was always a popular pioneering 2D engine.  But Lars regrettably doesn’t have the time to compete with professional 2D engines anymore – so he is ceasing development.

Peter “Shtif” Štefček’s Genome 2D is an immensely powerful 2D engine incorporating some very clever capabilities that out-performs Starling.

MadComponents – the new Flex?

Ages ago, I demonstrated how to wrap MadComponents inside Flex wrappers and how this might be the basis for a new lightweight Flex Mobile component set.  As far as the developer is concerned, this approach could maintain many of the conventions that they’re used to – as well as interoperability with existing components.

Nobody seemed interested in the idea before.

Now with my foray into Stage3D acceleration – I’ve been talking about this idea again.  Admittedly, I’ve been pretty scathing about Mobile Flex in the past – and in it’s current bloated form, I still think it’s horrendous.  But I’d be willing to work with the Open Flex community (free time permitting) – if they wanted to move towards a much more lightweight and Stage3D-capable Mobile Flex framework based on MadComponents.

So what does MadComponents3D do?

MadComponents3D incorporates a set of optional “Stage3D acceleration” classes that work with the existing MadComponents framework.  There are eight new capabilities:-
1. List And Picker Scrolling.
MadComponents3D list acceleration works with existing <scrollVertical>, <list>, <picker>, <groupedList>, <dividedList>, <tickList>, <tickOneList>, and <longList> components.  It exploits a variable motion blur shader to make list scrolling appear more natural.
2. Navigation slides
Smooth sliding transition beween a list, and the next page.
3. Page Transitions
Slide, Cube, Door, Flip, Swap, and Trash page transitions.
4. Page Flipping.  (Note the reflections – another custom AGAL shader).
5. Grid Navigation.
Inspired by the behaviour of Windows8 Metro, this even incorporates live tiles, and flip-around forms.  (If anyone has the time – I’d love to see a demo that LOOKS like Metro, with the same colour scheme and icons.  Easy to do – the appearance is all based on versatile MadComponents layouts).
6. A new Slide-Out-Navigation component
7. Pop-Up windows that pop-up.
8. Scalable panels that combine a panel resize with a cross-fade to switch between two different UI forms.

UPDATE: New MC3D Components added since I first wrote this

9. Folding Pages (Inspired by the FlipBoard App).

fold

 

 

10. Really Cool Wheel Menus!

wheels

Coming soon…

In subsequent instalments to this series, I’ll be describing exactly how these MadComponents3D classes work so you can incorporate them in your own projects.
In the meantime, for community discussions about MadComponents3D, come and join the facebook page:-http://www.facebook.com/groups/336764056350846/

If you’d be interested in helping to write and maintain code for MadComponents / MadComponents3D – please let me know.  If not – please blog about tis project and help to spread the word.
Also, don’t forget to leave a “star”, or “g + 1″ recommendation on the Google Code site.
Advertisements

5 Comments

Leave a Comment
    • Daniel Freeman / Oct 21 2012 1:46 am

      No. Someone on the Facebook group suggested we need one too. If anyone wants to write one – that would be great! (I don’t have the time).

  1. flashisobar / Oct 20 2012 3:57 pm

    Awesome Work~~~~

  2. Elliot Geno / Oct 24 2012 8:22 pm

    You mentioned texture upload durations… Have you tried ATF? I realize the tooling hasn’t been available. But I see you on the betas. Thibault dropped a version of it for us to get started. (Not officially… just to placate us.)

    • Daniel Freeman / Oct 25 2012 2:43 am

      Hi Elliot,

      ATF is a good solution to pre-made static textures. Again, it’s for games. The situation here is a DisplayObject UI, rasterised as a bitmapdata first, and then uploaded to a texture.

      Only I want to recycle Texture strips to reduce the Texture memory overhead of large scrolling content. So it happens INSIDE the “Game-loop”. If a scrolling UI is scrolled quickly – you can imagine many texture strips need to be recycled – and that’s when this situation begins to choke UI performance.

      What I can do is synthesise that APPEARANCE of rapidly scrolling content – masked with heavy motion blur – without actually uploading real rows. But I’ve found that texture upload is so slow that the scrolling content doesn’t have to be moving very fast before I need to do that. It’s a compromise and it’s not a very natural UI experience.

To discuss MadComponents/MC3D, join the Facebook group!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: