Skip to content
April 10, 2012 / Daniel Freeman

Drawing MadComponents with Stage3D

A few users have asked if it might be beneficial to render the MadComponents UI using Stage3D, or to incorporate a 2D framework such as Starling.  Rather than using the display list.

Everything is possible – “It’s only software” - so maybe the question should be – would the performance gains be worth the development effort?

Provided that you use GPU render mode, the performance of MadComponents is adequate.  But if for any reason we were unable to use GPU mode, this would certainly tip the scales in favour of Stage3D rendered components.

Stage3D graphics, and direct render mode, is impressive.  Yet nothing I’ve seen is impressive in any way that would benefit MadComponents or enterprise mobile app development.

Frameworks such as Starling are excellent for games.  Games are Adobe’s main focus now.  They’re great for pushing sprites around the screen, and particle effects.  But could frameworks like this be applied to the lists, buttons, and forms that make up a mobile enterprise app UI?

Not really.  Frameworks such as Starling are good at bitmaps.  But a MadComponents UI is “elastic”.  The components adapt their size depending on the size of the screen, and its orientation.  Even if you could use nine patch images, MadComponents allows a developer the flexibility to customise the colours and shapes of the components – but bitmaps are fixed.

So none of the current frameworks that have been built on top of Stage3D would be any use to us.  If we were to exploit the power of Stage3D, for MadComponents, then we would have to get deeply into AGAL and shaders, and build our own dedicated graphics rendering layer.

MadComponents builds up the appearance of the UI components using vector graphics.  Shapes, and gradients combined in such a way to render the appearance of buttons, switches, sliders, list rows, etc.

Let’s start with a simple shape.  Consider a circle.  How might we develop a shader to render a circle?

We’re going to build a quad out of two triangles, and write a shader to render the circle inside the quad.


protected static const INDICES:Vector.<uint> = Vector.<uint> ([ 0, 1, 2,	0, 2, 3 ]);

protected static const VERTICES:Vector.<Number> =

   Vector.<Number> ([
//	X,  Y,  Z,		r, g, b,		u, v,
	-1.0,	-1.0,	0.0,	1, 1, 0,		0, 0,
	 1.0,	-1.0,	0.0,	1, 0, 1,		1, 0,
	1.0,	1.0,	0.0,	0, 1, 1,		1, 1,
	-1.0,	1.0,	0.0,	1, 1, 1,		0, 1
]);

This is a pretty standard approach.  Notice that I’ve assigned a colour to each corner of the quad, and the U,V values can be used to tell the fragment shader exactly where it is inside the quad.  The interpolated values of UV are like the coordinates of the particular pixel that we’re processing.
[ If you're not clued-up on Stage3D, you're going to need to find some tutorials to understand the rest of this.  Alternatively, you can just look at the pictures - and see the results I've been able to achieve so far with my experiments.  I had to write this in a hurry, as I'm catching a flight to Australia later.]
To draw a circle, we would measure the distance between the centre of the quad (0.5, 0.5) and if this distance is less than or equal to the radius of the circle, then we turn the pixel “on”, otherwise we turn it “off”.
Here is the vertex shader:
“m44 op, va0, vc0\n” + // 4×4 matrix multiply
“mov v0, va1\n” + // uv
“mov v1, va2\n” // rgb
 

And here is the fragment shader:-

 
“sub ft0.xy, v0.xy, fc4.xx\n” + // vector from centre
“mul ft0.xy, ft0.xy, ft0.xy\n” + // square
“add ft0.x, ft0.x, ft0.y\n” + // add x and y
“slt ft0.y, ft0.x, fc4.y\n” + // < radius
“mul oc, v1, ft0.yyyy\n”
 

fc4 is set up like this:

_context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.<Number>([ 0.5, 0.25, 0.0, 0.0 ])); // radius, radius squared

And here’s the result:

That’s a circle, but the edges are aliased.  Let’s modify the fragment shader to correct for this.  This time, we’re going to set the alpha/transparency value of each fragment, and we’re going to employ a little feathering to anti-alias the shape.
Here’s a new fragment shader:
“mov ft1, v1\n” +
“sub ft0.xy, v0.xy, fc4.xx\n” + // vector from centre
“mul ft0.xy, ft0.xy, ft0.xy\n” + // square
“add ft0.x, ft0.x, ft0.y\n” + // add x and y
“sub ft0.y, fc4.y, ft0.x\n” + // (1 + radius squared) – sum
“pow ft1.w, ft0.y, fc4.z\n” + // feathering to anti-alias circle
“mov oc, ft1\n” // output colour
 

Let’s set up our constants:

_context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.<Number>([ 0.5, 1.245, 200, 1.0 ])); // va4  centre, 1 + radius squared – frig, feathering, 1.0
 
 
 Don’t forget to allow transparency:-
_context3D.setBlendFactors(Context3DBlendFactor.SOURCE_ALPHA, Context3DBlendFactor.ONE_MINUS_SOURCE_ALPHA);
_context3D.setDepthTest(false, Context3DCompareMode.LESS);
And now our result looks like this:-
 
 

Much better.

Now consider a MadComponents UI.  What kind of shapes would be need to write shaders for?  Well, fortunately, we can consider most components as consisting of rounded-rectangles.  As circles, and rectangles are special cases of rounded rectangles.  Actually, there are a lot of semi rounded rectangles.  That is only the top corners, or the bottom corners are rounded.  Look at MadComponents grouped-lists,  for example, and you’ll see semi rounded rectangles.

So how do we write a shader to render a rounded rectangle.  Let’s start with a fragment shader to draw the corner curves.  We can accomplish this by modifying the circle shader.

“mov ft3, v1\n” +
“sub ft0.xy, v0.xy, fc4.xx\n” + // vector from centre
“sub ft0.z, fc4.x, v2.x\n” + // 0.5 – curve radius
“abs ft1.xy, ft0.xy\n” + // absolute
“sub ft1.xy, ft1.xy, ft0.zz\n” + // relative to the centre of a corner circle
 
“mul ft2.xy, ft1.xy, ft1.xy\n” + // square
“add ft2.x, ft2.x, ft2.y\n” + // add x and y
“sqt ft2.x, ft2.x\n” + // square root
 
“add ft2.z, fc4.w, v2.x\n” + // 1 + radius
“sub ft2.z, ft2.z, ft2.x\n” + //
“pow ft3.w, ft2.z, fc4.z\n” + // feathering to anti-alias circle
 
“sat ft3.w, ft3.w\n” + // 0 <= alpha <=1 (may not be necessary)
“mov oc, ft3\n” // output colour
 
 
 
 

Now, we can incorporate a test to detect if we’re in a corner or not.  And if we’re not in a corner, we can ensure that the fragment alpha is >=1.

Ok, let’s cut to the chase now.  The above renderers only work for “square” quads.  We want to modify the curvature of the corners to compensate for other aspect ratios.  Also, we’d like to introduce a parameter with only curves at the top, or only curves at the bottom.  Or curves in all four corners.

Here’s the complete program:-

package
{

	import com.adobe.utils.AGALMiniAssembler;
	import flash.display.Sprite;
	import flash.display.Stage3D;
	import flash.display.StageAlign;
	import flash.display.StageScaleMode;
	import flash.display3D.Context3D;
	import flash.display3D.Context3DProgramType;
	import flash.display3D.Context3DVertexBufferFormat;
	import flash.display3D.IndexBuffer3D;
	import flash.display3D.Program3D;
	import flash.display3D.VertexBuffer3D;
	import flash.display3D.Context3DCompareMode;
	import flash.display3D.Context3DBlendFactor;
	import flash.events.Event;
	import flash.geom.Matrix3D;

	public class Main6 extends Sprite {

		protected static const INDICES:Vector.<uint> = Vector.<uint> ([ 0, 1, 2,	0, 2, 3 ]);

		protected static const VERTICES:Vector.<Number> =

			Vector.<Number> ([
			//	X,		Y,		Z,		r,	g,	b,		u,	v,		cx, cy, flag
				-1.0,	-0.5,	0.0,	1,	1,	0,		0,	0,		0.2, 0.4, 1.0,
				1.0,	-0.5,	0.0,	1,	0,	1,		1,	0,		0.2, 0.4, 1.0,
				1.0,	0.5,	0.0,	0,	1,	1,		1,	1,		0.2, 0.4, 1.0,
				-1.0,	0.5,	0.0,	1,	1,	1,		0,  1,		0.2, 0.4, 1.0
			]);

		protected static const N:int = 11;

		protected var _context3D:Context3D;						// 3d graphics window on the stage
		protected var _shaderProgram:Program3D;					// compiled shaders
		protected var _vertexBuffer:VertexBuffer3D;				// vertexes used by our mesh
		protected var _indexBuffer:IndexBuffer3D;				// indexes of each vertex of the mesh
		protected var _modelMatrix:Matrix3D = new Matrix3D();	// transformation matrix

		public function Main6(screen:Sprite = null) {
			if (screen)
				screen.addChild(this);

			stage.scaleMode = StageScaleMode.NO_SCALE;
			stage.align = StageAlign.TOP_LEFT;

			stage.stage3Ds[0].addEventListener(Event.CONTEXT3D_CREATE, onContext3DCreate);
			stage.stage3Ds[0].requestContext3D();
		}

		protected function onContext3DCreate(event:Event):void {
			_context3D = Stage3D(event.currentTarget).context3D;

			if (!_context3D)
				return;

			_context3D.enableErrorChecking = true;
			_context3D.configureBackBuffer(stage.stageWidth, stage.stageHeight, 0, false);

			_context3D.setBlendFactors(Context3DBlendFactor.SOURCE_ALPHA, Context3DBlendFactor.ONE_MINUS_SOURCE_ALPHA);
			_context3D.setDepthTest(false, Context3DCompareMode.LESS);

			_vertexBuffer = _context3D.createVertexBuffer( VERTICES.length/N, N);
			_vertexBuffer.uploadFromVector(VERTICES, 0, VERTICES.length/N);
			_context3D.setVertexBufferAt( 0, _vertexBuffer,  0, Context3DVertexBufferFormat.FLOAT_3 );
			_context3D.setVertexBufferAt( 1, _vertexBuffer,  6, Context3DVertexBufferFormat.FLOAT_2 );
			_context3D.setVertexBufferAt( 2, _vertexBuffer,  3, Context3DVertexBufferFormat.FLOAT_3 );
			_context3D.setVertexBufferAt( 3, _vertexBuffer,  8, Context3DVertexBufferFormat.FLOAT_3 );

			_context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.<Number>([ 0.5, 0.0, 200, 1.0 ]));	// va4  centre, radius squared, 0.0, 0.0

			_indexBuffer = _context3D.createIndexBuffer( INDICES.length );
			_indexBuffer.uploadFromVector(INDICES, 0, INDICES.length );

			_modelMatrix.identity();
			_modelMatrix.appendScale(0.5, 0.5*stage.stageWidth/stage.stageHeight, 0.5);
			_context3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, _modelMatrix);

			var vertexShaderAssembler:AGALMiniAssembler = new AGALMiniAssembler();
			vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,
				"m44 op, va0, vc0\n" +	// 4x4 matrix multiply
				"mov v0, va1\n" +		// uv
				"mov v1, va2\n"	+		// rgb
				"mov v2, va3\n"			// curve
			);

			var fragmentShaderAssembler:AGALMiniAssembler = new AGALMiniAssembler();
			fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,
				"mov ft3, v1\n" +
				"sub ft0.xy, v0.xy, fc4.xx\n" +		// vector from centre
				"sub ft4.xy, fc4.xx, v2.xy\n" +		// 0.5 - curve radius

				"abs ft1.xy, ft0.xy\n" +			// absolute
				"sub ft1.xy, ft1.xy, ft4.xy\n" +	// relative to the centre of a corner circle

 				"div ft4.z, v2.x, v2.y\n" +			// aspect ratio of the curved corners
				"mul ft1.y, ft1.y, ft4.z\n" +		// modify circles by aspect ratio

				"mul ft2.xy, ft1.xy, ft1.xy\n" +	// square
				"add ft2.x, ft2.x, ft2.y\n" +		// add x and y
				"sqt ft2.x, ft2.x\n" +				// square root to give distance from centre of circle

				"add ft2.z, fc4.w, v2.x\n" +		// 1 + radius
				"sub ft2.z, ft2.z, ft2.x\n" +		//
				"pow ft3.w, ft2.z, fc4.z\n" +		// feathering to anti-alias circle

				"slt ft4.xy, ft1.xy, fc4.yy\n" +	// Are we in a corner?
				"add ft4.z, ft4.x, ft4.y\n" +		//
			 	"add ft3.w, ft3.w, ft4.z\n" +		// If not, alpha>1.0

				"mul ft0.z, ft0.y, v2.z\n" +		// Option to completely fill top or bottom section
				"slt ft0.w, ft0.z, fc4.y\n" +		//
				"add ft3.w, ft3.w, ft0.w\n" +		// If fill, alpha>1.0

				"sat ft3.w, ft3.w\n" +				// 0 <= alpha <=1 (may not be necessary)
				"mov oc, ft3\n"						// output colour
			);

			_shaderProgram = _context3D.createProgram();
			_shaderProgram.upload( vertexShaderAssembler.agalcode, fragmentShaderAssembler.agalcode);
			_context3D.setProgram ( _shaderProgram );

			_context3D.clear(0,0,0);
			_context3D.drawTriangles( _indexBuffer);
			_context3D.present();
		}
	}
}

And here’s the result:-

Here, we’ve set a flag parameter to 1.0 to specify curves at the top.  We can set the flag to -1.0 for curves at the bottom, or 0.0 for curves on all four corners.

Note that this shader is also capable of drawing rectangles, squares, circles, and ellipses, which are all special cases of rounded rectangles.

As an example, I’ve applied my shader to build a composition of shapes to make up a button.  Rendered using Stage3D:-

If buttons, are possible, so are all the other components.  This is all still in the experimental phase.  But it certainly WOULD be possible to do the graphics UI rendering of MadComponents, using Stage 3D graphics.  I’d anticipate that performance could benefit as a result.  But it will might take quite a bit of development effort to move from these experiments, to a new UI renderer engine within MadComponents.

Any volunteers?

Facebook Page

For community discussions about MadComponents, please join the facebook page:-

http://www.facebook.com/groups/336764056350846/

About these ads

8 Comments

Leave a Comment
  1. Keith / Apr 10 2012 12:28 pm

    Thanks for the post. You answered a question I was starting to ask just days ago. I’ve not started my first MadComponents project, but I was wonder if something like foxhole would be better than madcomponents. Although foxhole seems immature at this point I didn’t know if it would perform better.

  2. Bevis (@coodoo88) / Apr 10 2012 1:24 pm

    This is just a quick thought that I’m wondering is it possible to take a combined path, like rendering the component in vector off screen, then grab the bitmapData out of it and hand over to Starling, in this sense, Starling is mainly responsible for bitmap placement….etc, and vector display list are responsible for drawing the component to correct size and styles.

    -j

    • Daniel Freeman / Apr 11 2012 11:19 pm

      I do indeed intend to render pages as bitmaps, and move page-sized quad textures around for smooth page transitions. I can just use Stage3D. Starling has a lot of functionality I won’t use – and is intended more for people who don’t want to get into AGAL.

  3. sasmaster / Apr 11 2012 6:05 pm

    You can use shaders for UI as long as your shapes are geometric primitives.But if you want to draw some more freestyle stuff on GPU you have to triangulate your vector graphics.

    • Daniel Freeman / Apr 11 2012 11:14 pm

      Have you tried doing that? Using shaders for freestyle vector graphics? Straight line shapes are obviously easily constructed with vertexes. I was thinking that my circle shader could possibly be modified to approximate a quadratic bezier. But I’m going to write a nine patch shader too.

  4. Pierre Chamberlain / Jun 20 2012 12:15 pm

    Daniel, very nice tutorial!

    Is MadComponents currently available as a whole UI suite of Stage3D Components? Or is it still only available as traditional DisplayList UI components?

    If you know of any Stage3D UI Frameworks, especially one that supports Charting components as well, I would really appreciate if you could forward me some information! Thanks!

    -Pierre

  5. Harry / Jan 14 2013 9:18 am

    Hi Daniel,

    Great work and explanation.
    I know that AGAL can be a drag, however I recently found this Adobe project that might make it easier for you or any other volunteers to achieve a stage3D rendering engine for your components (it might be handy if in the future Adobe decides to put most of its resources towards that direction and GPU rendering is let bit back). You can find the source code here

    https://github.com/adobe/glsl2agal

    and a demo link in the readme section.

    Also in the next version of AGAL I read that conditional structures will be added!

    So thanks for the great work and good luck,
    Harry

To discuss MadComponents/MC3D, join the Facebook group!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 89 other followers

%d bloggers like this: