Code Slinger: A DP Blog

FlexUnit 4 in 360 seconds

Posted At: May 3, 2009 10:05 PM | Posted By: Michael Labriola
Related Categories: DP News, FlexUnit, Fluint, Projects

About a year back I remember reading a blog post called JUnit in 60 seconds. At the time I pondered how great it would be to have these features in Flex. Little did I know that today I would be writing this post introducing the Flex world to FlexUnit 4.

So, first a little background. FlexUnit 4 is the name for an upcoming release of FlexUnit. It represents the best features of the FlexUnit project combined with the best features of the Fluint project. It is built on top of a newly created foundation designed to support the latest techniques used in the JUnit testing community, but written for the specific requirements and needs of the Flash Player. Top all that off with an extensibility layer that encourages developers to create new types of test runners and extensions while simplify the process of integrating the results into IDEs and continuous integration environments, and it should give you an idea why I am excited about this release.

If it sounds interesting to you as well, the public alpha of this upcoming release is now available as a turnkey test project you can download from the adobe open source site. I hope you will use it to learn and explore the new features, and to provide feedback about any bugs you find along the way. However, it is an alpha, so locations and names of classes, signatures of methods and even which features are supported are all subject to change before release. It is not be advisable to use this as your production testing system.

Speaking of features, let’s jump in. There is a bit more content to go through than the original 60 second tutorial, so, I think you will need 4-6 minutes to get through it all. To keep things consistent, many of these examples are adapted right from the JUnit in 60 seconds site referenced above.

  1. Test Metadata
    Test cases are now marked with a piece of metadata named [Test]. Your tests no longer need any special name (prefixed with test, etc.) Also, the need for specific Test and Suite classes disappears. Your classes no longer need to inherit from any framework class. Here are a couple of sample tests. [Test] public function addition():void { Assert.assertEquals(12, simpleMath.add(7, 5)); } [Test] public function subtraction():void { Assert.assertEquals(9, simpleMath.subtract(12, 3)); }
    Because your test classes no longer inherit from a class in the FlexUnit framework, you will notice that the assert functions you used in the past (assertEquals, assertTrue) are now referenced as static functions of the Assert class; more on the new ways of asserting later in this post.

  2. Before and After
    Sometimes you need to setup your test environment (or fixture) for your tests. In the example above, you need to ensure your simpleMath reference exists before the tests are run. In previous versions of FlexUnit and Fluint you could override a setup() or teardown() method to accomplish this goal. FlexUnit 4 introduces Before and After metadata which accomplishes a similar goal. Any methods marked with Before will be run before each test method. Any methods marked with After will be run after each test method. This also means you can have multiple methods that run before or after the test.

    [Before] public function runBeforeEveryTest():void { simpleMath = new SimpleMath(); } [Before] public function alsoRunBeforeEveryTest():void { simpleMath1 = new SimpleMath(); } [After] public function runAfterEveryTest():void { simpleMath = null; simpleMath1 = null; }
    If you do choose to use multiple before or after, you can control the order that these methods execute using an order parameter. So, for example [Before(order=1)], [Before(order=2)].

  3. BeforeClass and AfterClass
    Methods marked with Before and After will run before and after each test method respectively. BeforeClass and AfterClass allow you to define static methods that will run once before and after the entire test class. Like Before and After, you can define multiple methods for BeforeClass and AfterClass and can control the order. [BeforeClass] public static function runBeforeClass():void { // run for one time before all test cases } [AfterClass] public static function runAfterClass():void { // run for one time after all test cases }
  4. Exception Handling
    Test metadata can also have an expects parameter. The expects parameter allows you to indicate that a given test is expected to throw an exception. If the test throws the named exception it is considered a success, if it does not, it is considered a failure. This prevents us from having to write tests wrapped in a try block with an empty catch. [Test(expects="flash.errors.IOError")] public function doIOError():void { //a test which causes an IOError } Or
    [Test(expects="TypeError")] public function divisionWithException():void { simpleMath.divide( 11, 0 ); }
  5. Ignore
    Ignore metadata can be added before any test case you want to ignore. You can also add a string which indicates why you are ignoring the test. Unlike commenting out a test, these tests will still appear in the output reminding you to fix and/or complete these methods. [Ignore("Not Ready to Run")] [Test] public function multiplication():void { Assert.assertEquals(15, simpleMath.multiply(3, 5)); }
  6. Async
    In previous versions of FlexUnit it was difficult to have multiple asynchronous events and to test code that was event driven but not always asynchronous. Fluint provides enhanced asynchronous support including asynchronous setup and teardown, but every test carried the overhead of the asynchronous code to facilitate this feature. FlexUnit 4 allows the developer to specify which tests need asynchronous support using the async parameter. When provided, the async parameter enables the full asynchronous support provided by Fluint for that particular test. When the async parameter is specified you may also specify an optional timeout for the method. [Before(async,timeout="250")] public function setMeUp():void { } [After(async,timeout="250")] public function allDone():void { } [Test(async,timeout="500")] public function doSomethingAsynchronous():void { //Async.proceedOnEvent( testCase, target, eventName ); //Async.failOnEvent( testCase, target, eventName ); //Async.handleEvent( testCase, target, eventName, eventHandler ); //Async.asyncHandler( testCase, eventHandler ); //Async.asyncResponder( testCase, responder ); }
    In addition to the async parameter, there are several new Async methods, each of which can also take individual timeouts, handlers and passThroughData.

  7. Hamcrest
    Earlier I alluded to new assertions. Thanks to the hamcrest-as3 project we now have the power of Hamcrest assertions. Hamcrest is based on the idea of matchers which match conditions for your assertions. For example:

    [Test] public function testGreaterThan():void { assertThat( 11, greaterThan(3) ); } [Test] public function isItInHere():void { var someArray:Array = [ 'a', 'b', 'c', 'd', 'e', 'f' ]; assertThat( someArray, hasItems("b", "c") ); }
    For more information on hamcrest:

  8. Suites
    FlexUnit 4 has a concept of test suites just like FlexUnit and Fluint. Test suites are just a collection of classes that represent tests or even other suites. A suite is defined by the [Suite] metadata. However, in FlexUnit 4, you also need to provide one additional piece of metadata called [RunWith] which instructs the test runner to execute the tests defined below using a specific class. The [RunWith] metadata forms the basis of the extensibility layer which will be discussed shortly.

    [Suite] [RunWith("org.flexunit.runners.Suite")] public class FlexUnitIn360 { public var t1:BasicMathTest; public var t2:MyTheory; }
    The test cases and any nested test suites, are simply defined as public variables. There is no need to instantiate them or mark them in any other way. FlexUnit’s test suite code understands how to recursively parse this class and find the tests.

  9. User Defined Metadata Parameters
    It’s often extremely useful to include additional pieces of information which are relevant to your development process when defining tests. So, for example, you might want to provide a detailed description of what a test is supposed to prove. This description could then be displayed if the test fails. Or perhaps you would like to note that a test relates to a give issue number in your bug tracking system. These custom parameters are stored by the framework when encountered during the test and can be used in reporting the success or failure later.

    [Test(description="This one makes sure something works",issueID="12345")] public function checkSomething():void { }
  10. Theories, Datapoints and Assumptions
    This is probably the largest single new feature as it introduces a whole new way of testing. A developer can create theories, which are ‘insights’ into the way a given test should behave or over a large, potentially infinite set of values. In other words these are tests that take parameters. The parameters are defined in properties, arrays or can be retrieved from functions or other external sources. A complete description of this feature can and will take a lot of documentation, however, if you are up for reading a bit of theory, this document will introduce the ideas . Here is a quick sample of using these new techniques:

    [DataPoints] [ArrayElementType("String")] public static var stringValues:Array = ["one","two","three","four","five"]; [DataPoint] public static var values1:int = 2; [DataPoint] public static var values2:int = 4; [DataPoints] [ArrayElementType("int")] public static function provideData():Array { return [-10, 0, 2, 4, 8, 16 ]; } [Theory] public function testDivideMultiply( value1:int, value2:int ):void { assumeThat( value2, greaterThan( 0 ) ); var div:Number = simpleMath.divide( value1, value2 ); var mul:Number = simpleMath.multiply( div, value2 ); Assert.assertEquals( mul, value1 ); } [Theory] public function testStringIntCombo( value:int, stringValue:String ):void { //call some method and do something }
    In this case, there are datapoints defined by static properties as well as method calls. The framework introspects the datapoints and uses this data combined along with any type specified in the ArrayElementType metadata. This information is used in combination with the theory method signatures to call each theory for each possible combination of parameters.

  11. RunWith
    FlexUnit 4 is nothing more than a set of runners combined to run a complete set of tests. A runner is a class that implements a specific interface and understands how to find, execute and report back information about any tests in a given class. Each time a new class is encountered, FlexUnit 4 works through a list of possible runners and attempts to identify the correct one to execute the tests contained in the class.
    The RunWith metadata allows you to override the default choice made by the framework and specify a different class to act as the runner. This feature allows developers to write entirely new types of runners, with support for new features, which can work directly with the existing framework and report their results back through the same interface.
    In the case of the suite, you are instructing the framework to run this class in a specialized runner that simply finds the correct runner for all of the classes it contains.

    [RunWith("org.flexunit.runners.Suite")]

  12. Adapters
    Using the flexibility of the multiple runners discussed above, the new FlexUnit 4 framework has legacy runners built in for both FlexUnit 1 and Fluint tests. This means that FlexUnit 4 is completely backwards compatible; all existing FlexUnit and Fluint tests can be run, and even mixed into suites with FlexUnit 4 tests without any code changes.

    Further, supplemental runners are in development for FUnit and several other testing projects

  13. User Interface Facade
    Lastly FlexUnit 4 provides a UIComponent testing facade which allows you to add or remove components from the display list. This allows you to accurately test component methods in a real runtime state. This feature creates a foundation for other projects to extend into areas of integration and functional testing without the need for extensive rewrites or modifications.

    [Before(async,ui)] public function setUp():void { //Create a textInput, add it to the testEnvironment. Wait until it is created, then run tests on it textInput = new TextInput(); Async.proceedOnEvent( this, textInput, FlexEvent.CREATION_COMPLETE, 200 ); UIImpersonator.addChild( textInput ); }

If you made it this far, I hope you download the alpha and start playing with it immediately. If you have significant time to devote to serious testing and debugging of the framework, contact me and I will be happy to invite you to the ongoing private beta program.

Stay tuned for some exciting news and, if you have the opportunity, be sure to make it 360|Flex for my session about the new framework. Plus, you never know, 360 is an exciting place, we may just have more to tell you by that time.

Cheers and enjoy,
Labriola

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • PDF
  • RSS
  • email
  • Technorati

Comments (43)

43 Responses to “FlexUnit 4 in 360 seconds”

  1. Aaron says:

    Can’t wait to dig into this! Thanks so much for the great work.

  2. Conrad Winchester says:

    WOW :-) This is fantastic – will start using immediately.

    Do you have any plans for combiing with a mocking framework?

  3. Jago Macleod says:

    I am excited to see the powerful features of fluint pulled into the FlexUnit project! I have to admit that it looks like a lot of metadata… But looks like the right direction. Looking forward to playing around with it !

  4. Jan K says:

    Looking VERY good!

  5. Jago Macleod says:

    Great to see the asynchronous aspects of fluint coming to FlexUnit! Does this include full blown SequenceRunners? I have to admit, it looks like a heavy dose of metadata, but excited to play around with it.

  6. Constantiner says:

    Looking great! But can you explain how it relates to the official Adobe’s FlexUnit framework? Is it new version of it?

  7. Brennan says:

    Any ETA on the new release date?

  8. @Brennan – No eta as of yet. We just hit alpha yesterday, so we need to gain some feedback.

  9. Justin Reidy says:

    REALLY excited to see this! The excuses for not testing a Flex project keep going down like dominoes. Thanks for all your hard work.

  10. Understood. Would be cool to also have instance methods that run before and after the tests like setUp() and tearDown() did.

  11. Chad Upton says:

    Wow. I thought Fluint was awesome, you’ve really outdone yourself here.

  12. @Ben:

    That’s what Before and After do (mirror setup and teardown), they are for instances

    BeforeClass and AfterClass are the static ones.

    … you get both now

    Mike

  13. Borek says:

    This is fantastic news! Can’t wait to get my hands on the FlexUnit 4 release, Flex really needs some unit testing love.

    Thanks for your effort,
    Borek

  14. Looks awesome Mike! I’m still a bit of a testing noob, and maybe its simply for JUnit compatibility, but the stipulation that [BeforeClass] and [AfterClass] have to be static methods seems odd and potentially frustrating. Any chance there will be support for instance methods that run at the beginning and end of the test class?

  15. A few quick answers:

    @Ben: Need to be static BeforeClass happens before the constructor
    @Conrad: There are several people working on mocking with the new framework
    @Constantiner: This will be the official FlexUnit framework once released

  16. Ruprict says:

    Will the existing fluint ant tasks work with the alpha bits (HA! "alpha bits")?

    (hope this is not a dumb q…)

  17. Ruprict says:

    So, not sure if this is material for the Google Group as y’all are in alpha, but the assert counts don’t work in the runner in your TurnKey project. I understand…alpha…but I figured I’d mention it anyway.

    Just messed with the UIImpersonator stuff. Very cool.

  18. @Ruprict: Existing fluint ant tasks will not work, but there are revised tasks coming soon

    As far as the assert count, this is a known. Fluint and the JUnit didn’t have this exact concept but FlexUnit 1 did.. so we need to finish that marriage.

  19. todd says:

    I look forward to having ONE unit testing software package to rule them all. It seems thus far, every six months, I have to choose between half-dozen half-baked solutions, or take an existing one and extend it somehow.

    This one, however, looks excellent! Good work.

  20. Tim Robles says:

    This is exciting stuff. I’m really glad to see unit testing and a officially supported framework progressing, as this looks to be a fairly significant release. I’d definitely be interested in seeing how the runners themselves integrate things like Cruise Control. I’d also love to contribute if you need some additional beta testers.

  21. Gary Mc says:

    Great news Michael ! I saw your Dense & Hot presentation at WebDU last year. Its great someone of your experience is putting some serious effort into this crucial library. I did try Fluint, but after a few hours just couldn’t get it to work, so I’m hoping you’ll be able to resolve some of the usability hurdles too.

    In the past I have written (but not released) forks of the DUnit and NUnit test runners, and now here’s the beginnings of my very rough hacked FlexUnit http://buzzware.com.au/Images/XRunnerCode.zip

    Here’s my basic requirements for a test runner, which I’ve achieved some of. The basic goal is to enable TDD such that writing tests is not a special activity – it is a comfortable and efficient way to write code and execute code. All code in development needs to be executed either in the context of the app, or something else, so it may as well run in a test without the usual series of clicks required to ‘test’ code in the app. The quality benefits then come for free.

    * tests compiled in "modules" http://livedocs.adobe.com/flex/3/html/help.html?content=modular_2.html
    * tests loaded and executed by a precompiled Air runner application. This enables use of the console, and allows eg test data to be loaded from local files.
    * execute from the OS commandline, with no mouse activity required. Results, progress, errors & trace dumped to stdio. I can then write the tests and code, compile & execute the test, see the output and fix the code all without touching the mouse or leaving my editor (SlickEdit) which has a built in console. I still use Flex Builder for GUI design and full debugging when required.

    * when eg. assertEquals(‘testing A’,'A’,'B’) the following line is sent to trace :

    assertEquals failed: Was ‘B’ (Expected ‘A’) testing A

    * when eg. assertEquals(‘testing A’,'A’,'A’) the following line is sent to trace :

    assertEquals OK: Value ‘A’ testing A

    This is great for debugging, as the output for the asserts provides a history of important values before things went wrong, which may save single stepping through it again. It annoys me when test runners merely output a dot when a test or assert runs succesfully. Why not output all that information I’ve put into assert statements ?

    With NUnit I had all this working great, and the output was written in XML to be reformatted into a printable report for presentation to a QA committee when justifying changes (it was a medical device company).

    Also, have you seen the ruby RSpec syntax ?

    eg. a.should == b

    kinda hard to implement I suppose but cute. Rspec also has the ability to specify a requirement eg

    should "accept A & B and give output C"

    which is a placeholder for a future test to be implemented, and the text is simply output as given.

    Cheers

    Gary

  22. Gary,

    Sorry you had problems with Fluint before. FlexUnit is similar though, just a library, so be sure to let me know if you have suggestions on better docs, etc. to make it all easier.

    Almost all of the features you requested in your post are already present in Fluint, so they will absolutely be present in the new version. Full debugging is a bit of a problem as AIR doesn’t presently allow us to create a debug version of a SWF inside of a deployed AIR app.

    As far as your assertion requirements, look at hamcrest-as3 when you have a chance. That is the direction we have proceeded and I like it quite a bit.

    Labriola

  23. JinyongZhang says:

    Great work Michael!
    But there are still the same small error in the official aricle(http://opensource.adobe.com/wiki/display/flexunit/FlexUnit+4+feature+overview),i suggest that it should be corrected now!
    I found that the version of flexUnit with flash builder beta is 4 but it can not support some latest features above.I would like to ask thas flash builder will fully support flexunit when release?
    Look forward to more surprises.

  24. JinyongZhang says:

    Great work Michael!
    But there are still the same small error in the official aricle(http://opensource.adobe.com/wiki/display/flexunit/FlexUnit+4+feature+overview),i suggest that it should be corrected now!
    I found that the version of flexUnit with flash builder beta is 4 but it can not support some latest features above.I would like to ask thas flash builder will fully support flexunit when release?
    Look forward to more surprises.

  25. Ignacio says:

    Great article Michael!
    I just point out that syntax of expected exceptions is wrong (or perhaps changed in latest version).
    Instead of "expecting" is "expected" like in:
    [Test(expected="ArgumentError")]

  26. Ignacio,

    This was a mistake, however, in the lastest version (in SVN) expects and expected both work.

    That new version will be available in a beta 2 release soon.

  27. craig says:

    I see you said to look at for your presentation at 360|Flex…I searched all the PDF’s up on SlideShare (http://www.slideshare.net/search/slideshow?lang=en&submit=post&q=+360flexindy&commit=search) and couldn’t find it..where can we get the presentation?

    Thanks!

  28. Joel says:

    looks good, haven’t finished reading it but FYI: the code samples DO NOT RENDER properly in Google Chrome browser.
    thanks for posting this excellent blog.
    jr

  29. Can I do Async Theories with this framework?

  30. Andrew,

    No, they will not work reliably at this time. It is on our roadmap though for the near future.

    Mike

  31. Mike,

    Glad to hear you guys have plans for that. +1 vote for the feature of doing async Theories.

  32. Loren Norman says:

    This is excellent news, I’m diving in and reviving our test suite to make use of FlexUnit 4 right now. It’s also very tempting to write my own runners, since I prefer the RSpec style of testing, but that’s getting ahead of myself.

    I see there’s no doc directory in the download, and (correct me if I’m wrong, but) I don’t think you can run asdoc on a .swc file. Is the API for FlexUnit 4 publicly available somewhere? Or the source code?

    Thanks!

  33. We are only in beta 1 so the docs directory does not yet exist.

    You can find all of the code for the project at the adobe open source repo.

    Basically, goto opensource.adobe.com, find the FlexUnit project and it will give you a source path in SVN. We are working out of the 4.x branch.

    I am avoiding posting the URL not to be difficult but because we really don’t need search engines crawling the repo :)

  34. Gary says:

    Re RSpec style testing and FlexUnit, this should be a big step closer to happening with this project :
    http://visionmedia.github.com/jspec/
    Remember ActionScript is pretty much a superset of JS.

  35. Loren Norman says:

    I’ve used JSpec extensively, actually, and I’d say you’re both right. JSpec doesn’t represent any grand usefulness to AS3 as it is, but it is a good example of how such a test runner might be written for AS3. (Also check out Screw Unit.)

    The point is to embrace the fact that functions are first class citizens, and make a closure-based test-runner. Writing tests this way is much more expressive, even if AS3 is a bit verbose (ie look at RSpec or Shoulda in Ruby and cry.)

    Here’s what I imagine it could look like in AS3 with such a runner (hope this formats well):

    context(function("When a user visits the Settings panel"):void {

    should(function("be able to see the Volume control"):void {

    // volume control test

    });

    should(function("be able to adjust the FX volume"):void {

    // FX volume test

    });

    // etc…

    });

    You can imagine how the testdoc output would look.

  36. Gary McGhee says:

    Michael, what about "ActionScript 3.0 is based on ECMAScript, the international standardized programming language for scripting. ActionScript 3.0 is compliant with the ECMAScript Language Specification, Third Edition (ECMA-262). It also contains functionality based on ongoing work on ECMAScript Edition 4, occurring within the ECMA standards body." from http://www.adobe.com/devnet/actionscript/articles/actionscript3_overview.html ?
    Also I was amazed at the success I had here : http://www.buzzware.com.au/_blog/Coffee_into_Code/post/Pure_Actionscript_HTML_Parser_(with_thanks_to_John_Resig)

  37. Gary:

    Have to disagree with you somewhat there. At this point actionscipt and javascript may have a common ancestry but, especially in terms of use, have less in common than ever in their past.

  38. Russ Schmidt says:

    Michael,

    Would you please post the preso deck from your MAX 2009 session about TDD? Unfortunately that session was one of the few that didn’t have a video recorded. Thanks!

  39. ahabman says:

    Is there any good documentation for FlexUnit 4? All the links on http://opensource.adobe.com/wiki/display/flexunit/Developer+Documentation refer to outdated code.

  40. http://www.flexunit.org should be your first stop for FlexUnit 4 documentation.

  41. Amit Gupta says:

    I like the async support in FlexUnit4. I follow TDD and here’s where I run into problem. I have a proxy service class Service.as where I make my URLLoader calls and handle the events. When I write a Unit Test, I call the method in Service.as (eg. login() ) and my event handler is in Service.as is setting some property (eg. loggedIn flag to true/false). In my test case I want to assert on this property and make sure it is getting set properly. But I couldn’t figure out a way to do that. It seems I have to write an event handler in the test case itself which breaks away from TDD.

  42. Amy says:

    The turnkey build of FlexUnit seems to be a Flash extension (fxp). I don’t have Flash on my machine, so I’m at a bit of a loss as to how to use it. The Flex 3.5 version will not download. Any thoughts?

  43. Michael Labriola says:

    FXP is a Flash Builder extension. It is basically a zip file. If you want to see the code, download the FXP file, rename it to a .zip file and extract it.

    Mike

Leave a Reply