Strongly Typed: A DP Blog

Playing HLS Video in Flash

Posted At: August 22, 2012 12:02 PM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex, Video

**Updated with fixed links. Sorry about that!
GitHub


I’ve recently started work on a side project. It’s going to be a front end client for a media management server (Plex), which means it’s going to serve up lists of content and play back videos/music. One of the things that makes Plex a great media server is that it can transcode videos on the fly to optimize playback for the device that’s requesting the file. The problem with this is that the transcoding spits out HLS which is not playable in the vast majority of web browsers.

Luckily, there are a lot of smart people out there and I was able to find an open source HLS plugin for Adobe’s OSMF project. The only downside is that the plugin didn’t quite work right in the latest version of OSMF (2.0 at the time of writing). That’s OK though, because I know a lot about OSMF and I was able to update the plugin.

The plugin I’m using isn’t 100%. It plays video just fine, but I’m sure there’s a lot of parts of the HLS specification that aren’t supported. In particular, live streaming (especially DVR) may not work properly or at all. One major issue is that seeking isn’t working right now. I’ll look into that at a future time, as that’s something I most definitely need for my project.

The open source project I forked is available on Google Code here.
My version of the plugin that works with OSMF 2.0, along with a sample player.
See the example in action here.

So while this is far from complete, it’s a good start. More importantly, it helps to demonstrate that HLS playback is entirely viable in Flash which, in my opinion, is going to be super important going forward.

There are several other Flash HLS implementations which may do the job better. I wanted to use OSMF though, which is why I’ve got with the particular library that I did. Of note, here are two main projects that enable HLS in Flash.
A expensive, closed source OSMF plugin.
This is a branch of JW Player which supports HLS.

Comments (7)

Where is the iOS SDK?!

Posted At: April 12, 2012 9:26 AM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

In my previous posts about writing native extensions for iOS, I pointed out that it’s important to tell Flash Builder where the iOS SDK is located. When I installed Xcode on a Snow Leopard machine the SDK was located at:
/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk
I set up a brand new Macbook today which is running Lion and after installing Xcode the SDK was not in the same spot! After some digging I found it here:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.1.sdk
Whew, that was a mouthful.

Comments (4)

Building a Native Extension | Part 5 – Putting It All Together

Posted At: April 10, 2012 8:44 AM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

This is the last step of the native extension build process. Did you miss something or are you late to the party? Jump back to step 1.

Now that all of the supporting projects have been built and compiled, it’s time to put it all together.  For native extensions, that means building a .ane file.

Building the ANE

1) Put all of the files into a folder together.  The structure should look like:

build directory
     -extension.xml
     -SWC for main ActionScript library

     android
          -androidLib.jar
          -library.swf (main ActionScript library)
     ios
          -iOSLib.a
          -library.swf (main ActionScript library)
          -platformoptions.xml (if required)
     default
          -library.swf (default library)

2) Run the following command:

adt –package –storetype pkcs12 –keystore cert.p12 –storepass XXXX –target ane Name.ane extension.xml –swc Name.swc –platform iPhone-ARM –C iOS . –platformoptions ios/platformoptions.xml –platform Android-ARM –C android . –platform default –C default .

Exclude any platforms that you aren’t supporting.

The –C directory . indicates that all of the files in directory should be included in the ANE.  The command –C directory library.swf would make only the file directory/library.swf be included in the ANE.

Signing the ANE is optional, but Flash Builder will have a warning when using unsigned native extensions.  You can use any p12 to sign a native extension, including a p12 built from Flash Builder during the release build process, so it’s easy to sign the ANE.

3) Include the ANE in the application.

Go to the properties of your project.  Navigate to “Flex Build Path” and go to the “Native Extensions” tab.  Click “Add ANE…” and find the ANE you built.

At some point, either during this step or one of the following steps, Flash Builder should indicate that the package of the ANE must be included in the app descriptor file and that Flash Builder will automatically include it.  Say yes to this prompt.

Now that the ANE has been added, you must toggle it on for each platform you are supporting.  Expand the “Flex Build Packaging” item.

If supporting iOS, click on the “Apple iOS” item.  Go to the “Native Extensions” tab.  Be sure that the checkbox in the “Package” column next to your ANE is checked.  Also be sure to reference the iOS SDK (as mentioned previously) in the “Apple iOS SDK” input.  Click “Apply”.

If supporting Android, click on the “Google Android” item.  Go to the “Native Extensions” tab.  Be sure that the checkbox in the “Package” column next to your ANE is checked.  Click “Apply”.

Now click OK and clean your project.  You’re ready to implement the native extension now!

Note: If you make any changes that require you to rebuild the ANE, I suggest removing the ANE from the “Flex Build Path -> Native Extensions” tab and redo the steps in item #3 (this one!) to re-add the ANE.  I’ve has some caching issues where an old version of the ANE is used if I didn’t totally remove it and start the add process over from scratch.

Note: If you make changes to any of the libraries, be sure to update the files in the build directory!  It’s especially easy to overlooking extracting the library.swf file from the main ActionScript library’s swc file.  Also be sure to place this library.swf file in both the iOS and Android directories.

Note: It’s very possible to automate this process via ANT.  I don’t cover it here, but many other examples use ANT to perform the entire packaging process.  You can even extract the library.swf from the swc files using ANT.

Wrap-up

Creating a native extension has a lot of steps and it’s easy to overlook some of the finer details that are required to get the ANE working.  There are a lot of great examples out there so check them out and get an even better understanding of the power native extensions give you.

Comments (15)

Building a Native Extension | Part 4 – The Android Code

Posted At: April 10, 2012 8:36 AM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

Building the Android library was very interesting.  It was easier for me to write Java as it was more similar to ActionScript.  Some amount of debugging (log statements) is actually possible with zero work on the developer’s part which is a great addition (more on that later).  Also, using the AIR runtime extensions in Java is a much simpler process.  However, the Android operating system leaves a lot of work up to the developers, and you’ll find that some functionality is built into the iOS framework but that a solution will need to be built from scratch for Android.

The Android Library

Creating the Android library is pretty straight forward.  If you haven’t already, you should download the Android SDK and install the ADT Eclipse plugin into a plain copy of eclipse (not Flash Builder).  (You probably can install the ADT plugin into Flash Builder, but I personally found it easier to just keep them separate.)

1.  Open Eclipse and create a new Android project.  Name it whatever you want and choose the minimum SDK you wish to support.  Chances are you’ll want to pick 2.2 (the first version of Android that AIR supports).

2.  The AIR runtime extensions need to be added to the project.

Right click on the project and go to “properties”.

Click on the “Java Build Path” item and go to the “Libraries” tab.

Click on “Add External JARs” and navigate to the directory air_sdk_dir\lib\android.  Select the FlashRuntimeExtensions.jar file.

Press “OK”.

3.  Create a class that implements FREExtension.  The package and name for this class should match the initializer node for the Android-ARM platform in the extension.xml file. This class should have the methods createContext(), dispose(), and initialize().

The createContext() method accepts a string parameter which is the type of context to create.  This is the second parameter passed to the ExtensionContext.createExtensionContext() method.  The return value for createContext() should be a class that extends FREContext.

public FREContext createContext(String contextType) {
     return new VolumeExtensionContext();
}

The dispose() method will be called when the native extension is done being used by the developer.  Here you should do any clean up required.

The initialize() method is called when the native extension is finished being built.

4.  Create a class that extends FREContext.  This class should have the methods dispose() and getFunctions().

The dispose() method is just like the dispose method of the FREExtension.

The getFunctions() method returns a collection of FREFunction objects.  There should be one FREFunction for each method that is called from the main ActionScript library.  Each FREFunction is mapped to a string value, which is the name of the function.

public Map<String, FREFunction> getFunctions() {
     Map<String, FREFunction> functions = new HashMap<String, FREFunction>();

     functions.put("init", new InitFunction());
     functions.put("setVolume", new SetVolumeFunction());

     return functions;
}

5.  Create one class that implements FREFunction for each method invoked from the main ActionScript library.

These classes will each have a call() method.  This method is invoked when the function is called from the main ActionScript library.  This is where the heavy lifting happens in the native extension.  This is the place where you’ll add the native code to do whatever it is that your native extension does.

The call() method is passed an FREContext and an array of FREObject instances.  The FREContext is the extension context used to invoke the method.  The array of FREObject instances is a collection of the parameters passed to the method; one FREObject for each parameter.

FREObject is a java representation of an ActionScript object.  It is a wrapper class, so to get to the underlying value you must invoke methods of FREObject.  For example, in the setVolume() method we pull out the Number value that is the new volume.

public FREObject call(FREContext context, FREObject[] args) {
     Context appContext = context.getActivity().getApplicationContext();
     AudioManager aManager = (AudioManager) appContext.getSystemService(Context.AUDIO_SERVICE);

     double volume = 1;
     try {
          volume = args[0].getAsDouble();
     } catch (Exception e) {

     }

     int maxVolume = aManager.getStreamMaxVolume(AudioManager.STREAM_MUSIC);
     volume = volume * maxVolume;
     int index = (int) volume;

     aManager.setStreamVolume(AudioManager.STREAM_MUSIC, index, 0);

     return null;
}

The bolded line is where the value passed from ActionScript is extracted from the FREObject collection.

6.  Create the jar file.

Right click the on the Android project and select ‘Export’.  Expand the ‘Java’ item and select ‘JAR file’.  Be sure the project you’re exporting is checked in the “Select resources to export:” box.  Choose where to save the jar file and click Finish.

 

A note about debugging:

Similar to the console in Xcode, the Android Eclipse IDE provides a way to log out to a console.  You won’t be able to hit breakpoints but you can see log statements from the native code.

When using Log.d(TAG, String); statements the output will show up in the LogCat console when the AIR application using the extension is running on the device and the device is plugged in via usb.

How to open the LogCat window: While Eclipse is running, select the “Window” menu option. Then navigate to “Show View” -> “Other”. Expand the “Android” item and then select the “LogCat” item. Press “OK”.

The LogCat window shows all of the logging happening on the device, so you will want to filter (by pressing the green plus sign) by tag to see only your specific logging.

Continue to part 5

Comments (0)

Building a Native Extension | Part 3 – The iOS Code

Posted At: April 10, 2012 8:13 AM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

The iOS project is the part that, in my opinion, is the hardest.  There are plenty of mistakes that are very easy to accidentally make.  Hopefully this post will help everyone else avoid the same mistakes that I made.

The iOS Library

1.  Open Xcode and make a new project.  In the New Project dialog, select iOS -> Framework & Library project option, then select “Cocoa Touch Static Library”.

2.  You’ll see a .h file and a .m file named after your project.  You don’t need the .h so you can delete that.  After you delete the .h you should empty out the .m file since the default code isn’t needed anymore.  You’ll also want to click on the Project in the bar on the left side and search for “prototype” in the Build Settings.  Toggle the value of “Missing Function Prototypes” to no.  This is just to suppress warnings.

3.  Download the AIR sdk if you haven’t already.  Navigate to “AIR_SDK_DIR/include”.  There is a file named “FlashRuntimeExtensions.h”.  Drag that file into Xcode and drop it onto your project in the left side bar.  This includes the file in your project.  (It doesn’t actually copy the file though, so if you delete/move the original Xcode won’t find it anymore.)

4.  Open the main .m file and add #import "FlashRuntimeExtensions.h" to the top of the file.  Now you can access the AIR runtime classes and get the architecture for the extension set up.

5.  In the main .m file create a method called ContextInitializer.  This method is called when the extension context is created in AIR.  In this method you need to create a collection of functions that the main ActionScript library will call.  Here’s an example:

void ContextInitializer(void* extData, const uint8_t* ctxType, FREContext ctx, uint32_t* numFunctionsToTest, const FRENamedFunction** functionsToSet)
{
     *numFunctionsToTest = 2;
     FRENamedFunction* func = (FRENamedFunction*) malloc(sizeof(FRENamedFunction) * 2);

     func[0].name = (const uint8_t*) "init";
     func[0].functionData = NULL;
     func[0].function = &init;

     func[1].name = (const uint8_t*) "setVolume";
     func[1].functionData = NULL;
     func[1].function = &setVolume;

     *functionsToSet = func;
}

You can see that two functions are set up, init and setVolume.  We invoked these methods earlier in the VolumeController class.

6.  In the main .m file create a method called ContextFinalizer.  This is called when the extension context is being garbage collected.  Here you should clean up anything that is used by your native code.

void ContextFinalizer(FREContext ctx)
{
     return;
}

7.  Create two more methods.  These methods should match the values in the initializer and finalizer nodes in the iPhone-ARM platform of the extension.xml file.  The initializer method will set which methods to call when doing the actual initialization.

void VolumeExtensionInitializer(void** extDataToSet, FREContextInitializer* ctxInitializerToSet, FREContextFinalizer* ctxFinalizerToSet)
{
     *extDataToSet = NULL;
     *ctxInitializerToSet = &ContextInitializer;
     *ctxFinalizerToSet = &ContextFinalizer;
}

void VolumeExtensionFinalizer(void* extData)
{
     return;
}

8.  Create the methods reference in the collection made in step #5.  These methods are where you should add the native code to do whatever it is that your native extension does.

Each method will accept the same parameters.  The main properties you’re worried about are the FREContext, which is extension context, and the collection of FREObject instances, which are the parameters passed to the native function.  Each FREObject is a C representation of an ActionScript object.  The FREObject class is a wrapper, so you have to pull the value out using methods on FREObject.  See the below code for an example:

FREObject setVolume(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[])
{
     double newVolume;
     FREGetObjectAsDouble(argv[0], &newVolume);

     [[MPMusicPlayerController applicationMusicPlayer] setVolume: newVolume];

     return NULL;
}

9. Generating the compiled library.

Click on the project on the left hand side of the screen.  Click on the “Build Settings” tab.

Do a search for “products path”.  The directory specified in the “Per-configurations Build Projects Path” is where the compiled library will be put.

To change the directory just double click the current path and enter in the one you’d prefer.

After that’s done, press “command + b” which will make Xcode compile the library.  You’ll find a .a file in the directory that was specified above.

A note about debugging:

Debugging in native extensions isn’t the easiest thing, but both IDEs for iOS and Android provide a way to log out to a console.  You won’t be able to hit breakpoints but you can see log statements from the native code.

When using NSLog(NSString) statements the output will show up in the console available through the Xcode Organizer when the AIR application using the extension is running on the device and the device is plugged in via usb.

How to open the console: While Xcode is running, expand the “Window” menu option and select “Organizer”. Find your device in the left-hand list and hit the arrow to expand its contents. Select the “Console” option. The NSLog statements will show up here.

iOS Problems and Trap Doors


 

Making sure you support iOS 4.* devices.

By default Xcode sets the deployment target to match the SDK you are using.  Xcode now only comes with the iOS 5.0 SDK which means that by default your library project targets 5.0.

Click on the project name in bar on the left hand side.  In Build Setting do a search for “deployment” and change the “iOS Deployment Target” to be 4.0.

You’ll have to take this one step further by adding an xml file specifying the minimum required iOS version when packaging the ANE.  See the item “Using External Frameworks and Libraries” for more information.

Always Debug on a Mac

I do my development on a Windows PC.  So, naturally, after I built the native extension on the Mac (since Xcode is only available for OSX), I tried to compile a test application using an ANE on my PC.  This would generally crash and burn.  Sometimes compiling would fail.  Oftentimes the application just didn’t work as expected, or would crash for no apparent reason.  I had tried this prior to creating the Volume extension and the test extension was pretty basic, but it just didn’t work when compiling from a Windows PC.

I don’t know exactly why this happens, but it was something that multiple people in my office had trouble with.  Especially coupled with the next few pitfalls, you’ll find it’s impossible not to compile the builds on a Mac.

Using More Than One Native Extension

Using more than one native extension in the same application can lead to some unnecessary headaches if you aren’t careful. If the ContextInitializer and ContextFinalizer functions that are specified in the extensions initializer function (the one specified in the extension.xml initializer node) share the same name as the ContextInitializer and ContextFinalizer of a different native extension, only first native extension loaded will work. I think this is because Objective C isn’t letting the new functions be created within the scope of the extension because they already exist from the previous extension. I’m assuming functions are stored in some sort of global scope; I’m not an expert on Objective C though, so that’s just my initial guess.

That being said, you should name the internal initializer and finalizer functions uniquely. I’m guilty of not doing that in this extension, which is why I’m writing this note right now. Here’s an example of what I mean.

void MySuperUniqueContextInitializer(void* extData, const uint8_t* ctxType, FREContext ctx, uint32_t* numFunctionsToTest, const FRENamedFunction** functionsToSet)
{
     ...
     return;
}

void MySuperUniqueContextFinalizer(FREContext ctx)
{
     return;
}

void VolumeExtensionInitializer(void** extDataToSet, FREContextInitializer* ctxInitializerToSet, FREContextFinalizer* ctxFinalizerToSet)
{
     *extDataToSet = NULL;
     *ctxInitializerToSet = &MySuperUniqueContextInitializer;
     *ctxFinalizerToSet = &MySuperUniqueContextFinalizer;
}

Use the iOS 5.0 SDK when packaging the IPA

By default the AIR SDK compiles the IPA using the iOS 4.0 SDK.  If you want to use iOS 5.0 SDK features, or if you want to link in external libraries (see next item) you’ll have to tell Flash Builder where the iOS 5.0 SDK is located.  Also, as mentioned in the previous item, an application using an iOS native extension will just work better if you link in the SDK.

Go into the properties of your flex project.  Navigate to “Flex Build Packaging -> Apple iOS”.  Click the “Native Extensions” tab.  In the “Apple iOS SDK” input type the location of the SDK.  The default location is “/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk”.

If you are compiling via the command line, you can use the –platformsdk option to specify the location.

adt -package -target ipa-app-store -provisioning-profile ./myProfile.mobileprovision -storetype pkcs12 -keystore ./Certificates.p12 -storepass XXX myApp.ipa myApp-app.xml Main.swf -extdir ext/ -platformsdk /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk/

See here for more information.

Using External Frameworks and Libraries

AIR doesn’t link in shared iOS frameworks or libraries, even some of the ones included with the iOS SDK.  Here is a list of frameworks that AIR links in by default:

  • CoreFoundation
  • UIKit
  • MobileCoreServices
  • CoreGraphics
  • SystemConfiguration
  • AudioToolbox
  • CFNetwork
  • QuartzCore
  • CoreLocation
  • CoreMedia
  • CoreVideo
  • AVFoundation
  • Foundation
  • OpenGLES
  • Security

If you want to use any other library or framework you have to list them in an XML which gets compiled with the ANE.  One reason this is required is because Xcode links in external libraries at compile time of the main application.  So when you build a library project in Xcode the resulting .a file doesn’t contain any code from referenced libraries/frameworks.

If the frameworks are not included with the iOS SDK by default, you must copy them into the iOS SDK folder.  (You can also specify custom directories to search for frameworks in the XML file, however you have to specify the path as relative to the iOS SDK directory, so I find it easier to just copy the libraries/frameworks into the iOS SDK directory.)

Inside of the iOS SDK directory, copy the library/framework files into either the System/Library/Frameworks folder or the usr/lib folder.

Create an XML file with the following structure.

<platform xmlns="http://ns.adobe.com/air/extension/3.1">
     <sdkVersion>4.0</sdkVersion>
     <linkerOptions>
          <option>-ios_version_min 4.0</option>
          <option>-framework Twitter</option>
          <option>-liconv</option>
     </linkerOptions>
</platform>

This file does two important things.  First it specifies the version of iOS that you support.  Secondly, it tells the compiler which frameworks it should look for to compile into the ANE.  If referencing a framework (you can tell it’s a framework because the folder will be Name.framework) use the option value –framework name.  If it is a library (you can tell it’s a library because the file should be a *.dylib) then use the option value –name.

When compiling the ANE, use the –platformoptions option to specify the location of the xml file.

adt -package -target ane myextension.ane extension.xml -swc mySwc.swc -platform iPhone-ARM library.swf libmylib.a -platformoptions myplatformoptions.xml

See here for more information.

Using Entitlements

As of AIR 3.1 you can specify custom entitlements for your application by specifying them in the app descriptor file.

<iPhone>
     <!-- A list of plist key/value pairs to be added to the application Info.plist -->
     <InfoAdditions>
          <![CDATA[
               ...
          ]]>
     </InfoAdditions>
     <!-- A list of plist key/value pairs to be added to the application entitlements -->
     <Entitlements>
          <![CDATA[
               <key>keychain-access-groups</key>
               <array>
                    <string>...</string>
                    ...
               </array>
               ...
          ]]>
     </Entitlements>
</iPhone>

See here for more information.

Using Attached Devices

When using an attached device, such as a barcode scanner, you have to specify an entitlement in the app descriptor file.

Here’s the information from the iOS documentation:

To declare the protocols your app supports, you must include the UISupportedExternalAccessoryProtocols key in your app’s Info.plist file. This key contains an array of strings that identify the communications protocols that your app supports. Your app can include any number of protocols in this list and the protocols can be in any order. The system does not use this list to determine which protocol your app should choose; it uses it only to determine if your app is capable of communicating with the accessory. It is up to your code to choose an appropriate communications protocol when it begins talking to the accessory.

As the last section detailed, you can add the UISupportedExternalAccessoryProtocols entitlement to your application descriptor file.

Continue to part 4

Comments (15)

Building a Native Extension | Part 2 – The ActionScript Code

Posted At: April 9, 2012 5:00 PM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

In this post we’ll discuss the parts of the native extension that are written in ActionScript.  This includes in the main library and the default implementation.

The ActionScript Library

The main library is an ActionScript library which serves as an interface to the AIR application.  This library is also responsible for interacting with the extension context generated by AIR.  The extension context is used as a bridge between AIR and the native platform.

1.  Create a Flex Library project.

2.  Create a Controller class that extends EventDispatcher.

public class VolumeController extends EventDispatcher {
      public function VolumeController( enforcer:SingletonEnforcer ) {
           super();
      }
}

 

 

3.  Initialize the extension context.

extContext = ExtensionContext.createExtensionContext("net.digitalprimates.volume", "" );
if ( !extContext ) {
     throw new Error( "Volume native extension is not supported on this platform." );
}

 

 

The extension context is the bridge between the AIR application and the native platform.  The createExtensionContext method accepts two parameters.  The first is the package of the native extension; this is defined in the extension.xml which we’ll cover in a minute.  The second is an optional parameter that defines the particular behavior desired from the native extension.  For simple native extensions this won’t apply, but in more complex native extensions this serves as a valuable way to separate the native code.  For example, if you had a native extension to send notifications you may have a local implementation and a push implementation.

The volume native extension is pretty basic so we pass the package path and leave the second parameter empty.

If the result from createExtensionContext is null there isn’t an implementation in the ANE for the native platform that the AIR application is currently running on.  In that case you should throw an error.

4. Expose the interface methods and call native methods on the extension context.

The volume extension has two native methods, init() and setVolume().

private function init():void {
     extContext.call( "init" );
}
public function setVolume( newVolume:Number ):void {
     if ( isNaN(newVolume) )  {
          newVolume = 1;
     }
     if ( newVolume < 0 ) {
          newVolume = 0;
     }
     if ( newVolume > 1 ) {
          newVolume = 1;
     }

     extContext.call( "setVolume", newVolume );
     systemVolume = newVolume;
}

 

 

The VolumeController is a singleton, since it doesn’t make sense to need more than one thing controlling the system volume.  The init() method is called when the singleton instance is created.

The setVolume() method expects a value between zero and one, so some basic validation is performed and then the volume is passed to the native extension.

5.  Create the extension.xml file.

The extension.xml defines where the code for each platform lives.  This is how the native extension knows which platforms are supported and how to access the native libraries.

<extension xmlns="http://ns.adobe.com/air/extension/3.1">
     <id>net.digitalprimates.volume</id>
     <versionNumber>0.0.1</versionNumber>
     <platforms>
          <platform name="Android-ARM">
               <applicationDeployment>
                    <nativeLibrary>libAndroidVolumeLib.jar</nativeLibrary>
                    <initializer>net.digitalprimates.volume.VolumeExtension</initializer>
               </applicationDeployment>
          </platform>
          <platform name="iPhone-ARM">
               <applicationDeployment>
                    <nativeLibrary>libIOSVolumeLib.a</nativeLibrary>
                    <initializer>VolumeExtensionInitializer</initializer>
                    <finalizer>VolumeExtensionFinalizer</finalizer>
               </applicationDeployment>
          </platform>
          <platform name="default">
               <applicationDeployment/>
          </platform>
     </platforms>
</extension>

For now just leave the Android and iPhone platform nodes empty.  Once the native libraries are generated you can fill them in.

The id is the same id that is requested when the extension context is built.

In the platforms node you should put one platform node for each platform you plan on supporting.  Here I’m supporting Android and iOS.  Other possible platforms are OSX, Windows, and AIR TV.

The nativeLibrary node points to the location of the native library.

The initializer points to the initializer class for the native library.

The finalizer points to the finalizer class for the native library.

We’ll dicuss initializers and finalizers more in a few minutes.

The default platform defines an ActionScript implementation to use on any platform that isn’t defined in the extension.xml.  Generally this will be some sort of stub implementation as it’s often difficult to reproduce native behavior in ActionScript (sort of the whole reason for writing a native extension in the first place).  I think it’s important to add a default implementation so that your application can be tested in the Flash Builder simulator.  We’ll cover how to create the default implementation later.

At this point to main library is done.  We have defined the interface to interact with the native extension, we have connected to the extension context bridge, and we’ve created the extension.xml file to define the available platforms and their options.

The Default Library

The default library is the code that will be used on any platform that isn’t specifically supported.  The main reason I’m including this is so that the native extension will work in the Flash Builder simulator; it would be pretty painful if you had to deploy to the device every time you wanted to see a small change.  However, it’s also important if you’re creating an extension that’s targeted specifically at one platform but the main application supports multiple platforms.

Creating the library is actually pretty easy, as the default library is an exact copy of the main ActionScript library but with all of the methods stubbed out.  It’s important that every class that exists in the main ActionScript library also exists in the default library.

In the Volume extension I let the default library continue to update the systemVolume property whenever setVolume() is called.  Also, the systemVolume property defaults to a value of 1.  This lets the native extension ‘work’ in the simulator, emulating what would happen if the user never touched the hardware volume buttons.  The native extension is not able to actually change the volume of the simulator though, so only UI pieces will appear to work.

See the VolumeDefault project for more information.

Instructions:

1) Make a new library project.

2) Create a copy of each class from the main ActionScript library project.

3) Stub out the methods for each class.  Be sure that there are no references to an extension context.

Continue to part 3

Comments (0)

Building a Native Extension | Part 1 – The Problem and the Plan

Posted At: April 9, 2012 3:48 PM | Posted By: Nathan Weber
Related Categories: actionscript3, Adobe Flex, as3, flash, flex

I’ve been writing a lot of native extensions lately.  The project I’m working on now requires all kinds of custom behavior.  It’s been an interesting ride.  I’ve learned a lot, for sure, particularly since I haven’t coded extensively with iOS or Android.  I’ve learned some things and I’d like to share them.  I won’t only be sharing the knowledge though, I also have a fully functional (and useful) native extension for everyone to use.

The Problem:

Changing the volume of a video while it’s playing in Flash is pretty easy, particularly if you’re using OSMF.  Just change MediaPlayer.volume, and if you’re running in a browser that’s it.  Sadly, when running on mobile that doesn’t produce the behavior that users are going to expect.

Both Android and iOS have a system wide media volume.  Pressing the hardware volume buttons while media is playing causes the media volume to be adjusted.  The volume is applied to the entire AIR application and overrides the volume of the AIR application.  This leads to a problem where the AIR application thinks that it is at full volume, but the system media volume is muted.  If you want to use a slider control in your application to control the audio (or have any sort of UI/logic that is based off of the volume) you’re in trouble as you will be representing the wrong volume.

The Solution:

Create a native extension that interacts with the native layer to modify the system media volume, used instead of the usual methods in AIR to change the volume.

The native extension must have two parts.  The first is a method that can be called in AIR that will adjust the system volume.  The second is the ability to notify AIR whenever the user changes the system volume, either by using the hardware volume buttons or by using a different application.

The Finished Product:

Building the native extension has five parts.  These parts are the ActionScript library, the iOS native code, the Android native code, and a default implementation written in ActionScript.  The next several blog posts will walk through each part, discussing how to build the code and any pitfalls to consider.

If you’d like to find out more about how the native extension was built, grab the entire source code here https://github.com/nweber/SystemVolumeNativeExtension and use it as a reference as you read through the next several blog posts.

If you’re only interested in grabbing the finished extension you can grab that here https://github.com/nweber/SystemVolumeNativeExtension/blob/master/Volume.ane.  Something to note, if you plan on using the extension on iOS you must compile the IPA (both debug and release) from a Mac.  You must also specify the location of the iOS SDK in the native extension settings for iOS in Flash Builder.  If you’d like more information on why that is or how to do it, see the part 3 blog post.

Continue to part 2

Comments (9)