Persuasion Interactive Media

Frame Rate Consistency in Flash Player

Leo A. Geis — Sun, 17 Feb 2013 16:20:36 +0000

Conventional Wisdom specifies that Timer Objects-not Frame Rates-be used for timing ActionScript functionality. Frame rates are unreliable.

Each "Frame" is essentially a block of time specified via the stage.frameRate:Number Property or via any of several controls in the IDE. Frame Rate is a "dynamic" Property and may be changed programmatically to manage computational loading or produce effects such as Time Posterization.

This demonstration should be useful for understanding Frame Rate as a perceptual and performance metric in Flash Player-it should not be used as a precedent for AIR.

The top portion of the window contains a Slider to specify the Frame Rate of the flashing black box and the oscillating circle. I have limited the excursion to 60 Frames per Second (FPS) for several reasons, one of which is a rendering limitation imposed by Browsers and the Flash Player Plugin. The flashing black box is white for a single Frame, then black for a single Frame, producing an "ultimate contrast" situation in a fixed position. The oscillating ball Translates (repositions) 10 Pixels each frame.

The "Actual Frame Rate" readout is an average, but it is easy to see inconsistency during the movement of the Slider, especially to high values. Without the averaging the number is nearly unreadable due to flickering. As the Frame Rate is decremented the readout lowers slowly: it is Frame-Based (not Time-Based) standard. It is also possible that you will see the Frame Rate deviate substantially from the rate you prescribe with the Slider.

We may conclude that Frame Rates in the Flash Player should not be used for critical timing issues.

Much of the concern over Frame Rates and actual human visual acuity is a diversion. Human vision and cognitive acuity is wildly variable, as are computer systems (including monitors). There are extraordinarily few applications in which a need for >30FPS or a temporal accuracy of +/-2% is indicated. Adobe recommends 24FPS for general use. The movie "The Hobbit" contains 48FPS video footage that has been widely criticized (because of the Frame Rate) in spite of its technical superiority.

The lower half of the window contains an opportunity to test your system under significant Floating Point & rendering load. You are obligated to check the box indicating that you understand that adding Perlin Noise to a high Frame Rate may cause your computer to hang (particularly if you are on a 2+ year old laptop or modest desktop). The publish settings for this particular Flash piece include Level 2 (GPU) utilization, but if your system doesn't support that feature you could end up rendering through the shell page, which is the slowest of all possibilities.

Look for artifacting in both the black box and the black ball, particularly "phase" artifacting in the box, and both fractures and ghosts with the ball. Your monitor's performance is a cardinal issue with these phenomena.

The Perlin Noise signature is:

bitmapHolder.bitmapData.perlinNoise(Math.random() * 10, Math.random() * 10, 2, 5, true, true, 7, false, null); on an ENTER_FRAME Callback.

The idea is to produce a computational burden to see how it affects your Frame Rate. I have purposefully limited the Octaves and denied variable Parameters to obviate unpleasant e-mails. You do have the option of opening multiple copies of this page (don't let any of them get buried) if you want to compound the computational load.

Spatial Animation Relative to the Cursor

Leo A. Geis — Sun, 16 Dec 2012 16:11:15 +0000

Clever animations are popular. The apparent complexity of animating using trigonometry and Matrix Math has resulted in "physics" programming being considered evidence of masterful code.

The following animation is a simple concept, but it adds engaging interactivity to a mundane object. The idea was to produce a Button Object that would respond to the cursor once the cursor was within a certain distance, and then return "home" after the cursor had move a certain distance away.

If your mouse (cursor) gets within the red circle the ball will start chasing it. The center of the ball will never travel further than the gray circle that will appear. The slower you coax the ball along (by moving your cursor leisurely) the further from center stage the ball will travel, but its center point will never exceed the gray circle.

Additionally, if your mouse position exceeds a certain distance from the ball, the ball will return to its home in the center of the stage even if it is nowhere near its excursion limit.

Another way of describing this control is that your mouse must be in both the red and gray circles to control the ball.

Anytime the cursor is exerting influence on the ball a red line will appear between the ball and the cursor as an indication of that vector-the length of that line will in fact indicate the "strength" of the attraction.

You are able to adjust the refresh rate of the animation using the slider beneath it. Testing low frame rates ensures there are no temporal "holes" in the code.

The point is that this is-by some standards-masterful code. If the graphics were "dressed up" as "eye candy" and I added some supplemental animations it would really be impressive. An array of these types of objects could be striking arranged in an arc above a medallion, or mesmerizing inside of various flocking and particle behaviors.

Also, useless. In fact, that sort of interactivity is actually hostile to the persuasive intent of many websites and presentations.

Cognition is like money-it's in a perpetual state of insufficiency. Adding wonderful animations to navigational devices or decorative elements actually adds a layer of abstraction to the engagement. In applications such as public relations, commercial real estate, and education the cardinal goal is not to win design awards-it is to educate and/or persuade. The software should be reliable and efficient toward that goal, not toward winning design awards or attracting other customers for the software developer.

Particularly in litigation, pointless "psycho-visual enhancement" is a liability.

Perlin Noise in Visualizations

Leo A. Geis — Sun, 04 Mar 2012 01:12:12 +0000

Perlin Noise can be very useful in animation but is rarely employed because of its complexity and the difficulty of visualizing what the BitmapData Method's Parameters will actually produce.

The applet below is useful for developing a fundamental understanding of what Octaves, Random Seed, and Channel Operators actually do. Change the various Parameters then click the "Generate" button to view the results.

This is a simplified exercise in Perlin Noise construction. Within the actual Method one has the ability to combine or separate Color Channels-functionality which is not included in this visualization. Additionally, the offsets Parameter is ignored here: it is very complex and typically a matter for experimentation.

Another interesting aspect of Perlin Noise within ActionScript is the fact that Perlin Noise resides in a the BitmapData Property of a Bitmap Object, meaning that it participates nicely with a variety of Display Object Methods and Properties such as Blending Modes, Filters, and Transforms. The use of Perlin Noise does not obviate the use of the Bitmapdata.noise() Method, nor does it frustrate any of the Flash Drawing API's Methods. Perlin Noise is very effective at adding an organic nature or randomized texture to animations.

Control over Interactive Asset Matrices

Leo A. Geis — Thu, 29 Dec 2011 00:03:25 +0000

A Data Matrix may be considered a collection of assets indexed in Y Axis rows and X Axis columns-and occasionally, in a third dimension such as Z Axis levels. The Software Developer must specify two or more Parameters to make the selection of an individual asset or functionality.

In a two dimensional situation the X and Y coordinates of the Cursor may be used to specify which asset to display or otherwise manipulate. In a 3-D environment Z Axis coordinates derived from the Mouse Wheel or other control may be used as an additional Parameter.

In the following visualization the X-Axis and Y-Axis values of the outlined square are converted from Global (Stage) Coordinates to the Local Coordinates and used to display numeric Row and Columnar identities. The Mouse Wheel is configured to navigate the Z Axis of the 3D Index. This functionality can control any Bitmap Property such as Scale, Rotation, or transform.matrix Parameters, or any BitmapData Property such as colorTransform Parameters or perlinNoise Parameters. The integer values of those Parameters are displayed to the right of the square. The square may be Scaled without corrupting its indexing functionality.

"Mouse Over" the square for the visualization to initiate. If your browser is scrolling click on this link to open the application in an independent window.

The point of the visualization is that assets may in fact be indexed in a Matrix: a software development practitioner is not limited to the Dictionary Class or using an Object to create an asset repository. It is in the event of the Z Axis indexing that this method really shines, though it does not offer the performance advantages realized within the more common Object or Vector Classes.

Consider that with 10 discreet X Axis entities, 10 discreet Y Axis entities, and 100 discreet Z Axis entities, this very simple, light (9.2KB) and unintimidating graphic is presenting a visual index of 10,000 assets in a space of several hundred pixels. If I had not limited the output of the X and Y metrics to integers from 1 to 10, it would represent 1,000,000 assets!

The controls themselves are not limited to the Cursor's spatial or zoom Properties: they can certainly be bound to (animated by) the time of day, an .xml/RSS feed, or the manipulation of other Display Objects, interactivity, or data constructs within or external to the .swf.

Granular Control over Audio Values

Leo A. Geis — Tue, 01 Nov 2011 01:36:27 +0000

ActionScript provides granular control over audio streams, including discreet control over audio Channels.

Digital sound distills down to "sampling." Sampling an audio stream 44,100 times per second produces 44,100 discreet sample values, which are stored as numbers. Those numbers may be formatted in 8 bits (a Byte: 256 levels), or in 16 bits (two Bytes: 65,537 levels), or more. The more and larger the numbers, the larger the file size necessary to store the data, and the faster the bitstream necessary to transfer the data conveniently.

A principle known as the Nyquist Frequency postulates that to sufficiently capture data for human sensory consumption (et. al.), a data set must be sampled at roughly twice its frequency. Standardized human aural acuity can sense sound up to nearly 20,000 cycles per second ("Hertz"), so to satisfy the Nyquist Frequency sound should be sampled at around 40,000 Hertz, and in fact, the standard high quality audio sample rate is (in multiples of 11,250 Hertz) 44,100 Hertz. However, not all sound produces such high frequency information, and the sampling frequency may be moderated when superflouous sampling does little but increase the file size. This is very common with telephonic (voice-only) sound. Smaller file sizes and more modest bitrate requirements are appropriate when programming for mobile devices.

For a more in-depth review of audio compression strategies please see the Forensically Fit website.

Flash (ActionScript 3) offers a very robust Sound functionality that includes multiple Channels, Stereo, volume and panning controls ("Transforms"). Because Flash is tremendously adept at both mathematics and drawing it can be used to produce very sophisticated, Channel-specific graphical sound displays and interactive animations-even "Beat Detection" and various psychoacoustic manipulations. The example below is very simple-it simply generates a sort of Histogram for the sound values in the Left Channel of a stereo track of bird songs. More complex displays can be generated, including those as sophisticated as controlling the parameters of Perlin Noise, Fractals, and Particle Generators with the audio data values or their dynamics.

Granular Control Over Pixel Values: Mapping a Histogram

Leo A. Geis — Mon, 11 Jul 2011 17:47:07 +0000

ActionScript provides elemental control over pixels, including discreet control over color Channels within Pixels (Red, Green, Blue, and Alpha/Transparency Values). A Histogram is a statistical device for visualizing the distribution of data.

In Photoshop, advanced practitioners make many of their decisions based upon an image's Histogram. An image's appearance on the monitor-which is subject to wicked variablilities such as ambient lighting and the practitioner's visual acuity- can be unreliable.

In order to determine which controls-and the manner of their application-should be employed within Flash, Photoshop, After Effects, or Premiere Pro, it is prudent to both understand and consult an image's Histogram (or RGB Parade)

Flash (ActionScript) is capable of building Histograms using the flash.display.BitmapData.histogram() Method, which returns a Vector (a type of Array with homologous elements) Object. Specifically, the product Vector of the bitmapData.histogram() Method is a multidimensional Array of Arrays of Numbers. Numbers are desirable because ActionScript 3 has a very robust Top-Level Math Class with which to investigate the statistical character of the report and to manipulate it.

Select an image and adjust its "Brightness": the Histogram will respond.

Within ActionScript we have the ability to access Pixel Channels and to map them (in this case, to a Histogram). This is by no means any sort of limit on the manipulations that can be effected on a Channel by Channel or Pixel by Pixel basis. Blending Modes (such as the Complex Mode "Hard Light") use discreet Channel values throughout Adobe's Creative Suite. ActionScript also affords various Properties and Methods in a variety of Classes with which control Pixels.

These capabilities are not simply "artistic controls" of aesthetic value only: they can be used to make photographs and various graphics more communicative-to carry a greater cognitive impact. Often, the ability to quantify and animate these manipulations makes for forensically defensible persuasive artifacts with significant communicative advantages.

The entire matter really distills that these functionalities are controllable not only via the Sliders I have provided, but as explained in previous posts (1, 2, 3):

They can be controlled by a wide variety of parameters including:

Mouse position;

Mouse movement direction;

Mouse movement speed;

Mouse button clicks;

Keyboard input;

Time of day;

Touch screen input;

Voice commands and other audio artifacts;

Latitude and/or longitude;

Device orientation (many have accelerometers...);

Device movement;

Interactive devices such as buttons or selections.

The point is that interactive visualization is engaging and makes visual artifacts more communicative: interactivity devoted to support a specific cognitive impact is engaging and persuasive. Interactivity that is rapid to produce and economical-particularly within Commercial Real Estate, Litigation Exhibits, Public Relations, Construction, Engineering, and Education industries-is highly desirable.

Directionally Blurring Visual Artifacts

Leo A. Geis — Sun, 10 Jul 2011 00:15:41 +0000

Blur is perhaps the most underappreciated feature in our animation & visualization digital world. It smooths movement in our motion graphics, suggests directional movement in still images, graduates effects when applied to masks, and even compensates for quantization errors when we composite.

However, there is a problem with common Blur Filter algorithms: they can directly be applied only upon the X- and/or Y-Axis, meaning that blurs are confined to left-right or up-down. If you are animating an object at 45 degrees, such an axial or biaxial (composite) blur will be inconsistent with the motion.

In the following interactive visualization you may apply an X-Axis and/or a Y-Axis blur. When you do, the effect is obvious, and when you use the control to rotate the array of artifacts the blur remains loyal to it's native application axis. This is due to the way that ActionScript applies Convolution Filters-it actually converts them into bitmaps (rasters, vice vectors)-which makes sense considering their transparency (Alpha Channel) requirements and potential blending/compositing requirements. The individual artifacts are redrawn per their rotation, and the Convolution Filter/s are reapplied on the redrawn artifact's axis.

If you apply a blur to the array, another artifact will appear in the lower right corner of the Flash Window below. It will honor any rotational commands you give using the control, but the blur will rotate with the artifact to any intermediate value.

There are some considerable logical and coding complications in producing this animation. Obviously-from the behavior of the array when blurred and rotated-the blur is stubborn. Additionally, an object produced programmatically will generate its registration point at Local(0, 0, 0)-the upper left corner-which is hostile to rotating or transforming an object around it's center.

ActionScript offers a solution for the rotation challenge in the ActionScript 3 fl.motion.MatrixTransformer.rotateAroundInternalPoint() Method, but I recommend Greensock's solution in their TransformAroundCenterPlugin.as.

If you apply a discreet X-axis blur, the array looks like:

Notice that the blur is confined along the x-axis (left to right); horizontal surfaces are relatively unaffected.

If you then rotate the array 90° clockwise the blur is still aligned along the x-axis and has not remained loyal to the original axis implementation.

However, the demonstration artifact has maintained its blur along its original axis.

The importance of this capability in a broad spectrum of animations is difficult to overstate.

Please add a blur to the images and adjust the quality back and forth to see the difference it makes. In ActionScript the "quality" of blur, whether applied programmatically or within the Flash application (IDE) simply refers to the number of times the Blur Filter is applied to the target. Obviously, more iterations produce a smoother object with less artifacting, but there can be a very steep computational penalty. In an animated environment it is necessary to weight the virtue of high grade blurs relative to CPU loading, particularly on mobile devices and in complex interactive situations.

The single blur object in the lower right is actually a dynamically drawn copy of that on the extreme left of the top array, and has been increased in size 30% to exacerbate artifacting along its perimeter so that it will demonstrate the advantages of higher grade blur more recognizably.

Blur can actually facilitate low frame rates by easily and convincingly replacing intermediate frame renders in rapid animations.

Controlling the Convolution Filter Parameters of Visual Assets

Leo A. Geis — Fri, 08 Jul 2011 13:45:00 +0000

Photoshop, Flash, and After Effects practitioners will recognize the specific filters and controls demonstrated in this post. In fact, these are iconic Convolution Filters within various digital image editing applications.

Using the provided controls a user is able to discreetly invoke four different Convolution Filters and adjust particular parameters-control of which is provided by the accompanying slider. On the Blur Filter and Glow Filter the amount (radius) of the blur is variable while the quality (number of iterations) is static. On the Drop Shadow Filter the distance of the shadow is variable, and on the Bevel Filter the apparent depth of the Bevel (effected using displaced and contrasted Gradients) is variable.

The entire matter really distills that these functionalities are controllable not only via the Sliders I have provided, but as explained in previous posts (1, 2):

They can be controlled by a wide variety of parameters including:

Mouse position;

Mouse movement direction;

Mouse movement speed;

Mouse button clicks;

Keyboard input;

Time of day;

Touch screen input;

Voice commands and other audio artifacts;

Latitude and/or longitude;

Device orientation (many have accelerometers...);

Device movement;

Interactive devices such as buttons or selections.

Controlling the Color & Transparency of Visual Assets

Leo A. Geis — Tue, 22 Mar 2011 02:56:11 +0000

Using the provided controls a user is able to independently manipulate the three Color Channels (Red, Green, and Blue), as well as the Transparency (commonly referred to as "alpha," or "α" in the Greek Lower Case) of the artifact.
Because of the granular control over the Color Channels and Transparency we can also control the composite (all Channels together) image Brightness, even going so far as to neutralize any color or invert the image to the negative.

Manipulating color this way provides us a glimpse into color science.

The Color Gradient used is one of Hue only-there is neither Saturation nor Luminance variance. When converted to monochrome there is what appears to be an inconsistent patterning with the distribution of Hue. In fact, I programmed the conversion to monochrome using the Haeberli Vectors, which are the relative intensities of total Luminance as accounted for by the Red (about 31% of total pixel luminance), Green (about 61%), and Blue (about 8%) Channels in the digital sciences. In fact, there are other standards besides Haeberli's (some are more recent and some perhaps more defensible), but I believe that Haeberli enjoys the broadest acceptance. It is the variances in Luminance contributions that accounts for the apparently misregistered monochrome patterning.

Switching the unadulterated color Gradient to the negative provides an interesting redistribution of the Primary Colors (Red, Green, and Blue) centered upon the former bands of the Complementary Colors (Cyan, Magenta, and Yellow)-a useful demonstration for individuals learning the mechanics of the Color Wheel and Subtractive Color.

Another interesting matter is the way in which Brightness is controlled. In this visualization I used an ActionScript 3 ColorMatrixFilter Object, which is a comparatively complicated tool to invoke. It is actually a Matrix of 4 rows of 5 values and uses both additive and multiplicative functionalities to scale the brightness of the image properly-no matter what balance of Red, Green, Blue and Alpha (Transparency) you have prescribed with the controls.

The entire matter really distills that these functionalities are controllable not only via the Sliders I have provided, but as explained in the previous post:

They can be controlled by a wide variety of parameters including:

Mouse position;

Mouse movement direction;

Mouse movement speed;

Mouse button clicks;

Keyboard input;

Time of day;

Touch screen input;

Voice commands and other audio artifacts;

Latitude and/or longitude;

Device orientation (many have accelerometers...);

Device movement;

Interactive devices such as buttons or selections.

Controlling Visual Assets in Three Dimensions (3D)

Leo A. Geis — Fri, 18 Mar 2011 23:41:34 +0000

Flash (that is to say, it's underlying programming language ActionScript 3) is very talented. You may find it hard to believe, but...

...Photoshop and After Effects-which are the "Big Guns" in Adobe's Creative Suite in terms of complexity-can both be mostly written in ActionScript 3. That includes the scary parts like Convolution Filters and various Channel Operations (ChOps).

In spite of all that talent, Flash (ActionScript's integrated development environment, or "mother program") still has most of its traction within the .html/.php internet community who want to host video, make their menus bounce up and down, or make sparks fly out of their cursor.

For companies like Persuasion Interactive Media, ActionScript offers wonderful interactivite visualization functionalities including three-dimensional manipulation of visual assets-especially raster Objects such as photographs.

This is a very simple demonstration in which a user may manipulate a 2D object within the simulated 3D space. Use the various sliders to alter the parameters of the plane.

These characteristics: rotation, scale, and position, don't have to be controlled by sliders. They can be controlled by a wide variety of parameters including:

Mouse position;
Mouse movement direction;
Mouse movement speed;
Mouse button clicks;
Keyboard input;
Time of day;
Touch screen input;
Voice commands and other audio artifacts;
Latitude and/or longitude;
Device orientation (many have accelerometers, including various Droids and iPhones);
Device movement;
Interactive devices such as buttons or selections.