Adding Interactivity to VR-360 Content in Animate 2019

In the first part you learned how to create 360-degree VR content. In this article, you will learn how to add interactivity to it. In the process, you will learn how to capture events on assets in the 360-degree scene and use JavaScript APIs of Animate 2019 to perform actions.

In the previous article, you saw the first two steps in creating 360 degree VR content:

Step 1: Create a 360-degree view

In this step, we created a texture layer with a background image.

Step 2: Add objects to the view

In this step, we added images for assets, such as paintings and 3D objects, to the scene.

The focus of this article is on step 3.

Step 3: Add interactivity

The last step is adding interactivity, which is the intent of this article. Some of the typical interactions users will perform with objects in the scene are:

In the document types derived from glTF in Animate, if you want to capture events on an object you need to create a movie clip symbol. You would have noticed in the FLA file in Step 2 that movie clip symbols are created for each image (paintings and 3D arts) for which we want to capture mouse events. It is also important to name such movie clip symbols so that they can be accessed by name in the JavaScript code.

We are going to add the following interactivity to our application:

The following figure summarizes all the events and the actions in the first scene, which is picture_gallery_scene:

One interaction we get free, without writing any code, is navigation. The user can hold the mouse button and move the mouse to pan the camera.

For the interactions listed above, we will need to write JavaScript code.

Actions editor

JavaScript code can be written in the following three contexts in glTF document types in Animate:

  1. Global scripts. Variables and functions declared in the global section are available throughout the application. The reference to this in the global section, in most cases, refers to the browser’s window object.
  2. Scene scripts. Work at the scene level. This in script refers to the scene itself and you can access top-level named symbols in the script as this.symbol_name. Note that all objects that contain animation are of MovieClip type. This the scene object is also of type MovieClip.
  3. MovieClip scripts. Work at the movie clip symbol level. ‘This’ in the script refers to the movie clip.

Scripts in the context of scene and movie clips are written for a frame on a layer. To add a script for a frame, right-click on the frame in the timeline and select Actions option. This opens the Actions panel, with the context of actions listed in the left panel and the script editor in the right panel.

Here is a screenshot of Actions Editor when all scripts for this demo application are added:

Adding frame scripts

In this section we will implement JavaScript code to add interactivity to our application.

Adding interactivity to direction sign

Let’s add a script to manage the animation of the arrow in the direction sign symbol in the first scene. You would have noticed in step 2 that the arrow keeps animating. We want the arrow to animate only on mouse hover. We also do not want the animation to loop, so we will add a script to stop the animation in the first and the last frame of direction_1_symbol. In the timeline of direction_1_symbol, right-click on the first frame of ‘arrow’ layer and select the “Actions” option and add the following code:

this.stop()

Let’s now write the code to animate the arrow when the mouse hovers over the symbol. For this, we will need to add the script to scene picture_gallery_scene.

If your scripts are long, adding them to a separate layer makes them easy to manage. So we will add an empty layer called script_layer in picture_gallery_scene. Then right-click on the first frame of script_layer and open the Actions panel. We will add the mouse hover listener to direction_sign_1 (which is a movie clip, and the arrow is animated in that movie clip). As mentioned above, here this refers to the MovieClip object representing the scene. All the movie clips in the scene (or the current MovieClip) can be accessed as this.movie_clip_name – so direction_sign_1 can be accessed as this.direction_sign_1.

// add mouse handles for the directional 'direction_sign_1' movie clip
this.direction_sign_1.addEventListener(AnEvent.MOUSE_OVER, function() {
    this.play();
});
this.direction_sign_1.addEventListener(AnEvent.MOUSE_OUT, function() {
    this.gotoAndStop(1);
});

A couple of points to note in the above script:

When clicking the direction sign, we want to switch to the scene that shows the 3D Arts gallery scene, so we will add the mouse click handler for direction_sign_1.

this.direction_sign_2.addEventListener(AnEvent.CLICK, function() {
    anWebgl.gotoScene("picture_gallery_scene");
});

The gotoScene method takes the name of the scene as the argument. You can call up this method on anWebgl, which is a global object made available from Animate SDK (Software Development Kit).

Global scripts

As mentioned earlier, shared code across scenes and frames can be added to the global section. We have a few shared behaviors for information images, which pop up when any art is clicked. Information pop-ups are movie clips and we need methods to play, stop, and hide movie clips. We also need to stop looping of animation for the information pop-ups. We will add code to perform these actions in the global section because they are shared by MovieClips for all information pop-ups.

To add a global script, open the Action panel from any frame. The global section is in the top-left left panel. Expand the section and click on the Script node. In the Code Editor, add the following script:

// Stop the movie clip and makes it invisible
function stopMovieClip(mc) {
    mc.visible = false;
    // go to the first frame and stop
    mc.gotoAndStop(1);
}

// Plays the movie clip and makes it visible
function playMovieClip(mc) {
    mc.visible = true;
    mc.play();
}

// Function to be called when play head reaches the last 
// frame of information pop-up movie Clips.
function onLastFrameOfPopup(popup) {
    popup.stop();

    // here the assumption is that pop has 'close' movie clip, to close the popup
    popup.close.addEventListener(AnEvent.CLICK, function() {
        // 'this' here refers to 'close' symbol. 
        // We will access the popup symbol as parent of 'close' symbol
        stopMovieClip(popup);
        popup.removeEventListener(AnEvent.CLICK);
    })
}

// Function to add pop up effect to painting movie clips (MC) on mouse hover
// and restore the MC on mouse out
function addPopupEffectToArt (artMC) {
    // On mouse over add pop up effect
    artMC.addEventListener(AnEvent.MOUSE_OVER, function() {
        //in the event handler 'this' refers to the object on which the event listener
        //is registered. In this case, it is paintingMC
        this.scaleX = this.scaleY = 1.02;
    });
    
    // On mouse out, restore the Movie Clip
    artMC.addEventListener(AnEvent.MOUSE_OUT, function() {
        this.scaleX = this.scaleY = 1;
    });
}

Currently all information pop-ups start animation as soon as the scene is loaded, which is not expected. We will write scripts to fix this. We will also add scripts to show information pop-ups only when any art is clicked.

Go to the first frame of script_layer of the scene and append the following script (this frame already has the code we added earlier for managing events for direction_sign_1):

function onClickPaintingMC () {
    // depending on which painting is clicked, we want to
    // display popup for that painting and hide popups for 
    // the other two paintings
    
    // 'this' in this case would be the movie clip that was clicked
    if (this.name === anWebgl.root.paint_1.name) {
        playMovieClip(anWebgl.root.paint_1_popup);
        //hide other popuos
        stopMovieClip(anWebgl.root.paint_2_popup);
        stopMovieClip(anWebgl.root.paint_3_popup);
    } else if (this.name === anWebgl.root.paint_2.name) {
        playMovieClip(anWebgl.root.paint_2_popup);
        //hide other popuos
        stopMovieClip(anWebgl.root.paint_1_popup);
        stopMovieClip(anWebgl.root.paint_3_popup);
    } else if (this.name === anWebgl.root.paint_3.name) {
        playMovieClip(anWebgl.root.paint_3_popup);
        //hide other popuos
        stopMovieClip(anWebgl.root.paint_1_popup);
        stopMovieClip(anWebgl.root.paint_2_popup);
    }
}

// Hide and stop all popups movie clips
stopMovieClip(this.paint_1_popup);
stopMovieClip(this.paint_2_popup);
stopMovieClip(this.paint_3_popup);


// Add popup effect to paintings on mouse hover.
addPopupEffectToArt(this.paint_1);
addPopupEffectToArt(this.paint_2);
addPopupEffectToArt(this.paint_3);

this.paint_1.addEventListener(AnEvent.CLICK, onClickPaintingMC);
this.paint_2.addEventListener(AnEvent.CLICK, onClickPaintingMC);
this.paint_3.addEventListener(AnEvent.CLICK, onClickPaintingMC);

The function onClickPaintingMC is an event handler function to be called when a user clicks on a painting. So, inside the function, this points to the MovieClip for the painting for which this event handler was registered. We compare the name of the movie clip to find which painting (out of three paintings) the user has clicked. We then play movie clip for the pop-up for that painting, by calling playMovieClip function, and stop playing MovieClip for any other pop-up, by calling stopMovieClip. Note that functions playMovieClip and stopMovieClip are declared in the global section, as described earlier. Another point to note in this function is the use of anWebgl.root. As mentioned earlier, anWebgl is a global object of the SDK. To access the MovieClip for the current scene we can use the root property of anWebgl.

Outside onClickPaintingMC function, we stop animation of all MovieClips for pop-up images, by calling stopMovieClip. We then call addPopupEffectToArt (which is declared in the global section) passing each painting MovieClip to add small pop-up effect on the mouse hover. Lastly, we register ‘Click event listeners’ for MovieClip for paintings, passing onClickPaintingMC function that we declared at the top as the event handler.

At this point, if you test the application, you will observe that animation of pop-up images that was happening earlier on loading the scene has stopped, because of stopMovieClip function call in the above script. But if you click any painting, the pop-up keeps animating/playing. We need to fix this. We have already declared function onLastFrameOfPopup in the global script to stop animation of pop-up images. We now need to call this function on the last frame of MovieClip for pop-up images. You can either create script_layer for paint_*_popup_symbol movie clips, or use one of the existing layers. In any case, right-click on the last frame of the pop-up movie clip, open the Actions panel and add following script:

onLastFrameOfPopup(this);

We are now done adding interactivity for the first scene, which displays paintings. I will skip the description of adding interactivity to the second scene because the scripts are very similar. You can download the FLA file for this step and observe the similarities. Click here to see how the application looks at the end of this step.

Adding scene transition

Though our application is functional now, it does not give any feedback to the user when scenes are being loaded. Because of the size of large images, scenes in the application take time to download. It would be nice to show a “loading” message whenever the user is waiting for the scenes to be displayed. A simple way to show this message is to create a message div, with message or image displayed in it and show the div when a scene is being loaded. Once it is loaded, hide the div. We will declare showTransition method in the global scripts and invoke it from the event handler for “load event” for body tag.

// show/hide transition div
function showTransition(show){
    var element = document.getElementById('transition_div');
    
    // return if the div does not exist
    if (!element) return;
    
    element.style.display = show ? 'block' : 'none';
}

// register onload event for  tag
tags = document.getElementsByTagName('body');
if (tags) {
    var bodyTag = tags[0];
    bodyTag.addEventListener('load', function() {
        // show loading div on document load
        showTransition(true);
    });
}

In the above code we have assumed that the HTML page that loads the scene has a div with id transition_div. To complete the application, we will add the following HTML code, at the top of the body tag in the page that loads the scene:

div id="transition_div" style="position: absolute; 
width: 100%; height: 100%;
left: 0px; top: 0px">
  img src="loading.gif" align="middle"
    style="position: absolute;
    left: 50%; top: 50%;
    transform: translate(-50%, -50%)
    "/>				
div>

The div has id transition_div, and it contains ‘loading.gif’ image. Note that this change will have to be made directly in the HTML file published from Animate. This file is overwritten every time you publish the document. Currently specifying an HTML template is not supported in a glTF published output, so you need to add the above code every time you publish the document. You can download the final FLA file.

Displaying content in Div

In all the above examples, the content published from Animate occupies the entire page. If you want the content to be displayed in a limited area on the page, you can create a div with the desired width and height and set that div as the parent element of anWebgl.

div id="vr_content_div" style="width: 600px; height: 400px;">
   
div>

Then just before calling anWebgl.loadAssetFolder, set anWebgl.parentElement

anWebgl.parentElement = document.getElementById('vr_content_div');

Click here to see the demo.

Conclusion

Animation can be made much more engaging to users by adding interactivity. In this two-part series of articles, you have learned how to add interactions to the glTF extended document type in Animate, using a VR-360 application as an example. The APIs for glTF extended document type is designed to match with common JavaScript and ActionScript APIs wherever possible. Interactivity is mainly implemented by handling events on objects and then calling supported APIs. In glTF document types, you need to create a MovieClip Symbol for an object if it is going to be accessed from scripts. Scripts are added in the Actions panel. You can add scripts globally, at the scene level or at the movie clip level. Depending on the context, the “this” variable in the script points to different objects.

All demos in this article are created by my colleague in Animate Team, Arvind B.V. Assets for the demo are licensed from Adobe Stock.