Documenting the Design of Rich Internet Applications: A Visual Language for State

By Richard F. Cecil

Published: December 17, 2007

“Documenting the design of any page that uses Ajax is a challenge, because the page—and, more importantly, components on the page—can have different states.”

Ajax and Rich Internet Applications (RIAs) have revolutionized the way users interact with Web sites. However, documenting the design of any page that uses Ajax is a challenge, because the page—and, more importantly, components on the page—can have different states, depending on how users interact with the page’s components.

When I use the term state, I refer to a page or a page component that displays different information—content or user interface (UI) elements—depending on a user’s interactions or the information available to the system. For example, the following components have state:

  • a navigation bar whose links appear in boldface to indicate a user’s current location within a Web site
  • a link that changes from Register to Sign in depending on whether a user is logged in
  • an Ajax widget that lets a user add a tag to a picture

Since the information on a page or within a UI component on a page depends on the available information, rarely does all of the information that might appear within a component appear in that component at once. So, when we describe state, we have to identify both the information that is available for display and the conditions under which to display it. Said another way, as a user interacts with a UI component, what information should appear in that UI component?

So, how do we document both the state-to-state interactions on a page or in a component, as well as the types of interactions that modify their states? Designers have proposed various ways of bridging this gap—particularly storyboards and prototypes. Both of these techniques have their strengths.

Storyboards—much like scenarios—let you communicate the intent of a page’s functionality. However, unless its functionality is simple, documenting all of its conditions and their resulting states can be cumbersome.

“Prototypes let you demonstrate functionality to stakeholders and engineers, but prototypes are difficult to use as documentation.”

Prototypes let you demonstrate functionality to stakeholders and engineers, but prototypes are difficult to use as documentation. For example, if you design ten states for a UI component, how does an engineer know the conditions under which each of the ten states appears? You could annotate each condition, but doesn’t that defeat the purpose of the prototype?

What’s more, there are no simple prototyping tools that let designers effortlessly design state behavior. Most prototyping tools document page-to-page behaviors, not state-to-state behaviors. Unless a designer knows Flash or HTML, CSS, and JavaScript—it’s necessary to assign a production development resource to create the prototype, which means the interaction designer still needs a way to communicate the design.

While both storyboards and prototypes can be helpful in some cases, neither technique provides the kind of robust language an interaction designer needs for communicating state-to-state interactions.

Even some traditional documentation techniques that provide a language for designing interactions come up a bit short when it comes to documenting RIAs. In task flows, as they currently exist, there is no clear indication when a site should

  • transition to the next page or
  • simply reload a single component on the same page

With wireframes, it’s impossible to identify all the conditions under which specific information should appear, in a manner that a development team can easily understand. However, if we add a few new symbols to the language of wireframes and task flows, we can overcome this shortcoming.

Wireframes

I’ll start with wireframes. We need to add a symbol to our visual vocabulary: the region. Figure 1 shows a region.

Figure 1—Region symbol

Region symbol

A region symbol blocks out an area of a wireframe in which a UI component—in its various states—always appears. For example, in the wireframe of a page header shown in Figure 2, the Sign in / Welcome region might contain different information, depending on what is known about a user.

Figure 2—Header wireframe

Header wireframe

To help people better understand a page layout, always include each component in some state as an example. You should name each region, making it easier for stakeholders and engineers to locate additional information about a region.

Task Flows

“By adding some additional symbols to our visual vocabulary, task flows can also be a great tool for documenting state changes within a UI component.”

But how do we determine or communicate what information should appear within a given region of a page and when it should appear? That is the question.

Task flows, workflows, activity diagrams—whatever you call them—have served interaction designers well in documenting how users interact with a system. By adding some additional symbols to our visual vocabulary, task flows can also be a great tool for documenting state changes within a UI component.

For the purposes of this article, I’m going to extend Jesse James Garret’s visual vocabulary, but you can apply these same concepts to your preferred vocabulary.

First, we need to identify how a component responds to user interactions—that is, do its responses occur asynchronously or synchronously. Second, we need a way to represent each UI component—and its current state—in a task flow.

Asynchronous State Changes

For user interactions that do not require a page reload, asynchronous interactions—whether implemented in JavaScript, CSS, Ajax, or another RIA technology—use an arrow with a dotted line like that shown in Figure 3.

Figure 3—Arrow representing an asynchronous state change

Arrow representing an asynchronous state change

Synchronous State Changes

For user interactions that require an entire page to reload, synchronous interactions, use a single-line arrow, as shown in Figure 4. This is the standard arrow Jesse James Garrett uses in his visual vocabulary.

Figure 4—Arrow representing a synchronous state change

Arrow representing a synchronous state change

Putting It All Together: The Wireflow or the Taskframe

“For simple user interactions, your task flows can include your actual wireframes of a UI component and its various states directly within the task flow.”

So, how do we represent a UI component in a task flow? For simple user interactions, your task flows can include your actual wireframes of a UI component and its various states directly within the task flow. This reduces the time people spend flipping between task flows and wireframes.

Let’s look again at the example of the Ajax login component that appears in the header wireframe in Figure 2. The information that appears in the header region depends entirely on the state of the UI component, which the task flow shown in Figure 5 defines.

Figure 5—Task flow for an Ajax login component

Task flow for an Ajax login component

Notice that this task flow includes a new component container symbol. Figure 6 shows the component container symbol.

Figure 6—Component container symbol

Component container symbol

This container identifies a UI component’s name and represents its current state within a task flow. For example, the component name Sign in / Welcome refers to the name of the region I defined in the wireframe of the page containing this component and Signed in represents that component’s current state. Thus, we get Sign in / Welcome: Signed in.

Now, let’s walk through the combination wireframe/task flow for the Sign in / Welcome component step by step.

  • In the Sign in / Welcome: Unknown User state, the user clicks Sign in. A Sign in / Welcome: Sign in state replaces the Sign in / Welcome: Unknown User state. The dotted line indicates this is an asynchronous behavior—that is, the page should not reload.
  • Once a user provides his email address and password in the Sign in / Welcome component, then clicks Sign in, the site asynchronously sends the login request to the server, as the dotted-line arrow indicates.
  • Per Jesse James Garret’s visual vocabulary, a diamond indicates a system decision point that a user interaction initiates. In this case, the user-initiated login request results in one of the following:
    • If the user provided a matching email address and password, the Sign in / Welcome: Signed in state results.
    • If the user did not enter the correct email address and password, the Sign in / Welcome: Error Signing in state results.

This method works great for illustrating state interactions for most components, but what happens if the interactions get too complex and unwieldy to include the component wireframes within a task flow?

The Straight Workflow

“To accommodate very complex state interactions, we need to add another shape to our toolbox.”

To accommodate very complex state interactions, we need to add another shape to our toolbox: a symbol that concisely represents a UI component and its state within a task flow like that shown in Figure 7.

Figure 7—UI component symbol

UI component symbol

Much like the Web page symbol, this UI component symbol represents a specific state for a UI component on a page. As with the component container symbol, use the following naming convention: [Component Name: State Name]—for example, Sign in / Welcome: Signed in.

Now, let’s recreate the task flow for the login UI component, using the UI component symbol, as shown in Figure 8.

Figure 8—Task flow for the login UI component, incorporating the UI component symbol

Task flow for the login UI component, incorporating the UI component symbol

Why Use This Approach?

“Using wireflows can help you to think through your designs for RIAs more completely.”

Certainly, there are other ways of documenting your designs effectively. However, using wireflows can help you to think through your designs for RIAs more completely, represent them fully in your design documentation, and increase the clarity of your conversations about your designs with other members of your product team.

Here’s a Visio stencil you can use when following this approach: Visio Stencil

22 Comments

Good article, Richard, documenting RIAs certainly is a challenge. I like the idea of amalgamating task flows and wireframes; it looks like a useful shortcut for many situations. But as you point out, this will often get too complex to read. If we fall back on the UI component symbol in the task flow, presumably the storyboards for the UI must be elsewhere, so we haven’t reduced any work in this case, right? We still have to flick back and forward between task flow and wireframe, and we still have the cumbersome job of documenting every state in the UI storyboards.

Still, I really like the idea of wireflows and will certainly put them to use. Thanks.

I believe we not only have to use these kinds of models for documentation, but also as our programming tool.

We are taking a similar approach with Himalia.net, merging state machines, navigation, and presentation models with the aim of building the GUI.

We don’t care if it’s an RIA, a desktop, or an old-Web user interface. In all cases, the user interface can be described, designed, and specified in the same way. It’s all about the tasks the users do, the things the users see, and how it is connected with an external service layer.

We are going to release a new beta at the beginning of the next year. You may be interested in trying it.

Richard,

Thanks for the article. As someone who designs and builds complex RIAs, with complex non-linear work flows, I have been struggling to find the ideal modeling vocabulary. Things get even more interesting when you have complex server back-end functions. I currently use a hodge podge (technical term) of wireframes, UML activity, sequence, and communication diagrams and whatever else seems to fit the bill. This is surely an area ripe for discussion and exploration.

Hi there. In your article, you say “What’s more, there are no simple prototyping tools that let designers effortlessly design state behavior.”

We use a tool called Axure RP Pro (www.axure.com) that does exactly that. They have a tool for creating dynamic panels that can be shown or hidden based on a number of parameters. Personally, I’ve found it invaluable when getting clients and developers to review complex RIA interactions.

Hi and thanks for the article. Since we are implementing a similar documentation approach, I would like to point out a few concerns.

We separated our document into three types of diagrams: wireframes, actions, and mockups. Our current pack is around a hundred pages.

The major concern is the growth of complexity in the document with the growing number of active elements. This growth is far from linear.

Coming from the complexity are the high cost of change, duplication, and error proneness.

On the minor side:

  • Your tool of choice is Visio, as far as I can see, which can be perfect for analysts or UI designers, but not for all stakeholders.
  • Changing these diagrams on the fly—like during a workshop or teleconference—is hard. This often leads to increased per-version turnaround time.
  • It is harder to track changes in such diagrams compared to mostly text-based descriptions.
  • Maintaining this pack print ready is also quite hard.

Having said that, I must agree that this approach is far better and more productive than text-based variants.

John, I would solve this issue by reducing the task flows to document one activity. This increases the number of task flows, but decreases the complexity of each task flow.

I’ve also found that this approach makes it easier for stakeholders to understand the concepts I’m trying to communicate. If you do this successfully, I’ve found that you can take a complex page that would normally require the UI Component Symbol and use the UI Component Container instead.

Leo, let me know when you release the next version of Himalia. I’d like to take a look at it.—Rick

Steve, for complex server interactions, I let the Technical Lead / System Architect define those. I focus the system responses on accurately capturing what the system does without precisely defining how it does it. In this way, I can keep the flows relatively simple and not get into documenting a lot of system behavior—something I’m not really fond of doing. Now, it doesn’t always work this way—there are times when I just have to dig in and document a complicated system flow so that I understand how it affects the user interaction.

Iain, I’ll have to look at Axure more closely. I think tools like Axure and Intuitect have a lot of potential. My only concern with these tools, though, is that your designs can become limited by the tool. Visio is pretty much a blank slate. That said, an effective tool can also make the design and communication of the design easier. I’ll give Axure another chance.

Valentin, as the technology and resulting interactions get more complex, I think the specifications are going to continue to grow. I really have two responses to this: 1) Simplify. Do your customers really need such complex interactions or features? 2) Consider using agile methods. They reduce the need for documentation, but do require that you have some understanding of what you need to build—that is, you should already have your customer research completed before you begin an agile design and development process.

Nice article. I think it’s safe to say that most designers are taking this hybrid, best-of-breed approach to communicating RIA designs. However, the real communication challenge is understanding where you are in your project process, who you are communicating with, and how the assets will be used—for example, printed, via the Web, during design reviews. Determining the timing, audience, and the usage should lead to the most effective approach whether it be storyboards, wireframes, flows, PowerPoint decks, conceptual models, prototypes, or a combination of these. Prototypes can’t be printed. Wireframes don’t move. Flow diagrams demand a large canvas. Storyboards lack detail. But when these methods are used together, in the right sequence, they can make all the difference. As a designer I would never rely on a single method.

I’d also add that a project’s development process will influence the communication approach. The challenges described in this article are mostly due to the waterfall method. What’s great about more agile processes is that designs and development can get done in parallel. If designs are developed collaboratively, stakeholders should have a shared understanding of how a design should look and perform. The lesson here is that RIA products demand that designers be more collaborative in the design process and more creative and flexible in their communication methods.

We just switched from Visio to Axure RP Pro, and we have found Axure to be the perfect tool for designing RIAs. Instead of having dozens of pages—one per state—for one rich component, you create a dynamic panel on a page. The panel itself can have dozens of states, but if you are not editing your component, all states are nicely hidden.

Also, if you export the prototype to a Word document, all states and interactions are neatly printed in the document. I find it a satisfying and easy way to document an RIA.

Rick, thanks for your response.

In fact, we are trying to do both.

We can’t really reduce the number of features we deliver, since this is what makes our products stand out in the line of a dozen similar. Now we focus more on UX and how to make this bag of features fit into our not-so-RIA domain.

We do use some of the agile practices that match our projects’ size and complexity. This hundred-page document is a third extension to an existing backlog.

This seems very useful when it’s desirable to show the region’s appearance. However, given you’re talking about using rich elements to control the presentation of content —essentially, within-page navigation—I suspect that, most of the time, it’s more important to document the possible flows in the context of the larger information architecture, not the visual design of the regions or each task flow in isolation. Also, while flow charts have their uses, they can also lock the designer into a modal, step-wise manner of thinking that is incompatible with the open and flexible UIs we seek.

Maybe you should start with node-link maps rather than wireframes or flow charts. I’ve been experimenting with modified node-link diagrams, and for this purpose, it seems sufficient to simply distinguish synchronous from asynchronous content loading, using parentheses rather than an arrow to code this.

For example, your login structure would look like this. I’ve included registration and sign-off content.

I haven’t decided whether the parentheses approach allows for sufficient complexity, but note that it clarifies something that is ambiguous in your “straight work-flow” diagram: When a sign-in error is made, the sign-in text boxes are still available to the user.

Rick, great piece. Gives food for thought.

I’ve learned that every situation has a priority that determines the appropriateness of a solution. I’ve had to go the documentation focus, sadly, when trying to satisfy an audience for requirements review. (We’re talking people focused on the process, not the results.) I’ve also had an audience throw up their hands over the requirements details and say, “I’m lost. Just show me how it’s going to behave.”

So rather than address specific solutions, let me offer a vendor challenge. The prototype can generate the appropriate documentation if, as Serena did with Composer, it is all created in XML. (Documentation is a simple push of the button.) The bad part about the Serena tool is everything else. :) It does a lot of really great things, but there are key functions we rely on like drawing that it won’t let you do. [Sigh.] Inside info here: I’ve interacted with the product manager repeatedly in the past. They really need the attention of their one experience design PhD specialist, but they’re single-threading his time across a huge suite of tools. (I’m not sure who they unleashed on their site recently. It’s visually more appealing, but throws up barriers to intent. [Sigh, sigh.]

I need to tie it all to my requirements—not development or testing requirements—but the research / evidences. A good interface for collecting such artifacts is CD Tools, but it has tremendous limitations and is really a function that should be part of a larger solution. For increased flexibility, I’ve also reworked a great Excel template—here’s the original.

Prototyping:

  • I’m not an Axure fan. The ability for a developer to see, translate, and implement a simple link is too buried. Bottom line: Level of effort doesn’t contribute effectively to the next state.
  • If the destination environment is SharePoint, designers can implement some things themselves faster than they can draw up or prototype them.
  • I have hopes for, but haven’t field tested the new Microsoft tools, especially Expressions Blend.

No one is considering the whole experience of what it is we need to a accomplish and the various scenarios we have to throw at our tools. They keep reinventing reality. (Visio could have been built out, but Microsoft has abandoned it for the Expressions suite.)

Hi,

Great article. It is truly challenging to wireframe or prototype and document these kinds of rich interactions. I love your idea of wireflows. I think developers and business stakeholders would find it an easy way to understand the interaction and see what is intended.

You are most concerned with documenting, as in writing specifications. But the same issue is going to pop up when some must document a dynamic page as part of user assistance for that page.

You propose some new visual symbols for use in wireframes and prototypes. A user assistance writer is going to want such cues, beause they are going to have to be aware of all of the states for any dynamic objects on a page in order to write about the page.

More importantly for user assistance, however, is synchronizing the state of a page object with the content of user assistance for that page object that appears in the UI itself or when user assistance is invoked via a popup, alt tag, or any other means, even in a printed document.

As an advocate of embedded user assistance—whether applications are Web or desktop based—I would love to see some discussion amongst designers and developers about how Ajax or other rich media tools can solve that problem, too. We need the same dynamic contextual Help functions we can get in desktop applications to be developed for Web-based applications.

Right now, I spend too much time helping clients invent a tool for contextual Help and not enough focusing them on how helpful that Help is.

Some really interesting thoughts, Richard, on a tough problem. It’s funny how, in 2008, we’re back to flowcharts :)

I tend to agree with Tim; I find that my solution changes with the situation. Being a long-time coder, I often resort to prototypes with my own projects, but go more the wireframe route with clients or when quoting projects with 3rd-party vendors—especially designers—because they need everything in front of them. Of course, then I end up with multiple versions of each mockup—member, non-member, unregistered, signed in, and so on. I definitely think there’s some merit to your solution.

Excellent article. While these sorts of issues have been around for awhile now, obviously Web 2.0 and Ajax—and others, too ;-)—are becoming ever more prevalent.

Back in 2002/03, a very smart architect I was working with at the time got me to,reluctantly, start using UML to describe just these sorts of states and flows and so on. I am not any sort of UML expert by any means, but I have learned enough of the basics to be able to detail out a lot of the interactions being designed within the Web 2.0 landscape.

A few nice things you can do more easily with UML are as follows:

  • represent pages or parts of a page as a state
  • indicate sub-systems within a main flow
  • separate user decision points from system decision points
  • indicate messages back to self—that is, state changes such as module/portlet or page refreshes/states
  • flows within states and so on

Wireflows are a great alternative as well. We should also look to do more scenario buildouts or walkthroughs within our wireframes, too, especially when we know the interaction for a page or piece of the page can get complex. Showing the various scenarios along with solid written descriptions can go a long way toward a better understanding of the intent of a planned design and ultimately a more accurate implementation of said design.

To follow up what I said previously, I believe firmly that wireframes, task flows, and UI specifications/requirements all must still be used and delivered together in order for development to have a chance of understanding the intent of the planned design. None of them can successfully stand alone.

Thank you for your article and the following comments. I have a lot to learn about this topic.

Great writeup! I don’t know—I can’t help but feel some of the assertions herein are out of touch with the stakeholders. Stakeholders that want RIAs might find presentation of flowcharts and/or UML distasteful. Heck, developers hate looking at them. I’ve seen a room filled with glossy developer eyes when UML is broken out. It usually becomes a pretty quiet room of engineers. I’ve heard of people getting headaches from ‘em. Yes, the flowcharts and UML are actually simple, but there’s no denying humans find it difficult to engage with them in a valuable manner. From my personal experience I’d say the deficiency in the communication of UML and flowcharts is the different medium than the target. Drawing a boom-box and using it will never be the same. Yet we start with drawings.

That said, after decades of programming, I’m still leaning toward prototyping and mockups as the most effective tool for communicating requirements, but finding even this solution flawed. I’ve found prototypes can be risky—as the author mentioned, they can easily require deep Web-development chops. I’ve also seen things go awry when prototypes are developed in a team. Yet, for smaller projects—300 hours or under, with one or two developers—I’ve experienced success with the method (twice).

I would assert that effective “RIA prototype development training” might solve the problem for larger projects that require a prototype development team. This is off the radar for the industry in general. I strongly feel a best-of-breed solution might be IAs producing the flowcharts as Richard illustrates, with trained developers creating prototypes from those diagrams. Obviously the challenge here is obtaining funding. One or more persons in the business process usually have the incentive to shortchange either IA or prototyping—or both.

Finally, I’ve used at least one of the tools mentioned above, and it didn’t seem to provide enough power either. I’d like to hear more about stories of learning experiences obtained from their use before seriously considering one again.

Join the Discussion

Asterisks (*) indicate required information.