Develop a Monitoring System

Before the Activity

This activity assumes you have gone through the exercise of articulating goals, proposing actions to implement to achieve those goals, and selected which actions to commit to. Additionally, in order for a monitoring system to be reasonably considered, the goals and actions should be expressed in a manner that indicates clear and distinct changes to be observed and taken. Therefore, if you have not yet articulated or met the standards for reliable goals and standards, consider employing methods in the interpret and ideation phases of the design process. 

This activity draws heavily from theory in systems thinking. If you are unfamiliar with systems thinking, in particular the concept of cybernetics, consider reading systems thinking before continuing. 

Preparing Materials 

For the exercise, you will need the set of goals and actions for which you are hoping to develop a monitoring system available for each participant to view and arrange. If conducting the exercise virtually, consider a virtual whiteboarding tool with each goal and action pre-loaded. If in person, consider laying out each goal and action on their own sticky note or index card and displaying it on a wall or whiteboard. Depending upon the volume of goals and actions under consideration, consider preparing an additional document to distribute to each participant for reference. 

The core of the activity involves answering a set of specific questions related to topics of information. If helpful, make use of the metric development worksheet and consider adapting it as a template for several spaces on your whiteboard or wall. 

In addition to preparing content materials, consider preparing a metric development worksheet background to distribute to each participant. 

Receiving Input in Advance 

Depending upon the complexity of your team’s organizational context, it may be beneficial to begin to complete certain steps of the activity ahead of the session driven by input from the team. One key area you can consider for input is “Information Flows.” Consider preparing this section on your own if you have sufficient organizational context, otherwise consider sourcing this information from participants. 

During the Activity

After providing session context and setting ground rules for participation, introduce participants to the concept of a monitoring system, walking through the metric development worksheet background if helpful. Spend time answering questions and ensuring shared mental models of the concept of a system of monitoring and how the existing “goals” and “actions” are part of that model. 

Select a Goal 

To get started, select one goal and designate it in a space for everyone to see. Remind the group that the way we are going to understand our progress towards this goal is through observation and data related to situations of interest to us. Draw or refer to a box and indicate that what we will place in the box are stakeholders and parts of those situations that we would want to pay attention to in order to determine whether the change associated with our stated goal is occurring. Prompt the group to consider the “components” piece by asking, “Who/what are the stakeholders/parts of the situation in which you hope to observe change?” As participants share ideas, be sure to socialize the meaning of each idea and relate it to the other ideas as the components do not exist in isolation. Before moving on, summarize the various components for the group, provide and opportunity for additional ideas, and ensure shared understanding. 

Desired Outcomes 

Next, pivot to desired outcomes. Charge the group with now proposing what the desired change is we hope to see among each component. One by one, select a component and prompt the group, “what change do we hope or expect to see for [insert component]?” List the ideas and synthesize as necessary. Before moving on, do a gut check with the group by summarizing the desired outcomes and asking, “if we were to achieve our goal, does it sound right that we would know it by these desired outcomes?” As necessary, iterate on this section and the components. 

Observations 

Next, transition to observations. Charge the group with now identifying how we are going to come to know whether or not each of these changes has occurred. One by one, select a desired outcome and prompt the group, “What might we be able to systematically track to monitor this desired outcome?” List the ideas and synthesize as necessary. Before moving on, do a gut check with the group by summarizing the desired outcomes and asking, “if we were to achieve our goal, does it sound right that we would feel confident in knowing we were making progress by these observations?” As an optional step here, you might want to begin to categorize where observations are taking place simultaneously. 

Information Channels 

Next, transition to information channels. If this has not yet been sketched out, ask the group to list the different existing channels that deliver information about the systems related to the goal under consideration (these might include existing practice, platforms, pieces of software, etc.). One by one, select an observation and prompt the group, “is there an existing information channel through which we could systematically gather this observation/data?” List and describe it under the related channel or sort it into an area for which there are no existing channels. After going through all observations, review what might be travelling through each information channel in total and discuss the organizational implications of collecting this information compared to the current context. Next, look through the ideas for which there are no existing channels and ideate with the group around new channels that might serve these data. 

Actions 

Next, recap what has taken place so far. You have mapped out from your stated goal all the way down to what you might want to see as evidence of that goal being achieved and how you might collect that information. Now, introduce the set of actions that were proposed related to this goal before the session. Ask the group if these actions feel like the right levers to pull in some combination to drive towards the desired outcomes. As necessary, lead ideation around additional actions. 

Responsibilities 

Next, identify the monitoring and decision-making responsibilities for this system. Determine which roles might be responsible for monitoring which data and which roles might be responsible for executing certain actions in response to the data. Detail those assignments as a group and propose an organizationally appropriate method of coordinating an approach amongst stakeholders. 

Close 

Review the ideas the group has committed to and work with them to detail next steps. 

After the Activity

Next steps likely include preparing documentation around work committed to, new responsibilities for existing roles (the possible creation of new roles), and a communications plan to socialize this. Depending upon your role, synthesize workshop documentation and prepare for transition into next steps. 

• Record and store
• Regularly track 
• If need be, begin another design process to address