August 2016 - January 2017
I led user research and delivered interaction and visual design specifications for the Stakeholder Engagement feature in the PagerDuty web application.
PagerDuty is a SaaS product that aggregates information from IT monitoring tools and facilitates rapid incident response when those tools detect an issue. People who use PagerDuty typically work in IT operations teams responsible for maintaining business-critical infrastructure, services, and applications. A service disruption may result in lost revenue, damage to the service provider's reputation, or—depending on the nature of the business—lost revenue for the company’s customers, so they use PagerDuty to help them minimize downtime.
The PagerDuty service sits between monitoring tools and IT operations teams, enabling managers to centralize alerting policy while also allowing responders to customize how they get alerted when things break. The PagerDuty web app allows people to integrate their monitoring tools with the PagerDuty service and configure alerting policy. The mobile apps for iOS and Android enable people on-call to receive push-notification alerts, as well as acknowledge and resolve those alerts from their smartphones. In addition to push, PagerDuty can also send notifications via email, SMS, and phone call, providing teams with the flexibility and redundancy to achieve reliable alerting.
PagerDuty is designed to mobilize the appropriate technical responders to ensure speedy incident resolution. During this project, my team and I explored how those technical responders might use PagerDuty to keep stakeholders informed during incident response.
Customers talk about different kinds of stakeholders:
- Technical stakeholders, such as directors/VPs within IT
- need to be aware their teams are working on something critical
- need to manage expectations with the rest of the business, and sometimes externally
- may already be PagerDuty users
- Business stakeholders, such as service owners, line of business, or executives
- rely on IT infrastructure being up to do their jobs
- need to manage relationships with important clients or customers
- External stakeholders, such as clients/customers or vendors the organization relies upon to deliver its services
What makes informing stakeholders so difficult? Based on my prior research into our customers’ incident management practices, I had identified the following challenges:
- Stakeholders cannot simply follow along with the technical conference bridge/chat to understand how the incident impacts them and when they can expect a resolution. Following all the responder activity is time-consuming, and a lot of technical detail and discussion is incomprehensible or irrelevant to them.
- A lot of things need to get done during a major incident response. It is difficult for responders to prioritize the task of sending a status update at a given time relative to the other tasks they need to do/delegate to resolve the incident.
- A status update needs to be crafted so that its intended audience gets the key information they need from it. It is time-consuming to craft an appropriate message.
- Stakeholders interrupt the responders to ask for status updates, which increases time to resolve.
- Stakeholders want be aware of current status without much time investment, so they prefer to have status updates pushed to them rather than have to periodically check a status page or incident ticket in the system of record.
- Identifying internal stakeholders to notify is a manual process involving email distribution lists and lists of service owners on wiki pages.
- Responders need to easily share incident status with a large number of people at once.
- Customers don't want to pay the same amount for PagerDuty users whose only role in incident response is stakeholder.
To keep the project scope manageable, my team decided to focus on making it easier for responders to push timely, relevant updates to internal stakeholders so that they would be more likely to give those stakeholders appropriate information as needed.
I proposed a plan to validate early design concepts, analyzed competitors' offerings, sketched a number of concepts, and worked with my team’s product manager to identify and interview customers in our target market. We targeted interview participants with the following characteristics:
- incident responders who are responsible for informing stakeholders about an incident, aka communications manager
- incident stakeholders, including...
- (technical) managers of incident responders (very likely to be existing PagerDuty users)
- VP of Engineering and similarly high-level technical stakeholders (somewhat likely to be existing PagerDuty users)
- CEO and similarly high-level business stakeholders (unlikely to be existing PagerDuty users)
Since we already had some insight into the incident management practices of these customers from prior user research, this research phase focused on nailing down the details of what the solution needed to do in order to meet customers' needs. We agreed upon our learning goals and conducted interviews with customers over the course of a month. Since the product development team was distributed across two office locations, we synthesized the data together over video conference using a web-based tool. Including the entire team in the synthesis of the research data was a deliberate decision of mine and helped ensure everyone was aligned on our findings.
Our findings helped us make decisions that affected implementation, such as how many people the system would need to be able to notify simultaneously. Customer feedback also made me think harder about the PagerDuty onboarding experience for executive-types who have never used PagerDuty before and likely would seldom, if ever, log in.
While the developers worked on implementation and I provided UX reviews as needed, the product manager and I also signed up a small set of customers to provide early feedback on the experience in a "preview" release. As the developers shipped functionality, we gathered internal feedback first, then released the changes to preview participants. The team instrumented the feature so that we could track when and how the feature was being used. I used that quantitative data in conjunction with first-time experience usability tests and post-preview user surveys to prioritize and inform design iterations.
As we approached our target launch date for general availability, I considered how this new feature fit into the existing product, how it would change our customers’ workflows, and how it might change how the market values PagerDuty as a product. In close consultation with both marketing and sales, we introduced a read-only Stakeholder user role and license at a lower price point than a fully featured user license. In the run-up to launch, I communicated with product management, customer support, product marketing, and sales to provide a user-centric perspective on messaging for the feature.
The Stakeholder Engagement feature that launched in January 2017 had three parts:
- a new read-only user role of Stakeholder, associated with a new type of user license at a price point lower than that of the standard user license
- a new tab on the incident page in the web application where responders could easily add subscribers and publish status updates
- a new incident status page, featuring just the information that would interest stakeholders, in a responsive layout that would display well on mobile devices
Probably the most significant business impact this project had was that for the first time, PagerDuty could bill for different types of user licenses at different price points. This change complicated our pricing model but also signals to the market that PagerDuty is not just for engineers on call, that it can provide value to a broader audience within an organization. PagerDuty has closed deals thanks to the functionality that we delivered for this project, and the feature enjoys a healthy adoption level among our largest accounts. Yet customer feedback post-launch has also highlighted several opportunities for future enhancements. I believe that we still have a ways to go to fully meet customers' needs when it comes to engaging stakeholders and keeping them informed in real time, but this first step has already changed the conversations that sales and product have with customers and prospects. PagerDuty is no longer "just a paging tool."