How to Succeed with Self-Service Analytics Part 3: The Key to Self-Service: Trust but Verify

Self-service analytics goes awry when business users publish and distribute reports broadly without submitting to an agile review process. This chapter shows how to create a governance process to manage standard reports.

Most business users reflexively dislike governance of any kind. Governance processes slow them down, undermine their effectiveness, and make it more difficult to achieve business goals, they say. Yet, without governance, self-service goes awry quickly.

For governance to work, business users must embrace it. They must see it as an incentive, not an obstacle or punishment. When that happens, report authors gain both empowerment and influence, and the business avoids data chaos. There are several ways to create governance incentives, as we’ll see. Besides, governance must be lightweight and efficient: Service level agreements (SLAs) should be associated with each step in the governance process so business users know they won’t be caught in a permanent IT backlog.

Without proper governance, the promised land of self-service empowerment quickly descends into hellish data chaos. The goal is to implement lightweight controls that channel user empowerment without creating confusion and misalignment.

Report Governance

Gateway. The first step in governing self-service analytics is to establish a report governance gateway for the publication of approved standard reports for decision-making. Most companies start by standardizing a dozen or so enterprise reports that executives use. Some departments also create standard reports for local consumption.

An enterprise report gateway governs the distribution of dashboards and reports created by power users for the broader community of casual users. If power users (and IT developers) want their reports consumed by the enterprise, they must submit their report (or a change to an approved report) through the gateway. Business units and departments can also establish report governance gateways for their standard reports.

The gateway consists of three components:

1) A report governance review,

2) An optional production handoff

3) Platform permissions

Report Governance Review. Power users kick off the report governance process by submitting a form that describes the new report or requested changes to an existing standard report. Typically, the analytics director or a triage committee review the form for completeness and clarity and then passes it to a review board. With a typical SLA of 2 to 6 days, the board reviews the proposed changes for conformance with corporate data and design standards, then passes the request to a technical sub committee for review of the SQL code, APIs, and query performance. Once approved, the report receives a watermark or seal of approval with a time-stamp.

The diagram shows a generic process for governing enterprise and departmental reports. Reports that aren’t governed are stored in a personal workspace and shared according to platform permissions. Watermark. The seal of approval is the incentive that drives power users to embrace the governance process because it increases the demand for their reports. The seal is a powerful change agent that gives business users the confidence to trust the figures in the report. As one data analytics leader said, “Our watermark has changed behavior in our organization. When executives open a [self-service] report, they first look for the watermark in the bottom right corner. If it’s not there, they ask why not and question the data.”

“When executives open a [self-service] report, they first look for the watermark in the bottom right corner. If it’s not there, they ask why not and question the data.”

This doesn’t mean that every report needs a watermark. Far from it. “Non-certified” reports serve to answer point-in-time questions or are used by a department or workgroup outside of the governance domain. One data analytics leader described following the 80/20 rule: 80% of reports should be standard reports created or certified by the corporate BI team (or departmental teams), while 20% are non-standard reports created for ad hoc analysis.

Production Handoff. In the case that a power user submits a report in a non-standard format such as Excel, the review board decides whether the report should be rewritten in a standard format. If the new report contains data, not in the existing data warehouse or data lake, the review board again must decide whether to extend the data warehouse with new data before approving the report.

Some companies support an array of “enterprise standard” reports written in different tools and formats, while others do not. Often, the deciding factor is query performance and data security. Users won’t use a standard enterprise report if it is too slow, and the IT team won’t release a report if it can’t protect against unauthorized data access.

Platform Permissions. The best way to govern self-service activity is to set the default mode for publishing reports to “Self Only.” This means that business users can create or modify a report or dashboard but they can’t share it with others. The creator can store the report in a personal workspace for future reference, or they can print the report and show it to colleagues. But they cannot electronically send the report to someone else until they have the authorization to do so, either permanently or on a case-by-case basis.

The best way to govern self-service activity is to set the default mode for publishing reports to “Self Only.”

Controlling permissions to hit the “send” button on a report is a key to governance, but few companies consider this until it’s too late. Also, some BI tools don’t support personal workspaces, which makes it impossible to shut down the report floodgates in time.

Analyst Certification

Another way to incent business users to embrace governance is to require them to demonstrate their knowledge of corporate data, standards, and tooling before receiving permission to distribute reports, a license to a self-service tool, or access to corporate data repositories. This can be done through training courses or certification tests, or a combination. The data analytics leader quoted earlier also said, “We require our data analysts to pass two tests to get a license [to a self-service tool.]” With rights come responsibilities!

“We require our data analysts to pass two tests to get a license [to a self-service tool.]” With rights come responsibilities!

To get certified, business users should demonstrate knowledge of the company’s key metrics, filters, and hierarchies, and how they are defined and calculated, logically and in SQL. They should also be familiar with the company’s visual design standards and templates and know how to use them correctly. They should also know how to navigate the data warehouse or local data mart and use the self-service tool.

Some companies also require users to know basic SQL and statistical and numeracy concepts.

Some power users might object to certification—putting their “rights” ahead of any responsibilities—but over time the majority will see certification as a badge of honor. They will aspire to move through multiple tiers of certification to gain status, advance their careers, and perform rewarding work. This quickly becomes a self-reinforcing cycle as business users who have achieved “status” motivate others to do likewise.

Over time the majority will see certification as a badge of honor.

Self-service analytics without governance is a disaster in the making. But governance without empowerment is a bureaucratic nightmare. To balance the two, companies need to turn governance into an incentive that business users embrace because they get something they want—a seal of approval or access to a self-service tool or data.