How to Succeed with Self-Service Analytics Part 5: Tools and Technologies

Technology plays a critical role in making self-service a reality. The advent of AI-infused products have changed the equation for self-service analytics.

So far in this series, we’ve focused on the importance of governance processes and organizational models to achieve success with self-service analytics. But tools and technologies also play an important role.

For decades, software vendors proclaimed the dawning of the era of self-service analytics. The reality has rarely matched the hype. Data analytics products have always been harder to configure and use than most users anticipate. But now that’s changed.

The advent of machine learning (ML) has dramatically changed the equation for self-service analytics. ML has made data and analytics tools both more powerful and easier to use for a wide spectrum of users, automating tasks that once took one or more experts to accomplish. These tools turn business users into citizen data analysts, and data analysts into citizen data scientists and
data engineers.

The advent of machine learning has dramatically changed the equation for self-service analytics.

Self-service tools have permeated every aspect of data analytics. Both casual and power users have benefitted.

Casual User Tools. Data consumers now have AI-powered reporting and dashboard tools that automatically generate analyses in the background, displaying the most relevant relationships, including correlations, anomalies, and trends detected in a selected
chart, dashboard, or data set. Data explorers can now generate natural language queries by speaking or writing words into a search box as a way to kickstart an analysis. They also can create dashboards by pinning the results to a display. (For more information about these types of tools, see “AI is the New BI: How Algorithms are Transforming Business Intelligence and Analytics”.)

For casual users, these AI-infused tools are like having a personal data analyst in their pocket. They enable casual users to take on some of the functions of a data analyst, allowing data analysts to focus on more complex business
problems.

Power User Tools. Power users are perhaps the biggest beneficiaries of the self-service movement. In the past, power users relied on the IT department for data, and when that took too long, they used Excel and Access to query databases and stitch together the information into a report. The process was cumbersome and time-consuming and left little time for analysis.

Today, self-service tools empower data analysts to do the work formerly done by BI developers,
data engineers, data scientists and application developers. For instance, data catalogs enable data analysts to find, profile and annotate relevant data sets. New data preparation tools make it easy to combine data sources, clean and transform data, and share these workflows with others. And new visual analysis tools make it easy for data analysts to query data, visualize and analyze the results, compile into a dashboard, and publish for others to consume.

(See “Self-Service Triumvirate: The New Data
Analyst Workbench.”)

Data analysts who aspire to become data scientists now have self-service tools that make it easy to build predictive and prescriptive models. These so-called AutoML tools provide a cloud-based, graphical environment for building and deploying machine learning models, including automated data prep, feature generation, and model-build selection. The tools profile a data set, automatically select algorithms to test and apply the appropriate data preparation routines for each selected
algorithm. Most come with built-in APIs to score the models against production data and monitor the results over time.

Finally, data analysts can embed charts, tables, and an entire self-service environment into other applications (e.g., portals, cloud applications, custom or mobile applications) with new “low-code” or “no code” tools. They can also white-label a data analytics tool using a point-and-click interface or by configuring cascading style sheets.

Self-Service Shifts

The advent of powerful new self-service tools is shifting what business and technical users can do with data. Soon, this shift will turn into an expectation written into job descriptions and performance reviews.

For example, data consumers will be expected to query and analyze data and modify dashboards thanks to new AI-enabled BI tools. Data explorers will be expected to create new dashboards or modify analytics applications with new queries. Data analysts will reallocate time currently spent finding and preparing data to more value-added activities, such as analyzing data and creating lightweight ML models, thanks to a host of tools mentioned above. Data scientists will create their own data pipelines and deploy models into production thanks to codeless data pipelining tools. And data engineers will focus more time building a robust, self-service oriented enterprise data platform rather than creating data pipelines for data analysts and data scientists.

The Impact of a Self-Service Data Platform

The most impactful way organizations can facilitate self-service analytics is to build a robust data platform that provides domain specific data sets and is geared to empower business and technical users. As a data platform becomes more robust, business users and technical developers become more self-sufficient. They do more work themselves without the help from an upstream specialist. This frees up resources to focus on more value-added activities.

As a data platform supports more shared metrics and domain models, the more self-sufficient business and technical users become.

Data Refinery. A self-service platform provides appropriate access points for each type of business and technical user. To do this, the data team creates a “data refinery” that transforms raw data into various types of assets. The landing zone contains raw data extracted from a host of sources; the integrated zone combines data into flat, subject-area building blocks; a dimensional zone converts that data into a dimensional model with conformed dimensions and trusted data; and a domain zone creates a department-specific view of data pulled from the other three zones. Business users armed with dashboards, reports, analytic applications, and AI-based search tools access data through specially crafted business views that run against data in the refinery. Power users equipped with a bevy of self-service tools and appropriate permissions use a data catalog to find appropriate data in the refinery, a sandbox to explore that data, and data prep tools to extract and transform the data into custom data sets for analysis. A well-designed data platform makes self-service tools more powerful and users more productive.