From Spaghetti Scripts to Streamlined Services: Managing Data Sources with KPIs.do

The scene is all too familiar for growing tech companies. It starts with a simple request: "Can we track our monthly active users?" An engineer writes a quick script. It queries the production database, runs on a cron job, and posts the result to a Slack channel. Success!

Then comes the next request. "Now, can we see our monthly recurring revenue from Stripe?" Another script. "And what about our new user sign-ups from Salesforce?" A third script. Before you know it, you’re the unwilling owner of a tangled mess of "spaghetti scripts." This fragile ecosystem runs on hope, duct tape, and a forgotten EC2 instance. Every API key change, every database schema update, every silent cron job failure is a disaster waiting to happen.

This ad-hoc approach is not scalable, reliable, or secure. It creates technical debt and drains engineering resources that could be spent on your core product.

What if you could move beyond this chaos? What if you could manage your business metrics not as a collection of brittle scripts, but as a robust, streamlined, and automated service? This is the core philosophy behind KPIs.do: transforming scattered data integration into a coherent, developer-first workflow.

The Chaos of Traditional KPI Data Integration

The "spaghetti script" problem isn't just about messy code; it's a systemic issue with deep-seated pain points that hamstring data-driven teams.

• Brittle, Bespoke Connections: Each data source—whether it's a PostgreSQL database, the Stripe API, or a Google Sheet—requires custom code. You have to handle authentication, pagination, rate limiting, and error handling for each one. When an API provider deprecates an endpoint, your entire reporting pipeline breaks.
• The Maintenance Nightmare: These scripts are often undocumented and owned by a single person. When that person goes on vacation or leaves the company, a simple reporting bug can become a full-blown forensic investigation.
• Data Silos and Inconsistency: When metric logic is spread across multiple scripts, it's easy for inconsistencies to creep in. Is "revenue" calculated before or after refunds? Does "active user" mean logging in once a month or performing a key action? Without a central definition, you can't guarantee a single source of truth.
• Lack of Security and Centralized Control: Scattering database credentials and API keys across various scripts and environments is a significant security risk. There’s no central place to manage or rotate these secrets, making your critical data infrastructure vulnerable.

The KPIs.do Way: From Scripts to Services with 'Metrics as Code'

KPIs.do fundamentally changes the paradigm. Instead of writing imperative scripts that say how to fetch and process data, you use our API to declaratively define what your KPI is. This is the power of Metrics as Code.

You define the KPI's logic, its data sources, and its targets programmatically. Our agentic workflow platform takes care of the rest.

Think of it like Infrastructure as Code (IaC), but for your business metrics. Your KPI definitions live in your codebase, can be version-controlled with Git, and are deployed as part of your development lifecycle.

A Unified Data Connection Layer

With KPIs.do, you stop writing boilerplate integration logic. Instead of building custom connectors, you simply tell our platform where the data lives.

As our FAQ states: You can connect to virtually any data source, including databases (SQL, NoSQL), data warehouses (e.g., BigQuery, Snowflake), business applications (e.g., Salesforce, Stripe), or any custom API endpoint. Our agents handle the integration logic.

Let's see what this looks like. Imagine you want to calculate your Net Monthly Active Users by pulling active users from your application database and subtracting churned users from Stripe.

Instead of writing two separate scripts, you would define a single KPI service:

import { KPIs } from '@do/sdk';

// Define a new KPI service via the API
const kpiDefinition = await KPIs.create({
  id: 'net-monthly-active-users',
  name: 'Net Monthly Active Users',
  // 1. Declare your data sources, don't write connection logic
  dataSources: [
    { 
      id: 'active_users',
      type: 'sql', 
      query: 'SELECT COUNT(DISTINCT user_id) FROM user_logins WHERE login_date > NOW() - INTERVAL \'30 days\'' 
    },
    { 
      id: 'churned_users',
      type: 'stripe',
      query: 'events.list',
      params: { type: 'customer.subscription.deleted' }
    }
  ],
  // 2. Define the calculation logic once
  calculation: 'data.active_users - data.churned_users'
});

// 3. Let the agentic workflow handle the rest.
//    Just query the final status whenever you need it.
const kpi = await KPIs.get('net-monthly-active-users');

console.log({
  value: kpi.currentValue, // The final, calculated value
  status: kpi.status,      // 'on-track'
  trend: kpi.trend,        // 'increasing'
});

In this example, you aren't managing database clients or making fetch requests to the Stripe API. You simply declare what you need. The KPIs.do agents autonomously and securely connect to the sources, execute the queries, perform the calculation, and make the result available through a simple API call.

The Benefits of a Service-Oriented Metric Workflow

Adopting a 'Metrics as Code' approach with KPIs.do moves your team away from firefighting and towards strategic analysis.

1. Reliability & Resilience: Our managed platform handles retries, transient network errors, and API changes. If a data source connection fails permanently, you get an automated alert, ending the era of silent script failures.
2. Scalability: Need to add a third data source from Google Analytics? You don't write a new script. You simply add another data source object to your KPI definition. It's clean, simple, and scalable.
3. Centralized Security: Manage all your database credentials and API keys in one secure, unified environment instead of scattering them across your infrastructure.
4. Focus & Speed: Your engineers are freed from the plumbing of data integration. They can get back to building your core product, while your data analysts and business leaders can trust the metrics they rely on to make critical decisions.

Stop Writing Scripts. Start Building Services.

It’s time to retire the spaghetti scripts. Your Key Performance Indicators are too important to be left to a patchwork of brittle, unmanaged code.

By treating your metrics as first-class services, you build a performance management system that is as robust and scalable as your core product.

Ready to turn your key metrics into actionable services? Explore the KPIs.do platform and start managing your performance metrics as code today.