Durante el desarrollo del proyecto Goliat - Dashboard, una herramienta de código abierto para gestionar y optimizar despliegues de Terraform , surgió una necesidad específica: identificar los costos asociados a recursos en Azure que estén etiquetados con información específica, como el nombre de una organización o un identificador de proyecto.
Las etiquetas en Azure permiten clasificar recursos, lo que facilita la administración de costos por equipo, proyecto o entorno. Para aprovechar esta capacidad, desarrollé una solución que permite:
- Buscar resource groups (RG) que contienen una etiqueta específica.
- Obtener los costos de esos resource groups.
- Visualizar esos costos segmentados por grupo de recursos, ubicación y nivel de servicio.
Esta solución se compone de:
- Un endpoint en Astro para consultar Azure y devolver datos de costos filtrados por etiquetas.
- Un componente React para visualizar los datos de manera clara y segmentada.
¿Qué ofrece esta solución?
- Filtrado preciso : identifica y analiza solo los resource groups que contienen una etiqueta específica.
- Monitoreo en tiempo real : consulta actualizada directamente desde Azure.
- Visualización clara : muestra gráficos segmentados por:
- Resource group.
- Ubicación.
- Nivel de servicio (tier).
- Cacheo inteligente : guarda resultados en MongoDB para evitar consultas repetidas y mejorar el rendimiento.
1. Crear el endpoint en Astro
¿Qué hace el endpoint?
El endpoint se conecta a las APIs de Azure y realiza lo siguiente:
- Busca todos los resource groups dentro de las suscripciones disponibles.
- Filtra aquellos resource groups que contengan una etiqueta específica , como
organization
,workspace
oprojectId
. - Obtiene los costos asociados a esos resource groups utilizando el servicio de cost management de Azure.
- Devuelve los datos en formato JSON para que puedan ser consumidos por el componente React.
¿Dónde crear el endpoint?
- Crea un archivo en la siguiente ruta de tu proyecto Astro:
import { methodValidator } from "../utils/methodValidator"; import { log } from "../utils/logging"; import { ClientSecretCredential } from "@azure/identity"; import { SubscriptionClient } from "@azure/arm-subscriptions"; import { ResourceManagementClient } from "@azure/arm-resources"; import { CostManagementClient } from "@azure/arm-costmanagement"; import getDatabase from "../utils/mongoClient"; const TENANT_ID = import.meta.env.AZURE_TENANT_ID as string; const CLIENT_ID = import.meta.env.AZURE_CLIENT_ID as string; const CLIENT_SECRET = import.meta.env.AZURE_CLIENT_SECRET as string; const credential = new ClientSecretCredential( TENANT_ID, CLIENT_ID, CLIENT_SECRET, ); const getSubscriptions = async () => { log("Fetching subscriptions...", "DEBUG"); const client = new SubscriptionClient(credential); const subscriptions = []; for await (const subscription of client.subscriptions.list()) { subscriptions.push(subscription); } log(`Fetched ${subscriptions.length} subscriptions.`, "DEBUG"); return subscriptions; }; const getResourceGroups = async (subscriptionId: string) => { log(`Fetching resource groups for subscription: ${subscriptionId}`, "DEBUG"); const client = new ResourceManagementClient(credential, subscriptionId); const resourceGroups = []; for await (const rg of client.resourceGroups.list()) { resourceGroups.push(rg); } log( `Fetched ${resourceGroups.length} resource groups for subscription ${subscriptionId}.`, "DEBUG", ); return resourceGroups; }; const getCostData = async ( subscriptionId: string, resourceGroupName: string, ) => { log( `Fetching cost data for resource group: ${resourceGroupName} in subscription: ${subscriptionId}`, "DEBUG", ); const costClient = new CostManagementClient(credential); const result = await costClient.query.usage( `/subscriptions/${subscriptionId}/resourceGroups/${resourceGroupName}`, { type: "ActualCost", timeframe: "BillingMonth", dataset: { granularity: "None", aggregation: { totalCost: { name: "PreTaxCost", function: "Sum", }, }, grouping: [ { type: "Dimension", name: "ServiceName", }, { type: "Dimension", name: "ResourceLocation", }, { type: "Dimension", name: "MeterSubCategory", }, ], }, }, ); log( `Fetched cost data for resource group ${resourceGroupName}: ${JSON.stringify(result)}`, "DEBUG", ); return result; }; export const GET = async ({ request }: { request: Request }) => { const url = new URL(request.url); const organization = url.searchParams.get("organization"); const workspace = url.searchParams.get("workspace"); const projectId = url.searchParams.get("projectId"); log("Received GET request.", "DEBUG"); if (!methodValidator(request, ["GET"])) { log("Invalid method used. Only GET is allowed.", "ERROR"); return new Response(JSON.stringify({ error: "Method Not Allowed" }), { status: 405, headers: { "Content-Type": "application/json" }, }); } if (!organization) { log("Organization parameter is missing.", "ERROR"); return new Response( JSON.stringify({ error: "Organization is required." }), { status: 400, headers: { "Content-Type": "application/json" }, }, ); } if ((workspace && projectId) || (!workspace && !projectId)) { log( "Invalid parameters: Provide either workspace or projectId, but not both.", "ERROR", ); return new Response( JSON.stringify({ error: "Provide either workspace or projectId, but not both.", }), { status: 400, headers: { "Content-Type": "application/json" }, }, ); } try { log("Checking database connection", "DEBUG"); const db = await getDatabase(); const identifier = workspace || projectId || ""; if (db) { log( `Checking cached data for organization: ${organization}, identifier: ${identifier}`, "DEBUG", ); const collection = db.collection("azure_cost_data"); const cachedData = await collection.findOne({ organization, identifier, }); if (cachedData) { log( `Returning cached data for organization: ${organization}, identifier: ${identifier}`, "DEBUG", ); return new Response(JSON.stringify(cachedData), { status: 200, headers: { "Content-Type": "application/json" }, }); } } else { log("Database not available, proceeding without caching.", "DEBUG"); } log("Fetching subscriptions...", "DEBUG"); const subscriptions = await getSubscriptions(); const results: Record< string, { organization: string; identifier: string; totalAllRGs?: string; totalAllRGsByLocation?: Record<string, string>; totalAllRGsByTier?: Record<string, string>; resourceGroups: Array<{ resourceGroupName: string; totalCost: string; costByCategory: Record<string, string>; costByLocation: Record<string, string>; costByTier: Record<string, string>; }>; } > = {}; results[identifier] = { organization, identifier, resourceGroups: [] }; let totalCostSum = 0; let costCurrency: string | null = null; const totalCostByLocation: Record<string, number> = {}; const totalCostByTier: Record<string, number> = {}; for (const subscription of subscriptions) { const subscriptionId = subscription.subscriptionId; if (!subscriptionId) { log("Subscription ID is undefined.", "ERROR"); continue; } log(`Processing subscription: ${subscriptionId}`, "DEBUG"); const resourceGroups = await getResourceGroups(subscriptionId); const matchingResourceGroups = resourceGroups.filter((rg: any) => { const tags = rg.tags || {}; return ( tags.organization === organization && ((workspace && tags.workspace === workspace) || (projectId && tags.projectId === projectId)) ); }); log( `Found ${matchingResourceGroups.length} matching resource groups.`, "DEBUG", ); for (const rg of matchingResourceGroups) { try { const resourceGroupName = rg.name; if (!resourceGroupName) { log( `Resource group name is undefined for subscription ${subscriptionId}`, "ERROR", ); continue; } const costData = await getCostData(subscriptionId, resourceGroupName); log( `Cost data response for ${resourceGroupName}: ${JSON.stringify(costData)}`, "DEBUG", ); if (!costData || !costData.columns || !costData.rows) { log("No columns or rows in cost data result.", "ERROR"); continue; } const costIndex = costData.columns.findIndex( (col: any) => col.name === "PreTaxCost", ); const serviceNameIndex = costData.columns.findIndex( (col: any) => col.name === "ServiceName", ); const locationIndex = costData.columns.findIndex( (col: any) => col.name === "ResourceLocation", ); const tierIndex = costData.columns.findIndex( (col: any) => col.name === "MeterSubCategory", ); const currencyIndex = costData.columns.findIndex( (col: any) => col.name === "Currency", ); let rgTotalCost = 0; let currency = ""; const costByCategory: Record<string, number> = {}; const costByLocation: Record<string, number> = {}; const costByTier: Record<string, number> = {}; for (const row of costData.rows) { const cost = row[costIndex]; const serviceName = row[serviceNameIndex]; const resourceLocation = row[locationIndex]; const tier = row[tierIndex] || "No Tier Info"; currency = row[currencyIndex]; rgTotalCost += cost; costByCategory[serviceName] = (costByCategory[serviceName] || 0) + cost; costByLocation[resourceLocation] = (costByLocation[resourceLocation] || 0) + cost; costByTier[tier] = (costByTier[tier] || 0) + cost; } if (!costCurrency) { costCurrency = currency; } totalCostSum += rgTotalCost; for (const [location, cost] of Object.entries(costByLocation)) { totalCostByLocation[location] = (totalCostByLocation[location] || 0) + cost; } for (const [tier, cost] of Object.entries(costByTier)) { totalCostByTier[tier] = (totalCostByTier[tier] || 0) + cost; } const formattedTotalCost = new Intl.NumberFormat("es-ES", { style: "currency", currency: currency, }).format(rgTotalCost); const formattedCostByCategory: Record<string, string> = {}; for (const [serviceName, cost] of Object.entries(costByCategory)) { formattedCostByCategory[serviceName] = new Intl.NumberFormat( "es-ES", { style: "currency", currency: currency, }, ).format(cost); } const formattedCostByLocation: Record<string, string> = {}; for (const [location, cost] of Object.entries(costByLocation)) { formattedCostByLocation[location] = new Intl.NumberFormat("es-ES", { style: "currency", currency: currency, }).format(cost); } const formattedCostByTier: Record<string, string> = {}; for (const [tier, cost] of Object.entries(costByTier)) { formattedCostByTier[tier] = new Intl.NumberFormat("es-ES", { style: "currency", currency: currency, }).format(cost); } results[identifier].resourceGroups.push({ resourceGroupName, totalCost: formattedTotalCost, costByCategory: formattedCostByCategory, costByLocation: formattedCostByLocation, costByTier: formattedCostByTier, }); } catch (error) { log(`Error processing resource group ${rg.name}: ${error}`, "ERROR"); } } } if (results[identifier].resourceGroups.length === 0) { log("No matching resource groups found.", "ERROR"); return new Response( JSON.stringify({ error: "No matching resource groups found." }), { status: 404, headers: { "Content-Type": "application/json" }, }, ); } if (costCurrency) { results[identifier].totalAllRGs = new Intl.NumberFormat("es-ES", { style: "currency", currency: costCurrency, }).format(totalCostSum); const formattedTotalAllRGsByLocation: Record<string, string> = {}; for (const [location, cost] of Object.entries(totalCostByLocation)) { formattedTotalAllRGsByLocation[location] = new Intl.NumberFormat( "es-ES", { style: "currency", currency: costCurrency, }, ).format(cost); } results[identifier].totalAllRGsByLocation = formattedTotalAllRGsByLocation; const formattedTotalAllRGsByTier: Record<string, string> = {}; for (const [tier, cost] of Object.entries(totalCostByTier)) { formattedTotalAllRGsByTier[tier] = new Intl.NumberFormat("es-ES", { style: "currency", currency: costCurrency, }).format(cost); } results[identifier].totalAllRGsByTier = formattedTotalAllRGsByTier; } if (db) { log( `Caching data for organization: ${organization}, identifier: ${identifier}`, "DEBUG", ); const collection = db.collection("azure_cost_data"); await collection.updateOne( { organization, identifier }, { $set: results[identifier] }, { upsert: true }, ); } return new Response(JSON.stringify(results[identifier]), { status: 200, headers: { "Content-Type": "application/json" }, }); } catch (error: unknown) { log( `Internal server error: ${error instanceof Error ? error.stack : String(error)}`, "ERROR", ); return new Response(JSON.stringify({ error: "Internal server error" }), { status: 500, headers: { "Content-Type": "application/json" }, }); } };
Copia el código del endpoint en ese archivo. Este código manejará la búsqueda de resource groups etiquetados y la consulta de costos a Azure.
Configura las variables de entorno en el archivo
.env
:Instala las dependencias necesarias para conectar con Azure y MongoDB:
2. Crear el componente en React
¿Qué hace el componente?
El componente React se encarga de:
- Consultar el endpoint para obtener los costos de los resource groups filtrados por etiquetas.
- Mostrar el estado de carga y los posibles errores.
- Visualizar los datos en gráficos tipo doughnut usando Chart.js :
- Costos por resource group.
- Costos por ubicación.
- Costos por nivel de servicio (tier).
¿Dónde crear el componente?
- Crea el archivo del componente en:
import React, { useEffect, useState } from "react"; import { Doughnut } from "react-chartjs-2"; import { FaSpinner } from "react-icons/fa"; import { Chart as ChartJS, ArcElement, Tooltip, Legend } from "chart.js"; ChartJS.register(ArcElement, Tooltip, Legend); export default function WorkspaceCostsController({ organization, workspace }) { const [data, setData] = useState(null); const [loading, setLoading] = useState(true); const [error, setError] = useState(null); const fetchData = async () => { setLoading(true); setError(null); try { const url = `/api/private/azureCost?organization=${organization}&workspace=${workspace}`; const response = await fetch(url); if (!response.ok) { const errorText = await response.text(); throw new Error(`Error fetching data: ${errorText}`); } const jsonData = await response.json(); setData(jsonData); } catch (err) { setError(err.message); } finally { setLoading(false); } }; useEffect(() => { fetchData(); }, [organization, workspace]); if (loading) { return ( <div className="flex justify-center items-center h-64"> <FaSpinner className="animate-spin text-2xl text-gray-900 dark:text-white" /> </div> ); } if (error) { return <p className="text-red-500">Error: {error}</p>; } if (!data) { return ( <p className="text-gray-500 dark:text-gray-400">No data available</p> ); } const { totalAllRGs, totalAllRGsByLocation, totalAllRGsByTier, resourceGroups, } = data; const isDarkMode = document.documentElement.classList.contains("dark"); const textColor = isDarkMode ? "#F9FAFB" : "#1F2937"; const borderColor = isDarkMode ? "#4B5563" : "#D1D5DB"; const parseEuroValue = (val) => { if (!val) return 0; let numericString = val.replace(/[^\d.,-]/g, ""); numericString = numericString.replace(",", "."); const numberValue = parseFloat(numericString); return isNaN(numberValue) ? 0 : numberValue; }; const formatEuro = (value) => { return new Intl.NumberFormat("es-ES", { style: "currency", currency: "EUR", }).format(value); }; const generateRandomPastelColor = () => { const hue = Math.floor(Math.random() * 360); const base = `hsl(${hue}, 70%, 80%)`; const hover = `hsl(${hue}, 70%, 60%)`; return { base, hover }; }; const generateDoughnutData = (obj) => { const entries = Object.entries(obj || {}); const filteredEntries = entries.filter( ([_, val]) => parseEuroValue(val) !== 0, ); if (filteredEntries.length === 0) { return null; } const labels = filteredEntries.map(([key]) => key); const values = filteredEntries.map(([_, val]) => parseEuroValue(val)); const backgroundColors = []; const hoverBackgroundColors = []; labels.forEach(() => { const { base, hover } = generateRandomPastelColor(); backgroundColors.push(base); hoverBackgroundColors.push(hover); }); return { labels, datasets: [ { data: values, backgroundColor: backgroundColors, hoverBackgroundColor: hoverBackgroundColors, borderColor: borderColor, borderWidth: 2, }, ], }; }; const locationData = totalAllRGsByLocation ? generateDoughnutData(totalAllRGsByLocation) : null; const tierData = totalAllRGsByTier ? generateDoughnutData(totalAllRGsByTier) : null; let rgData = null; if (resourceGroups && resourceGroups.length > 0) { const rgObj = {}; resourceGroups.forEach((rg) => { if (rg.totalCost) { rgObj[rg.resourceGroupName] = rg.totalCost; } }); rgData = generateDoughnutData(rgObj); } const doughnutOptions = { responsive: true, maintainAspectRatio: false, layout: { padding: { top: 10, bottom: 10, }, }, plugins: { legend: { position: "bottom", labels: { color: textColor, }, }, tooltip: { bodyColor: textColor, titleColor: textColor, backgroundColor: isDarkMode ? "#374151" : "#ffffff", callbacks: { label: function (context) { const label = context.label || ""; const value = context.parsed; return `${label}: ${formatEuro(value)}`; }, }, }, }, }; return ( <div className="mt-6"> <div className="grid w-full grid-cols-1 gap-4 xl:grid-cols-3"> <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col"> <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2"> Cost by Resource Group </h3> <p className="text-sm text-gray-600 dark:text-gray-400 mb-4"> Cost breakdown by each resource group. </p> <div className="h-64"> {rgData ? ( <Doughnut data={rgData} options={doughnutOptions} /> ) : ( <p className="text-gray-500 dark:text-gray-400">No data</p> )} </div> </div> <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col"> <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2"> Cost by Location </h3> <p className="text-sm text-gray-600 dark:text-gray-400 mb-4"> Cost breakdown by resource region. </p> <div className="h-64"> {locationData ? ( <Doughnut data={locationData} options={doughnutOptions} /> ) : ( <p className="text-gray-500 dark:text-gray-400">No data</p> )} </div> </div> <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col"> <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2"> Cost by Tier </h3> <p className="text-sm text-gray-600 dark:text-gray-400 mb-4"> Cost breakdown by service tier. </p> <div className="h-64"> {tierData ? ( <Doughnut data={tierData} options={doughnutOptions} /> ) : ( <p className="text-gray-500 dark:text-gray-400">No data</p> )} </div> </div> </div> </div> ); }
Copia el código del componente en ese archivo.
Instala las dependencias necesarias para los gráficos y los íconos de carga:
3. Integrar el componente en una página de Astro
Para mostrar el dashboard de costos en tu aplicación Astro:
Crea una página Astro en:
Importa el componente y pásale los parámetros necesarios, como
organization
yworkspace
:
4. Probar la implementación
Inicia el servidor de desarrollo con el siguiente comando:
Accede a la página en tu navegador, por ejemplo:
Deberías ver los gráficos que muestran los costos de los resource groups filtrados por la etiqueta especificada.
Conclusión
Esta solución permite monitorear y visualizar los costos de Azure de una manera eficiente, enfocándose en aquellos resource groups que cumplen con criterios específicos a través de etiquetas. Esto facilita la gestión de recursos y la optimización de costos en proyectos complejos.
Si te interesa una herramienta integral para gestionar tus despliegues de Terraform y visualizar costos en Azure, te invito a explorar el proyecto Goliat - Dashboard, una plataforma de código abierto diseñada para optimizar tus operaciones de infraestructura.
Top comments (0)