Google Cloud Platform provides you with simple ways to access computing resources and services. Using Google Cloud services comes with expenses, so it’s crucial to keep a close eye on your spending.

In April, we recently found out that our spending on GCP services, where we host our projects and applications, is significant. In total, we spent $4,327.45 on GCP during that month. Our client requested us to review these expenses and identify the services that are consuming a large portion of our budget. After investigating our billing cycle for that month, we identified the following services as the main contributors to our expenses:

• Cloud functions
• BigQuery
• CloudSQL

Billing report graph for April:

We identified several action items to reduce our total spending on these services. After implementing all of them, we successfully reduced our costs by approximately $3000 per month.

Billing report graph for November:

However, since we continuously develop projects and plan to upgrade the architecture of our main application to accommodate 3x-5x more users, there is a possibility that our spending may increase in the future. To ensure we monitor our spending on GCP services, we have implemented a system to track our monthly expenses. We have set a specific budget limit and established different levels of alerts to notify us when we approach that limit.

As a solution, we introduced a new micro-service for tracking and monitoring the spending amount at the GCP service. This microservice elevates an alert whenever the overall budget surpasses or meets predefined thresholds. Doing this lets you stay on track with your budget and take action if spending gets out of hand.

Ways to get the GCP budget Alerts

It can be configured for billing accounts and projects, allowing customization based on a specified spending limit or the previous month’s expenditures. These alerts are activated when your spending exceeds a predefined percentage of the budget.

We can receive notifications conveniently through these services according to our preferences.

• E-mail.
• Cloud Pub/Sub events.

Upon reaching the threshold, the emails are sent to billing administrators and the billing account users. It’s important to note that the email template is fixed and cannot undergo customization. On the other hand, Cloud Pub/Sub events offer flexibility by serving as alert triggers, enabling the transmission of notifications to Slack. Unlike the fixed email template, the Slack message is customizable and can be adjusted as per your requirements.

I know you are thinking what is a Cloud Pub/Sub event?  

Cloud Pub/Sub event, must be a bit mysterious at first, but it’s essentially a way to receive updates efficiently. Think of it as a secret messenger silently carrying important messages about your budget and spending. One more advantage is it’s customisable.

Using Cloud Pub/Sub for Slack Alerts

Cloud Pub/Sub event is a way to receive updates efficiently. Think of it as a secret messenger silently carrying important messages about your budget and spending.

If we opt for Cloud Pub/Sub, an event will be published on a topic every 50 minutes approximately. This event will contain important information like the billing account ID, the current threshold limit that has been reached, the budget limit, and the total cost incurred so far for the month.

High-level architecture of the GCP budget alert service should be like this:

In the diagram above we can see how the solution works. Billing Alerts trigger events, which are sent to a Cloud Pub/Sub topic, and then a Cloud Function listens to this topic and handles the billing alert event. The cloud function sends a notification to the Slack Channel.

Setup for the micro-service

Before we start writing the code for our notifications, we need to do a few things first:

1. Create a new Cloud Pub/Sub topic called “billing-alerts” using Cloud Pub/Sub. This topic will be used to send billing alerts. The Topic will be a Pub (Publisher) and an upcoming cloud function acting as a Sub (Subscriber).
2. Go to the Billing section in the Google Cloud Console and set up your budget.
3. In the “Manage notifications” section of the budget setup, connect it to the “billing-alerts” channel we created earlier in step 1.
   Note: In step 1, we created a Cloud Pub/Sub topic named “billing-alerts”.

Set up Slack access: Create a new Slack application by following the instructions at https://api.slack.com/slack-apps. Make sure to keep the Slack Access Token handy, as we will need it later to configure access from our Cloud Function to Slack.

Steps to set up the budget for your project at Google Cloud console

1. Go to the billing section at the GCP console.

2. Click on the Budget & Alert tab on the left side of the page.

3. Click on the Create Budget button on the top.

4. Add a name for your budget, select the time range for the budget alerts, and then select your organization, Project, and Services for which you need the Alerts.

5. Then moving to the next step add the amount of budget as per your requirements or preferences.

6. Now, moving to the next step set the alert threshold rules, and select the PubSub topic we created earlier for “billing-alerts”.

Now the setup for the budget is completed.

We will use a cloud function to subscribe to the Billing Alerts Cloud Pub/Sub topic, we will write the event data to BigQuery, and send a notification to the Slack channel.

Navigate to Cloud Function in the Google Cloud Console and create a new Cloud function with the following values (We can change the values as per our expectations):

• Name: gcp-budget-alerts
• Trigger: Cloud Pub/Sub
• Topic: billing-alerts
• Source code: inline editor
• Runtime: Python 3.8

Function to execute: handle (Endpoint of the cloud function)

The codebase setup for the microservice

The open-source GitHub repository for the codebase setup of GCP Alert Service: gcp-budget-alerts-service. When code setup for the service is done. We just need to deploy the code to the cloud function and monitor the Pub/Sub event trigger.

Our alert message will look like this on Slack:

To ensure the alert messages are sent to the designated channel, it’s necessary to establish and set up a Slack bot as outlined in the repository’s prerequisites.

The alert message contains the percentage of the threshold used, the Total budget amount we fixed, the budget period, the budget name, and The billing account ID.

Note: We can customize the design and data on the alert as needed in the codebase for the GCP alert service.

Impact

Our client wanted to keep an eye on the investments and the spending on the whole project hosted on the GCP so that an informed decision can be taken in case of high spending. 
Implementing this solution helped in the monitoring of the amount of spending on the different services of GCP. As a result, our client now receives regular updates on the current spending amount every month. Furthermore, if the budget is approaching exhaustion, we can promptly take appropriate action.

Outcome:

Now, the client can make informed decisions about budget allocation and optimize resource usage on the GCP platform. With real-time alerts and monthly spending reports, the client has gained better financial visibility into the project. This proactive approach has not only prevented budget overruns but also enabled the client to allocate resources more efficiently, ensuring that the project stays within financial constraints.

Summary/Conclusion

Efficiently monitor your Google Cloud Platform (GCP) budget through Slack alerts. By setting up a micro-service that tracks and monitors your GCP spending, you can receive alerts when your budget surpasses predefined thresholds. 

Configure budget alerts for billing accounts and projects, customize the alerts based on spending limits or previous month’s expenditures and receive notifications via email or Cloud Pub/Sub events. 

The micro-service architecture involves using Cloud Pub/Sub to trigger events, and a Cloud Function to handle the events and send notifications to Slack. Set up the codebase by creating necessary files and dependencies, and deploy the code to the Cloud Function. 

This GCP Alert service improves cost management and financial control, allowing you to take immediate action and ensure efficient budget management for your GCP projects.