10
.
06
.
2024
10
.
06
.
2024
Backend
Ruby on Rails

Optimizing Cloud Infrastructure by $40 000 Annually

Cezary Kłos
Ruby Developer
Cezary Kłos - Optimizing Cloud Infrastructure by $40 000 Annually

Recently, a client approached us with a request to migrate their product application's infrastructure from their current cloud provider to Azure. The primary goal was to unify their cloud environments onto a single platform.

We took this opportunity to take a closer look at the existing architecture. It is not uncommon to see an over-provisioned architecture in the wild. There are many possible reasons for that. For example:

  • Improper assumptions about the application’s scale
  • Adding extra servers to accommodate dropped requests
  • Expanding memory to cover memory leaks
  • An inexperienced DevOps team
  • Lack of maintenance procedures

Often, these changes are meant to be temporary, but are forgotten after they outlived their purpose. Regardless of origin, the over-provisioned architecture means that you pay for something that you don’t use. In scale of a year it adds up.

In this case, we were able to save our client nearly $40 000 annually on cloud hosting.

Initial Assessment

We began by defining and evaluating the existing architecture, which included:

  • Two virtual machines and a load balancer
  • A dedicated bare-metal machine for the Postgres database
  • An additional virtual machine for a staging environment
  • CircleCI for running code linter and tests

The former architecture was significantly over-scaled for the project, resulting in unnecessarily high operational costs. This over-scaling stemmed from outdated assumptions that had not been re-evaluated. For instance, the database machine had 3 TB of storage, while the database's uncompressed dump was under one gigabyte. Given that the client's average throughput was just over 100 transactions per hour, it became evident that the existing infrastructure far exceeded operational needs.

Proposed New Architecture

Our next step was to design a new architecture tailored to the client's actual needs. The proposed streamlined architecture included:

  • A single virtual machine using containerized services for the application, background jobs, Postgres database, and Redis database
  • A second virtual machine mirroring the first for the staging environment
  • Retention of CircleCI

After presenting the proposed architecture and the estimated new operational costs to the client, we received approval to proceed with the migration. Our primary considerations were to prevent downtime and ensure data integrity.

Migration Strategy

Two main focus points guided our migration strategy:

  1. Data Migration Between Postgres Instances: We opted for the traditional dump and restore method due to the relatively small amount of data and because neither the old nor the new Postgres database faced the internet directly. This ruled out a replication-based strategy.
  2. DNS Changes Propagation Time and Data Integrity: DNS changes can take up to 48 hours to propagate. During this time, users could be routed to either the new or old application instance, each accessing different databases. It was crucial to prevent the old instance from accepting orders and saving them in the old database.

To address these concerns, the final migration procedure included additional steps and the use of a temporary subdomain to redirect traffic to the new application instance. After thoroughly testing the migration procedure in a staging environment, we scheduled a migration window with the client and executed the plan. Thanks to proper planning and testing, the process went smoothly.

Conclusion

We successfully completed the client's migration task, and as a result, the monthly operational costs for running the application were slashed from $3700 to $600. This experience serves as a reminder that pre-emptive scaling is not always necessary. Regularly verifying resource utilization and making adjustments can lead to significant cost savings. If you need assistance with optimizing your infrastructure, please let us know!

Cezary Kłos
Ruby Developer

Check my Twitter

Check my Linkedin

Did you like it? 

Sign up To VIsuality newsletter

READ ALSO

Recap of Friendly.rb 2024 conference

Insights and Inspiration from Friendly.rb: A Ruby Conference Recap

02
.
10
.
2024
Kaja Witek
Conferences
Ruby on Rails

Covering indexes - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Ruby on Rails
Postgresql
Backend
Ula Sołogub - SQL Injection in Ruby on Rails

The Deadly Sins in RoR security - SQL Injection

14
.
11
.
2023
Urszula Sołogub
Backend
Ruby on Rails
Software
Michal - Highlights from Ruby Unconf 2024

Highlights from Ruby Unconf 2024

14
.
11
.
2023
Michał Łęcicki
Conferences
Visuality
Cezary Kłos - Optimizing Cloud Infrastructure by $40 000 Annually

Optimizing Cloud Infrastructure by $40 000 Annually

14
.
11
.
2023
Cezary Kłos
Backend
Ruby on Rails

Smooth Concurrent Updates with Hotwire Stimulus

14
.
11
.
2023
Michał Łęcicki
Hotwire
Ruby on Rails
Software
Tutorial

Freelancers vs Software house

02
.
10
.
2024
Michał Krochecki
Visuality
Business

Table partitioning in Rails, part 2 - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Backend
Postgresql
Ruby on Rails

N+1 in Ruby on Rails

14
.
11
.
2023
Katarzyna Melon-Markowska
Ruby on Rails
Ruby
Backend

Turbo Streams and current user

29
.
11
.
2023
Mateusz Bilski
Hotwire
Ruby on Rails
Backend
Frontend

Showing progress of background jobs with Turbo

14
.
11
.
2023
Michał Łęcicki
Ruby on Rails
Ruby
Hotwire
Frontend
Backend

Table partitioning in Rails, part 1 - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Postgresql
Backend
Ruby on Rails

Table partitioning types - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Postgresql
Backend

Indexing partitioned table - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Backend
Postgresql
SQL Views in Ruby on Rails

SQL views in Ruby on Rails

14
.
11
.
2023
Jan Grela
Backend
Ruby
Ruby on Rails
Postgresql
Design your bathroom in React

Design your bathroom in React

14
.
11
.
2023
Bartosz Bazański
Frontend
React
Lazy Attributes in Ruby - Krzysztof Wawer

Lazy attributes in Ruby

14
.
11
.
2023
Krzysztof Wawer
Ruby
Software

Exporting CSV files using COPY - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Postgresql
Ruby
Ruby on Rails
Michał Łęcicki - From Celluloid to Concurrent Ruby

From Celluloid to Concurrent Ruby: Practical Examples Of Multithreading Calls

14
.
11
.
2023
Michał Łęcicki
Backend
Ruby
Ruby on Rails
Software

Super Slide Me - Game Written in React

14
.
11
.
2023
Antoni Smoliński
Frontend
React
Jarek Kowalewski - ILIKE vs LIKE/LOWER - Postgres Stories

ILIKE vs LIKE/LOWER - Postgres Stories

14
.
11
.
2023
Jarosław Kowalewski
Ruby
Ruby on Rails
Postgresql

A look back at Friendly.rb 2023

14
.
11
.
2023
Cezary Kłos
Conferences
Ruby

Debugging Rails - Ruby Junior Chronicles

14
.
11
.
2023
Piotr Witek
Ruby on Rails
Backend
Tutorial