Aurora and SkySQL are MySQL compatible cloud database services. This article compares the performance of the two systems using Reserva, a custom benchmarking tool I created to simulate a high volume digital payments system.
SkySQL’s performance advantage over Aurora is significant. In my tests, it was 30% faster while being 24% cheaper. You may want to evaluate whether those performance and cost advantages are worth giving up some of the nice features that Aurora provides.
Aurora is developed by Amazon and is only available on AWS. SkySQL is multi-cloud. A helpful SkySQL employee gave the following explanation for how the platform is affiliated with the MariaDB project:
SkySQL was developed under MariaDB corporation, but was spun-off in late 2023 to be a separate and distinct company from MariaDB PLC… though we maintain a business relationship with them and MariaDB Foundation which shepherds the open source project.
In short, SkySQL is developed by people who have worked on MariaDB for quite a while, but are now operating as a spin-off of MariaDB PLC.
Reserva
Reserva is a system I made to simulate a streamlined, high volume payments processing system. It is written in Go and supports arbitrary amounts of concurrency.
On start, Reserva loads all users and authentication tokens into memory, then attempts to make as many funds transfers as possible in a loop. Each funds transfer requires 4 round trips to the database and touches 6 tables. The workflow is as follows:
- DB is queried to authenticate and authorize the user who is requesting payment.
- DB is queried to get information about the card used and account to be debited.
- DB is queried to authenticate and authorize the user who will approve or deny the payment request.
- DB is queried to update both account balances and record the payment.
Reserva allows you to select whether or not you want deletes to be part of the workload. The results on this page do include deletes as part of the workload.
Benchmark setup
First, I analyzed the options for creating Aurora and SkySQL instances in each platform. I settled on two machines that should provide roughly equal raw-metal performance. Below is a description of each machine’s key configuration parameters.
SkySQL configuration
I deployed SkySQL in AWS’ London region with the following settings:
- 2 vCPUs
- 8 GB of RAM
- Single zone
- MariaDB version 11.4.3-1-1
Below is a screenshot of SkySQL’s UI during the instance creation process:
Aurora configuration
I deployed Aurora in AWS’ London region with the following settings:
- db.t4g.large instance
- 2 vCPUs
- 8 GB RAM
- No Aurora replica
- Aurora version 3.07.1
- MySQL 8.0 compatible
- Aurora I/O-Optimized disk
Below is a screenshot of AWS’ UI after setup:
Cost comparison
SkySQL will cost around $0.19 per hour, and Aurora will cost around $0.25 per hour. This makes SkySQL 24% cheaper on an hourly basis for similar hardware.
SkySQL’s cost estimate is shown right on the instance creation page and includes storage costs. Aurora also shows a cost estimate on the page, but comes with the following disclaimer:
Estimate does not consider reserved instance benefits and costs for instance storage, lOs, or data transfer.
I wish they would include some sane default value for storage costs in that estimate, but I guess the amount can vary so widely that it wouldn’t be useful. Owell.
SkySQL cost estimate
SkySQL charges $0.18 per hour for the CPU and RAM. They also charge $0.01277 for a 100GB disk.
Here is a screenshot of the cost estimate provided on SkySQL UI:
Aurora cost estimate
I decided to use the AWS cost calculator to get an estimate of how much the instance I created would cost, since the creation UI wouldn’t tell me the final cost after storage.
Here is a screenshot of the cost estimate provided on AWS’ cost calculator:
Populating data
Next we run the DB preparation SQL scripts to create:
- 10 banks with access to the payments system and corresponding users + permissions + auth tokens
- 1,000,000 accounts, each with a corresponding payment card
You can see the SQL setup scripts, along with the rest of the code, on Reserva’s Github page.
The test
I ran Reserva with the following settings for both databases:
- 32 concurrent requests at a time
- Maximum of 32 open database connections (idle or active)
- 2 hour duration
- Deletes enabled as part of the workflow
At the end of the test, I executed the following query to show how many payments were made over the course of the hour. The results of those queries are visualized below.
Aggregation query
SELECT
hour(CREATED_AT) AS hour_created_at,
minute(CREATED_AT) AS minute_created_at,
COUNT(*) AS NUMBER_OF_TRANSFERS
FROM
transfers
GROUP BY
hour_created_at, minute_created_at
ORDER BY
hour_created_at, minute_created_at;Results
As you can see, SkySQL was significantly faster overall. It had occasional dips that are probably due to checkpointing, but in my opinion, it’s unlikely that these dips would affect real-world use cases. Real-world applications rarely pin a DB server’s CPU usage at 100% for extended periods of time, so checkpointing wouldn’t be competing for resources as intensely as it is here.
It is worth noting that because deletes were enabled in this run, 1 out of every 20 funds transfer loops also deletes a random transfer. So, the actual amount of fund transfers processed for each database is 5% higher. For comparison purposes, the results are still valid.
I also aggregated the last hour of the test in a spreadsheet to create a simple “transfers per minute” bar chart, shown below. SkySQL is 30% faster in the time frame that I selected.
Conclusion
SkySQL is 30% faster and 24% cheaper than AWS Aurora in this particular use case. I plan on doing a full review of the platform soon, as it also offers some nice high availability and replication features.
