Reproduction steps now available for the 2018 Cloud Report

Reproduction steps now available for the 2018 Cloud Report

[THE 2021 CLOUD REPORT IS AVAILABLE. READ IT HERE]

CockroachDB is a cloud-neutral database, which means it eliminates dependencies on a particular cloud environment and gives you the flexibility and choice to run it anywhere you like. We are committed to this principle and in order to deliver on this promise, we systematically deploy and test CockroachDB clusters on the three leading US cloud providers: Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure.

While executing these tests as part of our 2.1 release a few months back, we noticed something curious: the throughput on AWS test clusters was 40% higher than GCP test clusters.

This surprising initial finding led to further internal investigation and experiments on cloud provider performance. We shared those findings in what became the 2018 Cloud Report.

Much like our product, the 2018 Cloud Report has taken additional shape and direction from the incredible input from cloud providers (GCP & AWS), the broad open source community and from our Cockroach community. We had a lot of great conversations and with this input, we made updates to the report to reflect some of the most common questions and concerns raised within the community. A huge thank you to everyone who has contributed to the conversation, and who is helping us improve our benchmark.

Today, we're releasing an update to the 2018 Cloud Report based on some of the conversations we've had with you.

One of the most common questions/requests was for more information on how we configured our experiments to get these numbers. While we used open source benchmarks to test CPU, network, and I/O, there are other factors that impact analyses like this--many of which are difficult to isolate and control. Getting to a similar configuration (for a benchmark) between cloud vendors isn't quite as trivial as it sounds.

Today's updated report includes the reproduction steps to the 2018 Cloud Report findings. These steps allow for copy-paste reproduction of our process to empower you to check our work.

Have additional questions? We would love for you to join our webinar on February 14 where we'll walk through why we decided to benchmark, what we learned from the exercise, and then outline the gotchas we encountered along the way. Reserve your spot now.

CockroachDB remains committed to our stance as a cloud-neutral database. We'll continue to benchmark GCP and AWS for internal stability and performance testing, but since we expect these results to change as companies continue to invest in future infrastructure, our benchmarks will continue to evolve. In future editions, we will expand our test efforts to Microsoft Azure, Digital Ocean, and other cloud platforms.

[THE 2020 CLOUD REPORT IS AVAILABLE. READ IT HERE]

Nathan VanBenschoten github link linkedin link

Nathan VanBenschoten has been working on CockroachDB for the past 7 years. His focus has been on the performance of CockroachDB’s transaction, replication, and persistence layers. Over the past 2 years, he collaborated with a small team focusing on global deployments of CockroachDB. Before Cockroach Labs, he was an intern on the Google-Wide Profiling team, working on Google’s continuous profiling infrastructure. He holds a bachelor’s degree in electrical and computer engineering from Northeastern University.

Keep Reading

AWS outperforms GCP in the 2018 Cloud Report

[THE 2021 CLOUD REPORT IS AVAILABLE. READ IT HERE]

Note: As of December 20, 2018, we have updated two sections in …

Read more
CockroachDB 2.1 is now 50x more scalable than Amazon Aurora

[For CockroachDB's most up-to-date performance benchmarks, please read our Performance Overview page] …

Read more
CockroachDB 2.0 has arrived!

CockroachDB debuted as the open source database that made it possible to build massive, reliable cloud …

Read more