Orchestrating Spotify With K8S
Spotify’s History
Spotify Technology S.A a Swedish audio streaming and media services provider , offering digital copyright restricted recorded music and podcasts, including more than 70 million songs, from record labels and media companies.
Unlike physical or download sales, which pay artists a fixed price per song or album sold, Spotify pays royalties based on the number of artist streams as a proportion of total songs streamed. It distributes approximately 70% of its total revenue to rights holders (often record labels), who then pay artists based on individual agreements.
Challenges
Launched in 2008, the audio-streaming platform has grown to over 200 million monthly active users across the world.
An early adopter of microservices and Docker, Spotify had containerized microservices running across its fleet of VMs with a homegrown container orchestration system called Helios.
By late 2017, it became clear that having a small team working on the features was just not as efficient as adopting something that was supported by a much bigger community.
Kubernetes : The Savior
What is Kubernetes
Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management.
It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. It aims to provide a “platform for automating deployment, scaling, and operations of application containers across clusters of hosts”.
Many cloud services offer a Kubernetes-based platform or infrastructure as a service (PaaS or IaaS) on which Kubernetes can be deployed as a platform-providing service. Many vendors also provide their own branded Kubernetes distributions.
How does it solve Spotify’s needs ??
The first point being Kubernetes was more feature-rich than Helios.
“We wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools”
as said by Jai Chakrabarti, Director of Engineering, Infrastructure and Operations at Spotify.
At the same time, the team wanted to contribute its expertise and influence in the flourishing Kubernetes community.
Impact
The biggest service currently running on Kubernetes takes about 10 million requests per second as an aggregate service and benefits greatly from auto-scaling, says Site Reliability Engineer James Wen. Plus, he adds, “Before, teams would have to wait for an hour to create a new service and get an operational host to run it in production, but with Kubernetes, they can do that on the order of seconds and minutes.
Conclusion
“We were able to use a lot of the Kubernetes APIs and extensibility features to support and interface with our legacy infrastructure, so the integration was straightforward and easy.”
JAMES WEN, SITE RELIABILITY ENGINEER, SPOTIFY
Thank You for reading
Let’s connect on LinkedIn->
