The Data Source #6 | Choose Your Own Environment ⚒
Welcome to The Data Source, your monthly(ish) newsletter covering the top innovation in data infrastructure, analytics and developer-first tooling.
Subscribe now and never miss an issue 👇🏼
Developer Efficiency: Heroku + Kubernetes + Infrastructure as Code
I’ve been thinking a lot about developer effectiveness as it relates to DevOps teams and believe there’s an opportunity for new tooling to enable developer self-service and relieve operational burden on DevOps, which can ultimately speed up software development and delivery. More specifically, I’ve been mulling over the next evolution of Platform as a Service (PaaS).
What’s interesting about Heroku and Platform as a Service (PaaS) for that matter is that the core idea behind the tool and one of the reasons why it is such a critical part of the software development tech stack is that it optimizes DevOps effectiveness. Heroku’s managed infrastructure made it simple and easy for developers to build, run and derive immediate value from their applications without having to manage the underlying infrastructure themselves. And, it does so by providing the right level of opinionation that dictates just how a developer is to architect their applications.
But one of its limitations is that when it comes to resource usage, Heroku can get pretty expensive quickly. In fact, cost is one of the main reasons why developers typically migrate off to a more robust development platform like AWS and utilize container orchestration platforms like Kubernetes for handling resource heavy and large scale workloads.
Most large organizations today are using Kubernetes as the de facto platform for architecting, configuring and deploying cloud-native applications. But running Kubernetes in prod still remains a hassle as it requires significant resources, time and skills. In fact, many organizations have been hiring dedicated DevOps teams and in-house Kubernetes experts to help operationalize their clusters running in prod.
Among the problems that developers tend to run into when running Kubernetes in prod is having to create and maintain numerous environments for adequate testing and debugging before releasing software. Pre-production and staging environments are critical for software development because they enable developers to test their code under a prod-like environment before deploying it. It’s a way to reduce the likelihood of any production outage risk and yet many developers don’t have access to a shareable environment that they can all collaborate on with one another. Despite the advent of Kubernetes, replicating production data and generating bespoke on-demand environments is hard where most developers don’t have a consistent and collaborative workflow for releasing software. What ends up happening is that developers are working off a single environment, siloed from other developers. And as everyone pushes their code to production. they queue up, sitting idling while waiting for their code to make their way to prod.
This is the current state of software releases today and from what I’m observing, there are quite a few players in the market currently that are looking to disrupt the market with their “Environments as a Service” offering to effectively tackle the challenges I articulate above and improve DevOps effectiveness.
Environments as a Service: DevOps efficiency powered by a consistent developer workflow for building and releasing applications across any platform
* A Work-Bench portfolio company
We’ve seen the emergence of Infrastructure as Code (IaC) as technology that enables developers to provision and manage infrastructure by writing code. What used to require a ton of manual work can now be executed in just a few lines of code. And, I believe the same concept can be applied to modern software development where the provisioning of environments will be automated through software.
Solutions that have emerged in the market recently are focused on making it as quick and easy as possible for teams to roll out their own infrastructures without having to deal with the complexities of manually setting it up themselves. These are essentially application productivity solutions that enable the provisioning of purpose-built ephemeral environments used for deployment, testing, staging, production, demos, and more, while delivering seamless development and delivery experiences. Besides enabling the creation / tearing down of infrastructure, these solutions also address a host of infrastructure-related issues around costs, compliance and security.
One aspect which seems to be a critical component of the space moving forward are prescriptive and opinionated workflows as they provide developers with a guided experience and framework through which they can produce high quality software very quickly and with confidence. I expect the Environments as a Service category to continue to evolve and I’m excited for what this means for the future of software delivery and engineering productivity.
Founders and DevOps practitioners, if this is a space you’re building in or are passionate about, I’d love to swap notes with you. As always, my Twitter DM and 📩 (email@example.com) are open!
Work-Bench is an early stage enterprise software-focused VC firm based in NYC with our sweet spot for investment being at the Seed II stage, which correlates with building out a startup’s early go-to-market motions. In the data world, we’ve been fortunate to invest in companies like Cockroach Labs, Arthur, Algorithmia, Datalogue, Alkymi and others.