Cory O’Daniel on Infrastructure as Code
Cloud adoption has outpaced the development of corresponding expertise in cloud operations, leading to a significant skills gap across the industry. In this regard, many organizations have had to abandon cloud initiatives because they lacked the necessary in-house expertise to manage them. As a result, critical projects are delayed, and costs and operational risks increase. Recognizing this gap, Cory O’Daniel co-founded MassDriver, a platform designed to simplify cloud infrastructure management for engineers without extensive cloud or operations experience. In this episode, Cory will explain how MassDriver allows teams to focus on building business value rather than getting bogged down in intricate cloud configurations.
Edited transcription
After working at small and large software development companies, Cory O’Daniel found a widespread lack of knowledge about cloud infrastructure. Looking to fill this gap, he co-founded MassDriver, a platform as a service (PaaS) for cloud DevOps.
As Cory explains, MassDriver came to be from his effort to “make it easier for engineers that may not have this traditional cloud or operations experience to manage the cloud without getting inundated in the eight pages of docs about how to operate RDS, […] how to write Terraform, and how to build your CI pipelines.” In this way, the platform aims to relieve engineers from the burden of setting up their development environments and let them focus on building business value.
What MassDriver does
According to Cory, all software companies, whether they realize it or not, have a platform team. Even in cases where the platform is outsourced, he observes that companies are increasingly viewing the platform as an internal product, rather than “something this other team is doing in service for you.” The key distinction, Cory understands, lies in whether they are “consciously building a platform or not.” Considering this platform as the foundation for their development and deployment, Cory understands conscious platform building begins “as soon as you start getting to the cloud, starting to think, ‘how can I make these reproducible, reusable components that my software sits on top of so it makes it easy for me to make new environments or change things out?’ “
Targeting a user base of developers with no operational experience, MassDriver offers a creative system for building and managing complex cloud platforms. Under the hood, MassDriver follows the practice of infrastructure as code (IaC) for managing and provisioning infrastructure by describing it through code. However, MassDriver’s interface allows users to set up infrastructure using diagrams. As Cory explains, “There’s a number of infrastructure-as-code modules that are published into the platform; You can publish your own as well.” Companies can diagram the infrastructure they need and MassDriver will “grab infrastructure as code modules and policy as code modules from the registry and start to provision using that.”
Drawing infrastructure makes it easy to spot the components of cloud environments and understand how they work together. “When you’re in MassDriver”, Cory explains, “you just drag your application out and it does the Terraform for you, does the role assignments, binds it to your Kubernetes workload.”
In this way, MassDriver balances developer flexibility with robust governance. It empowers developers to make architectural changes while abstracting away granular control over credentials, module selection, and tool inputs. MassDriver handles these underlying details, making it ideal for users who have a defined set of architectural requirements but lack the specific skills to implement them.
To achieve this flexibility and extensibility, MassDriver is built on a functional programming model with types, using a type system as its extension mechanism. As Cory explains, “If you’re using, let’s say Mongo Atlas, we don’t have a type for Mongo Atlas in MassDriver by default, but you can push new types into your registry.” Infrastructure code is encapsulated within typed “bundles.” Each bundle defines input types (configuration parameters), output types (results/artifacts), and the underlying infrastructure code, which is treated as a black box. Consequently, MassDriver itself “has no idea what it’s doing”; it simply understands types and their relationships.
Multi-layered testing strategy
Given the complexity of infrastructure deployments, MassDriver employs a multi-layered testing strategy. This strategy addresses the limitations Cory identifies in traditional testing approaches within the DevOps community. The DevOps community needs to be “a bit more test minded,” he explains; “One of the things I think is really missing from our community and the DevOps community is being a bit more test minded.”
Integration testing
Cory points out that tools like Terraform test often focus on unit testing individual resources, which doesn’t adequately capture the interconnected nature of real-world infrastructure. Testing a single component in isolation, such as a PostgreSQL database, fails to account for crucial dependencies like the VPC and other necessary configurations.
Therefore, MassDriver emphasizes integration testing, which tests different components together to ensure their cohesive functionality within a system—a more accurate reflection of production environments. As Cory suggests, Terraform test is better suited for validating code correctness rather than testing the interactions between different infrastructure components.
Black box testing
Since MassDriver treats all components as black boxes, at its core, it tests that inputs conform to the expected types and verifies that the resulting outputs adhere to them and their structures. This simplifies testing by focusing on system behavior rather than internal implementation details.
Provisioning testing
On the other hand, through provisioning testing, MassDriver ensures infrastructure setups function correctly in real environments, validating credential type interactions and compatibility with various published credential types. When infrastructure is provisioned, MassDriver captures the resulting outputs—such as configuration details, connection information, or resource identifiers—as Artifacts. For example, provisioning a database might result in an Artifact containing the connection string, port, and credentials.
Artifacts can then be used by other infrastructure components, creating a modular and reusable system where the output of one module becomes the input for another, effectively managing dependencies. A web application module, for instance, might rely on the database connection details (an Artifact) from a database module. Artifacts also facilitate infrastructure state tracking and versioning, enabling rollback to previous configurations and providing a change history for more robust management. In testing, Artifacts serve as known good outputs to validate the functionality of other components, ensuring that changes in one area don’t negatively impact others.
TerraTest for infrastructure code
To test their infrastructure’s code, MassDrive uses Gruntworks’ TerraTest, a popular testing framework for infrastructure as code. TerraTest automates the deployment and management of cloud resources within the test suite, verifies that provisioned resources meet expected criteria, and applies load and simulates failure conditions to ensure robustness.
Use-case-driven test cases
Because isolated component testing isn’t feasible, MassDriver uses use-case-driven test cases. Each infrastructure module is designed around specific use cases “that we design each module around,” such as deploying a database in a VPC with specific configurations. These use cases define the required inputs and expected outputs, which are then tested to ensure correct module behavior in real-world scenarios.
Additionally, MassDriver Github has example configurations for each module. For one thing, these examples provide users with pre-configured setups to quickly deploy infrastructure without having to understand every detail. However, these are used as well to test cases and validate the infrastructure modules, ensuring these modules work as expected in various configurations.
Dependencies testing
To test dependencies, MassDriver sets up complete environments with all necessary components, ensuring each piece of infrastructure works correctly within its context. Before testing a specific component, MassDriver uses its own platform to automatically provision required downstream dependencies. For example, testing a PostgreSQL database would first involve setting up the necessary VPC, subnets, and security groups. After provisioning dependencies, MassDriver applies a load to test how well the infrastructure handles real-world usage, such as applying load to a production database to test auto-scaling behavior. By provisioning a full stack of dependent services (e.g., a database, VPC, and application server), MassDriver performs integration tests to verify that all components work together correctly and that changes in one area don’t break the overall system.
Performance optimization
Finally, recognizing that base provisioning can significantly lengthen test completion times, MassDriver employs strategies like parallelizing tasks and adjusting configurations to mitigate this. They also use data approximations for certain tests, such as using smaller data sizes for database restores, to expedite complex operations.
License changes in open source
Organizations often begin using open-source software due to its free availability without initial financial commitments. This allows them to experiment, integrate the software into their systems, and assess its suitability before making potential investments or facing licensing restrictions.
In some cases, on the other side, developing the open source software, there are companies looking to eventually profit from it. As Cory puts it, “open source, by and large, is the purchase pathway for many enterprises.” Initially, enterprises often adopt open-source software due to its free availability and ease of initial integration. However, as their reliance on the software grows, they frequently recognize the value of the additional features, support, and stability offered by enterprise or paid versions.
This strategy is frequently employed by companies developing open-source software. The initial goal is to achieve widespread adoption through the free version, establish a user base, and demonstrate the software’s capabilities. Once enterprises become dependent on the software within their critical workflows, these companies can then offer a conversion path to paid, enterprise-grade versions. These paid versions typically include valuable enhancements such as dedicated support, improved scalability to handle larger workloads, enhanced security features, and compliance certifications.
Still, despite these might seem compelling reasons for enterprises to upgrade, many enterprises start by using open-source software for free over an extended period. They integrate it into their workflows, benefit from its functionality, and often delay transitioning to the paid or enterprise version until they are done dealing with the difficulty of managing the open-source version. Eventually, some enterprises decide to pay for the software, either reluctantly because they need additional features, support, or stability, or sometimes willingly when they recognize the value the software brings. However, most of them continue to use the free version.
According to Cory, when “99% of the people that use our thing use it for free, you’re incentivized to change a licensing model.” This pattern of delayed payment creates a situation where open-source software companies rely heavily on converting free users into paying customers. “You’re incentivized to make the open-source version worse,” he concludes. Consequently, companies initially releasing software under an open-source license and later changing that license impair those that have integrated their software into their operations.
As a result, Cory advocates for certain highly prevalent open-source projects to be managed by foundations rather than single organizations. He believes “there’s a point where something becomes so ubiquitous across our industry that it should not be controlled by a single organization.” Placing these projects under the stewardship of a foundation would prevent unilateral license changes and ensure the software remains community-driven, stable, and less susceptible to the whims of a single entity.
Moreover, Cory believes developers should be aware of this situation and ask themselves about the ownership of their tooling: “We should be opening an issue on their GitHub and saying, ‘Why isn’t this a part of the foundation?’ ” In this regard, he also believes there should be more “smaller foundations centered around specific technologies” and with a smaller scope.
DevOps relevance
Cory points out that DevOps teams often “don’t get the same budget as developers and are seen as a burden rather than enablers of business value.” He articulates the difficulty in demonstrating their impact, stating that “it’s hard to see what success we add to the business and the value that we create.”
Cory argues that DevOps teams need more budget to acquire platform tooling earlier, rather than building these tools in-house. This budget constraint forces DevOps teams to often rely on free versions of tools for longer periods before justifying or affording paid versions. Since DevOps is responsible for maintaining and optimizing infrastructure, this delay can hinder their operations. They may spend significant time managing and troubleshooting free tools instead of leveraging more robust, supported enterprise versions that could streamline their workflows. Cory suggests that if DevOps teams had the resources to adopt the right tools earlier, they could deliver greater business value by focusing on higher-level tasks rather than managing basic infrastructure issues.
The bottom line
Follow Cory O’Daniel on X at @coryodaniel and Linkedin to learn the latest on platform engineering. You can also listen to his Platform Engineering Podcast.
To learn more about OpenTufo, visit OpenTofu.org and follow @opentofuorg on X and Linkedin.
The post Cory O’Daniel on Infrastructure as Code appeared first on Semaphore.
Source: View source
