Manage Your Toolchain Before It Manages You

Executive Summary

As the sprawl of applications fueling businesses continues to grow, so do the toolchains and tools that support their development and operational performance. Application development and operations teams are left to contend with this complexity every day, struggling to see across tools and toolchains, ensure security across tools, and improve developer productivity. Out-of-the-box toolchain management systems are seen as the key to simplifying the complex world of toolchains, combining both tactical improvements around security, revenue, and quality, with strategic improvements around more easily implementing multicloud environments.

In March 2019, GitLab commissioned Forrester Consulting to evaluate current toolchain management practices at enterprise organizations. Forrester conducted an online survey with 252 IT professionals across the US, UK, France, and Germany to explore this topic. We found that many software development teams are overrun with tools and toolchains and struggle to maintain discrete toolchains. Out-of-the-box toolchain management solutions are seen as a potential solution to managing this complexity.

Key Findings

67% of respondents agree that handoffs between teams using different tools slows down delivery.

Multiple toolchains and numerous tools are used across the software delivery life cycle (SDLC).

Most businesses have two or more toolchains powering their SDLC, and, for the majority, each toolchain is comprised of six or more tools.
This complexity creates visibility, security, and productivity challenges for development and operations teams alike.

Toolchain managers struggle to contain the tool sprawl while at the same time, software delivery release cycles have not improved, creating more pressure for development teams to find ways to speed delivery.
Out-of-the-box toolchain management solutions are seen as a solution to the sprawl.

In fact, improved security, increased revenue, and improved quality were the top-seen benefits from firms that have already implemented out-of-the-box toolchain management systems.

Instant Poll

See How Your Company Compares

"Which of the following best describes the degree of integration your software delivery toolchain has today?"

Select one:

We have an out-of-the-box software delivery solution that is integrated end to end.
Our toolchain is integrated with a combination of plug-ins and scripts.
Our toolchain is integrated via well-defined APIs.
Our toolchain is integrated manually, via hard-coded custom integrations between tools.
Our toolchain is not integrated.

DID YOU KNOW?

Over half of toolchains are integrated with a combination of plug-ins, scripts, or manual hard-coded integrations between tools, a process that is not only time-consuming, but rife with potential errors.

Most toolchains are integrated in labor-intensive ways.

Beyond simply managing the vast scale of tools in a modern software delivery organization’s toolchain, most teams are left to integrate toolchains by manual means. Over a third of toolchains are integrated with a combination of plug-ins and scripts; one in five toolchains are integrated via manual, hard-coded custom integrations, a process that is not only time-consuming but rife with potential errors.

Visibility, Security, And Productivity Are Key Challenges In Supporting Multiple Toolchains And Tools

46% of respondents struggle with insufficient skills, expertise, or resources to integrate discrete tools; 44% struggle with insufficient skills, expertise, or resources to maintain discrete tools and tool integrations.

The complexity of this landscape — many discrete tools across a range of toolchains — results in significant challenges for application development and operations professionals alike. Not only do toolchain managers struggle to contain the tool sprawl, but software delivery release cycles have not improved, creating more pressure to find ways to speed delivery. Sixty-seven percent of respondents agreed that handoffs between teams using different tools slows down delivery. The key challenges are: visibility across toolchains; maintaining security across tools; and ensuring that IT resources are available to maintain toolchains.

IT professionals noted that a lack of visibility into the process of maintaining the toolchain is a key challenge.

Nearly 40% of respondents noted that this lack of visibility is a top process challenge they currently face (see Figure 4). In fact, this was the top-noted challenge for application development professionals (49%). These custom toolchains are creating unhappy surprises for teams in the form of complexity, maintenance, and support, all which rob teams of resources that could otherwise be applied to product development. In fact, the complexity of tools and toolchains creates significant governance challenges for software teams as well — 71% agree that governance and end-to-end visibility of software delivery are major challenges. Meaning, they are simply only able to do the basics of continuous integration/ continuous delivery and have little time for further innovation.
Ensuring security across the toolchain is a significant challenge for IT professionals as well.

Almost half of all respondents (45%) noted that maintaining security across the toolchain is a key challenge; each tool has its own diverse set of requirements which creates significant challenges for IT professionals to not only develop but also maintain. Forty-six percent of respondents agreed that they spend too much time and money integrating and maintaining this diverse security landscape for each tool.
Visibility and security challenges create additional strain on IT resources.

Businesses struggle to retain resources that can properly integrate and maintain tools and tool integration. Forty-six percent struggle with insufficient skills, expertise, or resources to integrate discrete tools; 44% struggle with insufficient skills, expertise, or resources to maintain discrete tools and tool integrations. Because tool chain integration is a manual process at the majority of organizations, IT professionals are pulled into the rote work that could otherwise be automated, creating not only job satisfaction issues but also retention challenges.

Figure 4: Process Challenges Faced By Teams

Base: 252 IT professionals with responsibility over tool chain management
Source: A commissioned study conducted by Forrester Consulting on behalf of GitLab, March 2019

Combining Tactical Solutions With Strategic Benefits, Out-Of-The-Box Solutions Can Drive Businesses Forward

Out-of-the-box toolchain management systems are seen as the key to simplifying the complex world of toolchains. In fact, improved security, increased revenue, and improved quality were the top-seen benefits from firms that have already implemented out-of- the-box toolchain management systems. Coupling these tactical improvements with the strategic imperative that most companies face in implementing multicloud environments, the benefits of an out-of-the-box toolchain management solution are clear.

Improved quality, security, and productivity are the top- anticipated benefits of an out-of-the-box solution.

IT professionals are eager to embrace an out-of-the-box solution that will allow them to not only operate at higher levels than before but to also free up their IT resources for higher-level work. Interestingly, the top-anticipated benefit for application development professionals was improved security, while operations professionals were most eager about the potential for improved quality (see Figure 6).

42% of organizations are currently using an out-of-the-box toolchain management system.

Figure 6: Anticipated And/Or Realized Benefits Of An Out-Of-The-Box Toolchain Management System

Click to see data by region

All
France
Germany
United Kingdom
United States

Base: 243 IT professionals with responsibility over toolchain management
Source: A commissioned study conducted by Forrester Consulting on behalf of GitLab, March 2019

Interestingly, a top benefit experience by those who have already adopted an out-of-the-box solution was increased revenue.

The second most stated benefit of out-of-the-box solution adoptees was increased revenue (46%), proving that improved security, quality, and productivity can all drive bottom-line growth at the same time (see Figure 7).
Crucially, investing in an out-of-the-box toolchain management system is a strategic endeavor as well.

Beyond tactical improvements around quality, security, and productivity, the benefit that was noted as having the greatest impact on businesses was the ability to deploy to any target environment/cloud. This flexibility is especially important as 77% of IT professionals agree that their organizations are moving to the cloud and they want to avoid cloud lock-in. With that in mind, leveraging a tool that is cloud-agnostic will provide the highest level of cloud independence and leveraging one that has that capability, out of the box, will accelerate a team’s ability toward achieving a multicloud strategy.

Figure 7: Benefits Realized By Those Who Have Already Adopted An Out-Of-The-Box Toolchain Management System

Base: 106 IT professionals with responsibility over toolchain management with an out-of-the-box toolchain management system
Source: A commissioned study conducted by Forrester Consulting on behalf of GitLab, March 2019

Key Recommendations

DevOps has been in existence for over a decade and during that time the number and variety of tools have exploded out into many directions, from automating delivery to automating security scanning, and it’s not slowing down. Yet as our survey shows, this expansion of tools has come at a cost in terms of complexity with maintaining each tool and complexity across the toolchain in order to maintain integrations of the growing number of tools. Further compounding this issue is that many teams have four or more toolchains to maintain. All of this complexity robs resources from the main purpose of DevOps, to help software teams accelerate the delivery of innovation to the marketplace in order to achieve business success.

Forrester’s in-depth survey of IT leaders about application delivery yielded several important recommendations:

Reduce your resource expenditure on toolchain maintenance.

Maintaining toolchains is necessary, but it shouldn’t consume whole teams, instead find an out-of-the-box solution that offers a backbone of capability that your team can build upon, rather than building one from scratch. Moving to a system with less complexity, yet no less capability, can free teams from managing their myriad sets of tools. An out-of-the-box toolchain that provides security, productivity, and fewer context jumps gives time back to engineering teams so that they can re-invest their time into working on innovation and customer success.
Improve security with a single DevOps solution.

Security is the No. 1 process challenge faced by IT pros. Toolchains that rely on credentials stored in scripts and multiple logins often have shortcuts added to reduce process time friction. However, those shortcuts create major security vulnerabilities and IT pros know this. Selecting a toolchain that is a complete solution allows identity and authentication to be managed in a uniform way across each process step, simplifying and strengthening toolchain security. It eliminates stored credentials and simplifies developer and IT interaction with the toolchain, making shortcuts and hacks no longer necessary.
Visibility into the toolchain is equally important.

Forrester’s report on value stream management highlights the needs for application development and delivery professionals to gain insight into the processes their toolchains are automating, in terms of cycle time, flow, and backlog, in order to draw insight as to where they can further optimize software delivery and quality.¹ Teams that have this type of visibility can better understand and prioritize improvements that are necessary to further automate their toolchain. This visibility also allows them to track their performance over time and can help provide better estimates moving forward.
Enable a multicloud strategy with a cloud-agnostic solution.

IT pros indicate that multicloud is their strategic direction. Therefore, selecting a toolchain that enables deployment to any cloud environment must be part of that strategic plan. While selecting an out-of-the-box toolchain isn’t the only way to enable this capability, it’s the quickest because it gives the development team a single tool to manage, build, test, and then deploy, all as a single motion rather than different tools for different environments.

Appendix A: Methodology

In this study, Forrester conducted a cross-industry online survey of 252 IT professionals in the US, the UK, France, and Germany. Survey participants included decision makers in IT operations, application development, enterprise architecture, PMO, CIO, QA/testing, and security roles at companies with more than 1,000 employees. Respondents were offered a small incentive as a thank you for time spent on the survey. The study was completed in March 2019.

Appendix B: Demographics

Appendix C: Endnotes

¹ Source: “Elevate Agile-Plus-DevOps With Value Stream Management,” Forrester Research, Inc., May 11, 2018.

Executive Summary

Table Of Contents

Key Findings

Multiple toolchains and numerous tools are used across the software delivery life cycle (SDLC).

This complexity creates visibility, security, and productivity challenges for development and operations teams alike.

Out-of-the-box toolchain management solutions are seen as a solution to the sprawl.

Software Organizations Currently Use Multiple Tools, Multiple Toolchains

The vast majority of software organizations use at least two discrete software delivery toolchains.

Figure 1: Number Of Software Delivery Toolchains Maintained By Software Organizations

Figure 2: Number Of Tools Comprising Each Toolchain

This complexity begets more complexity.

Despite this complexity, the majority of organizations rely on application development professionals to maintain these toolchains.

Figure 3: Teams Responsible For Maintaining Toolchains

Instant Poll

See How Your Company Compares

"Which of the following best describes the degree of integration your software delivery toolchain has today?"

DID YOU KNOW?

Most toolchains are integrated in labor-intensive ways.

Visibility, Security, And Productivity Are Key Challenges In Supporting Multiple Toolchains And Tools

46% of respondents struggle with insufficient skills, expertise, or resources to integrate discrete tools; 44% struggle with insufficient skills, expertise, or resources to maintain discrete tools and tool integrations.

IT professionals noted that a lack of visibility into the process of maintaining the toolchain is a key challenge.

Ensuring security across the toolchain is a significant challenge for IT professionals as well.

Visibility and security challenges create additional strain on IT resources.

Figure 4: Process Challenges Faced By Teams

Combining Tactical Solutions With Strategic Benefits, Out-Of-The-Box Solutions Can Drive Businesses Forward

Improved quality, security, and productivity are the top- anticipated benefits of an out-of-the-box solution.

42% of organizations are currently using an out-of-the-box toolchain management system.

Figure 6: Anticipated And/Or Realized Benefits Of An Out-Of-The-Box Toolchain Management System

Interestingly, a top benefit experience by those who have already adopted an out-of-the-box solution was increased revenue.

Crucially, investing in an out-of-the-box toolchain management system is a strategic endeavor as well.

Figure 7: Benefits Realized By Those Who Have Already Adopted An Out-Of-The-Box Toolchain Management System

Key Recommendations

Reduce your resource expenditure on toolchain maintenance.

Improve security with a single DevOps solution.

Visibility into the toolchain is equally important.

Enable a multicloud strategy with a cloud-agnostic solution.

Appendix A: Methodology

Appendix B: Demographics

Appendix C: Endnotes

About Forrester Consulting