Why we need IDPs

Christina Hupy

Dev relations

August 11, 2023

Share your Social Media

Table of content:

‍

DevOps, the set of practices, tools and cultural philosophy that automates and integrates the processes between software development and IT teams, has completely revolutionized the modern software delivery paradigm over the last 15 years. From its onset in 2007, DevOps has grown to be the predominant method for the software delivery lifecycle. GitLab’s 2023 Global DevSecOps report found that 56% of respondents reported using DevOps or DevSecOps methodologies (source) and market research shows the market value for DevOps is 10.4B USD in 2023 with projected growth to 25.5B in 2028 (source).

‍

While DevOps has profound benefits including faster software delivery, more frequent releases, increased flexibility, greater scalability and availability, better resource utilization and more, it has brought its own set of challenges. As operations shifted left, developers became increasingly burdened with a multitude of tools, managing infrastructure, and participating in workflows that would typically fall to other types of engineers - rather than writing code. In addition to the new job tasks developers face as part of the “You built it, you run it” approach, cloud technology became more prevalent ushering in yet a new era - the cloud native era. With the cloud-native era well underway, developers needed to learn more complex cloud native technologies such as Kubernetes. These new technologies not only added cognitive load, but slowed down the developer process with repetitive manual tasks for cloud resource management. The advancements didn't stop there - shift left security, microservices architecture, infrastructure Code (IaC) and GitOps - all piled on developers and DevOps managers plates. DevOps managers and engineering leaders began to realize that as the demand on developers increased, the benefits of DevOps were being diminished by resourcing bottlenecks, manual repetitive tasks (DevOps toil) context switching, cloud resource management and more. The evolution of DevOps has created pain points across the engineering organization.

‍

Today's challenges for engineering functions

DevOps Manager	Developer	Engineering Leadership
Inbound requests from developers exceed capacity of the DevOps team Requests often lack details requiring multiple back and forths further delaying time to solve Controls on resource parameters (access, cluster size, budget stream, etc) require addition work to identify Developer resources are deployed without standards requiring additional overhead for tracking, maintenance, and security Reporting is spread across multiple systems Accessing logs requires many steps Developers often do not spend the required time to find relevant information in internal documentation	Developers face resourcing bottlenecks when requesting the resources they need Developers are dependant on DevOps engineers to make progress Time is spent on defining requirements and waiting rather than writing code Finding information on best practices and standards (CI templates) is time consuming and often ignored Difficult to locate relevant information for the resource requests Developer experience is diminished	Lack of visibility to resource allocation and cloud costs across business streams Audit and security risks are higher because of lack of controls on provisioning systems Resourcing is a bottleneck to developer productivity and deadlines are missed Too much time spent on internal operations and not on development Loss of talent due to diminished developer experience Feature backlog Delayed releases

‍

As engineering organizations looked to find ways to ease the burden of the modern tech stack on DevOps Managers and developers themselves, Platform Engineering gained a presence as a full-fledged discipline. Platform engineering is the “discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era.” Platform engineering is focused on supporting developers so they can focus on code, not wrangling the tools they need to write, build, test, secure, deploy and monitor the code. Accordingly to Paul Delory, VP Analyst at Gartner. “Platform engineering emerged in response to the increasing complexity of modern software architectures”. Platform engineering was recognized in top ten tech trends for 2023 by Gartner and included it in The Gartner Hype Cycle.

‍

Gartner expects that by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery.

‍

As the discipline of Platform Engineering grew in popularity, Platform Teams and Internal Developer Portals (IDP) surfaced as ways to implement the different approaches to easing the tech stack burden. Both have shared goals of equipping developers with the tools and infrastructure they need, while stripping away the complexities. This setup boosts developer productivity by keeping their focus on code, not tool wrangling. The outcome of either is a set of “golden paths”, as defined by Spotify “a set of opinionated and supported paths to build something”. In the next Chapter/blog post, we’ll dig into how IDPs are solving for DevOps toil and becoming more critical than ever.

‍

Puppet’s 2023 State of DevOps: Platform Engineering Edition showcases the benefits – 94% of respondents are in widespread agreement that the adoption of Platform Engineering is helping their organization release the benefits of DevOps.

‍

Download ebook

IDP

Developer platform

Platform engineering

What’s Interesting ?

Kubiya next generation is here!

Artificial intelligence

IDP

Product

Creating a service directory with Backstage.io

IDP

Developer platform

Terms and Conditions

Vulnerability Disclosure Policy