What GTSG took from the event

Austin hosted Forrester’s Technology & Innovation September 10-12, 2023. While the conference focused on Generative AI, several GTSG core strengths were called for by other topics drawing fewer headlines:

• Workload placement strategy, especially in the context of modernization planning, and the management of technical debt
• Resilience: we remain laser-focused on such rudiments of business continuity as the business impact analysis, planning, and test – and, of course, automation of the process
• Technical currency and sourcing: GTSG has had the opportunity to remediate environments where outsourcing contracts failed to provide incentives to prevent software from becoming outdated.

For context, Forrester defines Generative AI as a “…set of technologies and techniques that leverage very large corpuses of data, including large language models like GPT-3, to generate new content. Inputs for generative AI may be natural language prompts or other non-code and non-traditional inputs.”

In these notes, we will summarize the key points we took from the event, noting where GTSG can help our consulting clients to address the issues.

The conference theme: “Powering Change” through economic uncertainty

Matt Guarini provided the executive perspective: CEOs may accept that economic uncertainty changes their expectations, but what won’t change is that they (and their boards) will still expect better growth than the competition.

CEOs have been investing in technology at a rate greater than GDP for at least two decades – of late, with a declining return on that investment. With the arrival of Generative AI, however, they recognize that investment is required to deliver the growth, improved customer experience and employee engagement they are charged with – while the talent supply is at a premium. Without this investment, Guarini states, they just won’t realize these favorable competitive comparisons.

He also utilized an extended analogy with the “Ford vs. Ferrari” movie, which we’ll net down to “it’s not just about speed (or tech): talent, culture, and processes are essential elements of success.”

Generative AI is atop the news, yet resilience and technical debt remain top of mind for IT execs.

The event was heavily influenced by the explosion of interest in Generative AI since late last year, not only in the many sessions that directly address it but throughout the sessions on such topics as resilience and technical debt. These topics are close to GTSG core competencies: resilience and dealing with technical debt through sourcing in the context of workload placement strategy.

Begin Now to Learn

CEO George Colony- drawing contrast with many previous technology trends, to which he said Forrester had advised a cautious approach – implores us to learn now – ‘not next month, now,’ to learn how to use Gen AI to win, serve, and engage with customers. Gen AI is the most important technology change of a lifetime; a rare confluence of events, a “technology thunderstorm” – triggered by the most fundamental change in user interface in decades. Gen AI enables us to “converse with a big pile of data” – for example, a “boring, opaque insurance policy.”

There are several ways to make Gen AI actionable…

Rowan Curran analogized getting started to his recent Alaskan vacation cruise – three ways to get where you’re going:

• “Leisure cruise:” fully managed services
• “Adventure cruise” (using vendor-provided SaaS large language models (LLMs) with the addition of prompt engineering skills to further customize)
• “Rafting:” deploy on your own infrastructure and build.

Your people and gear, Rowan said, will help you to determine which way to go.

… and governable.

Forrester had a client share his approach to developing a Gen AI strategy:

• “Top down” with Forrester and other experts – “what’s going on and how can we apply it” – and
• “Bottoms up” – asking business leaders for their input. The result was a 64-page white paper focused on “getting people on the cruise ship.”

He emphasized the importance of getting started, acknowledging that they had to begin with a risk-averse approach in a knowledge-heavy, inherently conservative organization. A heavy focus on governance featured requirements that:

• Only public data can be used in modeling
• Human validation is required for any data used in a deliverable to any customer (broadly defined as anything someone hands to someone else).

Other observations included

• The way in which we now interact with applications becoming less the divide we’re all familiar with between code and data – and the need for coding and data science to come together
• Applications and data: the cost of training data is not necessarily part of the budget
• Involve as many people as possible
  • ‘If you’re tired of repeating yourself, you’re starting to do it right’
  • Create a broad working group, including data science, legal and human resources
• There will be a great deal of uncertainty in the process of selecting models, but take a reasonable position, start, break it down into small steps and bring the team along with you on the cruise ship, rather than watching other cruise ships go by

Panel discussion – applications and risk

In a separate panel, Liz Herbert made note that everyone is positioning as an AI company – resulting in AI software spend of $64B by 2025.

Enterprise Applications: Chuck Gahun provided an overview of the areas of enterprise applications (digital experience, ERP/DOP, sourcing/procurement) employing AI. He emphasized the ongoing requirement for human intervention with anything even indirectly impacting the brand.

Risk: Alla Valente presented a funnel yielding risk appetite from goals, through risk capacity, through risk tolerance. We need a risk appetite that supports the business – but also protects it – for example, 99.7% accuracy in recipes is not enough when it comes to food safety.

Impact on technical debt: AI can help by identifying areas for improvement, replacing some of today’s systems and helping to heal aging systems – but it can also add to tech debt burdens by BYO AI adding to redundancy, straying from vendor roadmaps.

Resilience by Design

Naveen Chhabra analogized systems architecture to the design of some of the world’s most resilient buildings. Two featured are set in inherently risky environments (Dubai, where it’s all built on sand, and Tokyo, where there is the risk of earthquakes). The proper steps have been deployed to make these buildings strong and safe even under duress; resilient technology systems do the same, utilizing such technologies and processes as

• AIOps
• Automation and infrastructure as code

• Cloud-native
• Chaos engineering
• Distributed architecture
• Failover/HA

• Disaster recovery
• Scale-out architecture
• Serverless

Naveen “double-clicked” on chaos engineering and automation/infrastructure as code, emphasizing that both require cultural change.

Naveen defined Chaos Engineering as “[The] discipline of experimenting on a [distributed] system to build confidence in the system’s capability to withstand turbulent conditions in production.” He countered such myths as “it’s about ‘randomly breaking things,’ describing CE as a “hypothesis rooted in nonfunctional requirements, with a control group and test group…”

Automation: Naveen encouraged exploration across both greenfield and brownfield (90% of our environments) and calls composable infrastructure a “must.” As infrastructure admins are the new developers, we need to upskill constantly.

Observability and AIOps

Carlos Casanova covers AIOps and digital experience monitoring for Forrester. His message:

• Prevention not reaction: achievement of faster recovery is merely a “better reaction to failure”
• AIOps provides an all-encompassing outside-in view, expanding on observability to enable predictive and preventive action
• The data tells us companies that invest in data management, systems of insight, and differentiated experiences grow more than those that don’t.

The progressions from monitoring to observability to AIOps:

• Monitoring: foundational; we need it, but it provides no visibility beyond externalized data
• Observability: provides additional visibility into devices, and with rudimentary use of AI/ML can begin to automate known issues
• AIOps: brings insight into the user experience; by expanding the use of AI/ML and analytics to identify anomalies, we enhance human judgment and make more automation possible.

Carlos uses a medical analogy: the progression from triage – to remedial prescription medication – to prevention, for example (his illustrations), a glucose pump or sophisticated prosthetic gathering information to get ahead of the condition. He also used more everyday IT illustrations from retail sites and trading floors to illustrate the goal, which is, again, not simply faster recovery but resilience, which he defines as “withstanding a continuous onslaught with little or no external projection of the impact it has on your business or IT operations.”

Technical Debt

Over three-quarters of survey respondents are “concerned” or “somewhat concerned” with technical debt, which explains why Forrester held four sessions – addressing management, systems, sourcing, and a panel which covered risk management and the impact of AI technical debt.

Managing Technical Debt

Charlie Betz noted that the definition of technical debt has expanded from Ward Cunningham’s original use to include hardware, compliance, technical obsolescence, skills, security, and other considerations.

Executives should attune themselves to “symptoms” they hear beyond the term “tech debt” directly, including “buggy,” “redundant,” “fragmented,” “obsolete,” “monolithic,” and “insecure,” experienced as:

• Failure to capture new opportunities
• Costs of supporting a deteriorating portfolio
• Risks of breaches, leaks and outages
• A poor customer experience.

These comments echoed those heard on two recent GTSG engagements focused on upgrading payments networks.

Our management capability for technical debt must be operational (it’s never “one and done” but rather an ongoing capability) and as automated as possible.

We can understand our tech debt via a four-lifecycle model: application, platform, vendor technology, and asset. (Charlie also notes that we still have four lifecycles in the cloud; it’s not going away.)

He proposes progression in tech debt management maturity, from:

• Manual spreadsheets
• The basics of asset and application portfolio management/ processes
• Maturing processes, including CMDB
• Data governed and integrated – but missing decision support
• Automated, intelligent, operationalized digital management

He also presented data showing a strong correlation between CMDB adoption and tech-forward, high-performing organizations.

Technical Debt and Sourcing

Bill Martorelli presented with Akshara Naik Lopez. By way of introduction, they spoke of a public sector client who, upon a change in administration, to save costs, relieved a provider from the requirement to maintain currency on the platforms they were supporting. If you’ve read this far, you know the results: pay now or pay later.

What are clients asking? Over the past several years, Forrester has taken thousands of calls on sourcing RFPs, and hundreds about partners. Modern sourcing practices have evolved to value the need for agility and flexibility and the principle of co-creation and co-innovation.

The presentation recommends five primary principles:

• Recognize your technical debt and use discovery to quantify it
• Maintain freedom of movement across deployment options (on-prem vs private/public cloud)
• Standardize, don’t customize; modernize, don’t merely lift and shift
• Automate
• Avoid horizontal multi-sourcing – organize suppliers by value streams with consolidation in mind

They also noted a tendency to rely on modernization and transformation efforts to reduce tech clutter, but that’s no guarantee of solving the problem. Organizations spend significantly on modernizing applications that aren’t major contributors to tech debt (which, GTSG notes, makes perfect sense where the organization needs to upgrade capability to compete). Bill stated that organizations put almost half of their IT change spend into applications that they would just as soon retire.

They recommend competitive multi-partner approaches with shared risk/reward, focusing on as-a-service purchasing options which buy outcomes. Be specific about what “continuous innovation and transformation” means in the contract – and we must prioritize technological currency over cost control where there is a choice. (Unfortunately, GTSG has seen supplier incentives as the root cause of a client paying a fortune in unnecessary premiums for out-of-support software, which we were subsequently called upon to remediate.)

The Impact of Generative AI On Technology Services

Ted Schadler presented, remarking on how Gen AI reinvents the interaction with information (“how can it help, educate me, ask it how it knows what it knows, imagine it’s a credible witness”) – the first new user experience model in 60 years and the first new knowledge model in 53 years (from the advent of relational databases) – although we’re sometimes uncertain how it knows what it knows.

The immediate early impact on worker productivity is on those who have some knowledge but not enough to be experts. In software development, it’s not just coding; it’s story points, test scripts, better insight into the improvement of coding, etc. With TuringBots offering a potential 3x increase in improvement, quality, speed, and predictability, we can expect changes in pricing models.

Traditional outcomes have been productivity-based. The new approach is co-innovation, a strategy to maximize the value with mutual commitments and shared outcomes, pursuing the same goal with aligned incentives.

Gen AI elevates the need for co-innovation with partners, and service providers are investing massively (he mentioned KPMG, the Deloitte/NVIDIA collaboration, Accenture).

Productivity is only half the story; quality, speed, and predictability also improve – with implications on sourcing.

The implications include:

• More fixed price bids as Gen AI is used to reverse engineer (as we heard it, we were thinking legacy code modernization)
• More confidence leads to more output-based pricing
• We need to ask ourselves:
  • Are we getting what we need from our partners and committing our own resources to get the maximum from them?
  • Does our inventory of partners have what it takes?
• We need to structure agreements that motivate, not for every partner but for strategic providers

Predictions 2024

The session opened acknowledging that some of these come to pass, and some do not (metaverse being a fine example). A sampling of this year’s:

• 90% of Fortune 500s will be thrust into modernization by Gen AI, as only 10% of technology executives will achieve their growth targets by the end of 2023
• Only 19% of HR executives are confident in their people’s ability to deal with AI, yet 89% of companies use AI in hiring. What could go wrong? Well, they predict that
  • Companies will hire at least one person for a job that doesn’t exist, and
  • At least one position will be filled by someone who doesn’t exist

…because bots talking to bots doesn’t work very well.

• Over 50% of the data managed by enterprises will now be unstructured – and a great deal of it is employee data
• 65% of companies spend less than $2,500 on management development – how can managers lead what they don’t understand?
• 50% of companies turn to partners to capitalize on Gen AI – the downside of which is their sacrifice of investment in internal talent-building, putting them further at risk

* * * * *

We hope these notes have been of value to you. If you would like to talk further, please write us at Partners@GTSG.com.