Achieve Staggering Performance Gains – Even in a Mature and Outsourced Shop

The Challenge

Our client is a well-known automotive supplier.   Headquartered in Europe, more than a decade ago they outsourced the management of their IT operations as well as application development to a pair of global Tier 1 providers. The outsource included their mainframe operations which run their systems of record.

When we met the client, they expressed no dissatisfaction with the outsourcer but had been trying to find a mainframe technical performance expert to manage that tower. Experienced mainframers are in short supply, particularly among outsourcers, resulting in what we call “run/operate” vs. “manage/improve” behavior. In outsourcing, improvement incentives are “zero” to “negative,” depending on the contract. But for the client, the impact is significant on sizing, 4HRA, and therefore mainframe TCO.

The client was in the early period of what they described as a five-year plan to decommission most of, if not the entire mainframe environment. What they discovered early on, however, was that organic growth in their business continued, and the additional capacity requirements for decommission development continued to push them into MIPS increases across the mainframe environment. At a minimum, any viable solution had to stem the continual addition of MIPS. They had little hope of decreasing MIPS or providing headroom for organic and application development growth.

After several calls to vet our technical skill and success at other shops, the client asked us to come to work.

The Solution

The team decided a performance and workload management engagement was the place to begin. Our team started immediately, bringing hundreds of years of experience across hundreds of shops literally, in the mainframe towers relevant to this particular client. They began looking at the largest consumers of capacity; HSM, DB2, CICS, and other high-consuming services- asking a deceptively simple question – “what seems off?” – which only works at this level of experience.

While our approach has a traditional consultative structure, it is the insight enabled by this experience that jump-starts the analysis and accelerates time to benefit:

Identify & Assess
Opportunities

  • Apply our vast knowledge base: “we always see issues with these subsystems”
  • What seems outside the norms considering our experience?
  • Assign Risk/Reward Rating: complexity; component consumer base
  • Estimate benefit: is the consumer an island unto itself? multiple components or iterative uses per day
  • Measure in client’s terms: (per hour, 24 hours, month, MIPS v MSUs)

Planning &
Implementation

  • Who implements? if outsourcer, detailed instructions and backout required; IVP plans
  • Outage requirements (IPL, Subsystem, None)
  • Testing requirements
  • Baseline to avoid low-consumption periods
  • Implement: active participation regardless of performing organization

Post Mortem &
Closed Loop Feedback

  • Recognize Success: estimates versus actual; report daily/ monthly MIPS/MSU elimination; tie to financial saves
  • Annotate daily/monthly performance reporting
  • Close the loop: did execution expose other bottlenecks not visible previously? Any new opportunities exposed?
  • Restart the process

What we find is substantial, well beyond what run/operate teams can typically uncover and action. For this client, simply looking for what “fails the sanity check,” we produced over 50 recommendations in the first eight weeks. Some of these recommendations applied to a number of jobs and subsystems where similar inefficient practices had been utilized.

Client Benefits

A representative sample of savings includes:

  • Over 30,000 monthly MIPS in DFHSM; further DFHSM improvement of 21% CPU consumption reduction after the changes that resulted in 30,000 monthly MIPS
  • Over 5% of HSM CPU consumption on an HSM member not participating in space management functions
  • Over 895 daily and 26,500 monthly MIPS with a single program coding change
  • Over 4,100 monthly MIPS in IWSz; additionally, EXCP rates reduced from a sustained average of over 1,200 per second to a sustained average of 26 per second – a nearly 98% reduction
  • Over 1,900 monthly MIPS for a single program that was consuming over three minutes of CPU reduced to less than an average of ten CPU seconds per daily execution
  • Over 1.2 TB of DASD even though this was not our primary tasking
  • Over 20,000 monthly MIPS in a single IDMS CV from peaks in previous months plus a downward sloping trend line that approximated 2,000 monthly MIPS

As a result of our success in the North American operation, the client has extended our scope to the more extensive implementation in Europe.