Mainframe systems often sit at the heart of an organization’s infrastructure, silently powering critical operations behind the scenes. Yet despite their centrality, they are frequently under-tested or misunderstood. This stems not just from their legacy nature but from a skills gap and a tendency to treat them as untouchable black boxes. In this article, we take a deep dive into Mainframe security testing and best practices with our Security Consultant & Global Mainframe Lead, James Boorman.
Intro to mainframes
When it comes to mainframes, there’s a fair amount of confusion, even within technical teams. Strictly speaking, it refers to systems like IBM Z/OS, Unisys ClearPath, or certain Fujitsu models. Then there are others, like HPE NonStop, which aren’t technically mainframes but behave similarly enough that many professionals lump them in. So whether you’re calling it a mainframe or a mid-range system with mainframe DNA, the security challenges, operational behaviours, and strategic importance remain largely the same.
These applications are built up over time, often older than the people maintaining them, and are highly interconnected to comply with strict regulatory demands. What appears simple on the surface involves numerous systems working together to review customer information, verify transactions, and manage accounts.
Common attack paths in mainframe environments
Mainframes might seem like fortress-grade systems, but attackers know where to look. Most vulnerabilities stem from access control misconfigurations, making privilege escalation and unintended data exposure real concerns. Let’s break down the most common and practical attack paths.
Most vulnerabilities stem from access control misconfigurations.
- Surrogate chaining
One of the standout techniques is surrogate chaining, where an attacker abuses job submission permissions to run code under higher-privileged user IDs. If User A can act as User B, and User B can act as User C, then User A effectively gains access to User C’s privileges, even without direct rights. These chains grow in complexity and can be exploited if not carefully managed, especially in large systems with sprawling permission structures.
- Application breakouts
In online applications, attackers often look for a way to break out of the app’s restricted environment. By triggering errors or manipulating inputs, they aim to infiltrate the underlying command environment, such as the CICS region, thereby gaining broader control than intended. From here, it’s about probing what access is possible at the platform level, even if they entered with a low-privilege account.
- Network-based entry points
Network-based attack paths exist but are less common in real-world testing. Gaining entry without credentials or direct access is tough, given that most controls live inside the mainframe. Still, misconfigured services or stolen credentials via phishing can open doors, and these shouldn’t be overlooked.
- Permission hopping via shared jobs or datasets
Mainframe security often revolves around resource protection and access assignments. If an attacker can alter a job that runs under another user’s credentials, they inherit that user’s access. Misconfigured datasets, improperly secured USS (Unix System Services), and over-permissioned profiles can make these jumps surprisingly easy.
The bottom line? Mainframe security is less about flashy exploits and more about navigating complex webs of access. System owners must stay vigilant, regularly review permission hierarchies, and test not just individual applications, but how they connect across the environment. Because with thousands of users and highly interconnected apps, one weak link is all it takes.
Mainframe security testing: platform vs. application
Security testing for mainframes typically happens on two levels:
- Platform-level reviews: These are broad assessments, similar to a build review. They target the overall platform, checking subsystems and mainframe-specific controls. However, these massive systems with potentially hundreds of applications and thousands or tens of thousands of databases and tables, going deep isn’t practical. Understanding why each person has access to every given table would take mind-numbingly long. Instead, the assessment focuses on the wider, more global configuration of the operating system and its subsystems, identifying systemic misconfigurations and weaknesses at the cost of specificity.
- Application-level reviews: These are more focused and context-rich. Testers examine specific apps, like a payment review tool, along with the data it uses, who can access it, and how it behaves. This approach is typically suitable for assurance-driven compliance and provides insight into the security posture of the specific application and its resources. However, since mainframe applications rarely exist in isolation, this method misses both the wider solution and bigger system picture – especially when applications rely heavily on each other.
The gap? Strategic middle-ground assessments. These look at clusters of related applications, say, 40 systems that make up a payment engine, to understand how data flows between them, where interdependencies lie, and how attackers might move laterally. It’s a space where purple teaming or attack path mapping can add real value, combining offensive and defensive testing to simulate realistic threats across complex environments.
Why mainframes need holistic security testing
Mainframe systems often sit at the heart of an organization’s infrastructure, silently powering critical operations behind the scenes. Yet despite their centrality, they are frequently undertested or misunderstood. This stems not just from their legacy nature but from a skills gap and a tendency to treat them as untouchable black boxes.
The reality of mainframe testing
Mainframe security testing isn’t commonly practiced; it’s a niche skill set, and few professionals specialize in it. In many cases, organizations claim to test mainframes but restrict those efforts to unauthenticated network scanning. When services don’t respond or appear hardened, teams often walk away satisfied. The result? Superficial assessments that miss deeper, systemic vulnerabilities.
Organizations claim to test mainframes but restrict those efforts to unauthenticated network scanning.
Why a combined platform + application approach matters
The organizations that truly engage in effective mainframe testing take a more holistic approach. Rather than treating the system as a monolith, they separate platform-level configuration and application-level security. This dual perspective provides:
- Broad insight into how infrastructure-wide settings affect individual components.
- Granular visibility into application-specific risks, such as poorly configured authentication or insecure data storage.
- Coverage of legacy weaknesses, which often arise from outdated assumptions about how the systems work.
Testing both dimensions helps uncover attack paths that stem from decades-old code and documentation gaps, things that traditional audits would miss.
Testing frequency: Compliance vs. proactivity
Security testing cycles vary widely. Some enterprises only test mainframes to meet compliance thresholds, annually or once every few years, depending on system criticality. Others take a more proactive stance, building regular mainframe testing into their operational security roadmap.
Yet some organizations still neglect to perform routine, comprehensive mainframe testing. These systems, perceived as inaccessible, legacy-bound, or simply misunderstood, may not always be included in modern testing pipelines. The truth is, their complexity and obscurity increase risk, not reduce it.
Common misconceptions & hidden risks
One recurring theme in mainframe testing is the prevalence of assumptions:
- “The application passwords are encrypted” until closer inspection reveals they’re stored in plaintext.
- “Application IDs can only be used by one user” until a tester successfully impersonates another user.
- “No one can access this system” until someone tries… and succeeds.
These myths persist because documentation is missing, incomplete, or lost to time. Knowledge silos and shrinking subject matter expertise compound the problem.
Mainframe security: Best practices that make a difference
Despite their age, complexity, and niche status, mainframes continue to power critical infrastructure across industries. Their enduring presence doesn’t mean they’re impervious, but it does mean they require a tailored approach to security. Here’s what organizations should prioritize when safeguarding these systems.
1. Follow vendor guidance, and actually apply it
Leading vendors, such as IBM, provide extensive best-practice documentation for securing their platforms and subsystems. These resources offer practical architecture guidelines and proven security configurations. However, these recommendations are too often overlooked or partially implemented. The first security win is simply making sure they’re properly followed.
2. Perform regular, context-aware security testing
Mainframes shouldn’t be tested once every few years just to tick a compliance box. Security assessments must be recurring and adapted to each organization’s setup. Testing should span:
- Platform-wide evaluations to assess overall configuration and access control
- Application-level reviews that uncover business logic flaws and interface risks
- Strategic security audits tailored to the organization’s architecture and threat landscape
No two mainframe environments are identical, even if they run similar software. Testing must reflect how each system is uniquely deployed and configured.
3. Configure the security subsystem correctly
Mainframes use internal security subsystems to enforce access controls and define privilege boundaries. Misconfigurations here can undermine the entire platform. Organizations should:
- Audit subsystem roles, groups, and policies regularly
- Enforce least privilege across users and services
- Ensure segregation between test and production environments
Mainframes’ enduring presence doesn’t mean they’re impervious, but it does mean they require a tailored approach to security.
4. Lock down network access with firewalls & segmentation
Production mainframes should be isolated from internal networks wherever possible. Strict firewalling and zone segregation prevent lateral movement and unauthorized access. Air-gapped designs or tightly controlled access pathways are ideal for critical workloads.
5. Monitor & audit with intelligence
Basic logging isn’t enough. Security teams should implement intelligent monitoring, with alerts tied to behavioural anomalies. For example:
- A Db2 table accessed at odd hours
- Unexpected admin login from a new source
- Privilege escalation attempts outside approved workflows
Correlating these events across logs can surface hidden attack paths or misconfigurations.
6. Modernize authentication & access management
Many mainframes still rely on legacy password practices, think eight-character passwords with low complexity. But modern solution integrations do exist and should be adopted:
- Enable multi-factor authentication (MFA)
- Use privileged access management tools
- Implement just-in-time access for high-privilege accounts
- Rotate sensitive credentials on schedule or per-use
Security maturity here is uneven across organizations; modernization is low-effort with high payoff.
7. Encrypt sensitive data at rest
A surprising number of legacy systems still store sensitive data in plain text. It’s essential to apply modern encryption standards wherever sensitive data resides:
- Within databases and application tables
- In archival or regulatory storage
- Across communication channels (avoid legacy SSL/TLS versions)
Data at rest encryption protects against insider threats and improperly scoped access.
Industry-specific mainframe security practices
Mainframes are versatile systems, deeply embedded across sectors like finance, healthcare, aviation, insurance, and government. Even in the same industry, they’re configured and deployed very differently. While general security principles apply, certain industry contexts require more tailored approaches and awareness of regulatory frameworks.
Financial institutions: More than just compliance
In banking and finance, mainframes often handle core transaction processing and customer data. These systems may fall under compliance mandates such as PCI DSS when dealing with cardholder data. While PCI DSS provides a checklist-style framework for testing and encryption requirements, organizations shouldn’t default to minimum effort:
- Don’t just “tick the box.” Treat PCI DSS as a baseline, not a ceiling.
- Customize testing to reflect actual platform risk, not just what regulators expect.
- Ensure sensitive data like identifiers, credentials, and financial records are encrypted at rest and in transit.
It’s easy for legacy systems to be excluded from modern compliance pipelines simply because they’re “hard to test.” But complexity isn’t an excuse; it’s a red flag.
Aviation: Legacy systems that still fly
Airlines are known to rely heavily on mainframes for reservations, departure control, and crew scheduling. Despite their dependence, these systems are often considered untouchable. Though testing access might be challenging, aviation operators should prioritize:
- Network segregation of operational systems from passenger-facing applications
- Monitoring of batch jobs and scheduling access to detect potential abuse
- Updating authentication mechanisms across reservation platforms integrated with legacy backends
While not bound by PCI DSS across the board, aviation companies still handle sensitive data, such as passport numbers, personal information, and payment details, and must adhere to international privacy laws and standards.
Cross-industry takeaway: Avoid minimum viable testing
Whether regulated or not, every industry faces this decision: do we do what’s required, or what’s responsible? The best practice is simple but vital, don’t settle for the bare minimum. Legacy systems are just as critical as flashy new apps, often more so. Their opacity, age, and integration with newer platforms make them uniquely valuable and uniquely vulnerable.
Organizations should:
- Treat mainframes as active components of modern architecture
- Test regularly and purposefully, beyond compliance thresholds
- Integrate mainframe security into enterprise-wide strategies
How Reversec can help
Our mainframe security assessment provides a holistic view of your applications, ensuring end-to-end coverage that includes applications, network presence, operating system controls, and other interactions within the wider environment. This comprehensive approach enables you to apply layered security controls and significantly reduce your attack surface.
Blending scarce mainframe security skills with modern testing techniques, we tailor each assessment to your specific needs, providing maximum coverage and assurance. Our team also draws from industry-specific experience gained through work with finance, aviation, healthcare, and government clients.
