Production AI-driven host automation testing

Translator | Chen Jun

Reviewer | Chonglou

Code that converts traditional mainframe applications And data migration to modern technology architecture is considered a key step in the digital development of enterprises. In the pursuit of efficiency and scalability, this transition often involves migrating from traditional mainframe environments to more flexible cloud computing or on-premises solutions. Such transformations help enterprises achieve greater agility and innovation while reducing costs and improving security. This migration can also bring better resource utilization and faster market responsiveness to the enterprise. Enterprises need to plan and execute this transition carefully to ensure that the data migration and application reconstruction process goes smoothly to ensure business continuity and data security. By migrating traditional mainframe applications to modern technology architecture, enterprises can better adapt to rapidly changing market demands and achieve

However, as time goes by, Dynamic changes in the business environment will also make the migration process more complicated. This is not only a technical redeployment of resources, but also a fundamental change, so it often requires rigorous testing to ensure the equivalence of various functions, thereby maintaining the operational integrity and performance of the application. standardization.

Also, after migration, applications are often forced to undergo extensive modifications driven by new requirements, evolving business strategies, and changes in regulatory standards. And every modification, whether it is a minor adjustment or a major "renovation", must undergo rigorous testing. The most critical challenge is ensuring that new changes integrate harmoniously with existing functionality without causing unintended consequences or disruptions. It can be seen that the dual requirements of verifying new functions and retaining existing functions highlight the importance of automated test suites after migration.

In recent years, the wide application of generative AI (GenAI) has radiated to various fields. If it is introduced into the modernization process of mainframes, it is not only expected that enterprises will achieve significant improvements in software quality, operational efficiency, etc., but it will also bring about fundamental changes in software development and quality assurance concepts, allowing enterprises to ultimately obtain considerable returns on investment. .

I will discuss with you how to leverage appropriate automated testing methodologies, tools, and best practices to ensure that the inherent quality and performance of modern mainframe applications is maintained during a smooth technology transition.

Traditional Manual Testing Concept

For a long time, large mainframe environments have been unwilling to accept automated testing methods. The 2019 Global Survey conducted jointly by Compuware and Vanson Bourne shows that only #7% of respondents have adopted Automated test cases are enough to illustrate the industry's attitude towards this.

The Dilemma of Manual Testing

In contrast, manual testing is a traditional method commonly used by many companies. However, in the complex process of mainframe modernization, this approach is becoming increasingly inadequate and error-prone. After all, test engineers need to manually verify every scenario and business rule. This process is fraught with the possibility of human error. Moreover, given the high-risk and mission-critical nature of many mainframe applications, once we overlook a subtle error during the testing process, it may lead to serious production problems, substantial downtime, and even financial losses. Therefore, this shortcoming becomes particularly prominent. In addition, manual testing also has the following defects:

1. Omissions and inaccuracies: Manually processing a large number of test cases will increase the risk of missing key scenarios and inaccurate data verification.

2. Time-consuming: Manual methods require a lot of time to thoroughly test every aspect, so they are not efficient in fast-paced development environments.

3. Scalability problem: As the application expands and evolves, the workload required for manual testing will increase exponentially, making it even more difficult to effectively identify errors.

Some companies may intuitively think of expanding their manual testing team. However, this is not a feasible solution. This move creates cost inefficiencies and fails to address the inherent limitations of manual testing processes. Enterprises need to integrate automated testing processes through modern methods such as DevOps to improve efficiency and reduce errors.

The Necessity of Test Automation

Overall, by integrating automated testing processes into modern mainframe applications, enterprises can greatly improve the efficiency and accuracy of migrations sex. Of course, the adoption rate of automated testing in mainframe environments is not high. Some companies see this as a challenge, while others see it as a huge transformation opportunity. After all, adopting automation in testing is not just a technology upgrade, but a strategic move to reduce risk, save time, and optimize resource utilization.

This shift is critical for businesses hoping to remain competitive and efficient in a rapidly evolving technological environment. According to the "DevOps Status Report", automated testing can play a huge role in optimizing operational workflow and ensuring application reliability.

What is automated testing?

According to Atlassian’s definition, automated testing is to automatically complete those manually driven tasks through the use of software tools Review and validation process for software products. Its speed, efficiency and accuracy surpass the limitations of traditional manual testing methods. That is, automated testing helps accelerate application changes while ensuring that its quality and reliability are not compromised. Automated testing not only simplifies the verification process of new changes, but also monitors the integrity of existing functionality, thereby playing a key role in the seamless transition and continuous maintenance of modern applications.

In the process of optimizing the software testing process, the use of automated testing often requires initial manual investment, which involves test engineers' understanding of the intricate business logic underlying the application. This understanding is crucial to effectively generate automated test cases using frameworks such as Selenium. Although this stage consumes a lot of manpower, it is a basic work. After all, subsequent automated testing will significantly reduce the reliance on manual testing, especially in repetitive and extensive testing scenarios. Furthermore, once an automation framework is established, it becomes a powerful mechanism for continuous evaluation of applications. Its advantage is that it is good at identifying errors or bugs that may arise due to application changes.

Automated testing methods during mainframe modernization

In the field of software engineering, for large-scale migration or modernization of mainframe applications Automated testing often requires a comprehensive understanding of all business rules in an application in order to generate automated test cases for a huge code base that often consists of millions of lines of code. This is a complex and quite difficult task. Given that it is difficult to achieve 100% code coverage, we need to achieve a balance between test coverage depth and practical feasibility to ensure that critical business logic can receive sufficient test coverage.

In this case, emerging technologies such as GenAI provide a possibility. It can automatically generate automated test scripts to simplify the testing process of mainframe modernization projects and provide a more efficient, accurate, and scalable method for quality assurance in software development.

Wide Utilization of GenAI

Before we delve into how GenAI enables automated testing in mainframe modernization, let’s take a brief look at GenAI. Fundamentally, GenAI represents an aspect of artificial intelligence. It uses generative models to generate various texts, images or other media. These generative AI models are good at learning the patterns and structural elements of the input training data and then generating new data that reflects those characteristics. Obviously, such systems mainly rely on machine learning models, especially those in the field of deep learning.

Natural Language Generation (NLG) is a form of GenAI that is strongly relevant to the modernization of mainframes. It can produce human-like text with the support of large language models (LLM). Typically, LLMs are trained on large corpora of text data, enabling them to discern and reproduce the nuances and structure of language. This training therefore enables them to perform a variety of natural language processing tasks, including text generation, translation summarization, and sentiment analysis. Notably, LLM is also proficient in generating accurate computer program code.

Currently, well-known use cases of large language models include: GPT-3 (Generative Pre-trained Transformer 3), BERT (Bidirectional Encoder Representations from Transformers) and T5 (Text-to-Text Transfer Transformer). These models are usually built on deep neural networks, especially those using the Transformer architecture. Therefore, they show excellent efficacy in processing sequential data such as text. Large amounts of training data, including millions or even billions of words or documents, allow these models to master a wide range of programming languages. Not only do they excel at generating coherent and contextual text, but they also excel at predicting language patterns such as completing sentences or answering queries.

Currently, some large language models can also understand and generate text in multiple languages, increasing their usefulness globally, from powering chatbots and virtual assistants. Intelligence", to supporting application fields such as content generation, language translation and summarization, LLM's versatility can bring outstanding performance.

How to use GenAI to generate automated test scripts

In the field of software testing, LLM can help us extract business logic from the code of the application, And convert these rules into human-readable format, and then generate corresponding automated test scripts. At the same time, it can also help us select the necessary number of test cases to meet various potential coverage requirements of the code fragments.

Usually, using GenAI to generate automated test scripts for application code requires the following structured three-step process:

1. Use GenAI Extract business rules: As an initial stage, we need to use GenAI to extract business rules from the application. This process will judge the extracted rules according to the level of detail and interpret them in a human-readable format. In addition, GenAI also helps to gain a complete understanding of all potential outcomes of a given piece of code. This knowledge is critical to ensuring that accurate, relevant test scripts are created.

2. Use GenAI to generate automated test scripts at the functional level: Based on the extracted business logic, test engineers can have a comprehensive understanding of the application's functions so that they can be exploited at the functional level. GenAI to develop test scripts. This step involves determining the number of test scripts required and identifying scenarios that may be excluded. Of course, the code coverage of such automated test scripts is often decided collectively by the team.

3. Verification and reasoning added by Subject Matter Experts (SME): In the final stage, once the business logic has been extracted and the corresponding automated test scripts have been generated, the test Experts will verify these scripts and have the authority to add, modify or delete them. Such interventions address potential probabilistic errors that may arise from GenAI output and increase the quality certainty of automated test scripts.

The above process may seem complicated, but it can actually make full use of GenAI's capabilities to simplify the test script generation process and ensure the perfect combination of automation efficiency and human expertise. Among them, the participation of testers in the verification phase is particularly important. This will allow the output results generated by artificial intelligence to be based on actual, real-life application knowledge, thereby significantly improving the reliability and applicability of test scripts.

Summary

In summary, as a tool to improve efficiency, GenAI can generate automated test scripts through its NLG capabilities, and then Improve the accuracy and reliability of software testing processes for mainframe modernization. At the same time, GenAI needs to improve the output results generated by AI through a structured three-step process and ensure that the automated scripts are not only technically reasonable but also applicable in practice, thus reflecting the harmony and unity of AI capabilities and human expertise. This integration is clearly critical to address the complexity and dynamic requirements of modern mainframe applications.

Translator Introduction

Julian Chen, 51CTO community editor, has more than ten years of experience in IT project implementation and is good at Implement management and control of internal and external resources and risks, and focus on disseminating network and information security knowledge and experience.

Original title: GenAI-Driven Automation Testing in Mainframe Modernization , Author: sampath amatam)