In order to meet the needs of the ever-changing information age, telecommunication services are constantly changing. The business support system is updated once or twice a month with major versions, not to mention emergency versions. To ensure the normal stability of the system, performance testing is the last important guarantee before the version goes live. As the primary task of carrying out performance testing, the requirements analysis stage determines whether the performance test objectives are correct, whether the method is reasonable, and whether the measurement is appropriate. More precisely, the requirements analysis determines whether the test can produce the results that the customer wants.
The requirements analysis stage requires repeated discussions with customers, which is a cyclical process: from requirement capture -> requirements collation -> requirements verification -> re-requirement capture... This work requires not only understanding ability, design ability, but also An ability to interact and communicate with others.
1 communication
The first step of demand analysis: understand the pain and difficulty of the customer, so that the right medicine can be used to solve the problem for the customer. But often not every "patient" can tell exactly where the disease is. It needs to listen carefully and communicate patiently to gradually get to the root of the problem. For systems with many services and complex relationships such as telecommunications, first-time demand analysts often encounter the problem of inadequate business, which leads to over-obedience to customers. Customers can do what they say. In the end, they do very hard and tired. Can't make customers happy.
After many projects and close interactions with telecommunications customers, I conclude from the experience: Before communicating with customers, you must work hard in the telecommunications business to figure out business requirements, usage scenarios, operation steps, business system associations, and business execution. path. Ask customers why more, you can have a deeper understanding of these domains, think about problems from the customer's perspective, and more accurately understand the reasons why customers put forward business needs. In addition, in the process of dealing with customers, we discovered that the requirements obtained from customers’ mouths are only the tip of the iceberg in the entire test requirements, and there are some hidden requirements that we need to dig out by ourselves, for example: the customer’s unspoken requirements, the customer Unexpected demand.
1.1 Unspoken demand
For the customer's unspoken demand, it is not that the customer is deliberately showcasing the mystery and does not want to say it, but the common name has been agreed in the customer's business field. Just like people have to eat and sleep every day, there is no need to say it at all, and the industry is conscious Business rules to follow. However, as a demanding person who has just stepped into this field, he does not understand these rules. If the requirements mentioned by customers are recorded in a passive way, this part of the requirements will definitely be ignored, which is why after exhausting the test results, the results still cannot be satisfied by the customers, and a large number of them are proposed. The reason for the question. The correct way to encounter this kind of dilemma is to first understand the business from the customer, find out what type of business is, what are the business rules, daily user operating habits, peak hours, etc. Among them, the operating habits and experience of the user group are the most important. It reflects how much pressure the system needs to bear and what response speed it can achieve. Ask why, because customers are troubled by this problem. Collecting and sorting out the "pain points" from customers can deeply understand the root cause of the problem, learn relevant domain knowledge from it, and think about the problem from the customer's perspective, so as to understand the customer more accurately Why put forward their business needs.
1.2 Unexpected demand
The other is that the customer never thought of the demand. This kind of unexpected problem is actually a problem that has not yet appeared in the business demand stage, and does not mean that the customer does not need to solve or will not appear. Many demand managers often complain that customers' ideas are constantly changing and new content is constantly being added. The client is not a professional tester. When a new problem is discovered or the problem is fixed, it needs to be re-evaluated to verify the cause of the problem or the effect of the repair. These are things that customers cannot predict before they consider their needs. When the implementation reaches this stage, when problems arise, many needs they didn't expect at the beginning will be continuously raised. But at this time, the workload of performance testers has become very large, and the demand analysis work generated by passive acceptance will bury huge risks for project execution and subsequent work. Solving this type of problem requires requirement analysts to be proficient in system architecture principles, combined with a full understanding of business domain knowledge and requirements, to discover this part of the requirements in advance through analysis, formulate test models and test methods, and finally form a required test plan. Through the various stages of acquisition, understanding, analysis, design, and formation, a high-quality demand analysis can be obtained to effectively guide the entire test execution process and avoid various possible project risks. It can be said that the need for analysis is the blueprint of performance testing, which determines the quality and success of the test.
2Analysis & Design
After understanding the customer's real ideas and mastering the real needs, it is necessary to combine the testing thinking and environmental characteristics to in-depth analysis of testing methods and measurement indicators, set testing goals, and repeatedly confirm with customers. Because the test objectives must be clear and consistent from all parties to ensure that the test results are approved by the customer.
2.1 Business analysis
Business analysis must be established on the basis of familiarity with the business, understanding why this business is needed, what this business does, the people who use the business, and the objects it affects. These business backgrounds are often easily overlooked in heavy testing tasks, and this critical step must be kept in mind at all times, otherwise the gains outweigh the losses. For example: once upon receiving a certain request, a black and white list should be added to the group network at the beginning. The black list is for users who have subscribed to a certain type of business and restrict their business operations. Because I didn't understand the function of the beginning of the black and white list, I directly performed the test and found that the results in the beginning state were all the same, which was criticized by customers. After I figured out that it was a business rule for a specific user group, it took a few overnight to correct the result. This case shows that no matter how good the testing technology is, it cannot be used without knowing the business. In addition, being familiar with the business and understanding the business rules can prevent mistakes and improve work efficiency. For example, each performance test needs to prepare a large amount of test data. Knowing the business rules can easily find a shortcut to obtain the data, or find a way to obtain the data. When it is necessary for customers to come forward and coordinate, a clear business logic expression will also help increase customers' recognition of your work and increase the degree of cooperation.
In short, whether the test is recognized by the customer does not depend on the level of technical ability, but on whether the test is based on business principles, customer ideas and needs.
2.2 Determine the test type
After fully understanding the test purpose and completing the requirements analysis, you need to choose an appropriate test method. The pressure change model can be divided into 4 types of tests:
●Performance testing. The system performance between point a and point b is to verify whether the system can achieve the expected performance target within the range of resource availability.
●Load test. The system performance at point b is verified for a period of time under certain pressure until one or more indicators of the system reach the limit.
● Stress test. The system performance from point b to point d is verified under the condition of exceeding the safe load, and the system is continuously pressurized until the system can no longer accept requests.
● Stability test. The performance of the system from point a to point b is to verify that the system runs for a period of time under a certain pressure to detect whether the system is stable.
Performance testing usually uses a single business scenario test first, and then a comprehensive business scenario test.
The single business scenario is mainly used to test the basic performance indicators of the business, as a horizontal comparison of the same type of business, or the baseline indicator of the business, for comparison between versions. Single business scenarios are suitable for performance testing, load testing, stress testing, and stability testing;
Hybrid business scenarios are used to simulate business pressures or user usage scenarios running on the production line to test whether the overall performance of the system meets actual performance requirements. It is a business scenario that combines performance test points that have been screened by certain rules according to actual logical virtual user requests and concurrency. Mixed business scenarios usually include two or more script groups, which take a long time to execute. Hybrid scenarios are usually used in stability testing and load testing.
2.3.1 Key indicators
●Response time refers to the system response speed when operating the business, which directly affects user experience. This indicator generally has a design specification value or a feedback value of user experience.
●Business processing speed refers to how many transactions of this type can be processed per second by the system. Since the number of users in each city is different, the number levels are different, which can be calculated by the following formula:
●Get the peak monthly work orders of the business in the city, assuming that 80% of the work orders are processed within 4 hours:
(Total number of front desk work orders of the month×80%)/30 days/(4×60×60)
For example: the total number of monthly work orders for account opening business in a certain city front desk** is 2615537.
2615537×80%/30/(4×60×60)=4.84 pens/sec
● Concurrency. The broad sense refers to the number of services processed by the system at the same time, and the narrow sense refers to the number of services processed by the system at the same time. The latter is usually used as a reference indicator for the number of concurrency in a single business scenario.
●System resource consumption indicators: CPU, memory, disk IO. In some cases, business indicators are satisfying customers, but resource consumption indicators reflect that there may be risks. For example: When the processing speed and response time of the business are up to the standard, the CPU usage reaches more than 90%. If this situation continues for a period of time, or the number of users increases, the server is at risk of downtime and slower computing speed.
The customer's idea is usually that the wider the coverage, the better, but there is always a gap between the ideal and the reality, which is often limited by the window time left for testing. Before accepting test tasks, you must first establish access criteria principles, such as: completing the configuration confirmation of the test environment, agreeing on the time period for other development and testing work to use test resources, the number of use cases received each time, and the last receiving time point. Only with good access standards can it be possible to ensure the high quality of the test.
When measuring between time and coverage, in addition to the completion quality, it is more important to consider the risk and issue tracking mechanism. For example: the test result of a certain business module does not meet the standard, but the business must go online on time. There is a test -> tuning -> test verification cycle. Although the test script does not need to be modified repeatedly, the workload of preparing test data needs to be considered. The number of revisions, the last online time and the test results must be confirmed with the customer, and data support for risk judgment shall be provided, so that the customer has a clear understanding of the risk of going online and minimizes the probability of risk.
In addition, if there are multiple teams participating or supporting the time, it is necessary to clearly divide the responsibilities of each team, so that when an emergency occurs, each team can perform its own duties and work together to resolve the problem instead of messing around.
3. The plan is established
After the communication, analysis, and design phases, the resulting test plan needs to be documented and notified to all parties before the test can be carried out after being approved.
The following is the template content of the plan for your reference.
1. Test goal, explain what the purpose of the test is, or what effect needs to be produced;
1. Test background, explain business principles, generate background, and let testers understand the purpose of the test;
2. Test environment, describe the test environment architecture, server, press configuration, test tools, etc.;
3. Test scenarios, explaining the focus of the test business;
3.1 Test case, describing the operation steps of the use case, entering and exiting parameters, and business rules.
3.2 Test indicators, describe specific indicators and thresholds to determine whether the test passes;
4. Team division of labor, describing the scope and content of each participating team;
5. Execution time, specify the execution test time period and release time point of the environment;
6. The deliverable, the output after the test (such as: test report).