Table of Contents
Testing is an integral part of the development process as it helps ensure quality, security, and usability of software. One of its most important types is performance testing, aimed at evaluating how well your system performs in various conditions.
So, what is performance testing? How exactly do you carry it out and what metrics are you paying attention to during the process? Read below to find out.
What is performance testing?
The performance testing meaning can be defined as a method that helps evaluate speed, responsiveness, stability, and reliability of a software application. The main goal of the process is to ensure the quality of the app and its seamless functioning in various conditions. Since you cannot predict the exact load volume that might occur, performance testing helps prepare the software for future load spikes and guarantee its failure-free operation even during extreme loads.
In addition to double-checking the quality of performance, this method also helps identify the most common performance issues and bottlenecks, such as:
- Long load time: sometimes an app takes too long to load and that greatly impacts user experience;
- Low response time: a low response time means a user has to wait to get a response from the app. Same as with long load time, low response time impacts user experience negatively.
- Low scalability: the app cannot handle the growing load and cannot scale correspondingly, which results in failures.
- Low throughput: a system cannot handle concurrent requests efficiently and thus, the data is transferred slower than needed;
- Memory leaks: the consumed memory is not released properly, which leads to poor performance and potential system failures.
- Database bottlenecks: occur with too slow database queries or with wrong indexing.
- CPU bottlenecks: the CPU reaches its maximum capacity and cannot effectively execute tasks, thus impacting the performance.
Why performance testing is important
After understanding the performance testing definition, let’s look at its main benefits. Performance testing in software testing helps detect any existing issues and eliminate the factors that negatively affect the performance of the app. However, it’s not the only reason why organizations decide to conduct this process:
- Preventing system’s failures and downtime: performance testing is a great proactive measure that helps address existing issues and vulnerabilities in advance and timely eliminate them, thus making the system more stable and robust.
- Preparing for growth: if you plan to scale your app, this method is a highly effective way to check how ready the app is for future growth and what should be optimized to help it scale in a painless way.
- Running software diagnostics: with the help of performance testing, organizations can regularly check their software and what kind of bottlenecks are to be eliminated.
- Verifying the vendor’s claims: in order to check whether the system meets the stated specifications, organizations often run performance tests.
- Collecting information for stakeholders: the results of performance tests are often presented to stakeholders as a proof of software’s stability and reliability.
- Improving user experience: modern users expect their software products to be highly responsive and swift. So if an app does not meet these requirements and takes too long to load or respond, users will most likely abandon it. Needless to say that decreased user engagement has a direct and negative impact on a business.
Whatever reason you have for conducting performance testing, remember that the process requires a well-planned approach and several preparatory steps – we’ll talk about them in detail a bit later. For now, let’s look at the types of performance testing that are available for consideration.
The main types of performance tests
In order to accurately estimate the app’s performance in specific conditions, you need to emulate these conditions – this is where different methods come in. While there are two main types (load and stress testing), there are several of their variants that organizations use, depending on their specific business needs.
This testing type evaluates how well a system performs under a specific (and anticipated) load by simulating the needed number of concurrent users and/or transactions. In this way, developers can identify potential bottlenecks that stop the system from scaling and implement needed optimizations. An interesting thing about load testing is that it can be applied to specific functionalities (i.e., a checkout) since for some parts of the app stellar performance and high responsiveness are critical.
Stress testing is aimed at evaluating the system’s performance under extreme and unexpected load volumes. In this way, developers estimate the app’s capacity limit and determine the breaking point where the system can no longer perform effectively. In addition to defining the app’s maximal capacity, stress testing also helps understand how long it will take the app to recover and what resources (CPUs, memory) are used the most.
There are two subsets:
- Soak testing (endurance testing): the number of users (the load volume) increases gradually;
- Spike testing: the number of users increases suddenly.
This technique, as the name implies, tests the app’s ability to scale up and down in correspondence with the changing number of users. During this process, the number of users is specified and does not usually go beyond maximal limits. One of the biggest advantages of scalability testing is that it helps better plan the capacity addition for further performance improvement.
Capacity testing is also aimed at estimating how well the app handles a specific number of users. The main difference here is that capacity testing estimates whether the system effectively handles the load that it was designed to. So in capacity testing you do not really go beyond the expected maximum – instead, you test the app’s capacity and whether anything prevents it from effective functioning.
The main metrics to consider and measure
There is a great variety of metrics that are usually measured and evaluated during the performance testing. Below, we list several examples:
- Latency: the delay between the request initiation and the received response;
- Response time: the time it takes for a system to respond to a user’s request;
- Throughput: the rate at which the application processes transactions or requests per unit of time;
- Error rate: the percentage of requests that result in errors or failures;
- Bandwidth: the volume of data (bits per second) that moves between workloads;
- Memory: the amount of storage space available for a processor;
- CPU interrupts per second: a number of interrupts a processor gets per second;
- Peak response time: the longest time a system needs to respond while being under the peak load;
- Session amounts: the maximum number of active sessions that can be open at once;
- Average load time: the average amount of time it takes for a request to be delivered.
As you can see, most of the metrics revolve around the speed of a system’s response and the time it needs. Naturally, the faster the system responds, the better the user experience is.
Performance testing process
While every software project is individual and hence requires an individual approach, there are several core performance testing stages that are to be included in every performance testing approach.
Definition of objectives and requirements
First, it is important to define clear requirements towards the testing process and establish objectives that you want to achieve. This includes understanding the expected load and defining thresholds and important metrics.
Creation of a test plan
Next, the testing team designs a test plan that normally describes the testing strategy, defines the testing environment, success criteria, and various test scenarios. By preparing a detailed test plan in advance, you ensure that your further activities will align with the business objectives and will help measure the right metrics.
Design of test scenarios
Think of test scenarios as of simulations of real-world user interactions with the system. In this way, a test scenario includes a set of actions that a user might perform and covers multiple aspects, like peak load, stress conditions, etc. This is needed to identify how an app performs in both expected and unexpected conditions.
Configuration of test environment
Since testing does not happen in the production environment, you need to mirror it in the testing environment by properly configuring the latter. This includes configuration of hardware and software as well as network.
Execution of tests
Once everything is ready, the testing team performs the tests. The process includes the application of the simulated load and measurement and collection of defined metrics. This is important so the team later analyzes the system’s performance, assembles a report and, based on that, performs retesting.
As already mentioned, during the execution of tests, the team monitors real-time results and collects them. This data is then thoroughly analyzed to identify the bottlenecks and areas for improvement.
Once the defined improvements are implemented and the system is analyzed, the team performs retesting to see whether optimization indeed addressed the existing issues. Some of the activities involved in optimization often include changes in code, configuration tweaks, or augmentation of the infrastructure.
Reporting and creation of documentation
After the system is retested and its performance meets all needed criteria, the team creates an extensive report where it summarizes key findings and provides recommendations for further optimization. The creation of documentation is very important as it helps monitor changes in the system behavior and serves as a source of truth for any further activities, while also adding transparency to the internal processes.
Now that we’ve answered the “what is performance testing?” question, it’s clear that performance testing is a highly effective method for understanding how your app performs and what can be improved. However, it is important to accurately design the tests and prepare everything in advance so that the testing process brings benefits, not additional challenges.
Jedwali la Vifaa Upimaji ni sehemu muhimu ya mchakato wa maendeleo kama inasaidia kuhakikisha ubora, usalama, na usability ya programu. Moja ya aina yake muhimu ni majaribio ya utendaji, lengo la kutathmini jinsi vizuri mfumo wako hufanya katika hali mbalimbali. Kwa hiyo, ni nini upimaji wa utendaji? Jinsi gani hasa kufanya kazi na nini metrics wewe ni kulipa tahadhari wakati wa mchakato? Soma hapa chini ili uone. Uchunguzi wa utendaji ni nini? Utafiti wa utendaji wa programu ni njia ya kupima kasi, majibu, uthabiti na kuegemea kwa programu. Lengo kuu la mchakato ni kuhakikisha ubora wa programu na kazi yake seamless katika hali mbalimbali. Kwa kuwa huwezi kutabiri kiasi halisi cha mzigo ambao unaweza kutokea, upimaji wa utendaji husaidia kuandaa programu kwa spikes za mzigo wa baadaye na kuhakikisha kazi yake bila kushindwa hata wakati wa mzigo mkubwa. "Kupitia njia hii, unaweza pia kutambua matatizo ya kawaida ya utendaji na bottlenecks kama vile: ""Nyakati ya kupakia ya muda mrefu: wakati mwingine programu inachukua muda mrefu kupakia na hiyo huathiri sana uzoefu wa mtumiaji.""" Sawa na muda mrefu mzigo, muda wa chini ya majibu athari uzoefu wa mtumiaji hasi. - Scalability chini: programu haiwezi kushughulikia mzigo kuongezeka na haiwezi kiwango sawa, ambayo husababisha kushindwa. - Low throughput: mfumo hawezi kushughulikia maombi wakati huo huo kwa ufanisi na hivyo, data ni kuhamishwa polepole kuliko inahitajika; - kumbukumbu kuvuja: kumbukumbu kutumika si iliyotolewa vizuri, ambayo inaongoza kwa utendaji duni na uwezekano wa kushindwa mfumo. - Database bottlenecks: kutokea na maswali ya database polepole sana au na indexing makosa. CPU bottlenecks: CPU hufikia uwezo wake wa juu na haiwezi kutekeleza kazi kwa ufanisi, hivyo kuathiri utendaji. Kwa nini upimaji wa utendaji ni muhimu Baada ya kuelewa ufafanuzi wa upimaji wa utendaji, hebu tuangalie faida zake kuu. Utafiti wa utendaji katika upimaji wa programu husaidia kugundua shida zozote zilizopo na kuondoa sababu ambazo zinaathiri vibaya utendaji wa programu. Hata hivyo, si sababu pekee kwa nini mashirika kuamua kufanya mchakato huu: - Kuzuia mfumo kushindwa na downtime: utendaji kupima ni hatua kubwa proactive kwamba husaidia kushughulikia masuala yaliyopo na udhaifu mapema na kwa wakati kuondoa yao, hivyo kufanya mfumo imara zaidi na imara. Ikiwa una mpango wa kuongeza kiwango cha programu yako, njia hii ni njia bora ya kuangalia jinsi programu iko tayari kwa ukuaji wa baadaye na nini kinachopaswa kuboreshwa ili kusaidia kupanua kwa njia isiyo na maumivu. - Kuendesha programu utambuzi: kwa msaada wa majaribio ya utendaji, mashirika inaweza mara kwa mara kuangalia programu yao na nini aina ya bottlenecks ni kuondolewa. - Kuthibitisha madai ya muuzaji: ili kuangalia kama mfumo hukutana na vipimo vilivyotajwa, mashirika mara nyingi huendesha vipimo vya utendaji. - Kukusanya habari kwa ajili ya wadau: matokeo ya vipimo vya utendaji ni mara nyingi kuwasilishwa kwa wadau kama uthibitisho wa utulivu na kuegemea programu. - Kuboresha uzoefu wa mtumiaji: watumiaji wa kisasa wanatarajia bidhaa zao za programu kuwa na majibu ya juu na ya haraka. Kwa hivyo, ikiwa programu haifai mahitaji haya na inachukua muda mrefu kupakia au kujibu, watumiaji wataacha. Ni lazima tuseme kwamba kupungua kwa ushiriki wa watumiaji kuna athari ya moja kwa moja na hasi kwa biashara. Chochote sababu una kwa ajili ya kufanya majaribio ya utendaji, kumbuka kwamba mchakato inahitaji mbinu vizuri iliyopangwa na hatua kadhaa za maandalizi - tutazungumza juu yao kwa undani kidogo baadaye. Kwa sasa, hebu kuangalia aina ya vipimo vya utendaji ambayo zinapatikana kwa kuzingatia. Aina kuu za vipimo vya utendaji Ili kukadiria kwa usahihi utendaji wa programu katika hali maalum, unahitaji kuiga hali hizi - hapa ndipo mbinu tofauti zinaingia. Ingawa kuna aina mbili kuu (mzigo na mkazo wa majaribio), kuna aina kadhaa ambazo mashirika hutumia, kulingana na mahitaji yao maalum ya biashara. Aina hii ya majaribio inathamini jinsi mfumo unavyofanya vizuri chini ya mzigo maalum (na unatarajiwa) kwa kuiga idadi inayohitajika ya watumiaji wa wakati mmoja na / au shughuli. Kwa njia hii, watengenezaji wanaweza kutambua bottlenecks uwezekano kwamba kuacha mfumo kutoka scaling na kutekeleza optimizations zinahitajika. Jambo la kuvutia kuhusu upimaji mzigo ni kwamba inaweza kutumika kwa utendaji maalum (yaani, malipo) tangu kwa baadhi ya sehemu ya programu stellar utendaji na high responsiveness ni muhimu. Mtihani wa mkazo ni lengo la kutathmini utendaji wa mfumo chini ya kiasi cha mzigo wa juu na zisizotarajiwa. Kwa njia hii, watengenezaji kukadiria app ya's kikomo cha uwezo na kuamua kuvunja uhakika ambapo mfumo haiwezi tena kufanya kwa ufanisi. Mbali na kufafanua uwezo wa juu wa programu, upimaji wa mkazo pia husaidia kuelewa ni muda gani itachukua programu kupona na ni rasilimali zipi (CPUs, kumbukumbu) zinazotumiwa zaidi. Kuna subsets mbili: - Soak kupima (uvumilivu kupima): idadi ya watumiaji (uzito kiasi) huongezeka hatua kwa hatua; - Spike kupima: idadi ya watumiaji huongezeka ghafla. Mbinu hii, kama jina linavyodokeza, mtihani uwezo wa programu ya kuongeza na kupunguza katika mawasiliano na idadi ya kubadilisha ya watumiaji. Wakati wa mchakato huu, idadi ya watumiaji ni maalum na si kawaida kwenda zaidi ya mipaka ya juu. Moja ya faida kubwa ya upimaji scalability ni kwamba inasaidia bora mpango uwezo kuongeza kwa ajili ya kuboresha utendaji zaidi. Upimaji wa uwezo pia unalenga kukadiria jinsi programu inavyoshughulikia idadi maalum ya watumiaji. Tofauti kuu hapa ni kwamba uwezo wa kupima makadirio kama mfumo ufanisi kushughulikia mzigo kwamba ilikuwa iliyoundwa kwa. Hivyo katika uwezo wa kupima wewe si kweli kwenda zaidi ya inatarajiwa kiwango cha juu <unk> badala yake, wewe mtihani uwezo wa programu na kama kitu chochote huzuia ni kutoka utendaji ufanisi. Metrics muhimu kuzingatia na kupima Kuna aina kubwa ya metrics kwamba ni kawaida kipimo na tathmini wakati wa majaribio ya utendaji. - Latency: muda kati ya maombi ya awali na majibu kupokea; - Wakati wa kujibu: muda inachukua kwa mfumo kujibu ombi la mtumiaji; - Kiwango cha upatikanaji: kiwango ambacho maombi ya mchakato shughuli au maombi kwa kitengo cha muda; - Kiwango cha makosa: asilimia ya maombi ambayo kusababisha makosa au kushindwa; - Bandwidth: kiasi cha data (bits kwa sekunde) kwamba husafiri kati ya kazi; - Memory: kiasi cha nafasi ya kuhifadhi inapatikana kwa processor; - CPU interrupts kwa sekunde: idadi ya interrupts processor anapata kwa sekunde; - Peak majibu wakati: muda mrefu zaidi mfumo anahitaji kujibu wakati kuwa chini ya mzigo wa kilele. Kama unaweza kuona, wengi wa metrics kuzunguka karibu kasi ya mfumo wa majibu na muda inahitaji. Bila shaka, kadiri mfumo unavyoitikia haraka, ndivyo uzoefu wa mtumiaji unavyozidi kuwa bora. Wakati kila mradi wa programu ni mtu binafsi na kwa hiyo inahitaji mbinu ya mtu binafsi, kuna hatua kadhaa za msingi za majaribio ya utendaji ambazo zinapaswa kuingizwa katika kila mbinu ya majaribio ya utendaji. Kuamua malengo na mahitaji Kwanza, ni muhimu kuamua mahitaji ya wazi kwa mchakato wa kupima na kuanzisha malengo ambayo unataka kufikia. Hii ni pamoja na kuelewa mzigo inatarajiwa na kufafanua mipaka na metrics muhimu. Kuunda mpango wa mtihani: Timu ya mtihani huunda mpango wa mtihani ambao kwa kawaida unaelezea mkakati wa mtihani, hufafanua mazingira ya mtihani, vigezo vya mafanikio, na hali mbalimbali za mtihani. Kwa kuandaa mpango wa majaribio ya kina mapema, unaweza kuhakikisha kwamba shughuli zako za baadaye zitakuwa sawa na malengo ya biashara na kusaidia kupima vipimo sahihi. Kubuni ya matukio ya mtihani Fikiria matukio ya mtihani kama simulations ya ulimwengu wa kweli watumiaji mwingiliano na mfumo. Kwa njia hii, hali ya mtihani inajumuisha seti ya vitendo ambavyo mtumiaji anaweza kufanya na inashughulikia mambo mengi, kama vile mzigo wa kilele, hali ya mkazo, nk. Hii inahitajika ili kutambua jinsi programu inafanya katika hali zote mbili zinazotarajiwa na zisizotarajiwa. Configuration ya mazingira ya majaribio: Kwa kuwa majaribio hayafanyika katika mazingira ya uzalishaji, unahitaji kuiga katika mazingira ya majaribio kwa kuanzisha kwa usahihi mwisho. Hii ni pamoja na usanidi wa vifaa na programu kama vile mtandao. Utekelezaji wa vipimo Mara kila kitu ni tayari, timu ya kupima hufanya vipimo. Mchakato ni pamoja na matumizi ya mzigo simulated na kipimo na ukusanyaji wa metrics maalum. Hii ni muhimu hivyo timu baadaye inachambua utendaji wa mfumo, hukusanya ripoti na, kulingana na hiyo, hufanya upimaji upya. Kama ilivyoelezwa hapo awali, wakati wa utekelezaji wa vipimo, timu inawakilisha matokeo ya wakati halisi na kuwakusanya. Takwimu hizi kisha kuchambuliwa kikamilifu kutambua bottlenecks na maeneo kwa ajili ya uboreshaji. Mara baada ya maboresho maalum ni kutekelezwa na mfumo ni kuchambuliwa, timu hufanya retesting kuona kama optimization kweli kushughulikiwa masuala yaliyopo. Baadhi ya shughuli zinazohusika katika optimization mara nyingi ni pamoja na mabadiliko katika kanuni, Configuration tweaks, au ongezeko la miundombinu. Baada ya mfumo ni retested na utendaji wake hukutana vigezo vyote zinahitajika, timu inajenga ripoti ya kina ambapo ina muhtasari matokeo muhimu na hutoa mapendekezo kwa ajili ya optimization zaidi. Uundaji wa nyaraka ni muhimu sana kwani husaidia kufuatilia mabadiliko katika tabia ya mfumo na hutumika kama chanzo cha ukweli kwa shughuli zozote za baadaye, wakati pia kuongeza uwazi kwa michakato ya ndani. Sasa kwa kuwa tumejibu swali "ni nini upimaji wa utendaji?" ni wazi kwamba upimaji wa utendaji ni njia yenye ufanisi sana ya kuelewa jinsi programu yako inafanya na nini inaweza kuboreshwa. Hata hivyo, ni muhimu kwa usahihi kubuni vipimo na kuandaa kila kitu mapema ili mchakato wa kupima huleta faida, si changamoto za ziada.