Stopping Cyberattacks with AI
Math Plus Machine Learning Quickly Identifies and Deflects Malicious Traffic
The war in Ukraine has caused a dramatic rise in distributed denial-of-service (DDoS) cyberattacks worldwide, digital offensives that can bring down websites by overwhelming the targeted server with a deluge of internet traffic. Millions occur every year, with the number and size rising. Approximately a third of website downtime is due to DDoS attacks.
“DDoS cyberattacks are meant to generate chaos, disrupt institutions, and of course cause financial losses,” says Michał Karpowicz, a computer scientist and the director of research at NASK, Polish National Research Institute for Cybersecurity and AI. “And they are very common because they are relatively easy to generate.”
In a typical DDoS attack, a culprit uses many computers and online devices infected with specially designed malware. These devices can include gadgets on the Internet of Things (IoT)—kitchen appliances, security cameras, and thermostats—which are proliferating every day. Currently, the IoT contains more than 10 billion devices, a massive army awaiting recruitment. Attacks are typically one of two types, or a combination. In volumetric attacks, infected devices flood the targeted network with extreme traffic volumes at once, overloading its capabilities to serve regular users. In an application attack, they make requests that require the server to perform extensive computations, also overloading it.
Karpowicz describes the problem using a familiar scenario. Imagine you’re driving to the bank. There’s little traffic and everything is going smoothly until you reach the final intersection. Suddenly, everything jams up and you must wait. That’s a volumetric attack. Now imagine you have finally arrived at the bank and are standing in line for the teller. You notice that the customer in front of you has an unusual issue that requires the attention of every bank employee. Again, you must wait. That’s an application attack.
Malware can hide the source of the requests, making attacks hard to shut down. Services also offer attacks for hire, making them widely accessible. Such services have been launched by disgruntled employees, activists, market competitors, and nations. Karpowicz notes that during pandemic lockdowns, NASK detected the attacks launched from student networks at the beginning of lectures to halt online tests.
But recently Karpowicz has developed mathematical methods to detect such malicious requests and halt or redirect them. “When you are faced with a problem that is rooted in reality,” he says, “it gives you a lot of inspiration.”
Fingerprints on the Fly
Information on the internet, including requests from websites, consists of data packets. Each packet has a header that acts as a unique identifier describing its size, source, and destination. Typical defenses look at those headers to see if they’re on a ban-list. The defenses then redirect bad packets to harmless destinations.
The problem is that they must have this list beforehand, and attackers frequently change headers to escape detection. Karpowicz’s solution instead detects network traffic patterns in real time.
Imagine a series of vehicle traffic cameras at city intersections. If you look at a random sampling of video clips, you start to notice a sudden buildup of blue convertibles at a certain location. Suspicious. You can then look closely at the cars and learn more about them and potentially pull more of them over.
The method takes network data and translates it into signal data. The task then is to separate different signal sources. Karpowicz likens that to the cocktail party problem, where you’re at a party surrounded by conversations and must single out the words of the person speaking to you.
Here, he uses linear algebra. The signals, or packet headers, fill matrices. In a matrix, each row represents packets of a certain type over time, defined by packet size or source or some other factor.
In the vehicle traffic metaphor, you might have rows for blue vehicles, minivans, and cars from California. The values filling the cells in a given matrix indicate something that could be relevant to identifying an attack. In one matrix, each cell might contain the number of flags, and in another it might indicate bits per second. Karpowicz then deciphers which combinations of attributes are especially high in these metrics. Not just blue cars or convertibles, but blue convertibles.
Once he’s found those combinations of features, he translates them back to the networking world, generating rules for firewalls. In the vehicle traffic metaphor, if he now knows that blue cars of a certain make and model are causing the problem, he creates camera filters that spot those cars immediately. He calls his linear algebra–based method meta-factorization. A paper on meta-factorization is under review.
Karpowicz implements the calculations in MATLAB®. “Linear algebra algorithms are the fastest known algorithms in science,” he says, “making this method highly efficient.”
Detecting malicious packets is only part of the problem; defenders must also control them. Karpowicz says a solution came to him by accident. About a decade ago, he was working on a project to increase the energy efficiency of network devices. He wanted to predict traffic volume and redirect packet streams to routes that could handle them. While discussing the project with a colleague working on cybersecurity, the colleague said it might be used to fight DDoS attacks. “We were playing with the idea,” Karpowicz says, “and bit by bit, a new technology was being developed.”
In a paper published last year in the European Journal of Control, Karpowicz describes the result, a method called adaptive tuning. Typical network traffic control systems don’t receive feedback on their performance. They defend against attacks by stopping all the traffic somehow related to the attack. That is often too much, as it may also stop the legitimate traffic. Karpowicz suggests using traffic controllers that incorporate feedback so they know if they’re redirecting the right packets.
“Cybersecurity provided me with great scientific challenges, and in particular mathematical challenges,” he says. The theory and practicality fed off each other. “That’s what kept me going. I had this hunch that there’s something to it and there’s a problem that suggests solutions.” Together, the detection and control systems compose the basis of a service offered by NASK called FLDX, for which they now hold a patent. It can detect attacks within 5 seconds and start to mitigate them in 10 seconds.
NASK deploys FLDX across Poland, protecting nationwide networks using a distributed cluster of virtual machines. "What makes the entire solution unique is that it’s not only a cybersecurity system, but it’s also a research platform,” Karpowicz says. “It allows you to program your own algorithms for detecting and suppressing attacks in MATLAB. It allows you to access data that we collect in MATLAB, and you can use all the benefits of this technology to use data processing, signal processing, machine learning artificial intelligence (AI), based on traffic samples that we provide.”
Both the detection and adaptive tuning algorithms use machine learning, specifically a kind called semi-supervised learning, which doesn’t require much hand-labeled data for training. “We encode a little bit of expert knowledge in the learning algorithms and then let them do their job,” Karpowicz says. “And their job is to discover what is going on in the world.”
They find statistical regularities in large amounts of data, like spotting clusters of a certain type of car. “It is machine learning at its best,” he says. “We don’t need to begin by collecting huge data sets in order to tune the system before it becomes operational. It is ready to work the moment it is installed.”
The detector uses a “voting” system to identify anomalous data flows. It also optimizes matrices by solving sets of equations. Even if network behavior is nonlinear—where some factors have disproportional effects—linear equations can capture the bulk of the patterns.
Researchers and engineers can use MATLAB to access NASK’s data set, and because many technical universities already use it, “that’s something that is very convenient.” NASK’s data set is unique because it consists of traffic data from all across the country and has a high rate of data sampling.
NASK’s research looks at traffic dynamics, target vulnerabilities, and attack sources, frequency, and methods. And they use FLDX to protect public and commercial clients, including schools and vaccination enrollment services. Earning the trust of his cybersecurity colleagues has been “a great surprise,” Karpowicz says. “It takes time for scientists to become respected in the cybersecurity engineering department. You have to show that you understand what they are doing and that you do something useful.” But with FLDX, “you get access to this playground where you can actually make things happen. And that’s something very special about NASK. We have a very short path from the laboratory to the technology and deployment.”
Karpowicz is now collaborating with other institutions, including MIT and the University of Technology Sydney. “I want NASK to be known worldwide for cybersecurity and AI,” he says. It’s on its way.
"Kukomesha mashambulizi ya mtandao na AI Math + Machine Learning: Kuamua na kuondoa trafiki mbaya ""Vita vya Ukraine vimesababisha ongezeko kubwa la mashambulizi ya mtandao ya kukataa huduma (DDoS) ulimwenguni kote, mashambulizi ya dijiti ambayo yanaweza kuangusha tovuti kwa kuzidisha seva inayolengwa na mafuriko ya trafiki ya mtandao." Mamilioni hutokea kila mwaka, na idadi na ukubwa wake unaongezeka. Karibu theluthi moja ya muda wa kushindwa kwa tovuti ni kwa sababu ya mashambulizi ya DDoS. "Mashambulizi ya kibiashara ya DDoS yanakusudiwa kutokeza machafuko, kuvuruga taasisi, na bila shaka kusababisha hasara za kifedha,"" anasema Michał Karpowicz, mwanasayansi wa kompyuta na mkurugenzi wa utafiti katika NASK, Taasisi ya Kitaifa ya Utafiti wa Usalama wa Kibiashara na AI ya Poland." "Na ni ya kawaida sana kwa sababu ni rahisi kuzalisha. "" Katika shambulio la kawaida la DDoS, mhalifu hutumia kompyuta nyingi na vifaa vya mtandaoni vilivyoambukizwa na programu hasidi iliyoundwa maalum." Vifaa hivi vinaweza kujumuisha vifaa kwenye Mtandao wa Vitu (IoT) - vifaa vya jikoni, kamera za usalama, na thermostats - ambazo zinaongezeka kila siku. Kwa sasa, IoT ina zaidi ya vifaa bilioni 10, jeshi kubwa linalosubiri kuajiriwa. Kwa kawaida mashambulizi ni ya aina mbili, au ya aina mbili pamoja. Katika mashambulizi ya volumetric, vifaa vilivyoambukizwa hufurika mtandao uliolengwa na kiasi kikubwa cha trafiki kwa wakati mmoja, vikizidisha uwezo wake wa kuwahudumia watumiaji wa kawaida. Katika shambulio la maombi, wao kufanya maombi ambayo zinahitaji seva kufanya hesabu kubwa, pia overloading yake. Karpowicz anaeleza tatizo hilo kwa kutumia hali inayotambulika. Hebu wazia kwamba unasafiri kwa gari kwenda benki. Kuna trafiki kidogo na kila kitu kinakwenda vizuri mpaka ufikie makutano ya mwisho. Kwa ghafula, kila kitu hufadhaika na ni lazima usubiri. Hiyo ni shambulio volumetric. Sasa fikiria kwamba umefika benki na unasimama katika mstari wa mteja. Unatambua kwamba mteja mbele yako ana suala lisilo la kawaida ambalo linahitaji uangalifu wa kila mfanyakazi wa benki. Kwa mara nyingine tena, ni lazima usubiri. Hiyo ni shambulio la maombi. Programu hasidi inaweza kuficha chanzo cha maombi, na kufanya mashambulizi kuwa magumu kuzima. Huduma pia kutoa mashambulizi kwa ajili ya kukodisha, kufanya yao kwa wingi kupatikana. Huduma hizo zimezinduliwa na wafanyakazi waliokasirika, wanaharakati, washindani wa soko, na mataifa. Karpowicz anasema kuwa wakati wa kuzuiwa kwa janga hilo, NASK iligundua mashambulizi yaliyozinduliwa kutoka kwa mitandao ya wanafunzi mwanzoni mwa mihadhara ili kusimamisha vipimo vya mtandaoni. Lakini hivi karibuni Karpowicz amebuni mbinu za hisabati za kugundua maombi hayo yenye madhara na kuyasimamisha au kuyaelekeza. "Wakati unakabiliwa na tatizo ambalo lina mizizi yake katika hali halisi, ""anasema, ""linakupa msukumo mwingi. "" Habari za Fingerprints on the Fly kwenye mtandao, ikiwa ni pamoja na maombi kutoka kwa tovuti, zinajumuisha pakiti za data. Kila pakiti ina kichwa ambacho hutenda kama kitambulisho cha kipekee kinachoelezea ukubwa wake, chanzo, na marudio. Ulinzi wa kawaida kuangalia vichwa hivyo kuona kama wao ni juu ya marufuku-orodha. Kisha ulinzi huo huelekeza tena pakiti mbaya kwenye maeneo yasiyo na madhara. Tatizo ni kwamba lazima wawe na orodha hii mapema, na washambuliaji mara nyingi hubadilisha vichwa ili kuepuka kugunduliwa. Karpowicz's ufumbuzi badala yake hugundua mifumo ya trafiki ya mtandao katika muda halisi. Hebu wazia kamera kadhaa za usafiri wa magari kwenye makutano ya jiji. Kwa mfano, ikiwa unaangalia sampuli ya video, unaweza kuona idadi kubwa ya magari ya bluu kwenye eneo fulani. Ni jambo lenye kutilika shaka. Unaweza kuangalia kwa karibu magari na kujifunza zaidi juu yao na uwezekano wa kuvuta zaidi yao juu. Mbinu inachukua data ya mtandao na kutafsiri katika data ishara. Kazi basi ni kutenganisha vyanzo tofauti vya ishara. Karpowicz anafananisha hilo na tatizo la sherehe ya cocktail, ambapo uko kwenye sherehe umezungukwa na mazungumzo na lazima uchague maneno ya mtu anayezungumza nawe. Hapa, yeye hutumia linear algebra. Ishara, au vichwa vya pakiti, hujaza matrix. Katika matrix, kila safu inawakilisha pakiti za aina fulani kwa muda, iliyofafanuliwa na ukubwa wa pakiti au chanzo au sababu nyingine. Katika mfano wa trafiki ya magari, unaweza kuwa na safu kwa ajili ya magari ya bluu, minivans, na magari kutoka California. Thamani kujaza seli katika matrix fulani kuonyesha kitu ambacho inaweza kuwa muhimu kwa kutambua shambulio. Katika matrix moja, kila seli inaweza kuwa na idadi ya bendera, na katika nyingine inaweza kuonyesha bits kwa sekunde. Karpowicz kisha deciphers ambayo mchanganyiko wa sifa ni hasa juu katika metrics hizi. Si tu magari ya bluu au convertibles, lakini convertibles bluu. Mara baada ya yeye amepata wale mchanganyiko wa vipengele, yeye hutafsiri yao nyuma kwa dunia ya mitandao, kuzalisha sheria kwa firewalls. Katika mfano wa magari, kama anajua kwamba magari ya rangi ya bluu ya aina fulani na mfano ni kusababisha tatizo, yeye huunda filters kamera kwamba spot magari hayo mara moja. Yeye huita mbinu yake ya msingi wa algebra ya linear meta-factorization. Karatasi juu ya meta-factorization ni chini ya mapitio. Karpowicz kutekeleza mahesabu katika MATLAB®. <unk>Algorithms linear algebra ni algorithms ya haraka inayojulikana katika sayansi,<unk> anasema, <unk>kufanya mbinu hii ufanisi sana.<unk> Kugundua pakiti mbaya ni sehemu tu ya tatizo; watetezi lazima pia kudhibiti yao. Karpowicz anasema suluhisho lilimjia kwa aksidenti. Karibu miaka kumi iliyopita, alikuwa akifanya kazi kwenye mradi wa kuongeza ufanisi wa nishati ya vifaa vya mtandao. Alitaka kutabiri kiasi cha trafiki na kuelekeza mito ya pakiti kwenye njia ambazo zingeweza kuzishughulikia. Wakati wa kujadili mradi huo na mwenzake anayefanya kazi kwenye usalama wa mtandao, mwenzake alisema inaweza kutumika kupambana na mashambulizi ya DDoS. "Tulianza kufanya kazi na wazo hilo, na hatua kwa hatua, teknolojia mpya ilianzishwa,"" Karpowicz anasema katika makala yake iliyochapishwa mwaka jana katika jarida la European Journal of Control, akifasiri matokeo ya utafiti huo, ambayo ni mbinu inayoitwa ""adaptive tuning.""" Mifumo ya kawaida ya kudhibiti trafiki ya mtandao haipati maoni juu ya utendaji wao. Wao kujikinga dhidi ya mashambulizi kwa kuacha trafiki yote kwa njia fulani kuhusiana na shambulio. Mara nyingi hilo ni jambo kubwa sana, kwa kuwa huenda pia likazuia trafiki halali. Karpowicz anapendekeza kutumia udhibiti wa trafiki ambao hujumuisha maoni ili waweze kujua ikiwa wanaelekeza pakiti sahihi. "Usalama wa mtandao ulinipa changamoto kubwa za kisayansi, na haswa changamoto za hisabati", anasema. Nadharia na utendaji wa kazi ziliongezana. <unk>Hiyo ndiyo iliyonifanya niendelee. "Nilikuwa na hisia kwamba kuna kitu katika hilo na kuna tatizo ambalo linaonyesha suluhisho. "" Pamoja, mifumo ya kugundua na kudhibiti huunda msingi wa huduma inayotolewa na NASK inayoitwa ""FLDX,"" ambayo sasa wana hati miliki." Inaweza kugundua mashambulizi ndani ya sekunde tano na kuanza kuwakomesha katika sekunde kumi. NASDAQ inatoa huduma ya FLDX nchini Poland, kulinda mitandao ya kitaifa kwa kutumia kundi la mashine za virtual. "Kile kinachofanya suluhisho lote kuwa la kipekee ni kwamba si mfumo wa usalama wa mtandao tu, lakini pia ni jukwaa la utafiti", Karpowicz anasema. Inakuwezesha programu algorithms yako mwenyewe kwa ajili ya kugundua na kukandamiza mashambulizi katika MATLAB. "Kama vile, kwa mfano, ""Matlab"" ni mfumo wa kujifunza mashine, ambayo hutumia teknolojia ya kuhariri, kuhariri, na kujifunza mashine, na ina uwezo wa kuhariri data kutoka kwa mashine, na kwa hivyo ina uwezo wa kuhariri data kutoka kwa mashine, na kwa hivyo ina uwezo wa kuhariri data kutoka kwa mashine, na kwa hivyo ina uwezo wa kuhariri data kutoka kwa mashine." "Tunaboresha ujuzi wa wataalamu katika algorithms za kujifunza na kisha kuwaacha wafanye kazi yao", Karpowicz anasema. <unk>Na kazi yao ni kugundua nini kinaendelea katika dunia.<unk> Wao kupata utaratibu wa takwimu katika kiasi kikubwa cha data, kama spotting makundi ya aina fulani ya gari. "Ni kujifunza mashine katika bora yake", anasema. <unk>Hatuna haja ya kuanza kwa kukusanya seti kubwa za data ili kurekebisha mfumo kabla ya kuwa kazi. Ni tayari kufanya kazi wakati ni imewekwa. <unk> Detector anatumia <unk> kupiga kura <unk> mfumo wa kutambua mtiririko wa data isiyo ya kawaida. Pia optimizes matrices kwa kutatua seti ya equations. Hata kama tabia ya mtandao ni nonlinear-ambapo baadhi ya mambo na athari disproportionate-linear equations inaweza kukamata kiasi kikubwa cha mifumo. Watafiti na wahandisi wanaweza kutumia MATLAB kupata seti ya data ya NASDAQ, na kwa sababu vyuo vikuu vingi vya kiufundi tayari vinatumia, hiyo ni kitu rahisi sana. seti ya data ya NASDAQ ni ya kipekee kwa sababu inajumuisha data ya trafiki kutoka kote nchini na ina kiwango cha juu cha sampuli ya data. Utafiti wa NASK unaangalia mienendo ya trafiki, udhaifu wa lengo, na vyanzo vya shambulio, mzunguko, na mbinu. FLDX inatumika kulinda wateja wa umma na kibiashara, pamoja na shule na huduma za uandikishaji wa chanjo. Kupata uaminifu wa wenzake wa usalama wa mtandao imekuwa "mashangao makubwa", Karpowicz anasema. <unk>Inachukua muda kwa wanasayansi kuwa kuheshimiwa katika idara ya uhandisi wa usalama wa mtandao. "Lakini kwa FLDX, unapata ufikiaji wa uwanja huu wa michezo ambapo unaweza kufanya mambo yatokee.""" Na hiyo ni kitu maalum sana kuhusu NASK. "Kama ilivyoelezwa hapo juu, ""Kupitia utafiti wa maabara, Karpowicz amekuwa akishirikiana na taasisi nyingine, ikiwa ni pamoja na MIT na Chuo Kikuu cha Teknolojia cha Sydney." "Nataka NASK ijulikane ulimwenguni kote kwa usalama wa mtandao na AI", anasema. Ni juu ya njia yake.