Chinese Malware in Machine Learning Models

The new research, […] envisions a future where an organization may bring in an off-the-shelf machine learning model for any given task (say, a chat bot, or image detection) that could be loaded with malware while performing its task well enough not …


This content originally appeared on DEV Community and was authored by The Sharp Ninja

The new research, [...] envisions a future where an organization may bring in an off-the-shelf machine learning model for any given task (say, a chat bot, or image detection) that could be loaded with malware while performing its task well enough not to arouse suspicion.

By Radhamely De Leon: Researchers Hid Malware Inside an AI's 'Neurons' And It Worked Scarily Well

First, is it any surprise the Chinese did this research? If ever there was a case study for why ethics are necessary, China is exhibit A.

Second, this is done using steganography, which is a way to hide one message in another message. The message being concealed appears as noise in the data concealing it. This is how lossy compression formats are used to embed child porn in seemingly normal images. That there are over 6000 repositories on GitHub around this should be concerning to everyone.

Third, awareness needs to be front and center on the dangers of pre-generated ML Models. The truth is, nobody knows what's in them. ML models are typically not deterministic. Someone will have to figure out a way to prove mathematically how to determine if a model has been tampered with and if it is safe to use. Trusting the creator is more difficult than code because most of the time, the training is not deterministic and is highly susceptible to outside influences that cause each training run to produce slightly different neural networks. This eliminates a source of proof for an ML model being authentic or one hiding malware.

The only solution I see is for vendors to hash the neural net after building it and during packaging it and failing the packaging phase if they differ. It may even be necessary to encrypt the model as it is generated and to run the model from its encrypted state, only decrypting on the fly to retrieve nodes at runtime. Registering the hashes of models with a common registrar of know-good-models would help anti-virus software to block infected models from entering the network.


This content originally appeared on DEV Community and was authored by The Sharp Ninja


Print Share Comment Cite Upload Translate Updates
APA

The Sharp Ninja | Sciencx (2021-08-11T11:46:51+00:00) Chinese Malware in Machine Learning Models. Retrieved from https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/

MLA
" » Chinese Malware in Machine Learning Models." The Sharp Ninja | Sciencx - Wednesday August 11, 2021, https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/
HARVARD
The Sharp Ninja | Sciencx Wednesday August 11, 2021 » Chinese Malware in Machine Learning Models., viewed ,<https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/>
VANCOUVER
The Sharp Ninja | Sciencx - » Chinese Malware in Machine Learning Models. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/
CHICAGO
" » Chinese Malware in Machine Learning Models." The Sharp Ninja | Sciencx - Accessed . https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/
IEEE
" » Chinese Malware in Machine Learning Models." The Sharp Ninja | Sciencx [Online]. Available: https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/. [Accessed: ]
rf:citation
» Chinese Malware in Machine Learning Models | The Sharp Ninja | Sciencx | https://www.scien.cx/2021/08/11/chinese-malware-in-machine-learning-models/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts