This content originally appeared on HackerNoon and was authored by Computational Technology for All
:::info Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu).
:::
Table of Links
8 Evaluation of LifeHD semi and LifeHDa
9 Discussions and Future Works
10 Conclusion, Acknowledgments, and References
6 VARIANTS OF LIFEHD
While LifeHD is designed to cater to general IoT applications with streaming input and without supervision, real-world scenarios may vary. Some scenarios might have a few labeled samples in addition to the unlabeled stream, while others may require operation within strict power constraints. LifeHD offers extensibility to address these diverse needs. In this section, we introduce two software-based extensions: LifeHDsemi, which adds a separate processing path to manage labeled samples, and LifeHDa, which adaptively prunes the HDC model using masking to handle low-power scenarios.
6.1 LifeHDsemi
\
![Figure 7: Left: An example of the impact of masking on HDC. Test on CIFAR-10 [29]. Right: The intuitions behind the design of LifeHDa.](https://cdn.hackernoon.com/images/fWZa4tUiBGemnqQfBGgCPf9594N2-gm930yf.png)
6.2 LifeHDa
While HDC computation is typically lightweight, there may be instances of energy scarcity (e.g., when powered by a solar panel) that call for a balance between accuracy and power efficiency. Similar to neural networks, one approach is to prune the HDC model using a mask, retaining the most crucial HDC dimensions postencoding [25]. Dimension importance can be determined by aggregating all class hypervectors into one and sorting the values across all dimensions. Notably, direct reduction of the encoding dimension should be avoided, as it can degrade HDC’s expressive capability and end up with corruption. Fig. 7 (left) visually demonstrates the impact of masking in supervised HDC tasks: retaining the top 6000 bits incurs only a 3% accuracy loss compared to using the full 10K-bit precision.
\ While the concept of masking has been employed in prior HDC studies [25], they are not directly applicable to LifeHD due to their offline training setting with iid data. With streaming non-iid data in LifeHD, the set of observed cluster HVs may only represent a subset of the potential classes, and the less significant bits could become crucial as new classes are introduced.
\ We introduce LifeHDa, which enhances LifeHD through an adaptive masking approach applied to all cluster HVs in working and long-term memory. Let 𝐷𝑎 represent the target dimension for reduction. The rationale behind LifeHDa is depicted in Fig. 7 (right). Whenever an original LifeHD detects novelty, we temporarily revert to the full dimension 𝐷 for 2 batches, which is sufficient for LifeHD to consolidate new patterns in its memory. After these two batches, we assess the long-term memory cluster HVs by aggregating them and ranking the dimensions, and derive a mask retaining 𝐷𝑎 dimensions with the largest absolute values. This mask is then applied to the following batches of 𝜙 (𝑋) immediately after encoding, up until the next novelty is detected. Novelty detection is executed with the masked hypervectors. Importantly, LifeHDa can utilize the same novelty detection sensitivity as LifeHD since the most significant dimensions dominate the similarity check. In other words, the similarity results in LifeHDa using 𝐷𝑎 dimension are similar as using the full dimension. LifeHDa offers an adaptive HDC model pruning interface with minimal accuracy loss and overhead.
\
:::info This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.
:::
\
This content originally appeared on HackerNoon and was authored by Computational Technology for All
Computational Technology for All | Sciencx (2024-07-24T17:00:21+00:00) Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Variants of LifeHD. Retrieved from https://www.scien.cx/2024/07/24/lifelong-intelligence-beyond-the-edge-using-hyperdimensional-computing-variants-of-lifehd/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.