Swin-Vision Transformer for edge device

Linearized self-attention in Transformers for Jetson Nano achieving 14-20% memory and 2-3% latency savings

Leveraging (q, k, v) low-rank structures, applied LoRa approx. to the self-attention weight matrices and harnessed SVD in attention block for efficient matrix operations allowed optimizing vision transformers saving, 18% GFLOPS with 2% accuracy drop

~ More description and cool demos coming soon