Subject: Cutting-Edge Research in Signal Processing and Machine Learning

Hi Elman,

This newsletter explores recent advancements in signal processing and machine learning across wireless communications, medical diagnostics, and remote sensing.

General Overview

This collection of preprints showcases diverse applications of signal processing and machine learning. Several papers focus on enhancing wireless system performance through innovative antenna designs and signal processing techniques. Wu et al. (2024) Wu et al. (2024) introduce a rotatable antenna (RA) model, optimizing 3D antenna orientations to maximize SINR. For secure communication in NOMA networks, Mai et al. (2024) Mai et al. (2024) propose a secure beamforming design for fluid antenna systems (FAS) to enhance secrecy rates. Yao et al. (2024) Yao et al. (2024) also leverage FAS for improved spectrum sensing in cognitive radio networks. In the realm of mmWave technology, Rivetti et al. (2024) Rivetti et al. (2024) investigate clutter-aware target detection in cell-free massive MIMO ISAC systems, exploring the impact of clutter subspace exploitation and dedicated sensing streams on detection probability. Finally, Di et al. (2024) Di et al. (2024) tackle the joint optimization of UAV trajectory and precoder design for MIMO relay systems with finite-alphabet inputs.

Machine learning also plays a prominent role in enhancing signal analysis and system optimization. Tang et al. (2024) Tang et al. (2024) introduce bilateral signal warping (BSW) for improved ECG-based Left Ventricular Hypertrophy diagnosis by creating prototypes for normal and LVH heartbeats. Kela et al. (2024) Kela et al. (2024) develop practical deep learning schedulers for cellular radio resource allocation, showing improved performance over heuristic methods in 5G system-level simulations. Svirin et al. (2024) Svirin et al. (2024) utilize machine learning for signature diagnostics of three-phase motors, combining supervised and unsupervised approaches for improved accuracy. For enhanced security in IIoT, Meng et al. (2024) Meng et al. (2024) propose a Temporal Dynamic Graph Convolutional Network (TDGCN) for physical-layer authentication, leveraging IRSs and GNNs to capture spatio-temporal dynamics of CSI fingerprints. Ji et al. (2024) Ji et al. (2024) utilize transfer learning for noise reduction in automatic modulation classification, achieving better accuracy in low SNR scenarios. Alikhani et al. (2024) Alikhani et al. (2024) introduce the Large Wireless Model (LWM), a foundation model for wireless channels, generating channel embeddings that improve performance in various downstream tasks.

Addressing specific signal processing challenges, Ezzeddine et al. (2024) Ezzeddine et al. (2024) explore robust lossy compression of heavy-tailed sources. Cao et al. (2024) Cao et al. (2024) introduce dynamic angle fractional Fourier division multiplexing (DA-FrFDM) for PAPR reduction in multi-carrier systems. Luo et al. (2024) Luo et al. (2024) propose a novel nonlinear SCMA codebook design. Ozden et al. (2024) Ozden et al. (2024) introduce double media-based modulation (DMBM) for high-rate wireless communication. Kaplan et al. (2024) Kaplan et al. (2024) investigate reducing dynamic range in bistatic backscatter communication.

Practical applications are highlighted through novel system designs and datasets. Benmessaoud et al. (2024) Benmessaoud et al. (2024) create a high-quality ECG dataset. Kong et al. (2024) Kong et al. (2024) propose a time-varying spectral approach for characterizing ECG-derived skin nerve activity. MacWilliam et al. (2024) MacWilliam et al. (2024) develop a state-space estimation method for room impulse responses. Janowski et al. (2024) Janowski et al. (2024) present remote sensing datasets of the Puck Lagoon. Mahdavihaji et al. (2024) Mahdavihaji et al. (2024) analyze blockage prediction in mmWave/sub-THz communication. Chen et al. (2024) Chen et al. (2024) propose a sensing-assisted beam tracking scheme for THz communications. Lyu et al. (2024) Lyu et al. (2024) present a hybrid channel model for THz monostatic sensing.

Finally, several papers explore algorithmic advancements. Saini et al. (2024) Saini et al. (2024) discuss the Min-Max Framework for Majorization-Minimization (MM4MM) algorithms. Furudoi et al. (2024) Furudoi et al. (2024) investigate discrete-valued signal estimation. Lyu et al. (2024) Lyu et al. (2024) study waveform design for ISAC systems. Bui et al. (2024) Bui et al. (2024) analyze time-constrained federated learning in IoT. Kaneko et al. (2024) Kaneko et al. (2024) propose a multiscale graph construction method. Bozkus et al. (2024) Bozkus et al. (2024) analyze coverage in Q-learning. Li et al. (2024) Li et al. (2024) introduce SPDIM for domain adaptation in EEG. Kang et al. (2024) Kang et al. (2024) propose a split federated fine-tuning framework with differential privacy. Fan et al. (2024) Fan et al. (2024) introduce DEFINED for symbol detection. McKnight et al. (2024) McKnight et al. (2024) develop a self-supervised learning approach for defect segmentation. Liu et al. (2024) Liu et al. (2024) investigate resource allocation in AI-enabled wireless networks. Zhang et al. (2024) Zhang et al. (2024) study sum rate maximization for RSMA systems.

Paper Highlights

Large Wireless Model (LWM): A Foundation Model for Wireless Channels

Large Wireless Model (LWM): A Foundation Model for Wireless Channels by Sadjad Alikhani, Gouranga Charan, Ahmed Alkhateeb https://arxiv.org/abs/2411.08872

Caption: The architecture of the Large Wireless Model (LWM) uses a transformer encoder to generate channel embeddings from patched channel data. Pre-training utilizes Masked Channel Modeling (MCM) to reconstruct masked patches, and the resulting embeddings are then used for downstream tasks like beam prediction and channel classification. The diagram illustrates the flow of data from raw channel input through patch generation, embedding, transformer encoding, and finally to downstream tasks and pre-training objectives.

This paper introduces the Large Wireless Model (LWM), a groundbreaking foundation model for wireless channels. Designed to be task-agnostic, LWM can be applied to various downstream tasks without needing task-specific training. It achieves this by generating universal, rich, and contextualized channel embeddings (features) using a transformer-based architecture, drawing inspiration from successful foundation models in NLP and computer vision. LWM's pre-training utilizes a novel self-supervised technique called Masked Channel Modeling (MCM). This involves masking portions of the channel data and training the model to reconstruct the missing parts, allowing it to learn intricate spatial and temporal relationships within the data without requiring large labeled datasets, a significant advantage given the scarcity and cost of labeled wireless data.

The methodology involves processing the channel matrix H ∈ C^M×N by dividing it into patches, separating real and imaginary components, and flattening them into vectors. These vectors are further divided into P patches of length L = 2MN/P. The MCM technique masks 15% of the real part patches (80% fully masked, 10% replaced with random vectors, 10% unchanged), with corresponding imaginary patches masked similarly. A CLS (classification) patch, prepended to the sequence of channel patches, acts as a global context aggregator. These patches are linearly embedded, combined with positional encodings, and fed into the transformer encoder. The model minimizes the mean squared error (MSE) between the reconstructed masked patches and their original values: C_MCM = (1/|M|) Σ_i∈M ||W^dece^LWM_i - p_i||², where M is the set of masked patches, e^LWM_i is the embedding of the i-th masked patch, W^dec is the weight matrix of the linear layer, and p_i is the original value of the i-th patch.

The results show consistent improvements in classification and regression tasks using LWM embeddings compared to raw channel representations, especially with complex tasks and limited datasets. For instance, in beam prediction, LWM embeddings consistently outperform raw channels, achieving benchmark performance with significantly less data. In LoS/NLoS classification, LWM achieves a high F1-score with very few training samples. Furthermore, in robust beamforming, LWM maintains robust performance even with masked test channels.

The patch-based processing offers computational efficiency and captures both local and global relationships within the data, aligning well with the multi-head attention mechanism. Compared to autoencoders, LWM demonstrates superior ability to capture complex dependencies and generalize across tasks due to its self-supervised learning objective and use of self-attention.

LWM offers a robust, adaptable, and efficient framework for feature extraction and task-solving in wireless communication and sensing. Its potential to revolutionize the field lies in its ability to act as a universal feature extractor, readily adaptable to various downstream tasks with minimal retraining.

Time-constrained Federated Learning (FL) in Push-Pull IoT Wireless Access

Time-constrained Federated Learning (FL) in Push-Pull IoT Wireless Access by Van Phuc Bui, Junya Shiraishi, Petar Popovski, Shashi Raj Pandey https://arxiv.org/abs/2411.08607

Caption: This figure illustrates the time-shared FL framework with push-pull communication. The frame T_frame is divided into slots for broadcasting global parameters, reserved pull slots T_Q, and shared push slots T_c, where UEs contend using framed-ALOHA. This framework allows the PS to strategically pull updates from selected UEs while accommodating others to push their updates, balancing resource utilization and model generalization.

Federated Learning (FL) at the network edge faces challenges due to limited transmission resources and device heterogeneity. Existing device scheduling methods struggle to balance model performance with time constraints, particularly in diverse IoT environments. This paper proposes a novel approach using push-pull communication in time-constrained FL, enabling devices to both proactively transmit updates (push) and respond to server queries (pull). This aims to improve model generalization by incorporating diverse data distributions and accommodating stragglers in random access.

The system integrates push-pull communication within a time-shared FL framework. Each time frame is divided into pull-based and push-based communication slots. In the pull phase, the Parameter Server (PS) strategically selects a subset Z of User Equipments (UEs) based on a utility function to solicit their local model updates. Remaining UEs transmit updates in the push phase using framed-ALOHA, contending for available slots T_c = T_frame - T E[Z]. This dynamic interplay necessitates careful resource allocation and scheduling.

A utility-based analytical model investigates performance trade-offs and resource requirements. The utility function U(.) evaluates each UE's contribution to global model accuracy using an approximation of Shapley Value (GTG-Shapley). The PS aims to minimize the global loss function F(x; [K]) subject to constraints on selected UEs (Σ_k=1^K α_k < Z), latency (T_cost ≤ T_max), and target accuracy (x ∈ arg min F(x; [K]) ≤ θ_th). The analysis considers local training latency (T_k,comp) and transmission success probability in the push phase.

Experiments on MNIST and CIFAR-10 datasets with K=200 UEs demonstrate the approach's effectiveness. Balanced push-pull slot allocation achieves higher accuracy and faster convergence than random selection or centralized approaches. The proposed method outperforms existing greedy Shapley-based selection in terms of latency, particularly for high target accuracy. Increasing push UEs negatively impacts accuracy due to collisions, highlighting the importance of balanced resource allocation.

Conclusion

This newsletter highlighted two impactful papers addressing critical challenges in signal processing and machine learning. The Large Wireless Model (LWM) introduces a novel approach to learning robust representations of wireless channels, enabling efficient adaptation to diverse downstream tasks with limited data. This work opens up exciting possibilities for building more intelligent and adaptable wireless systems. Complementing this, the research on push-pull federated learning presents a practical solution for training models in resource-constrained IoT environments. By strategically balancing push and pull communication, this approach effectively manages limited resources while ensuring model generalization and timely convergence. Both papers offer significant contributions to their respective fields, paving the way for further advancements in wireless communication, sensing, and distributed machine learning.