ECG-Language-Models
A research framework for fine-tuning and evaluating ECG-language models, which combine electrocardiogram signals with text for tasks such as ECG question answering.
This framework supports multiple model architectures, training objectives and data representations for ECG-language models, with distributed training built in. It expects ECGs to already be preprocessed by a companion ECG-Preprocess tool, and links to a separate ECG-Neural-Networks project for anyone who wants to pretrain an ECG encoder first rather than start from a language-model backbone.
It includes a Rust-based byte-pair-encoding tokenizer, built with maturin, for a symbolic ECG data representation, and loads a standard set of published Hugging Face datasets covering ECG question-answering and instruction data built on PTB-XL, MIMIC-IV-ECG and other sources, with support for mixing multiple datasets in one training run by specifying several dataset names at once.
It was developed and tested with PyTorch 2.9 and CUDA 12.8 on H100 NVL GPUs, and the project describes its current status as beta, inviting contributions of new features, models and datasets.