Pretraining for Sample-Efficient
Neural Interfaces

Ben Tang, Zachary Spalding, Gregory B. Cogan

Duke University

Simple methods trained on large amounts of data have produced powerful general models in language, audio, and vision. We hope the same can be done for the brain.

We introduce MAPA, a masked autoencoder for intracranial neural data. It learns directly from unlabeled recordings, producing neural representations that generalize across subjects.

Pretraining reduces labeled-data requirements

Across subjects, MAPA needs ~164 labeled trials to reach the accuracy that takes 3,500 trials without pretraining.

MAPA’s encoder is pretrained and then frozen for all evaluations. MAPA and the frontend baseline share their frontend and readout, so the gain comes from the pretrained encoder alone.

Each regime uses its own y-axis range.

Macro AUROC against the number of labeled trials on a log axis. The arrow marks the label saving against the frontend baseline fit on the full training set. MAPA is above the baseline at every label count we test, and the gain does not close as labels grow.

MAPA sets a new state of the art

MAPA sets a new state of the art across all three regimes of the Neuroprobe benchmark without fine-tuning: within-session, cross-session, and cross-subject.

Fit the readout on one subject. Test on other subjects.

Zero marks the frontend baseline. All three regimes use the same y-axis scale; unavailable results are omitted. Lap+STFT denotes Laplacian re-referencing followed by a spectrogram.

Neuroprobe leaderboard. Macro AUROC over 15 tasks. Only the best entry of each published method family is shown.

Generalization to held-out subjects

The strictest test of cross-subject transfer fits the readout on an anchor subject and evaluates on subjects held out from pretraining. A gain requires learning neural representations that generalize across subjects.

In the cross-subject regime, MAPA improves macro AUROC over the frontend baseline on all held-out subjects.

Macro AUROC by encoder depth on subjects excluded from pretraining.
Macro AUROC by encoder depth in the cross-subject regime, on the two subjects excluded from pretraining. Layer 0 is the frontend baseline.

Spatial encodings for cross-subject transfer

MAPA encodes each contact’s brain region and the relative positions of contacts along an array, giving the model spatial information shared across subjects.

Two spatial properties shared across subjects: arrays with differing placement sample the same neuroanatomical regions, and the same pattern of activity falls at different positions along the array in each subject.
(a) Arrays with differing placement across subjects sample the same neuroanatomical regions. (b) The same pattern of activity falls at different positions along the array in each subject.

MAPA architecture

MAPA is an extension of Masked Autoencoders (MAE) to intracranial neural recordings. The encoder, decoder, and objective follow the standard MAE design. We change only the patching and the two spatial encodings to fit iEEG data.

The masked autoencoder architecture: patches over contacts, bands and time are masked, an encoder sees the visible patches, and a small decoder reconstructs the removed ones.
During pretraining, a large random subset of patches over electrode contacts, frequency bands, and time is masked out. The encoder operates on the visible patches only, and a small decoder then processes the full set of encoded patches and mask tokens to reconstruct the frequency bins of the removed patches. Our two spatial encodings enter both the encoder and the decoder, and the decoder is discarded after pretraining.

Conclusion

Now is the time to scale. Intracranial recordings are accumulating en masse across institutions and experimental paradigms, and we can bring them together to train powerful general models for the brain.

And perhaps, a model trained on thousands of brains could not only bring us extraordinary neural interfaces but also tell us something about what it means to be human.

Citation

@misc{tang2026pretraining,
      title={Pretraining for Sample-Efficient Neural Interfaces},
      author={Ben Tang and Zachary Spalding and Gregory B. Cogan},
      year={2026},
      eprint={2609.13507},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2609.13507},
}

Acknowledgments and Disclosure of Funding

This work used the Delta and DeltaAI systems at the National Center for Supercomputing Applications through allocation CIS261108 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. This research used both the DeltaAI advanced computing and data resource, which is supported by the National Science Foundation (award OAC 2320345) and the State of Illinois, and the Delta advanced computing and data resource, which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta and DeltaAI are joint efforts of the University of Illinois Urbana-Champaign and its National Center for Supercomputing Applications. Z.S. and G.B.C. were supported by NIH R01DC019498. G.B.C. was supported by NIH R01NS129703.

We thank Geeling Chau, Christopher Wang, Shivashriganesh P. Mahato, Saba Hashemi, and Andrii Zahorodnii for helpful discussions.

Figure