Reconceptualizing neural function as
high-dimensional brain state dynamics

Adina Wagner, M.Sc.
mas.to/@adswa
Institute for Experimental Psychology,
HHU Düsseldorf


Psychoinformatics lab,
Institute of Neuroscience and Medicine, Brain & Behavior (INM-7)
Research Center Jülich

Consider an example

Delayed Decision Making

Mental representations

Dynamic population codes represent working memory content.

(adapted from Meyers, 2018)

This can be reconceptualized as trajectories in a high-dimensional space.

Investigating these trajectories during decision making might reveal underlying brain states and their transitions.

Project outline

We can model brain activity in an $n$-dimensional space, where $n$ is the number of measurements (e.g., voxels/sensors/electrodes).
Assumption:
When the same event is experienced, exact brain activity may differ anatomically, but should correspond to similar cognitive processes.
Idea:
To functionally align brain activity, we align the vector representations of the signals.
Aim:
Assign meaning to the axis of the shared space.

Study and data overview

  • Acquired 2016/17 at OvGU
    Magdeburg by Kaiser et al.
  • 22 participants
  • 510 trials; delayed decision making task
  • 9 pre-learned magnitude (frequency)/
    probability (angle) combinations
  • MEG acquisition on Elekta Neuromag System (306 channels)

The Shared response model (SRM)

Each recording's $i$'s data is a matrix of dimensions sensors $\times$ time-points $X_i$.
SRM (Chen et al., 2015) models neural responses as a recording-specific base $W_i$
and shared components over all recordings' responses $S$.

SRM identifies common activity patterns across recordings (e.g., participants), and provides a
method to transform activity into a lower-dimensional shared latent component space.

\[\begin{aligned} min_{w_i, s}\sum_i{\|X_i - W_iS \|}^2_F \\ s.t. W^T_iW_i = I_k \end{aligned} \]

SRM Simulation

Generate a ground-truth
signal and 306 "sensors"
of pure noise. 		For a proportion of sensors,
weight the signal (random
weight [0, 1]) and add it.
 Repeat to generate
artificial time series
for $N$ recordings.

A shared response model fit on this data can recover the hidden signal well:

The shared components contain the signal.

The weights used in subject-specific signal generation show a
high correlation to the subject-specific transformation bases.

Transforming raw signal into the shared space yields
consistent components resembling the signal.

SRM Simulation

With random offset in subjects...

...the signal recovery is impeded:

Components capture mixed and partial signals.

There is overall no clear relationship between model weights and ground truth.

Across subjects, different components capture the signal.


This would impede a consistent interpretation of latent factors.

A spectral variant of the SRM

Do not use MEG time courses to train the model, but their spectral transformation.

Subject bases can then transform unseen time-resolved data
into a time-resolved shared space.

Model weights recover the signal weights consistently.

These transformations consistently reconstruct the signal in the same components.

Analysis overview

    Data wrangling
  • Raw data organized
    to BIDS Standard
  • Preprocessed raw data
    • Analyses:
  • Behavioral Analysis
  • SRM analysis
  • Temporal decoding analysis
  • Temporal generalization analysis

Trustworthy research needs data management

  • Domain-agnostic command-line tool, built on top of Git & Git-annex
  • 10+ year open source project (100+ contributors), available for all major OS
  • Major features:
    • Version-controlling arbitrarily large content
    Version control data & software alongside to code
    Transport mechanisms for sharing, updating & obtaining data
    Consume & collaborate on data (analyses) like software
    (Computationally) reproducible data analysis
    Track and share provenance of all digital objects
adapted from https://dribbble.com/shots/3090048-Front-end-vs-Back-end Halchenko, Meyer, Poldrack, Solanky, Wagner et al. (2021). DataLad: distributed system for joint management of code, data, and their relationship. Journal of Open Source Software 6 (63), 3262 joss.theoj.org/papers/10.21105/joss.03262

Research Data Management: Conceptual work

"Research Objects"
(Bechhofer et al., 2010) 		DataLad Dataset "FAIR digital objects"
(De Smedt et al., 2020)

"The goal of Research Objects is to
create a class of artefacts that can
encapsulate our digital knowledge
and provide a mechanism for sharing
and discovering assets of reusable
research and scientific knowledge." 		"A stable actionable unit that
bundles sufficient information
to allow reliable interpretation
and processing of contained data.
PIDs and metadata of FDOs are
open; access to FDO content may
be subject to authentication."


Hanke, Pestilli, Wagner et al. (2021). In defense of decentralized research data management. Neuroforum, 20(1) www.degruyter.com/document/doi/10.1515/nf-2020-0037

Research Data Management: Conceptual work

Exhaustively Versioned

Actionable Metadata

Modular structures for reuse

Portable, self-contained units

Reusable

Hanke, Pestilli, Wagner et al. (2021). In defense of decentralized research data management. Neuroforum, 20(1) www.degruyter.com/document/doi/10.1515/nf-2020-0037

Research Data Management: Technical solutions

Wagner, Waite, Wierzba, et al. (2022). Fairly big: A framework for computationally reproducible processing of large-scale data. Scientific data, 9(1), 80 www.nature.com/articles/s41597-022-01163-2
  • Scalable framework for large-scale processing
  • Exhaustively tracks & links analysis dependencies (opt. compressed, encrypted)
  • Can yield (computationally) reproducible results, suitable for individual recomputation on consumer hardware, as fully portable units
  • Proof-of-principle processing of UK Biobank dataset
  • Adapted into third-party processing frameworks (Zhao et al., 2023)

Research data management: Education

  • Online RDM handbook,
    continuously developed
    since 2019; ~60 contributors
  • RDM trainings and
    workshops since 2020
  • Print edition of RDM
    handbook released 2023
Availability correlates with
software popularity 		~3 times higher web traffic
than technical documentation
Wagner et al. (2020). The DataLad Handbook. Zenodo.doi.org/10.5281/zenodo.7640431
Project overview Trial overview

Simulating possible decision strategies

Better-than-chance performance; Consecutive evaluation starting with either of the stimulus properties aligns best with participants' actual performance.

Random choice Magnitude-based Probability-based
Expected-Value-based Mag.-then-Prob.-based Prob.-then-Mag.-based

Top row: Participants actual performance; Bottom row: Simulated strategy
(last two panels have no random element and yield single, fixed value).

Latent components capture some features

The shared space contains information about some trial properties,
e.g., motor response or left vs. right visual stimulation.

Decision phase

Delay phase

But no direct mapping to stimulus properties during the delay.

representations fade in the delay

    Temporal decoding of stimulus properties in the delay:
  • Custom temporal decoding pipelines, 3 with dimensionality reduction transformers:
    SRM, spectral SRM, PCA
  • Hyperparameter tuning (GridSearch)
Decoding reward probability of option 1
    No stable decoding
    of stimulus properties
    in the delay across
    pipelines.

Preparatory decision signals in the delay?

    Temporal generalization analysis
  • Assess temporal generalization of
    preparatory decision representations
    based on trial characteristics in
    the delay.

  • Training: Response-centered data. Labels
    according to 1) actual choice, 2) choice derived
    from first option's properties     , 3) choice derived
    according to subject-specific property weights.
  • Testing: Stimulus presentation and delay.

Preparatory decision (magnitude)


Chance level: 50%

Preparatory decision (probability)


Chance level: 50%

The delay period might contain a representation
of a decision-related signal.

Contributions

"What" is represented?
  • Stimulus features
  • Initial evidence of a preparatory
    decision signal. Hunt et al., 2013: "Evaluation in action-related
    spaces" during sequential stimulus presentation

    • "How" is it represented?
  • Non-stable representation of
    stimulus properties.
    Stokes, 2015: Activity-silent working memory;
    Trübutschek et al., 2019:
    Activity-silent non-conscious working memory
    		•
    Reusable research outcomes
  • Transformed Kaiser et al.'s data into a FAIRer digital research object
  • Demonstrated feasibility of SRM and spectral SRM on MEG data
  • Domain agnostic educational resource on research data management
  • Computational framework for large-scale processing
    		•

    Thanks ♥️

    Gerhard Jocham
    Jan Hirschmann
    My thesis commission

    The Coco Lab
    Luca, Antonia, Mani,
    Hannah, Christiane
    Monja, Eduard, Lina
    Anna, Armin     		The PsyInf people
    Michael, Laura,
    Alex, Michał,
    Stephan, Tosca
    Christian, Olaf,
    Gosia, Manu

    Yarik, Kyle, JB,
    Camille, Remi, V.
    Ljerka, Daniela     		The INM-7
    Simon Eickhoff

    My parents, Alex,
    Michelle, Svea, Thorge,
    Merle, Fynn, Gunnar,
    Oma & Opa, Lotti,
    Iggy, Matti,
    my parents-in-law

    Free and open source software, and the people behind it
    The open science movement

    Für Oma(†)

    Questions

    Additional slides

    Experimental Stimuli

    Modeling property importance

    Preprocessing

    Reaction times

    Temporal decoding

    Temporal decoding

    Temporal decoding

    Temporal decoding

    RDM

    Previous Analyses