An important goal of clinical neuroscience is to move towards personalized therapies that can target specific neural circuit dysfunctions that lead to neuropsychiatric disorders. To accomplish this goal, it is necessary to understand brain function across multiple scales, from single neurons to large-scale brain networks, and their dynamic interaction with the environment, hence, requiring neurophysiological investigations across animals (single-cell and local-field-potential recordings) and humans (whole-brain recordings). To address these needs we have developed Simulink brain signal interface (SimBSI), an open-source graphical environment  for the rapid prototyping of animal and human brain-computer interfaces (BCIs). SimBSI is designed as a library on top of the graphical programming environment of Simulink (MATLAB), with three goals in mind. 1) To provide a flexible cognitive platform for developing human and animal experiments by using Simulink’s Stateflow programming. 2) To allow for flexible data acquisition by including multiplatform drivers for standard instrument communication protocols (including the Lab Streaming Layer). 3) To allow for real-time analysis and control of neural circuit dynamics by leveraging Simulink’s DSP and Control toolboxes in addition to a customized neuroimaging module. With this library, we hope to ease the development of individualized BCI-based therapies while contributing a tool for deepening our understanding of the neurobiological and environmental basis of neuropsychiatric disorders.

In the simplest terms, the Brain-Computer Interface (or BCI) allows the brain to collaborate with a device and interact directly with the environment. BCI is widely considered a ‘disruptive technology’ for the next-generation human-computer interface in wearable computers and devices. In particular, there are incredible potential real-life applications of BCI in augmenting human performance for people in health and aged care. Despite this, there are limitations. Human cognitive functions, such as action planning, intention, preference, perception, attention, situational awareness, and decision-making, although omnipresent in our daily lives, are complex and hard to emulate. Yet, by studying the brain and behavior at work, a BCI plays an incredibly important role in natural cognition. Discover the latest thinking in the realm of the Brain-Computer Interface in this lecture. Listen the current status of BCI and discusses its three major obstacles: the shortage of wearable EEG devices, the various forms of noise contamination that hinder BCI performance, and the lack of suitable adaptive cognitive modeling. This talk will introduce the fundamental physiological changes of human cognitive functions at driving and explain how to combine the bio-findings and AI techniques to develop monitoring and feedback systems enhance driving safety.

The assessment of functional brain activity in everyday life situations represents the new frontier for cognitive neuroscience investigations. This becomes particularly important to investigate prefrontal cortex (PFC) function and dysfunction that cannot be properly studied in typical laboratory settings. In fact, the physically restrained and artificial environments such as a fMRI scanner can influence our behavior and reduce the ecological validity of our measurements. This can lead to a disagreement between measurements taken in the lab and in real-life, especially in case PFC lesions. Thanks to the recent technological advancements, we have now new mobile and wearable functional Near Infrared Spectroscopy (fNIRS) devices that allow imaging functional brain activity in more ecologically-valid situations such as outside the lab and on freely-moving people. However, the use of fNIRS in more naturalistic contexts presents several challenges, including the design of appropriate functional activation protocols, the technology limitations, the localization and inference of functional brain activity, and the impact of systemic physiological changes. In addition, in case of unstructured cognitive tasks, the identification of functional events in real-world experiments can be inaccurate. In this talk, I will present a feasibility study on the use of fNIRS to monitor PFC hemodynamics and oxygenation on people freely-moving outside the lab while undertaking an ecological cognitive task investigating executive functioning. In particular, the issues associated with real-world neuroimaging and possible solutions to overcome them will be discussed, and a novel algorithm for the identification of functional events in unstructured cognitive experiment will be presented.

Research in multi-modal bio-sensing has traditionally been restricted to well-controlled laboratory environments. Such bio-sensing modalities used to measure electroencephalogram (EEG), electrocardiogram (ECG), pupillometry, eye-gaze and galvanic skin response (GSR) are typically bulky, require numerous connections, costly, hard to synchronize, and have low-resolution and poor sampling rates. Thus, they are not practical for routine use by unconstrained users in real-world environments. Furthermore, they lack a method to automatically tag cognitively meaningful events. Developing a research-grade wearable multi-modal bio-sensing system would allow us to study a wide range of previously unexplored research problems in real-world settings. We present a novel multi-modal bio-sensing platform capable of integrating, synchronizing and recording multiple data streams for use in real-time applications. The system is composed of a central compute module and a companion headset. The compute node collects, time-stamps and transmits the data while providing an interface for a wide range of sensors including EEG, ECG, GSR, photoplethysmgram (PPG), full-body motion capture, and eye gaze among others. Though some of the integrated sensors are designed from the ground-up to fit into a compact form factor, we validate the accuracy of the sensors and find that they perform similarly to, and in some cases better than, alternatives. By providing a wearable platform that is capable of measuring numerous modalities in the real world and that has been benchmarked against state-of-the-art tools, we hope to expand the deplorable questions in MoBI research.

Most technologies for the non-invasive recording of human brain activity do not tolerate motion during signal acquisition very well. This poses an obvious dilemma to the field of behavioural brain sciences. Unfortunately, recently developed mobile EEG systems, while portable, are not necessarily mobile, that is, they do not feature motion-robust signal acquisition. Another problem is that these systems are clearly visible and therefore cannot be used in daily-life situations. A transparent EEG, on the other hand, would not only be portable and motion-tolerant, it would also feature low visibility and generally minimal interference with daily-life activities. The recording of brain-electrical activity from the outer ear and around the ear may be an important step towards reaching this ambitious goal. I will summarize our work on developing mobile and transparent EEG technology, with a strong focus on reporting the limitations and possibilities of smartphone EEG acquisition. Examples will include ear-EEG acquisition with flex-printed cEEGrid sensors, sleep-EEG, and the subsequent memory effect captured on smartphone.

In this talk we will present and demonstrate the NeuroPype Suite Academic Edition, a suite of desktop applications made available for free to the scientific community with the goal of accelerating and streamlining the creation, deployment, and sharing of pipelines for real-time or batch processing and decoding of unimodal and multi-modal sensor data (e.g. EEG, ExG, intracranial electrophysiology, actigraphy, eye tracking, audio and video signals, etc). The suite includes a version of NeuroPype, an extensible Python-based dataflow programming environment, containing 250+ modular data processing and visualization routines (“nodes”) that can be configured and linked together to create and run pipelines for signal processing and analysis, BCI and signal classification, neuroimaging, closed-loop feedback and control, and more. NeuroPype integrates seamlessly with the open-source Lab Streaming Layer (LSL) protocol for data acquisition, synchronization, and I/O, and works out of the box with LSL-compatible hardware devices. An extensible framework enables users to add their own custom processing nodes in Python, while a RESTful API allows external applications to interface with NeuroPype to create, run and configure pipelines in real-time. While NeuroPype can be operated entirely programmatically, the suite also includes Pipeline Designer, an open-source visual programming application based on Orange which provides an intuitive drag-and-drop GUI for pipeline design and configuration through NeuroPype’s API. We will illustrate both local deployment of NeuroPype pipelines, as well as deployment on the NeuroScale cloud platform for streaming access to/from mobile and other internet-connected devices and for scalable batch processing.

In my lab, we investigate the mechanisms of human social cognition with the use of naturalistic interactive protocols including embodied artificial agents (humanoid robots) and cognitive neuroscience methods. Experimental protocols with humanoid robots allow for more ecological validity than standard screen-based stimuli, thanks to the embodied presence of the robot. At the same time, they also allow excellent experimental control. We are interested in the behavioural characteristics of the robot that evoke the mechanisms of social cognition (such as joint attention, action prediction). In this talk, I will present a collection of studies that examined how joint attention was influenced by factors such as:

• real-time mutual gaze and gaze avoidance,
• contingency of the robot’s gaze behavior,
• expectations regarding action sequences.

These studies have been conducted with the use of the humanoid robot iCub, designed at our Institute (IIT). We integrate iCub in interactive protocols in which we measure – during interaction– participants’ EEG, gaze behavior (with a wireless mobile eyetracker) and performance. Our results show that embodied presence of an agent casts a new light on social cognitive mechanisms. For example, gaze cueing effects (behavioural and EEG) are modulated by whether the robot engages participants in mutual gaze or avoids gaze (prior to the gaze cueing procedure). We propose that through the use of artificial agents and realistic interactive protocols, we can learn how human (social) cognition works in real life.

Are event related potentials, well investigated under laboratory conditions, a signature of cortical processing during natural behavior? We explore this question with a fully mobile recording setup. It integrates and synchronizes an EEG system, a mobile eye tracker with
pupil- and world-cameras, as well as a step-sensor. These data are compared to recordings with more restricted behavior as well as classic fully controlled laboratory conditions. With a focus on the N170 ERP we streamline the data analysis using deep neural networks to categorize
elements like faces in the participant’s surrounding.

We find that widely reported effects are not as robust as they seemed. However, free viewing of static images and passive presentation leads to comparable results. Finally we present data on the step-wise bridge the gap between lab and real world recordings.

Simon Ladouce, David I. Donaldson, Paul Dudchenko, & Magdalena Ietswaart

Current knowledge about the neural correlates of attentional processing comes largely from lab-based research. As a result, little is known about how these processes respond in the face of complex environments. In this talk, a series of experiments using mobile EEG to examine attentional processing in the real-world will be presented. We used a neural marker of attention, the Event Related Potential (ERP) P300 effect. In Experiment 1 we found that attention allocated to the detection of infrequent target stimuli is reduced when human participants walk down a familiar hallway compared to when they stand still. Experiment 2 extended this finding by demonstrating that this reduction in the neural marker of attention is not caused by the act of walking per se, but rather is associated with movement through the environment. Experiment 3 identified the independent processing demands driving reduced attention to target stimuli. Taken together, these findings demonstrate the potential of a real-world approach to brain imaging, to reveal detectable reductions in attentional processing when participants are engaged in real-world behaviour.

Human activities are rarely restricted to isolated tasks and consequently linked to several cognitive and perceptual processes. Especially in older adults, effective resource allocation in parallel processing is crucial as aging is often associated with interdependent compensation mechanisms due to age-related cognitive and sensory declines. Everyday tasks like the maintenance of a stable and secure gait can require increased cognitive control leaving fewer resources for concurrent tasks such as scanning the traffic for approaching cars or the environment for obstacles. Although techniques to record neurophysiological data during realistic locomotion are nowadays possible using Mobile Brain-Body Imaging (MoBI) approaches, the interdependencies of visual perception and motor performance and the underlying brain dynamics are not yet understood in detail. In this talk, I will present results from two dual-talk studies investigating peripheral visual perception and motor task performance in older and younger adults. Age-related differences in performance, gait and posture data as well as neurophysiological measures and their interdependencies with different levels of motor-activity (sitting, standing, walking) will be discussed. In addition, experiences and benefits of MoBI-research with older adults will be outlined.

The use of high-density electrophysiological recordings in individuals while they are ambulating makes it clear that the neural circuitry of cognitive control is differentially engaged as a function of active mobility.  That cognitive operations require reconfigurations of control circuitry during activity raises the possibility that such reconfigurations may not be properly engaged in unhealthy aging; that is, those who are beginning to develop the prodromal phases of cognitive impairment, Alzheimer’s Disease or related dementias. In a series of studies, we have established clearly differential neural processing in elderly individuals as a function of gait parameters and environmental sensory challenges. We will discuss these results and the potential utility of MOBI in establishing endophenotypes of aging-related cognitive decline.

In my lab, we investigate the mechanisms of human social cognition with the use of naturalistic interactive protocols including embodied artificial agents (humanoid robots) and cognitive neuroscience methods. Experimental protocols with humanoid robots allow for more ecological validity than standard screen-based stimuli, thanks to the embodied presence of the robot. At the same time, they also allow excellent experimental control. We are interested in the behavioural characteristics of the robot that evoke the mechanisms of social cognition (such as joint attention, action prediction). In this talk, I will present a collection of studies that examined how joint attention was influenced by factors such as:

• real-time mutual gaze and gaze avoidance,
• contingency of the robot’s gaze behavior,
• expectations regarding action sequences.

These studies have been conducted with the use of the humanoid robot iCub, designed at our Institute (IIT). We integrate iCub in interactive protocols in which we measure – during interaction– participants’ EEG, gaze behavior (with a wireless mobile eyetracker) and performance. Our results show that embodied presence of an agent casts a new light on social cognitive mechanisms. For example, gaze cueing effects (behavioural and EEG) are modulated by whether the robot engages participants in mutual gaze or avoids gaze (prior to the gaze cueing procedure). We propose that through the use of artificial agents and realistic interactive protocols, we can learn how human (social) cognition works in real life.

Are event related potentials, well investigated under laboratory conditions, a signature of cortical processing during natural behavior? We explore this question with a fully mobile recording setup. It integrates and synchronizes an EEG system, a mobile eye tracker with
pupil- and world-cameras, as well as a step-sensor. These data are compared to recordings with more restricted behavior as well as classic fully controlled laboratory conditions. With a focus on the N170 ERP we streamline the data analysis using deep neural networks to categorize
elements like faces in the participant’s surrounding.

We find that widely reported effects are not as robust as they seemed. However, free viewing of static images and passive presentation leads to comparable results. Finally we present data on the step-wise bridge the gap between lab and real world recordings.

Simon Ladouce, David I. Donaldson, Paul Dudchenko, & Magdalena Ietswaart

Current knowledge about the neural correlates of attentional processing comes largely from lab-based research. As a result, little is known about how these processes respond in the face of complex environments. In this talk, a series of experiments using mobile EEG to examine attentional processing in the real-world will be presented. We used a neural marker of attention, the Event Related Potential (ERP) P300 effect. In Experiment 1 we found that attention allocated to the detection of infrequent target stimuli is reduced when human participants walk down a familiar hallway compared to when they stand still. Experiment 2 extended this finding by demonstrating that this reduction in the neural marker of attention is not caused by the act of walking per se, but rather is associated with movement through the environment. Experiment 3 identified the independent processing demands driving reduced attention to target stimuli. Taken together, these findings demonstrate the potential of a real-world approach to brain imaging, to reveal detectable reductions in attentional processing when participants are engaged in real-world behaviour.

Human activities are rarely restricted to isolated tasks and consequently linked to several cognitive and perceptual processes. Especially in older adults, effective resource allocation in parallel processing is crucial as aging is often associated with interdependent compensation mechanisms due to age-related cognitive and sensory declines. Everyday tasks like the maintenance of a stable and secure gait can require increased cognitive control leaving fewer resources for concurrent tasks such as scanning the traffic for approaching cars or the environment for obstacles. Although techniques to record neurophysiological data during realistic locomotion are nowadays possible using Mobile Brain-Body Imaging (MoBI) approaches, the interdependencies of visual perception and motor performance and the underlying brain dynamics are not yet understood in detail. In this talk, I will present results from two dual-talk studies investigating peripheral visual perception and motor task performance in older and younger adults. Age-related differences in performance, gait and posture data as well as neurophysiological measures and their interdependencies with different levels of motor-activity (sitting, standing, walking) will be discussed. In addition, experiences and benefits of MoBI-research with older adults will be outlined.

The use of high-density electrophysiological recordings in individuals while they are ambulating makes it clear that the neural circuitry of cognitive control is differentially engaged as a function of active mobility.  That cognitive operations require reconfigurations of control circuitry during activity raises the possibility that such reconfigurations may not be properly engaged in unhealthy aging; that is, those who are beginning to develop the prodromal phases of cognitive impairment, Alzheimer’s Disease or related dementias. In a series of studies, we have established clearly differential neural processing in elderly individuals as a function of gait parameters and environmental sensory challenges. We will discuss these results and the potential utility of MOBI in establishing endophenotypes of aging-related cognitive decline.

Mobile Brain/Body Imaging (MoBI) is rapidly gaining traction as a new imaging modality to study how cognitive processes support movement in the real world.  Issues associated to synchronization of multiple streams and the presence of artifacts (e.g., cable movement, electrode-gel coupling, EMG, blinks, saccades, eye bounce etc.) however often restrict the availability of the MoBI paradigm to high-end research  facilities. Regarding multi-stream data synchronization, we tested the effectiveness of a fallback strategy (spikes alignment at the beginning and ending of a recording), which enables to achieve MoBI-grade synchronization of EEG and EMG, when other strategies such as Lab Streaming Layer (LSL) cannot be used, e.g., due to the unavailability of proprietary Application Programming Interfaces (APIs), as it is often the case in clinical settings. We show that synchronization cannot rely only on the equipment sampling rate advertised by manufacturers: repeated spike delivery can be used to test online synchronization options and to troubleshoot synchronization issues over EEG and EMG.

The talk will also cover advancements on the integration of Reliable Independent Component Analysis (RELICA) to MoBI preprocessing pipelines. RELICA helps enhancing the removal of artifacts and reliably disentangling information from noise, thus ensuring that components retained for further analyses are both stable (i.e., not a result of algorithm instability, noise or mechanical artifacts) and dipolar (i.e., fitted by an equivalent dipole with low residual variance).

Mobility stress tests such as dual-task walking are particularly suited to unmask subtle gait changes in adults aged 65 and older. This is important, as quantitative gait markers are independent predictors of negative outcomes such as falls and cognitive decline. Further, structural neuroimaging highlights cortical contributions by linking gait variability in aging to atrophy in medial areas important for lower limb coordination and balance. Advancing our knowledge of the electro-cortical underpinnings of such complex cognitive-motor behavior will provide relevant clinical insight. Hence, our goal is to further EEG-based Mobile Brain/Body Imaging (MoBI) as a clinical research tool to determine changes in cognitive, sensory, and motor coupling with advanced age and neurological disorders such as multiple sclerosis.  We employ a 3D infra-red camera system to monitor gait and posture during ambulation while high-density electrophysiology is simultaneously recorded. We vary sensory load by manipulating full-field optical flow stimulation as well as task-load, as participants will also perform cognitive tasks while walking in this environment.  I will provide an overview of our work addressing the test/retest reliability of EEG signals while walking, the use of event-related potentials to map age differences, and power spectral density of localized ICA sources to assess cortical network activity during dual-task walking. In addition, I will present ongoing efforts to determine electro-cortical signals associated with increased gait variability in aging. We believe MoBI will provide new insights to enhance the mobility and quality of life of older individuals.

Understanding the electrocortical dynamics of locomotor tasks such as recumbent stepping and cycling can provide insight for prescribing gait rehabilitation therapies when gait rehabilitation involving walking is not practical. The use of perturbations during locomotor tasks such as cycling and recumbent stepping has only recently begun to be explored as another approach for promoting locomotor adaptation. As such, there is a need to understand the underlying electrocortical dynamics driving locomotor adaptation during walking and other locomotor tasks. The purpose of this study was to determine the electrocortical correlates of adapting to mechanical perturbations on a robotic recumbent stepper. Electroencephalography (EEG) was recorded as healthy young adults performed recumbent stepping with 4 types of perturbations, which were brief increases in resistance at step initiation and mid-step of each leg. For each perturbation type, the protocol was 2 minutes of unperturbed stepping (baseline), 6 minutes of perturbed stepping, and ending with 2 minutes of unperturbed stepping. Perturbations were applied during each stride except 1 out of every 5 strides was a “catch” trial where no perturbation was applied. After stepping for just 6 minutes with one type of perturbation, stepping patterns shifted away from the baseline to a new sustained stepping pattern. By the end of perturbed stepping, sources in the anterior cingulate showed increased theta (4-7 Hz) band synchronization that corresponded with the timing of the perturbation. These results suggest that increased anterior cingulate theta synchronization may underlie perturbation-driven adaptation during recumbent stepping towards new sustained stepping patterns.

Intact gait function is a prerequisite to ensure mobility and independence for people. Cortical contributions to lower limb function are relevant, since brain injuries and impairment of the corticospinal tract can cause motor disabilities. Nonetheless, we are just only beginning to understand the involvement of cortical dynamics in generation and control of gait.

I will describe the use of electroencephalographic (EEG) source imaging techniques to reconstruct and study cortical dynamics during gait in humans. In particular, I will show that the combination of EEG source imaging, advanced computational techniques for artefact suppression, and motion tracking is necessary to enable mobile brain imaging (MoBI) during gait.

We used these MoBI techniques to study the modulation of cortical oscillations in healthy volunteers as they walked on a treadmill supported by a robotic gait orthosis. We distinguished state transitions between standing and walking from dynamic modulations within the gait cycle. Both of these perturbations were focally located in central sensorimotor regions, i.e. in accordance with leg motor cortical areas. Interestingly, sustained- and gait-phase related activities were associated with two distinct frequency-specific networks. First, movement-state related networks, which upregulate cortical excitability in sensorimotor leg areas. Second, gait-phase related networks, which modulate their frequency-specific synchrony in relation to the gait cycle.

In ongoing and future work, we aim to use these two types of gait-related cortical dynamics for real-time decoding of gait events in healthy volunteers and people with spinal cord injury. Our ultimate goal is to develop a therapeutic system where decoded gait events trigger spinal cord stimulation protocols to support gait rehabilitation after spinal cord injury.

Wearable technology is a rapidly expanding field with great untapped potential for mobile human experiments. Small, non-invasive sensors record biomechanical and physiological signals in real-time as subjects move freely. Combining wearables with traditional neuroimaging methods provides novel insight into brain-body dynamics. Furthermore, wearable technology is itself beginning to move into the area of direct brain monitoring and neuromodulation. In just one segment of the wearables market, fitness bands, dozens of novel sensors have emerged that provide complementary data for neuroimaging experiments. Heart rate and respiratory rate is computed with photoplethysmograph. Galvanic skin response provides insight into minute-to-minute emotional valence. Accelerometry allows for the reconstruction of gait patterns or even the classification of movement type during mobile experiments, to name just a few applications. Furthermore, wearables themselves are beginning to tap into or modulate brain signals.  Today, commercial wearables utilize electroencephalography (EEG) and transcranial magnetic stimulation (TMS). While these technologies are in their early stages, they may one day provide a lower cost alternative to traditional neuroimaging.

With all this potential for insight, wearables are increasingly being incorporated into big data initiatives. This talk will examine the state-of-the-art of wearable sensing for mobile neuroimaging applications. It will provide examples of how novel sensors have been used to gain insight into brain-body dynamics and to tackle big data questions in neuroscience.

A common problem with mobile EEG is motion artifact. As humans walk and run, they induce increases and decreases in EEG spectral power across a range of frequency bands. We have created a novel noise-cancelling dual layer EEG system that devotes one electrode to pure motion artifact recording and a second electrode to a mix of biological signal and motion artifact recording. Using either time-series or spectral frequency subtraction, it is possible to reproduce a much higher fidelity representation of the true electrocortical signals. I will describe the dual-layer EEG system and discuss how we validated the system with an electrical head phantom and robotic motion platform. The approach is feasible for a wide range of EEG electrodes and could greatly improve the ability to study human brain dynamics in active real-world tasks.

Mobile Brain/Body Imaging (MoBI) is rapidly gaining traction as a new imaging modality to study how cognitive processes support movement in the real world.  Issues associated to synchronization of multiple streams and the presence of artifacts (e.g., cable movement, electrode-gel coupling, EMG, blinks, saccades, eye bounce etc.) however often restrict the availability of the MoBI paradigm to high-end research  facilities. Regarding multi-stream data synchronization, we tested the effectiveness of a fallback strategy (spikes alignment at the beginning and ending of a recording), which enables to achieve MoBI-grade synchronization of EEG and EMG, when other strategies such as Lab Streaming Layer (LSL) cannot be used, e.g., due to the unavailability of proprietary Application Programming Interfaces (APIs), as it is often the case in clinical settings. We show that synchronization cannot rely only on the equipment sampling rate advertised by manufacturers: repeated spike delivery can be used to test online synchronization options and to troubleshoot synchronization issues over EEG and EMG.

The talk will also cover advancements on the integration of Reliable Independent Component Analysis (RELICA) to MoBI preprocessing pipelines. RELICA helps enhancing the removal of artifacts and reliably disentangling information from noise, thus ensuring that components retained for further analyses are both stable (i.e., not a result of algorithm instability, noise or mechanical artifacts) and dipolar (i.e., fitted by an equivalent dipole with low residual variance).

Mobility stress tests such as dual-task walking are particularly suited to unmask subtle gait changes in adults aged 65 and older. This is important, as quantitative gait markers are independent predictors of negative outcomes such as falls and cognitive decline. Further, structural neuroimaging highlights cortical contributions by linking gait variability in aging to atrophy in medial areas important for lower limb coordination and balance. Advancing our knowledge of the electro-cortical underpinnings of such complex cognitive-motor behavior will provide relevant clinical insight. Hence, our goal is to further EEG-based Mobile Brain/Body Imaging (MoBI) as a clinical research tool to determine changes in cognitive, sensory, and motor coupling with advanced age and neurological disorders such as multiple sclerosis.  We employ a 3D infra-red camera system to monitor gait and posture during ambulation while high-density electrophysiology is simultaneously recorded. We vary sensory load by manipulating full-field optical flow stimulation as well as task-load, as participants will also perform cognitive tasks while walking in this environment.  I will provide an overview of our work addressing the test/retest reliability of EEG signals while walking, the use of event-related potentials to map age differences, and power spectral density of localized ICA sources to assess cortical network activity during dual-task walking. In addition, I will present ongoing efforts to determine electro-cortical signals associated with increased gait variability in aging. We believe MoBI will provide new insights to enhance the mobility and quality of life of older individuals.

Understanding the electrocortical dynamics of locomotor tasks such as recumbent stepping and cycling can provide insight for prescribing gait rehabilitation therapies when gait rehabilitation involving walking is not practical. The use of perturbations during locomotor tasks such as cycling and recumbent stepping has only recently begun to be explored as another approach for promoting locomotor adaptation. As such, there is a need to understand the underlying electrocortical dynamics driving locomotor adaptation during walking and other locomotor tasks. The purpose of this study was to determine the electrocortical correlates of adapting to mechanical perturbations on a robotic recumbent stepper. Electroencephalography (EEG) was recorded as healthy young adults performed recumbent stepping with 4 types of perturbations, which were brief increases in resistance at step initiation and mid-step of each leg. For each perturbation type, the protocol was 2 minutes of unperturbed stepping (baseline), 6 minutes of perturbed stepping, and ending with 2 minutes of unperturbed stepping. Perturbations were applied during each stride except 1 out of every 5 strides was a “catch” trial where no perturbation was applied. After stepping for just 6 minutes with one type of perturbation, stepping patterns shifted away from the baseline to a new sustained stepping pattern. By the end of perturbed stepping, sources in the anterior cingulate showed increased theta (4-7 Hz) band synchronization that corresponded with the timing of the perturbation. These results suggest that increased anterior cingulate theta synchronization may underlie perturbation-driven adaptation during recumbent stepping towards new sustained stepping patterns.

Intact gait function is a prerequisite to ensure mobility and independence for people. Cortical contributions to lower limb function are relevant, since brain injuries and impairment of the corticospinal tract can cause motor disabilities. Nonetheless, we are just only beginning to understand the involvement of cortical dynamics in generation and control of gait.

I will describe the use of electroencephalographic (EEG) source imaging techniques to reconstruct and study cortical dynamics during gait in humans. In particular, I will show that the combination of EEG source imaging, advanced computational techniques for artefact suppression, and motion tracking is necessary to enable mobile brain imaging (MoBI) during gait.

We used these MoBI techniques to study the modulation of cortical oscillations in healthy volunteers as they walked on a treadmill supported by a robotic gait orthosis. We distinguished state transitions between standing and walking from dynamic modulations within the gait cycle. Both of these perturbations were focally located in central sensorimotor regions, i.e. in accordance with leg motor cortical areas. Interestingly, sustained- and gait-phase related activities were associated with two distinct frequency-specific networks. First, movement-state related networks, which upregulate cortical excitability in sensorimotor leg areas. Second, gait-phase related networks, which modulate their frequency-specific synchrony in relation to the gait cycle.

In ongoing and future work, we aim to use these two types of gait-related cortical dynamics for real-time decoding of gait events in healthy volunteers and people with spinal cord injury. Our ultimate goal is to develop a therapeutic system where decoded gait events trigger spinal cord stimulation protocols to support gait rehabilitation after spinal cord injury.

Wearable technology is a rapidly expanding field with great untapped potential for mobile human experiments. Small, non-invasive sensors record biomechanical and physiological signals in real-time as subjects move freely. Combining wearables with traditional neuroimaging methods provides novel insight into brain-body dynamics. Furthermore, wearable technology is itself beginning to move into the area of direct brain monitoring and neuromodulation. In just one segment of the wearables market, fitness bands, dozens of novel sensors have emerged that provide complementary data for neuroimaging experiments. Heart rate and respiratory rate is computed with photoplethysmograph. Galvanic skin response provides insight into minute-to-minute emotional valence. Accelerometry allows for the reconstruction of gait patterns or even the classification of movement type during mobile experiments, to name just a few applications. Furthermore, wearables themselves are beginning to tap into or modulate brain signals.  Today, commercial wearables utilize electroencephalography (EEG) and transcranial magnetic stimulation (TMS). While these technologies are in their early stages, they may one day provide a lower cost alternative to traditional neuroimaging.

With all this potential for insight, wearables are increasingly being incorporated into big data initiatives. This talk will examine the state-of-the-art of wearable sensing for mobile neuroimaging applications. It will provide examples of how novel sensors have been used to gain insight into brain-body dynamics and to tackle big data questions in neuroscience.

A common problem with mobile EEG is motion artifact. As humans walk and run, they induce increases and decreases in EEG spectral power across a range of frequency bands. We have created a novel noise-cancelling dual layer EEG system that devotes one electrode to pure motion artifact recording and a second electrode to a mix of biological signal and motion artifact recording. Using either time-series or spectral frequency subtraction, it is possible to reproduce a much higher fidelity representation of the true electrocortical signals. I will describe the dual-layer EEG system and discuss how we validated the system with an electrical head phantom and robotic motion platform. The approach is feasible for a wide range of EEG electrodes and could greatly improve the ability to study human brain dynamics in active real-world tasks.

Early brain injury, as seen in cerebral palsy (CP), transforms the nervous system and motor development in often very individualized and unpredictable ways. While static structural images of grey and white matter can document differences between children with and without CP and relate location and severity of abnormalities to functional abilities, these provide little insight into brain activation during everyday motor tasks in an individual child. MoBI techniques, such as EEG and fNIRS, hold great potential for uncovering the neural mechanisms underlying motor development and coordination and how these may be disrupted in CP with the ultimate goal to design more effective biologically based and personalized motor training paradigms. We will discuss some of our initial results in age-matched cohorts with and without CP during a range of upper and lower extremity motor tasks from squeezing a ball to walking using NIRS or EEG with a primary focus on relating cortical activation to muscle activation. Intriguing differences in the maturation of the sensorimotor system across groups are also emerging. We will briefly discuss how MoBI in pathological conditions also conveys novel challenges for the field due to distorted anatomy and behavioral limitations as well as the growing interest in applying these techniques in combination, in very young infants, and as outcome measures in clinical trials.

In this talk we survey and make explicit various control types for artistic brain-computer interfaces. We distinguish between passive, selective, direct, and collaborative control and illustrate them with examples. An intentional choice by the user or a control determined by the mood of a user are quite different ways of interacting with an artistic brain-computer interface. We distinguish different types of mental activity that are related to control and we make a comparison with similar issues as they emerged in human-computer interaction. Other issues such as ‘agency’ and ‘dominance’ will be touched upon. We take the opportunity to shortly discuss the contents of a book on artistic brain-computer interfaces that will appear in 2019 and in which the various control mechanisms will be exploited.

Jesus G. Cruz-Garza, Girija Chatufale, Jose Luis Contreras-Vidal

Human creativity was explored in the context of the exquisite corpse protocol – a collaborative, chance-based game made famous by the Surrealists in the 1920s. The players attempt to create a “body” consisting of a head, torso, and legs – 15 minutes for each segment. Three visual artists worked together, by observing the edge of the previous composition to begin their own. Artists were instrumented with wireless 62-channel EEG, inertial measurement units, and video cameras in a public setting.

A classical machine learning approach using kSVM was compared to a deep learning approach using convolutional neural networks (CNN). The kSVM input were time and frequency-domain features in 1s windows with 50% overlap. The CNN took the EEG time windows of 62 channels as input. A four-class classification problem yielded over 79% accuracy across subjects, with similar performance in the kSVM and CNN methods. The most relevant EEG features were found in frontal (1-4 Hz, 8-12 Hz), central (8-12 Hz, 30-50 Hz), and posterior (8-50 Hz) brain networks, consistent both in the predefined set of features and those found automatically by the deep learning framework. Input-perturbation with output probability correlation was used to visualize the salient features. Data samples with the highest probability per class were compared with ‘baseline eyes open’ to understand what the CNN learns in each class.

The ultimate goal is cataloging of dynamic brain patterns associated with mental states underlying creativity, and developing computational models that take real-time neural EEG input and predict evolving behavioral actions.

We report on how people acquire a sense of spatial relation to each other and to our media rich environment.  We call this sense spatiality, after Merleau-Ponty’s treatment which resorts neither to physical abstractions external to subjective experience, nor to purely mental phenomena. Note: (1) Relationality motivates Leibnizian, Riemannian approaches rather than the Euclidean model in conventional computational abstractions.  (2) The sense of space requires taking subjective experience as primary data rather than subject-independent measures (such as clock time or “objective” sensor data), which requires phenomenologically informed methodology.  (3) Since people and events always vary, spatiality is inextricable from temporality.

When we see a child about to jump over a skip rope held by friends, we see characteristic pre-accelerations that entrain the person to the anticipated time-varying, spatial relations.  Since people’s experiences are mediated via bodies in physical environments, and given that people’s experiences are dynamical, it’s natural to particularize the question of spatiality to the question of how bodies move across inhomogeneous extension to produce a sense of rhythm.   We will suggest that (1) rhythm is not sense data; (2) biosocial rhythm is not perfectly periodic; and (3) rhythm is not unidimensional.

We invite colleagues from neuroscience and bio-engineering to construct experiments in whole-body interaction of ensembles of three or many people, joining brain data with movement or gestural data modulating the rich but precisely reproducible qualia (“feedback”) of responsive media environments.

Synchronizing movements between individual performers is a central aspect of performing dance and music. In two experiments, we investigated performing and perceiving synchrony in live dance performances. In a live theatre setting, participants performed a set of movement tasks that were either performed as a group or individually. During execution (dancers) and observation (spectators) of these tasks, we assessed movement synchrony based on performer acceleration and spectators’ psychophysiological responses using wrist sensors. We also recorded continuous ratings of aesthetic appreciation and perceived group characteristics. We show that movement synchrony is associated with group affiliation among performers and predicts spectators’ heart rate and enjoyment. Our findings point to an evolutionary function of dance – and perhaps all performing arts –  in communicating social signals between groups of people.

Recent developments in EEG recording and signal processing have made it possible to record in an unconstrained, natural movement task, therefore EEG provides a promising approach to understanding the neural mechanisms of upper-limb reaching control. This study specifically addressed how EEG dynamics in the time domain encoded finger movement directions (directional tuning) and posture dependence (movement reference frames) by applying representational similarity analysis. High-density EEG covering the entire scalp was recorded while participants performed eight-directional, center-out reaching movements, thereby allowing us to explore directional selectivity of EEG sources over the brain beyond somatosensory areas. A majority of the source processes exhibited statistically significant directional tuning during peri-movement periods. In addition, directional tuning curves shifted systematically when the shoulder angle was rotated to perform the task within a more laterally positioned workspace, the degree of tuning curve rotation falling between that predicted by models assuming extrinsic and shoulder-based reference frames. We conclude that temporal dynamics of neural mechanisms for motor control can be studied noninvasively in humans using high-density EEG and that directional sensitivity is not limited within the sensorimotor areas but extends to the whole brain areas.

Motor unit (MU) recruitment strategies are thought to produce safe, economic contraction of skeletal muscle via the Central Nervous System (CNS) to protect muscle integrity and whole-body health (1).  Extensive studies by colleagues and ourselves have provided further knowledge surrounding MU recruitment strategies employed under different environmental and pathological conditions; i) Subconcussion demonstrate cortico-spinal inhibition (2) and altered MU recruitment strategies (Di Virgilio in preparation); ii) Multiple Sclerosis patients demonstrated reduced MU recruitment with increased muscle fibre conduction velocity (MFCV) (3); iii) exercise induced hyperthermia reduced MU recruitment and preserved MFCV (4); iv) eccentric overload reduced firing rates of high threshold MU (5) and; v) damaging eccentric exercise showed recovery of force coupled with higher threshold motor unit firing (6). These novel studies have provided greater information regarding motor unit recruitment strategies but the mechanisms of brain function, via the CNS, remain elusive. The rapid evolution of mobile cognition technologies such as mobile; EEG, EMG; electrogoniometry; and foot force pressure sensors are now providing further opportunities to explore brain function in relation to neuromuscular recruitment strategies.  The benefit of these technological advances in our understanding are, for the time being, limitless particularly as there is now increasing evidence of a mismatch between data obtained in the lab versus that in the field.  Understandably this has significant implications for translation of lab studies into that of the clinical and sports performance areas.  This provides a wealth of applications and opportunities for neuromuscular science of the future.

References

1. Noakes TD, St Clair Gibson A, Lambert E V. From catastrophe to complexity: a novel model of integrative central neural regulation of effort and fatigue during exercise in humans: summary and conclusions. Br J Sports Med. 2005 Feb 1;39(2):120–4.
2. Di Virgilio TG, Hunter A, Wilson L, Stewart W, Goodall S, Howatson G, et al. Evidence for Acute Electrophysiological and Cognitive Changes Following Routine Soccer Heading. EBioMedicine. 2016 Nov;13:66–71.
3. Scott SM, Hughes AR, Galloway SDR, Hunter AM. Surface EMG characteristics of people with multiple sclerosis during static contractions of the knee extensors. Clin Physiol Funct Imaging. 2011 Jan;31(1):11–7.
4. Hunter A, Albertus-Kajee Y, St Clair Gibson A. The effect of exercise induced hyperthermia on muscle fibre conduction velocity during sustained isometric contraction. J Electromyogr Kinesiol. 2011 Oct;21(5):834–40.
5. Balshaw TG, Pahar M, Chesham R, Macgregor LJ, Hunter AM. Reduced firing rates of high threshold motor units in response to eccentric overload. Physiol Rep. 2017 Jan 20;5(2):e13111.
6. Macgregor L & Hunter AM. High-threshold motor unit firing reflects force recovery following a bout of damaging eccentric exercise. PLOS ONE (in press).

The effects of exercise-induced fatigue on cognitive functions such as working memory, attention and visual-information processing speed vary and depend upon exercise duration and intensity, and the type of cognitive tasks assessed (Moore et al., 2012). To explain the detrimental effects of exercise upon cognition Dietrich and Audiffren (2011) proposed the reticular-activating hypofrontality hpothesis, which makes a decrease in frontal neural activity responsible, in favor of brain regions associated with sensory and motor processes. In young adults, the empirical evidence for hypofrontality during exercise has been mixed, indicating either negative or positive effects of exercise (e.g., Chu et al., 2017; Davranche et al., 2015; Del Giorno et al., 2010; Pesce et al., 2003; Schmit et al., 2015; Wang et al., 2013). Most of these studies used shorter and mostly less challenging exercise protocols; no studies have examined prolonged exercise protocols until exhaustion (> 20 min). The aims of this study are: (1) to examine the effect of a concurrent cognitive task on fine motor task performance in adults (before, during and after a prolonged motor fatiguing protocol), (2) to determine whether the effect varied with different difficulty levels of the concurrent task, and (3) to measure oxygenation of the prefrontal cortex associated with different levels of task difficulty. We examined this knowledge gap by measuring cognitive control and cerebral oxygenation during high-intensity interval exercise (HIIE) or a moderate-intensity continuous exercise (MCE) protocol. Participants were monitored using prefrontal near-infrared spectroscopy during the completion of a modified (digital) Trail-Making-Test (TMT) at rest or while cycling on a stationary bicycle at either 4 x 4min high-intensity bouts at 90% of the VO₂max, separated by 3min of moderate intensity (60% of the VO₂max) or 30min at 60% of the VO₂max. Cognitive performance was performed 2min into the exercise protocol at 60% of the VO₂max, again after the exercise intervention, and after a 10min recovery period. In addition to the typical TMT (A serial connection of numbers (1–25); B serial connection of numbers (1–13) and letters (A–L) in an ascending number-letter sequence (1-A-2-B- etc)) we have included a motor speed trail-tracing-task allowing us to calculate dual-task effects. Furthermore, the digital version of the TMT (e.g., time in circles, pauses, lifts, time between circles) allows us to isolate cognitive processes believed to be important in TMT performance. In this talk, we will present and discuss behavioral and neural data from young adults, and athletes with and without prior concussion.

Andreas Mierau, Alexander Niklas Häusler, & Sergio-Oroz-Artigas

World football/soccer is considered as one of the most popular sports in the world. Soccer players like Christiano Ronaldo, Lionel Messi and Neymar consistently impress millions of people world-wide with their skills.   However, despite recent progresses in mobile brain imaging, only very little is known about the neurophysiological processes that support high achievement in soccer. Further advancing our knowledge about such processes is important for a better understanding of neural plasticity associated with participation in different sports, and it may also have important implications for the further development of effective training interventions. We addressed this knowledge gap by measuring electroencephalographic (EEG) activity in skilled soccer players and non-soccer athletes during preparation of various ball kicking tasks differing with respect to ball dynamics (stationary vs. approaching) and the relevant performance measure (accuracy vs. speed matching) in an ecologically valid setting. Using independent component analysis and clustering, a central, a left parietal and a parieto-occipital cluster of components were identified. More accurate kicking performance in soccer players was associated with a significantly different pattern of event-related spectral perturbations in a cluster-dependent and task-specific manner. This study indicates that is feasible to identify cortical regions relevant for the preparation of complex soccer skills in a real-word environment with as few as 15 active EEG channels. Furthermore, the results provide new insights into brain processes associated with skilled soccer actions.

Guy Cheron and Axelle Leroy

A modern neuroscience consensus concluded on the importance of three basic research questions: “what makes individual brains unique; how brain’s many components orchestrate learning and perform a task; and how to leverage the brain’s plasticity to protect and restore brain function?”(Underwood, 2016). In this context, the reactivity of the EEG signals has been mainly studied in response to sensory input or during cognitive task but less often during motor behaviour. As the EEG signals represent the dynamics of the brain states resulting from synchronous neuronal activity of local field potentials distributed into temporal and spatial coordinated networks of neurons, it is important to quantify in the EEG signals the part of this activity devoted to its downstream impact on motor behaviour. Based on the idea that the uniqueness of the individual brain is all the more evident that it is accompanied by an exceptional performance, we recently begun the experimental search of psychological “flow”. This singular brain state emerges from an action requiring clear goal and a perfect match between specifics skills and challenge (Csikszentmihalyi, 1975; Mao et al., 2016; Cheron, 2016). Amongst different sports, the tightrope walker activity appeared as particularly attractive because the highly restrictive field of action requiring optimal balance control permanently exerted at the edges of the fatal fall. What about the brain of a tightrope performer during this performance? How to reach the brain dynamics of a subject situated on a cable at an altitude of 30 meters? We here report this analysis on Oliver Zimmerman’s brain by using high density electroencephalography (EEG), coupled to electro-oculography (EOG), electrocardiography (ECG) and electromyography (EMG) recordings before and during walking on a long cable (100 m) placed at an altitude of 30 meters. The neuronal generators of the different EEG oscillations were studied by means of inverse modelling (swLORETA) showing along this performance the respective contribution of different cortical areas, the basal ganglia and the cerebellum.

Since its inception in the 1940’s, the field of art therapy has intuited the connections between artistic expression and brain processes with the identification of three primary tenets, all of which can be underscored with neuroscience principles: (1) the bilateral and multidirectional process of creativity is healing and life enhancing; (2) the materials and methods utilized affect self-expression, assist in self-regulation, and are applied in specialized ways, and (3) the art making process and the artwork itself are integral components of treatment that help to understand and elicit verbal and nonverbal communication within an attuned therapeutic relationship (King, 2016). However, without empirical evidence to prove these tenets, art therapists rely on interpretive frameworks, which are often anecdotal, idiographic, and do not allow generalizations to be made for larger populations. The purpose of this talk is to discuss trans-disciplinary research on how brain science and artistic processes inform one another to support the overall health and amelioration of disease for patients in need of psychological and medical care. Exploring the biological basis of creative arts and neuroscience through the use of Brain-Computer Interface and Mobile Brain-Body Imaging (MoBI) techniques will promote a greater understanding for the capacities of the creative arts therapies to be considered an effective and data-driven medical and mental health profession. Simultaneously, creative arts therapists are positioned to uniquely inform research scientists of the implications of non-verbal, sensory based and symbolic expression within a therapeutic and clinical context.

Art therapy interventions have been found to result in positive health outcomes like improved mood, self-efficacy and creative agency, as well as, lowered anxiety and stress. In addition, in narrative responses, participants have indicated that the therapeutic interaction with an art therapist helps them view experiences through new perspectives, experience positive emotions, develop a stronger sense of identity, externalize debilitating ruminative processes, distract from negative perceptions, and, improve focus and attention. Despite these outcomes, mechanisms of change through art therapy remain poorly understood. Mobile brain imaging technologies like functional near-infrared spectroscopy (fNIRS) could help identify some of these mechanisms especially since they generate measurements of processes in naturalistic settings including those of art-making in the context of a therapeutic session. FNIRS has been used to assess reward perception as related to different drawing tasks like doodling, coloring and free drawing. It has also been used to determine patterns and brain signatures in psychological health conditions like schizophrenia, depression, and, suicidality. The use of fNIRS is still emergent and has been limited by the inconclusive interpretations of the hemodynamic response. This presentation will describe art therapy treatment approaches for diagnoses like for eating disorders, mood disorders, and post-traumatic stress and examine if and how MoBI technologies like fNIRS might be used to understand mechanisms of change. The presentation seeks to generate discussion around clinical constructs from art therapy and how they can be linked to the technological capabilities of MoBI.

According to Damasio, when the current state of the body is conveyed to the brain by afferent input from the body, the resulting brain activation patterns represent unconscious emotions, which are experienced as subjective feelings. This implies that deliberate control of motor behavior could help regulate feelings through proprioceptive input. Indeed, we demonstrated that execution, observation and imagination of various whole body emotional expressions differentially activate emotional processing regions in the brain and enhance specific emotions. These principles are used in dance-movement therapy, when therapists observe and mirror clients’ movements to empathize with them, or encourage clients to engage in specific movements, to help them experience and process associated emotions. But which movement evokes which feeling? Using Laban Movement Analysis we identified unique sets of movement components (characteristics) whose execution enhances different emotions. Moreover, movements composed of components associated with specific emotions were recognized above chance level as expressing those emotions, even when movers did not intend to express emotions, and   emotion elicitation using recall of autobiographical memories led to whole-body emotional expressions composed of mainly those same movement components. An additional study using Kinect and machine learning techniques suggested a biofeedback system able to identify these movement components from people’s movements based on their 3D data. Based on these past findings and methodologies, we will present our vision and invite colleagues to join a future collaborative study aimed to further uncover the underlying brain mechanisms for movement-emotion interaction, using mobile EEG, 3D data, machine learning, and Laban experts as movers.

More info about Tal Shafir can be found here: https://www.talshafir.com/

This project explores the relationship between EEG and artistic engagement to examine how neural oscillations and modifications may contribute to meaningful self-expressive communication processes. It investigates how biodata and neurofeedback can be integrated into the architectural design of digital spaces that are designated for artistic expression, and how these applications might be valuable for evaluating meaning-making and learning outcomes for current users of Brain-Computer Interfaces (BCI) and Mobile Brain Body Imaging (MoBI) systems. Conceptually, these efforts are designed to promote collaboration, artistic self-awareness, rehabilitation and recovery for people with limited mobility and limited resources for self-expressive forms of communication. Additionally, this work explores how applied communicative theoretical frameworks and Human-Computer Interface (HCI) concepts may further promote potential therapeutic opportunities. It looks to extend the lens for transdisciplinary efforts to include art, technology and therapy, and questions how this integration may enhance communication within existing channels of language behaviors for users. With a focus on using complex technologies to provide innovative contributions to therapeutic care, it asks how we may pragmatically apply the principles of attention and engagement within this space, and translate these notions into action. Ultimately, this work poses the question of how technology can best be used to mediate social expression.

Gráinne McLoughlin & Jason Palmer

In young adulthood, the brain begins the final stretch of its development. This process is crucial to making the leap to an adult mature state. However, there are individual differences in social and attentional processing, which are key to improved well-being and quality of life. Application of advanced signal processing to temporally-rich EEG data can provide highly sensitive markers of cognitive function that may have a genetic influence. We have previously shown this in a twin study through the identification of functional measures that explain more genetic variance than standard EEG. It is currently unclear how brain function in early adulthood influences emotional wellbeing, behavior and adaptive functioning and how much genes and environment contribute to this relationship. To answer these questions, we applied signal processing techniques to mobile EEG data from a large population-based twin study. Our sample consists of 480 individuals (240 twin pairs) aged between 21 and 24 from across the UK. Our findings mirror earlier results in that we find very distinct differences between ASD and ADHD in social processing measures, yet some overlap in attentional processing. Our key finding is that deficits in social processing share genetic influences with wellbeing and social adjustment in early adulthood. We anticipate that this work will pave the way for identifying optimal treatment targets for these disorders and the design of more specific interventions to prevent emotional and mental health problems from developing. The innovative use of mobile EEG has the potential to transform neurophysiological research and routine clinical evaluation for psychiatric disorders.

Motor imagery (MI) with neurofeedback is a promising add-on therapy for motor recovery after stroke. Regular training with MI neurofeedback should facilitate compensatory plasticity and motor rehabilitation, but due to the machinery involved, translating regularity into training protocols has been difficult so far. Based on a number of studies in healthy younger and older adults we implemented a frequent and efficient neurofeedback training that was run at patients’ homes. Patients imagined a simple hand movement and neurofeedback that was based on data collected with a wireless electroencephalogram (EEG) system. Three chronic stroke patients practiced every other day over a 4-week period and participated in pre- and post-training assessments of behavior, function, and structure. One patient showed a substantial clinical improvement of upper limb motor function that was associated with a significant change in EEG lateralization and paralleled by additional changes in function and structure. Two patients showed no clinical improvement in motor function. However, for them, EEG activity induced by MI of the paretic hand became likewise more lateralized towards the ipsilesional hemisphere over the course of training, yet to a smaller degree. Though preliminary, these results show great promise for the benefit of mobile, wireless EEG for neurorehabilitation applications.

Mental states, like boredom or fatigue, are among the main accident hazards in many work environments. Attentional withdrawal may lead to impaired information processing and consequently to processing errors. Understanding mechanisms and causes of reduced task engagement is therefore an essential issue for work safety. In the laboratory, increased power in the lower frequency bands of the EEG has turned out to be a valid indicator of attentional withdrawal. Applying these measures to more realistic scenarios, by means of mobile EEG, revealed a broadband reduction of oscillatory power. This broadband reduction, however, appears to mask specific spectral effects, as it seems to reflect primarily neural noise. Based on recent data we try to evaluate new measures, which are tailored towards representing cognitive effects in specific frequency bands. Our aim is to estimate mental states in realistic environments and to be able to detect phases of reduced attentional engagement.

Christiane Glatz, Marie Lahmer, Makoto Miyakoshi, Lewis Chuang

Whilst driving, we attend to the road to ensure that we stay on it. Nonetheless, unexpected events might demand our immediate attention instead; for instance, the sudden appearance of collision hazards or jaywalkers along the road. As we approach such objects, they loom, which is to say that they increase in retinal size. Also, they might emit looming sounds, which increase in loudness. Our brains respond preferentially to looming stimuli; past research has primarily focused on multisensory integration or crossmodal influences. In my talk, I will present our investigations on how looming sounds could influence visual attention in steering environments. The first study was performed in a driving simulator whereby we found that looming sounds promoted faster braking times to the unexpected appearance of collision objects, relative to comparable sounds. EEG analyses revealed that differences in the activity of BA6 underlie this performance benefit, suggesting that looming sounds heighten arousal and preparatory activity for braking responses. In a second study, we show that auditory looming cues resulted in faster discrimination of peripheral visual targets during a continuous visuo-motor steering. EEG analyses suggested two complementary networks of preferential activity for looming over static auditory cues (BA23, BA19, and BA7) and for static over looming auditory cues (BA8, BA45, and BA10). Respectively, they suggest that looming cues promote voluntary spatial orienting and experience less inhibition in prioritising this over the primary steering task. To sum, looming sounds help us to attend appropriately to objects that we might collide with during steering.

Human behaviour is a contributing component in up to 94% of vehicle crashes [1]. A common “solution” is to attribute these crashes to human error and settle with that. A more productive way forward is to define human behaviour as a product of its settings, where a suitable vehicle and working environment will enable the drivers to act with a minimized probability of error. It comes without saying that in this latter approach, a detailed picture of driver behavior is needed. This detailed picture is usually obtained by evaluating driving performance, reaction times and glance behavior, where foveal vision is used as a proxy for attention [2]. To get a fuller picture of what is happening inside the driver’s head, we have found it useful to complement these measures with EEG data. This presentation will provide an overview of our EEG research in car driving. This includes the development of a close to real-time algorithm for artifact handling that has been tailored to active individuals in a driving setting [3], an investigation of local sleep in car drivers where signs of sleep need (theta content) in source localised motor-related parts of the brain preceded lane departures [4], the relation between cognitive overload and eye fixation related potentials (EFRP) [5], and finally that underload reduces attention allocation when driving semi-automated vehicles [6]. All in all, EEG information has helped us to get a richer understanding of driver behavior which eventually will help us reduce the number and severity of crashes.

References

Today, five out of ten diseases worldwide resulting in long-term disability are related to the central nervous system. Due to the immense complexity and inter-individual variability of the human brain there are still no effective treatment options for many serious neurological and psychiatric disorders such as stroke, major depression, schizophrenia or dementia. Recent advancements in sensor technology and computational capacities resulted in the development of brain/neural-machine interfaces (BNMIs) that translate electric, magnetic or metabolic brain activity into control signals of external devices, robots or machines. Moreover, novel transcranial magnetic and electric brain stimulation (TMS/TES) systems were developed allowing for direct modulation of brain activity. However, current BNMIs are limited by the low information extraction rate constraining fluent direct brain-computer interaction. Furthermore, as simultaneous assessment of brain oscillations during TES was regarded unfeasible due to stimulation artefacts, current TES systems can only deliver “open-loop” stimulation unrelated to the underlying dynamic brain states resulting in highly variable TES effects.

The talk will describe how combing both techniques into a neuroadaptive BNCI might overcome these limitations and lead to new and more effective treatments strategies for neurological and psychiatric disorders. Besides addressing the feasibility of assessing brain oscillations during TES, the talk will also provide an overview of how BNMIs can be taken out of the lab, e.g. to restore activities of daily living after quadriplegia and improve motor function after stroke. The next steps towards the development and application of neuradaptive BNMIs will be depicted, and possible neuroethical dimensions discussed.

Stroke rehabilitation has to address many factors that are impaired by brain lesion caused by brain hemorrhage. Patients show diverse symptoms reaching from movement disorders, impaired sensation and cognitive deficits, which must be addressed by therapy. To compensate movement disorders, exoskeletons can be applied. They can sense movements and support them to enable a patient with smallest remaining muscular activity to regain control over a disabled limb. Electromyogram and force measurements can be used to adapt the support to the patients need. In case that no control of the limb is left, brain activity can be analyzed by embedded brain reading (eBR) to infer on the intention of the patient and to trigger movements implicitly. For a successful therapy not only assist as needed is required but further, patients must be able to follow instructions – mental stress must be avoided to assure optimal therapy conditions. Again, eBR can be used to detect task load on a patient while he or she is performing an ongoing action, e.g., performing a specific arm movement to infer whether a patient (while performing the movement) is still able to understand instructions by the therapist or a serious game. This presentation will show new therapy approaches for upper limb rehabilitation made possible by means of a newly developed exoskeleton (1) with highly adaptive embedded control (1,2) combined with movement intention recognition based on EEG (3), EMG (4) and eye tracking data (5) as well as task load detection based on P300 related activity under simple and multi-tasking conditions (6).

(1) Elsa A. Kirchner, Niels Will, Marc Simnofske, Luis M. V. Benitez, Bertold Bongardt, Mario M. Krell, Shivesh Kumar, Martin Mallwitz, Anett Seeland, Marc Tabie, Hendrik Wöhrle, Mehmed Yüksel, Anke Heß, Rüdiger Buschfort, Frank Kirchner (2016), Recupera-Reha: Exoskeleton Technology with Integrated Biosignal Analysis for Sensorimotor Rehabilitation, In 2. Transdisziplinäre Konferenz “Technische Unterstützungssysteme, die die Menschen wirklich wollen”, 12.12.-13.12.2016, Hamburg, Elsevier, pages 504-517, Dec/2016.

(2) Hendrik Wöhrle, Marc Tabie, Su-K. Kim, Frank Kirchner, Elsa A. Kirchner (2017), A Hybrid FPGA-Based System for EEG- and EMG-Based Online Movement Prediction, In In Sensors – Open Access Journal, MDPI, 2017 Jul 3;17(7). pii: E1552. doi: 10.3390/s17071552.

(3) Anett Seeland, Marc Tabie, Su-K. Kim, Frank Kirchner, Elsa A. Kirchner (2017), Adaptive multimodal biosignal control for exoskeleton supported stroke rehabilitation, Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), 5-8 Oct. 2017, Banff, AB, Canada, DOI: 10.1109/SMC.2017.8122987

(4) Luis M. V. Benitez, Niels Will, Marc Tabie, Steffen Schmidt, Matias Jordan, Elsa A. Kirchner (2013), Exoskeleton Technology in Rehabilitation: Towards an EMG-Based Orthosis System for Upper Limb Neuromotor Rehabilitation, In: Journal of Robotics 2013(10):1-13, DOI: 10.1155/2013/610589

(5) Elsa A. Kirchner, Jan Albiez, Anett Seeland, Mathias Jordan, Frank Kirchner (2013), Towards Assistive Robotics for Home Rehabilitation, In Proceedings of the 6th International Conference on Biomedical Electronics and Devices, (BIODEVICES-13), 11.2.-14.2.2013, Barcelona, Feb/2013.

(6) Elsa A. Kirchner, Su-K. Kim (2018), Multi-Tasking and Choice of Training Data Influencing Parietal ERP Expression and Single-Trial Detection—Relevance for Neuroscience and Clinical Applications, In: Frontiers in Neuroscience, volume 12, pages 188, DOI: 10.3389/fnins.2018.00188