Deep brain stimulation (DBS) has been applied as an effective therapy for treating Parkinson's disease or essential tremor. Several open-loop DBS control strategies have been developed for clinical experiments, but they are limited by short battery life and inefficient therapy. Therefore, many closed-loop DBS control systems have been designed to tackle these problems by automatically adjusting the stimulation parameters via feedback from neural signals, which has been reported to reduce the power consumption. However, when the association between the biomarkers of the model and stimulation is unclear, it is difficult to develop an optimal control scheme for other DBS applications, i.e., DBS-enhanced instrumental learning. Furthermore, few studies have investigated the effect of closed-loop DBS control for cognition function, such as instrumental skill learning, and have been implemented in simulation environments. In this paper, we proposed a proof-of-principle design for a closed-loop DBS system, cognitive-enhancing DBS (ceDBS), which enhanced skill learning based on in vivo experimental data. The ceDBS acquired local field potential (LFP) signal from the thalamic central lateral (CL) nuclei of animals through a neural signal processing system. A strong coupling of the theta oscillation (4-7 Hz) and the learning period was found in the water reward-related lever-pressing learning task. Therefore, the theta-band power ratio, which was the averaged theta band to averaged total band (1-55 Hz) power ratio, could be used as a physiological marker for enhancement of instrumental skill learning. The on-line extraction of the theta-band power ratio was implemented on a field-programmable gate array (FPGA). An autoregressive with exogenous inputs (ARX)-based predictor was designed to construct a CL-thalamic DBS model and forecast the future physiological marker according to the past physiological marker and applied DBS. The prediction could further assist the design of a closed-loop DBS controller. A DBS controller based on a fuzzy expert system was devised to automatically control DBS according to the predicted physiological marker via a set of rules. The simulated experimental results demonstrate that the ceDBS based on the closed-loop control architecture not only reduced power consumption using the predictive physiological marker, but also achieved a desired level of physiological marker through the DBS controller.

IntroductionAccording to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition.MethodsIn the present study, we explored a 134-person dataset consisting of learners’ feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions.ResultsResults showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners’ performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test.DiscussionTogether, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon. No use, distribution or reproduction is permitted which does not comply with these terms. Introduction According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition. Methods In the present study, we explored a 134-person dataset consisting of learners’ feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions. Results Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners’ performance level. pmc-status-qastatus 0 pmc-status-live yes pmc-status-embargo no pmc-status-released yes pmc-prop-open-access yes pmc-prop-olf no pmc-prop-manuscript no pmc-prop-legally-suppressed no pmc-prop-has-pdf yes pmc-prop-has-supplement no pmc-prop-pdf-only no pmc-prop-suppress-copyright no pmc-prop-is-real-version no pmc-prop-is-scanned-article no pmc-prop-preprint no pmc-prop-in-epmc yes pmc-license-ref CC BY section-at-acceptance Learning and Memory Introduction The process of acquiring motor skills is typically marked by rapid improvements in performance observed early in practice followed by smaller adjustments when the learner achieves a higher skill level (e.g., power law of practice; Newell and Rosenbloom, 1981 ). To expand our understanding of this relationship, in the present study, we focus on the application of reinforcement learning to explain motor skill acquisition by adopting a mechanistic approach to investigate one of its main drivers, reward-prediction errors. In human research, reward-prediction errors have been studied through the measure of the reward positivity (RewP), an event-related potential (ERP) component derived from the electroencephalogram (EEG). This figure is a slightly modified version of Figure 1 presented in Bacelar et al. (2022) . Permission to reproduce this figure has been obtained by the authors. Procedures Acquisition phase To determine baseline skill level, a 10-trial pretest without feedback was carried out before the acquisition phase. (Participants were allowed to see the target for 10s before initiating the pretest). (D) Model predictions for the relationship between single-trial RewP and single-trial RE as a function of average RE (in the graph represented by the variable performance level ). Aggregate RewP and retention Results of the analysis of the relationship between aggregate RewP and retention showed no Thus, depending on the learner’s skill level, a practitioner or a coach might decide to rely on different error correction strategies to better support their learners. For example, they might use knowledge of results , a type of augmented feedback that informs the learner about the success of an action in reference to a goal (e.g., you missed the target by 32 cm; Schmidt and Lee, 2019 ) to assist high-performing learners who can rely on a stronger internal model to compute reward-prediction errors based on the outcome feedback, and select and execute movements that maximize the reward-prediction errors. In line with the challenge-point framework ( Guadagnoli and Lee, 2004 ), another strategy that an instructor might consider is adjusting the difficulty of the task at hand based on how challenging the task is for the learner, which is directly associated with the learner’s current skill level, and also the goals of practice (e.g., practice-to-learn versus practice-to-maintain; Hodges and Lohse, 2022 ). According to this framework, learning is enhanced when individuals practice within their range of optimal difficulty. At least at the behavior level, recent evidence suggests that reinforcement learning approaches may lead to better motor skill retention compared to other modes of learning ( Magelssen et al., 2024 ; Truong et al., 2023 ). Thus, we encourage future investigations into the application of reinforcement learning theory and its neural mechanisms to explain motor skill learning.

A neural network controller is proposed for the motion control of robot manipulators with force/torque feedback signals. This controller is trained with reinforcement learning algorithms and a model is extracted from the synaptic weights within the neural network. This model is continuously refined by the feedback signals to ensure its validity even in a stochastic and non‐stationary environment. With this model and the real‐time force/torque feedback data, the robot can acquire a fine skill for a particular assembly task for which it is trained.

Glucagon-like peptide-1 (GLP-1) is a nutrient-responsive hormone classically associated with glucose homeostasis and food intake control, yet its receptor is broadly expressed throughout the central nervous system (CNS) in circuits governing complex cognitive processes. Here, we synthesize emerging evidence from preclinical models and human studies demonstrating that GLP-1 receptor (GLP-1R) signaling modulates multiple cognitive domains, including reward and motivational processes relevant to obesity and substance use disorder, affective-related behaviors, and learning and memory. We propose a unifying framework in which GLP-1R signaling acts as a key interoceptive indicator of energy status, dynamically modulating cognitive and behavioral output in accordance with metabolic state. In animal models, GLP-1R activation dampens effort-based seeking for palatable food and addictive drugs alike, exerts bidirectional effects on affective behavior (e.g., anxiety-like behavior), and promotes synaptic plasticity, learning, and neuroprotection. Clinical studies further indicate that GLP-1R agonists alter neural responses to reward-related cues, influence mood-related outcomes, and are associated with reduced risk of cognitive decline, although results pertaining to benefits in neurodegenerative disease remain mixed. Collectively, these data position GLP-1R signaling as a metabolic-cognitive interface linking internal energy status to reward processing, affective regulation, and memory, and highlight the importance of disentangling direct central actions from indirect secondary metabolic effects when evaluating the therapeutic potential of GLP-1-based interventions for psychiatric and cognitive disorders.

Menschen üben motorische Fähigkeiten, wie das Greifen einer Kaffeetasse oder das Fangen eines Gegenstandes mit großer Leichtigkeit aus. Sogar schwierigere und komplexe Aufgaben wie Fahrrad fahren oder Tischtennis spielen sind oft bis zu einem gewissen Grad schnell zu erlernen. Auch wenn viele dieser Fähigkeiten nur auf einer kleinen Anzahl elementarer Bewegungen beruhen, ist der Mensch dennoch in der Lage eine Vielzahl unterschiedlicher Aufgaben zu bewältigen. Roboter hingegen sind immer noch festgelegt auf eine bestimmte Anzahl motorischer Abläufe, die in wohl definierten Arbeitsumgebungen ausgeführt werden. Die Modellierung und das Lernen motorische Fähigkeiten ist daher ein wichtiger Aspekt in der Robotik. Mathematische Modelle der motorischen Kontrolle des Menschen können daher genutzt werden um Roboter zu entwickeln, die in der Lage sind komplexe Aufgaben in einem von Menschen bewohnten Umfeld zu bewältigen. Solche Modelle können der Schlüssel zu robusten, effizienten und menschenähnlichen Bewegungsabläufen sein. Im Gegenzug kann die Reproduktion von menschenähnlichen Bewegungsverhalten auf Robotern auch nützlich sein, um diese mathematischen Modelle zu verifizieren. Auch wenn biomimetische Modelle eine große Hilfe sein können, um die Lücke zwischen Mensch und Roboter zu schließen, stellen sie dennoch einen fixen Plan dar, der auf eine bestimmte Anzahl von Szenarien begrenzt ist. Eine wichtige Eigenschaft des Menschen ist jedoch die Fähigkeit, motorische Abläufe an neue