Neurocognitive consequences of hand augmentation · 1 day ago · Neurocognitive consequences of...

Neurocognitiveconsequencesofhand

augmentationPaulinaKieliba1,DanielleClode1,RoniOMaimon-Mor1,2,TamarR.Makin1,2*

1InstituteofCognitiveNeuroscience,UniversityCollegeLondon,17QueenSquare,London

WC1N3AZ,UK2WINCentre,UniversityofOxford,OxfordOX39DU,UnitedKingdom

Abstract

Fromhandtoolstocyborgs,humanshavelongbeenfascinatedbytheopportunitiesafforded

byaugmentingourselves.Here,westudiedhowmotoraugmentationwithanextrarobotic

thumb(theThirdThumb)impactsthebiologicalhandrepresentationinthebrainsofable-

bodiedpeople.Participantsweretestedonavarietyofbehaviouralandneuroimagingtests

designed to interrogate the augmented hand’s representation before and after 5-days of

semi-intensivetraining.TrainingimprovedtheThumb’smotorcontrol,dexterityandhand-

robotcoordination,evenwhencognitiveloadwasincreasedorwhenvisionwasoccluded,

andresultedinincreasedsenseofembodimentovertheroboticThumb.Thumbusagealso

weakenednaturalkinematichandsynergies.Importantly,braindecodingoftheaugmented

hand’smotorrepresentationdemonstratedmildcollapsingofthecanonicalhandstructure

following training, suggesting that motor augmentation may disrupt the biological hand

representation. Together, our findings unveil critical neurocognitive considerations for

designinghumanbodyaugmentation.

.CC-BY 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted June 16, 2020. . https://doi.org/10.1101/2020.06.16.151944doi: bioRxiv preprint

https://doi.org/10.1101/2020.06.16.151944

http://creativecommons.org/licenses/by/4.0/

IntroductionTherearemanythingsthatwecoulddobetterifwehadmorefingersinourhand.Engineers

arecurrentlydevelopingextraroboticfingersandevenentirearmsaimedtoaugmentour

bodies by expanding our natural motor repertoire (1-5). Despite rapid advancements in

augmentativetechnologies, littlenotice isgiventothecrucialquestionofhowthehuman

brainmightsupportthem.Theaugmentativedevicesaimtochangethewayweinteractwith

theenvironment,whichentailschangestohowwemoveandoperateourbiologicalbody.

Hereweaskedwhetherthehumanbraincanaccommodatemotorcontrolofanextrarobotic

finger,focusingonitsimpactontheneuralrepresentationofthebiologicalhand.

Handrepresentationintheprimarysensorimotorcortexofthebrainhasawell-established

functionalstructurethatdevelopsveryearlyon(6,7).Itishighlyconsistentwithin(8)and

across(9)participantsandispreservedevenafterseverelossofmotorfunctionsduetoe.g.

stroke (9), spinal cord injury (10), disability (11) or even hand amputation (12-14). Hand

representationhasbeensuggestedtoreflectdailyhanduse(9),withstudiesshowingthatit

may be altered under constrained circumstances.Most notably inmusicians’ dystonia, a

clinicalconditioninvolvingincreasedfingerenslavement,theindividualisedrepresentationof

singlefingershasbeenshowntocollapse(15,thoughsee16).

Herewetrainedable-bodiedpeopletouseanextraroboticthumb(theThirdThumb,created

byDaniClode(17),hereafter“Thumb”)overthecourseof5days,includingbothlab-based

andin-the-wilddailyuse.TheThumbisa3D-printedsupernumeraryroboticfinger,withtwo

degreesoffreedom,controlledwithpressureexertedwiththebigtoes,designedtoextend

the natural repertoire of hand movements (Figure 1A-B). During training, we tracked

(biological)fingercoordinationandcompareditwithnormalhanduse.Wetestedforchanges

inmotorcontrolandembodimentoftheThumb,aswellashand-Thumbcoordinationbefore

andaftertraining.Augmentedparticipantswerecomparedtoacontrolgroupthatunderwent

similartrainingregimewhilewearingtheThumbwithoutbeingabletocontrol it.Wealso

examinedhowthesensorimotorandbodyrepresentationoftheaugmentedhandchanged

following Thumb training. We hypothesised that successful hand-robot cooperation and


https://doi.org/10.1101/2020.06.16.151944


subsequent change of finger co-use will update both biological and artificial body

representation.

Results

Improvedmotorcontrol,hand-ThumbcoordinationandThumbembodiment

followingdailyThumbusage

Wefirstcharacterisedmotorperformanceoftheaugmentedhandthroughoutthe5daysof

usage.Augmentationparticipantscompletedfivedailyin-labtrainingsessions(1.58±0.22hr;

mean±std)andwereadditionallyencouragedtousetheThumboutsidethelab(2.61±1.18hr;

self-reported). The average use time, as quantified by the automatic usage logs, was

2.95±0.84hrperday,outofwhichatotalof1.37±0.49hrinvolvedactiveThumbmovement.

Figure1.Experimentaldesign.(A-B)TheThirdThumbisa3D-printedroboticthumb.Mountedontheside of the palm (1), the Thumb is actuated by twomotors (fixed to a wrist band), allowing forindependent control over flexion/abduction. The Thumb is powered by (2) an external battery,strappedaroundthearmandwirelesslycontrolledby(3)twoforcesensorsstrappedunderneaththeparticipant’sbigtoes.(C)Experimentaldesignfortheaugmentationgroup.(D-E)Examplesofthein-lab training tasksused forhand-Thumbcollaboration (building a Jenga tower), shared supervision(holdingaplasticcupwhileextractingamarblewithaspoon)andThumbindividuation(stackingtapeswiththeThumbwhilethebiologicalhandisoccupied).Participantsshowedsignificantperformanceimprovementsacrosstrainingsession.Asterisksdenotesignificanteffectoftimeat***p<0.001.

Duringdailytrainingsessions,participantswerepresentedwithavarietyofreaching,grasping

andin-handmanipulationtasksdesignedtocreatedifferentchallengesfortheThumbusage

(see Supplementary Video 1 for examples). Augmentation participants showed significant


https://doi.org/10.1101/2020.06.16.151944


improvementonallthetrainingtasks(maineffectoftimeforalltasks:p<0.001,ηp2>0.5Figure

1D-E&SupplementaryFigureS1).Motorcontrolwasfurtherassessedusingahand-Thumb

coordinationtask,requiringparticipantstoopposetheThumbtotheirbiologicalfingers.Even

thoughcontrollingtheThumbwiththebigtoesmayseemunusual,participantswereableto

successfullyperformthehand-Thumbcoordinationtaskevenatbaseline(Figure2B),though

thisperformancewassignificantly improvedafter training.Significant improvementswere

observedbothduringdailytraining(F(4,76)=28.24,p<0.001,ηp2=0.6Figure2A-C),andwhen

comparingtheperformancepre-andpost-the5daysoftrainingusingasequentialvariation

of the same task (see Methods). Here, augmentation participants showed significant

improvements, not only with vision (t(19)=8.96, p<0.001, ηp2=0.81), but also when

blindfolded (t(19)=7.40, p<0.001, ηp2=0.74, Figure 3A), indicating improved Thumb

proprioception.Importantly,theparticipant’smotorperformancewiththeThumbwasnot

impacted by increased cognitive load incurred during a dual-task involving numerical

cognitionandmotorcontrol,performedduringthefirstandlastdaysofthetraining.Thiswas

demonstratedbythe lackofsignificantcognitive loadxsession interaction(F(1,16)=0.003,

p=0.959) and a non-significantmain effect of the cognitive load (F(1,16)=2.465, p=0.136;

Figure2D).Thiswasfurtherconfirmedusingaone-tailedBayesiant-tests,whichyieldeda

BayesianFactor(BF)of0.13and0.14forthefirst/lastdayoftraining, indicatingmoderate

evidence in support of the null hypothesis. As such, participants learned to operate the

Thumbunderavarietyofcircumstances,extendingbeyondtheirspecifictraining.

WealsoassessedtheperceivedsenseofembodimentovertheThumbfollowingthetraining

period,relativetobaseline.Participantswereaskedtorespondtostatementsrelatingtokey

embodimentfeatures(18,19).Augmentationparticipantsreportedasignificantincreaseof

embodimentineachofthefourcategories(bodyownership:t(13)=6.57,p<0.001,ηp2=0.77;

agency: t(13)=4.07, p<0.001, ηp2=0.56; body image: t(13)=5.215, p<0.001, ηp2=0.68;

somatosensation:t(13)=6.032p<0.001,ηp2=0.74;Figure3B).

Astheimprovementsdescribedabovecouldbeskewedduetotaskrepetition,wealsotested

agroupof11controlparticipants,whounderwentsimilarpre-andpost-testsandtraining

regime(seeMethods)butworeastaticversionoftheThumbforthesamedurationoftime-

4.11±1.06hrperday(t(29)=0.526,p=0.6,BF=0.39),outofwhich2.93±1.34outsideofthelab-


https://doi.org/10.1101/2020.06.16.151944


basedtrainingsessions(t(29)=-0.697,p=0.49,BF=0.42forwear-timegroupcomparison).The

control group showed significantly lower improvement on the pre-post sequential hand-

Thumb coordination (significant effect of group revealed by ANCOVA - with vision:

F(1,27)=22.86,p<0.001,ηp2=0.44;blindfolded:F(1,27)=11.96,p=0.002,ηp2=0.28)anddidnot

embody the extra Thumb to the same extent as the augmentation group (agency:

F(1,21)=10.013,p=0.009,ηp2=0.285;bodyimage:F(1,21)=11.16,p=0.012,ηp2=0.26).Forbody

ownership, the group effect was trending (F(1,21)=4.07, p=0.057, ηp2=0.16), while for

somatosensationscores,thegroupcomparisonwasnonsignificant(BF=0.48).Theseresults

indicatethatactiveusage iscritical fordevelopingproprioceptionandembodimentofthe

roboticThumb.

Figure2.Trainingoutcomes.(A-C)Augmentationparticipantsshowedsignificantdailyimprovementonthehand-Thumbcoordinationtask.(D)MotorperformancewiththeThumbwasnotimpactedbyincreasedcognitive loadduring the first and last trainingdays. (E)Handkinematicsdata collectedduring the training sessions. The first principal competent (synchronisedmovementacross all fivefingers) captured less variance in the augmentation group compared to controls, indicating lesssynchronisedmovements.(F-G)Theaugmentationgroupshowedlowerinter-fingercoupling,relativetocontrolsduringThumbuse,indicatingchangetothenaturalfingercoordination.Thebarsdepictgroup means, error bars represent standard error of the mean. Individual dots correspond toindividualsubjects’averageinter-finger(D1-D5)coordinationscoresaspredictedbythelinearmixedmodel(seeMethods).Asterisksdenotesignificanteffectsat*p<0.05and***p<0.001.


https://doi.org/10.1101/2020.06.16.151944


Handaugmentationimpactsmotorcontrolofthenaturalhand

Next,weinvestigatedtheimpactofhandaugmentationonmotorcontroloftheaugmented

(right) hand. We first examined the complexity of the hand movements (i.e. kinematic

synergies)duringThumbuse,capturedwithaCybergloveduringthein-labtraining.Wefound

that in general more principal components (kinematic synergies) were needed in the

augmentationgroup,comparedtothecontrolgroup,toexplainthe80%ofthetotalvariance

of the hand movements (F(1,22)=5.52, p=0.03, ηp2=0.2, Supplementary Figure S3B). This

differencewas however strongly driven by the amount of variance explained by the first

principalcomponent,correspondingtothecoordinatedflexionofallfingers(Supplementary

Figure S3A). Indeed, the variance explained by this inter-finger synergy was significantly

decreasedintheaugmentationgroupcomparedtocontrols(F(1,22)=6.27,p=0.02,ηp2=0.22,

Figure 2E), while no difference was found between the first and the last day of training

(F(1,22)=2.57,p=0.12,BF=0.62).Sincetheremainingprincipalcomponentsrepresentmore

intricatefingermovements,thedecreaseofvarianceexplainedbythefirstkinematicsynergy

suggestsmore finger individuation in the augmentation group. To uncovermoredetailed

changesinfingercoordination,wethenlookedatthedegreeofkinematiccouplingbetween

individualdigitpairs.Hereagain,nodifferencesinfingercoordinationwerefoundbetween

thefirstandthelastdayoftraining(maineffectoftime:F(1,23)=1.3,p=0.27,BF=0.17).For

the augmentation group, this finding indicates that the strategies implemented for

incorporatingtheThumbintothemotorrepertoireduringday1weregenerallypreserved

throughouttraining.ConsistentwiththePCAresults,wefoundsignificantdifferencesinfinger

coordination implemented across groups (group x finger-pair interaction: F(9,414)=2.66,

p=0.005),withanoveralldecreaseininter-fingercouplingintheaugmentationgrouprelative

tocontrols(maineffectofgroup:F(1,23)=6.98,p=0.01,Figures2F-G).Togethertheseresults

indicateapotentialbreakageofnaturalfingersynergiesduringThumbuse.

Changesininter-fingermotorcontrolwerefurtherinvestigatedthroughforceenslavement

(involuntaryforceproductionbynon-intendedfingers),measuredpre-andpost-Thumbuse

(Figure3C-D).Whilenosignificantchangestotheoverallpatternoftheforceenslavement

werefound,intheaugmentationgrouptherewasatrendtowardsincreasedenslavement

causedbythebiologicalthumbfollowingextraThumbuse(F(1,17)=3.36,p=0.08,firstcolumn


https://doi.org/10.1101/2020.06.16.151944


of the pre-post difference matrix in Fig3D). However, this effect was not robust, as

demonstrated by a lack of significant time x group interaction when comparing the

augmentationgroupwithcontrols(F(1,27)=1.5,p=0.23,BF=0.57).

Figure3.Behaviouralcorrelatesofhandaugmentation.(A)Augmentationparticipantsshowedgreaterimprovement than controls on a hand-Thumb coordination task conducted before and after thetraining period. Participants showed improved performance even while blindfolded, indicatingincreasedThumbproprioception.(B)Self-reportedThumbembodimentincreasedsignificantlyintheaugmentationgroupfollowingThumbtraining.(C-D)Participantsperformedindividuatedkeypresseswithaninstructedfinger,whiletheforcesexertedwiththenon-instructedfingersweremeasuredtoobtain a 5x5 force enslavementmatrices.Matrices presented here depict the group average. Nosignificantchangeintheaverageenslavementwasobserved.Asterisksdenotesignificanteffectsat*p<0.05,**p<0.01,***p<0.001.

Wealsoexaminedpotentialchangestobodyimage(perceptionsandattitudesconcerning

one'sbodyrepresentation;(20)).Usingatactiledistancestask,wefoundnosignificantpre-

topost-changesintactilebiases(t(18)=0.164,p=0.87,BF=0.24).Similarly,wedidnotobserve

anysignificantchangesincurredbyThumbusageusingavisualhandlateralisationtask.While

participantsdidgetfasteratvisualhandlateralisation(F(1,16)=6.89,p=0.018),thiswasthe

casefor judgementsmadebothfortheaugmentedandthenon-augmentedhand(handx

session interaction F(1,16)=0.019, p=0.89, BF=0.326). These findings indicate that hand

augmentation does not influence all aspects of body representation, with body image

remainingstableirrespectiveofThumbusage.


https://doi.org/10.1101/2020.06.16.151944


CanonicalrepresentationstructureshrinksfollowingextraThumbuse

Havingobservedthataugmentationparticipantsshowchangedpatternoffingercoordination

whencomparedtothecontrols,wesoughttounderstandwhetherThumbusagecanalter

the biological hand representation in the sensorimotor cortex. As such,we used fMRI to

compare neural hand representation before and after Thumb use. During the scans,

participants were required to make individuated finger movements with their biological

fingers.NotethatduetoMRIsafetyconsiderations,participantswerenotwearingtheThumb

during the scans.Herewealso tested thenon-augmented (left)hand,providingawithin-

participantcontrol.

First,weusedaunivariateapproachtointerrogateactivitylevelswithinthe(independently

localised)corticalterritoryofthebiologicalhands.Wefoundthattheaverageactivitywas

decreasingfromthepre-topost-scan(maineffectoftime:F(1,18)=7.89,p=0.012,ηp2=0.3).

However, this decrease was not specific to any of the hands (hand x time interaction:

F(1,18)=1.66,p=0.21,BF=0.37).

Figure4.Canonicalhandrepresentationshrinksfollowinghandaugmentation.(A)ThesensorimotorROIwasdefinedusinganatomicallydefinedM1handROIincludingBroadmannareaBA4(Freesurfersegmentation), partiallyoverlapping the central sulcus. (B)Groupmeandissimilaritymatrixof theright(augmented)handpre-andpost-training.(C)Theaverageinter-fingerdissimilarityoftheright(augmented),butnot the left (non-augmented)handdecreasedsignificantly followingThumbuse.


https://doi.org/10.1101/2020.06.16.151944


The bars depict group mean, error bars represent standard error of the mean. Individual dotscorrespond to individual subjects’ average distance as predicted by the linear mixed model (seeMethods). (D) Multidimensional scaling (MDS) depiction of the left and right (augmented) handrepresentational structures. Ellipses indicate between-participant standard errors. Darker coloursrepresentthepostscan,whereaslightercoloursrepresentthepre(baseline)scan.Red=D1,Yellow=D2, Green = D3, Blue = D4, Purple = D5. Asterisks denote significant effects at * p<0.05 and ***p<0.001.

To investigatechangestotheaugmentedhand’srepresentationalstructure,weestimated

thedissimilaritybetweenactivitypatternselicitedby individual fingers’movements inthe

sensorimotor cortex, asmeasured using cross-validatedMahalanobis distance (21). Small

inter-fingerdissimilarityvalues indicatethattherepresentationof thetwofingers ismore

similar/overlapping, while larger dissimilarity implies more individuated finger

representation.Augmentationparticipantsshowedsignificantlyreduceddissimilarityofthe

augmented (right) hand representation in the sensorimotor cortex following Thumb use

(t(25.1)=2.3,p=0.03,Figure4).Thisshrinkingeffectwasspecifictotheaugmentedhand,as

demonstratedbyasignificanthandxtimeinteraction(F(1,722)=12.89,p<0.001,Figure4).

Thereductionofinter-fingerdissimilaritywasdiminishedinafollow-upscan,conducted7-10

daysaftertheendoftraining,collectedfromasubsetofavailableparticipants(n=12).This

wassupportedbymoderateevidence(BF=0.31)foranulldifferencebetweentheaverage

distancesmeasuredduringpre-andfollow-upsessions,indicatingthattheshrinkageofthe

canonicalhandrepresentationalstructuredependsonrecentThumbuse(notehoweverthat

withthissubsetofparticipantstheinitialdifferencebetweenthepreandpostsessionswas

ambiguous – BF=0.58). We also examined the inter-finger representational structure’s

typicality, that is the correlation of the individual’s representational dissimilarity matrix

(RDM)withagroupaverageRDM,computedfromthepre-data,andfoundnosignificantpre-

topost-differences(BF=0.67).ThesefindingsshowthatusingtheextraThumbnotonlyalters

themotorcontrolofthebiologicalhand,butalsoimpacttherepresentationalsimilarityof

individual fingers’ representation in the brain. Crucially, this effect was observed while

participantswerenotusingorevenwearingtheThumb.


https://doi.org/10.1101/2020.06.16.151944


Effectiveconnectivitybetweenhandandfeetmotorareasremainsinvariant

Finally,weexploredtherelationshipbetweenthehand’sandtoes’(controllingtheThumb’s

movements)representationinthesensorimotorcortex,butfoundnosignificantdifferences

followingtraining,relativetobaseline.First,weusedaunivariateapproachtoexaminefeet-

specificactivitywithin theaugmentedhandterritory,and found that ithadnot increased

significantly (t(18)=0.809, p=0.43, BF=0.27). Next, we examined the functional coupling

betweenthesensorimotorhandandfeetareas,usingresting-statefunctionalconnectivity

analysis. We extracted the mean time course from the augmented and non-augmented

hands’ territories, as well as from the feet territory, and used partial correlations to

investigate the resting-state coupling between the augmented hand and the feet (while

accountingforconnectivitybetweenthetwohands;t(19)=0.774,p=0.45,BF=0.3).Together,

theseresultssuggestthatwhileaugmentationmightpromoteplasticitylocally(i.e.between

fingers),theglobalhomunculuslikelyremainslargelyunchanged.

DiscussionHere,weprovidethefirstcomprehensivedemonstrationofsuccessfulmotorintegrationofa

roboticaugmentationdevice(theThirdThumb)andexplorehowaugmentationimpactsthe

user’s sensorimotor hand representation in the brain. After only 5 days of Thumb use,

participants showed significant improvements in six-fingered motor performance across

multipletasks.InadditiontoindividuatedcontroloftheextraThumb,participantswereable

to integrate Thumb motor control with the movements of their natural hand, requiring

collaboration, shared supervision and hand-robot coordination. Motor performance was

greatly improved even without visual feedback and remained stable under increased

cognitive load.We further show that hand augmentation resulted in increased sense of

embodimentovertheThumb-akeygoalforsuccessfulaugmentation(22).Bydemonstrating

successful adaptation tomotor augmentationunderdiverseecological taskdemands,our

findingsconstitutealeapbeyondearlierpioneeringproof-of-conceptaccountsofsuccessful

usageofextraroboticfingers(1,4,23-25)orarms(3,5)underrestrictedcircumstances.


https://doi.org/10.1101/2020.06.16.151944


Importantly,successfuladoptionofaugmentativetechnologiesreliesnotonlyontheuser’s

proficiencyinoperatingtheroboticdevice.Afurtherchallengeforaugmentationistoensure

thatthedeviceusagewillnotimpactusers’abilitytocontroltheirbiologicalbody,especially

whiletheaugmentativedevice isnotbeingused.Therefore,acriticalquestionforanyone

interested in safemotor augmentation is whether it would incur any costs to the user’s

biologicalbodyrepresentation.Thisconcernisrootedinpreviousresearchofbrainplasticity,

demonstratingthatourmotorexperienceshapesthestructureandfunctionofthenervous

system(9).Assuch,sincemotoraugmentation isdesignedtochangethewaywe interact

withtheenvironment,itisreasonabletopredictthatitwillreshapetheneuralbasisofour

biologicalbody.Withthatinmind,ourinvestigationwasfocusedonchangesincurredtothe

biologicalbody representationevenwhile theThumb isn’tbeingoperated. This approach

allowsforourfindingstobeeasilygeneralisedtoanyotherformofhuman-robotinterfaces.

Traditionally, body representation in the sensorimotor cortex is considered to be highly

adaptive even in the adult brain (26, 27) however recent research contributes a new

perspectiveonitsmalleability(12,13).Toolshavebeensuggestedtoupdatethebiological

bodyrepresentation,forexamplebytool-body integration(28-30).Yet,toolsarenormally

usedtoreplacethecapacityofthehand,ratherthantoaccompanyit.Therefore,whenusing

atool,oneisnotrequiredtoradicallyaltertheirhandfunction(forexample,theuserwill

chooseagripforthetool’shandlethatfitsthenaturalsynergiesofthefingers).Assuch,tool-

use does not entail an updated representation of the hand itself. Conversely, motor

augmentationinvitestheusertoreinventthewaytheyusetheirownbody.Thischallengeis

morecloselyakintotheacquisitionofanewandcomplexmotorskill–e.g.learningtoplay

thepiano.Here,researchhasdemonstratedthatlong-termtrainingleadstochangeinfinger

representations (31). Specifically, trained pianists (over the course of many years)

demonstrate reduced representational selectivity (lower representational dissimilarity)

relative tonovices.Thisevidence furtheremphasises theneedtoexaminehow long-term

motoraugmentationcanimpactthebiologicalhandrepresentation.

Here,weusedavarietyofpre-topost-measurestostudychangesinbodyrepresentation

following5daysofThumbusage.Whilesomeaspectsofbodyrepresentation(bodyimage,

largescaleconnectivityprofile)werefoundtobestable,semi-intensiveThumbusage(2.3–


https://doi.org/10.1101/2020.06.16.151944


6.3hoursperday)also resulted inmild,yet significantchanges to thesensorimotorhand

representation. First,motor integrationof theThumb resulted inbreakageof thenatural

fingercoordinationpatterns(kinematicsynergies)duringThumbuse,previouslyconsidered

asakeydeterminateofsensorimotorbrainorganisation(9,32,33).Suchanabruptchangein

fingercoordinationmay‘disorganise’theexistinghandrepresentation.Evidenceforaltered

sensorimotorcontrolofthebiologicalhandwasalsoapparentaftertraining,whentheThumb

wasnotbeingused,orevenworn.Specifically,weobservedareductionofrepresentational

selectivity (i.e. dissimilarity) between representational patterns of the 5 fingers, akin to

increased inter-finger overlapwhichhadbeenpreviously used as amarker for decreased

motorcontrol(15).Thiseffectmaybereversible(asindicatedbythefollow-upscantaken7-

10daysafterThumbusagehadceased),oreventransient.Yet,thisevidencenevertheless

suggest that motor augmentation might incur some costs to the augmented hand

representation. It is therefore crucial to consider what plasticity mechanisms might be

triggered during Thumb use to cause the observed changes in the biological hand

representation.

The “maladaptive” plasticity account highlights the fact that neural resources devoted to

handrepresentationarelimited,andthuschangestotheexistinghandrepresentationmight

incuracostonmotorcontrol(15).Here,theadoptionofanextraroboticthumbbyanadult

withastablehandrepresentationwillpromoteachangetoexistingbrainorganisation.We

used computational simulations to explore the feasibility of multiple potential plasticity

mechanisms that could trigger theobserved fMRI results (reduction in pairwise distances

betweenfingers;SupplementaryFigureS2).First,basedoninformationprocessingtheories

(34, 35), the integration of a new additional finger into the hand’s motor control could

impingeontheexistingrepresentationofthebiologicalfingers.Second,thechangeinfinger

coordination,observedduringtraining,mayalsoleadtoabruptchangesinexcitabilityprofiles

that can trigger homeostatic plasticity mechanisms and promote increased tonic, and

relativelywide-spreadinhibition(36).Thirdly,changestofingercoordinationduringThumb

usecanresultinincreasedfingerindividuation,leadingtoincreasedcorticalrepresentation

of individual fingers via Hebbian learning (cortical magnification, (37)). By simulating

“neuronal activity”over a fixed-sizeROI split into finger specific areas (seeMethods),we

foundthateachoftheseprocessesisconceptuallycapableofcausingtheobservedreduction


https://doi.org/10.1101/2020.06.16.151944


inrepresentationalselectivity(SupplementaryFigureS2).Itisalsopossiblethatthesedistinct

mechanismsinteractwitheachotherastheymayallberelevanttotheexperienceofusing

roboticmotoraugmentation.This is instarkcontrasttoarecentreportof individualswho

werebornwitha6th(fullyoperational)fingerandcouldharnessprocessesofdevelopmental

plasticity to establish normalmotor control across all six fingers (38). Finally, it is worth

consideringadaptive,ratherthanmaladaptiveplasticityprocessesthatcouldbeinvolvedin

developingmotor controlover anewbodypart. For instance,by inducingnewkinematic

synergies,learningtousetheextraThumbmaybepushingthenetworkoutsideofitsexisting

manifold (39), to allow for formation of new neural activity patterns directed at optimal

representationoftheThumbrelativetotherestoftheaugmentedhand.

To conclude,emerging technologies,designed toassist, substituteandevenaugmentour

motor abilities hold tremendous promise for transforming the lives of both disabled and

healthycommunities.Thisvisiondependsnotonlyontheexcitingtechnologicalinnovations,

italsocriticallyreliesonourbrain’sabilitytolearn,adaptandinterfacewiththesedevices.

Therefore, as technology becomes more integrated with the human body, we see new

challenges and opportunities emerging from neural and cognitive perspectives. Critical

questions arise as to how such human-machine integration can be best achieved, given

expectedneurocognitivebottlenecksofbrainplasticity.Here,wedemonstratethatsuccessful

integrationofmotoraugmentationcanbeachieved,withpotentialforflexibleuse,reduced

cognitiverelianceandincreasedsenseofembodiment.Importantly,though,suchsuccessful

human-robot integrationmayhaveconsequenceson someaspectofbody representation

andmotorcontrolwhichneedtobeconsideredandexploredfurther.


https://doi.org/10.1101/2020.06.16.151944


Methods

Participants

36healthyvolunteers(23females,meanage=23.1±3.89,allrighthanded)wererecruited

frominternet-basedadvertisementsandrandomlyassignedtoeitheraugmentation(n=24,

14females,meanage=22.9±4.12)orcontrol(n=12,9females,meanage=23.5±3.55)

group.Allparticipantswereright-handed,betweentheagesof18-35,didnothaveanyknown

motordisordersandreportednocounterindicationsformagneticresonanceimaging(MRI).

Professionalmusicianswereexcludedfromthestudy.Ethicalapprovalwasgrantedbythe

UCL Research Ethics Committee (REC: 12921/001). All participants gave their written

informedconsentbeforeparticipatinginthestudy.

Duetoschedulingconflicts,1controlparticipantand3augmentationparticipantsdropped

out of the study. Additionally, due to technical problems during data collection, 1

augmentationparticipantwasdiscardedfromthestudy.

Experimentaldesign

To assess the effects of hand augmentation on body representation, we implemented a

longitudinalexperimentaldesign(seeFigure1C),involving8experimentalsessionsconducted

across7-9days.Allparticipantsundertook(i)a1-hourfamiliarisationsession,introducingthe

equipmentandthebehaviouraltasks;(ii),a4-hourbaseline(pre-test)sessionconsistingof

behavioural testingandanMRI scan; (iii)52-hour trainingsessionsconductedover the5

subsequentdays(1sessionperday);(iv)afinal4-hourpost-testsessioncorrespondingtothe

baseline session. Additionally, 12 of the participants from the augmentation group also

undertookasecondaryfollow-upMRIsessionconducted7-10daysaftertheendoftraining.

Due toscheduling issues,1augmentationparticipantand1controlparticipantcompleted

only4/5trainingsessions.

Allstudyparticipantswereaskedtowearanextraroboticthumb(theThirdThumb;Figure

1A)ontheirright-handthroughouttheday.ParticipantswereinstructedtoweartheThumb


https://doi.org/10.1101/2020.06.16.151944


duringthein-labtrainingsessionsandtocontinuewearingitoutsideofthelabforatleast4

hoursperday.TheaugmentationgrouphadfullmotorcontrolovertheThumbandneeded

toactivelyuseittocompletethetrainingtasks.Theywerealsoencouragedtouseitasmuch

aspossibleoutsideof the lab for a free-styleenvironmentexploration (‘in thewild’). The

controlgroupworeastatic(not-moving)versionoftheThumbandcompletedthetraining

taskswithout being able to control it. Due to initial equipment issues, the first 2 control

participantsdidnotweartheThumbduringtraining.

TheThirdThumb

TheThirdThumbisa3DprintedroboticthumbdesignedbyDaniClode(17)toextendthe

abilitiesofthebiologicalhandbyincreasingthenaturalrepertoireoffingersynergies.The

Thumbiswornovertheulnarsideoftherightpalm,oppositetotheuser’snaturalthumb

(Figure1A).Itisactuatedbytwomotors,allowingthecontrolover2independentdegreesof

freedom-flexion/extensionandadduction/abduction.Themotorsaremountedonawrist

strap(Figure1B-1)andpoweredbyanexternalbatterypackwornaroundthearm(Figure1B-

2).ThemovementoftheThumbiscontrolledwithpressuresensorstapedunderneaththe

big toes of the user’s feet. The pressure sensors are powered by the external batteries

strappedaroundtheankles(Figure1B-3).Awirelesscommunicationprotocolisusedtosend

thesignalfromthepressuresensorstothemotorsactuatingtheThumb.Pressureexerted

withtherighttoecontrolsflexionoftheThumbwhilethepressureexertedwiththelefttoe

controls the abduction. The extent of Thumbmovement is proportional to the pressure

applied.

Usagemeasures‘inthewild’

TomonitorThumbusageoutsidethelab,self-reportedweartimeandThumbusageexamples

werecollecteddailyfromallwearers/users.Dailyreportswereaveragedacrossdaysandan

independent samples t-test was used to test for differences in wear-time between the

augmentedandcontrolgroup.Inaddition,bothpressuresensorswereequippedwithaSD-

carddata logger.While theThumbwason,both sensorswere logging the corresponding

motor’spositionandtheassociatedtimestamptotheSDcards.Iftheparticipantturnedthe

Thumb off during the day, the recording was paused and resumed after themotor was


https://doi.org/10.1101/2020.06.16.151944


restarted.Thoserecordingswereusedtofurtherquantifythenumberofhoursparticipants

spentusingtheThumbperday.Usetimewasdefinedasthetimespentwearingtheextra

ThumbwiththemotorsoftheThumbswitchedon,whilemovementtimewasdefinedasthe

timespentactivelyexertingpressurewiththebigtoeswhiletheThumbwasswitchedon.Due

toinitialequipmentissues,thesensors’datafromfirst3augmentationparticipantswerenot

recorded.

Trainingprotocol

During the training sessions, participants were asked to complete a set of reaching and

graspingtasks.Thesetasksweredesignedtobepurposefullychallengingwhenperformed

withonlyonehand.Thetaskexecutionwasrestrictedtotheaugmented(right)hand.The

augmentationgroupwasinstructedtousetheextraThumbtocompletethetrainingtasks.

Thecontrolgroup,wearingthestaticversionoftheThumbwasinstructedtocompletethe

trainingtasksusingonlytheirnaturalfingers,i.e.withoutusingtheThumb.Forallofthetasks,

participantswereseatedatadeskfacingthecamerarecordingtheirhandmovements.Each

taskwasconductedfor10-15minutesandrepeatedon2-4separatetrainingdays,withthe

exceptionofthehandrobotcoordinationtask(seebelow),whichwasperformedduringeach

ofthetrainingsessions.

Collaborationtasks

In the collaboration tasks, participants had to use the extra Thumb in collaborationwith

another fingertopickupmultipleobjects.Thecollaborationtasks includedbuildingJenga

tower,sortingDuploblocks,sortingshapesandmanipulatingmultipleballs.

BuildingaJengatower

A mixed jumble of wooden Jenga blocks was placed in a shallow box. Augmentation

participantswereinstructedtopickup2Jengablocksatatime,usingtheThumbtoholdor

supportoneoftheblocks,andbuilda2x2Jengatower.Controlparticipantswereinstructed

tousetheirbiologicalfingerstoholdandsupportbothofthepickedupJengablocks.The

experimenterrecordedthenumberof2-blockfloorsbuiltin1minute.


https://doi.org/10.1101/2020.06.16.151944


SortingDuploblocks

ParticipantswerepresentedwithasetofcolourfulDuploblocksandfourcolour-codedboxes.

The goal of the task was to sort all the Duplo blocks into the matching colour boxes.

Augmentation participantswere instructed to pick up four blocks at a time (one of each

colour),withoneblockbeingheldorsupportedwiththeextraThumb.Controlparticipants

wereinstructedtousetheirbiologicalfingerstoholdandsupportalloftheDuploblocks.The

experimenterrecordedthetimetakentocompletethetask

Sortingshapes

Participantsweregivenawoodenboxwithdifferently shapedholesanda setofwooden

blocksmatchingtheshapes.Pickinguptwoblocksatatime,onewiththeirbiologicalfingers

andonewith theextraThumb,augmentationparticipantshad to sort theblocks into the

correspondingholes.Controlparticipantswereinstructedtousedtheirbiologicalfingersto

holdandsupportbothoftheblocks.Theexperimenterrecordedthetimetakentocomplete

thetask.

Manipulatingmultipleballs

3smallfoamballswereplacedinshallowboxesinfrontoftheparticipants.Allparticipants

wereaskedtopickupall3ballswiththeirrighthand,startingfromtheballintherightmost

box,andtoplacethemdowninadifferentconfiguration.Augmentationparticipantswere

instructedtousetheextraThumbtopickupandholdoneoftheballs.Controlparticipants

were instructed to pick up and hold all of the balls with their biological fingers. The

experimenterrecordedthenumberofshufflesperformedin1minute.

Sharedsupervisiontasks

Inthesharedsupervisiontasks,participantshadtousetheextraThumbtoextendthenatural

gripofthehandandtofreeuptheuseoftheirbiologicalfingers.Sharedsupervisiontasks

includedpickingupmultiplewineglasses,pluggingcablesandstirringcups.


https://doi.org/10.1101/2020.06.16.151944


Wineglasses

5plasticwineglasseswereplacedupside-downinfrontoftheparticipants.Allparticipants

wereinstructedtopickupall5glasseswiththeirrighthandandtoplacethemuprightina

row,onthemarkedpositions.Augmentationparticipantswereinstructedtousetheextra

Thumb to hold one of the glasses. Control participantswere instructed to only use their

biologicalfingers.Theexperimenterrecordedthetimetakentoplaceall5oftheglasseson

the correct positions, the number of glasses thatwere dropped and the accuracy of the

placement.

Pluggingcables

Participantsweregiven4longUSBcablesandaUSBhubwith4separateports.Allparticipants

wereinstructedtopickuptheUSBhubandpluginall4USBcables,whileholdingthehubin

theair.AugmentationparticipantswereinstructedtousetheextraThumbtocompletethe

task. Control participants were instructed to only use their biological fingers. The

experimenterrecordedthetimeneededtopluginalloftheUSBcables.

Stirringcups

3smallmarbleswereplacedin3plasticcups.Allparticipantswereaskedtopickuponecup

atatimeandscoopoutthemarblewithaplasticspoon,whilstholdingthecupintheair.

Augmentation participants were instructed to hold the cup using the Thumb. Control

participantswereinstructedtoonlyusetheirbiologicalfingers.Afterthefirstdayoftraining,

the cups were filled with the Styrofoam balls to increase the difficulty of the task. The

experimenterrecordedthetimetakentoscoopouteachofthemarbles.

RoboticThumbindividuationtask

Stackingtaperolls

IntheindividuationtaskparticipantshadtoworkonthefinemotorcontroloftheThumb,

whilehavingtheirhandfullyoccupiedwithatask-irrelevantobject.Specifically,participants

weregiven6taperolls,afoamballandawoodenpolefixedtothedesk.Whileholdingthe

ballwiththeirbiologicalfingers,augmentationparticipantshadtousetheextraThumbto


https://doi.org/10.1101/2020.06.16.151944


pick up the tape rolls and place them on the wooden pole. Control participants were

instructedtousetheirbiologicalthumbtopickupthetapeswhileholdingtheballwiththe

remainingdigits. Theexperimenter recorded the time taken toplaceall the tapeson the

woodenpoleandthenumberofdroppedtapespertrial.

Hand-Thumbcoordination(Thumbopposition)

Tomonitor daily changes in hand-Thumb coordination, we used a finger opposition task

(previouslyused tomonitorhand function inhealthyandclinical cases,aswellas robotic

fingeroperation,(4,12,13)).Thistaskwasconductedatthestartofeachtrainingsession.

Participantswereseatedinfrontofacomputerscreenthatdisplayedtaskstimuli.

AugmentationparticipantswereinstructedtomovetheThumbtotouchthetipofarandomly

specifiedfingerontheaugmented(right)hand.Controlparticipantsperformedamodified

versionofthistaskinwhichtheywereinstructedtousetheirbiologicalthumbtoopposethe

remainingdigitsof the righthand.Allparticipantswere instructed toattempt tomakeas

manysuccessfulhitsaspossiblewithina1minuteblock.Participantscompletedatotalof10

blocks.AMATLABscriptwasusedtorandomlyselectatargetfinger(thumb,index,middle,

ringorlittle)andtodisplaythefingernameonthecomputerscreeninfrontoftheparticipant.

Theexperimentermanuallyadvancedtheprogramtothenextstimuluswhentheparticipant

successfullytouchedthetipofthetargetfingerwiththeextraThumborwithbiologicalthumb

(hit);orwhenawrongfingerhasbeentouched(miss).

Toquantifytheimprovementoftheaugmentationgrouponeachofthetrainingtasks,the

outcomemeasureofeachtrainingtaskwasaveragedforeachparticipantandeachtraining

session. As different participants had slightly different training regimes, in terms of

distributionoftasksacrossthedays,wesortedtheaveragescoresbasedontheorderoftask

repetition(i.e.1st,2nd,3rdtimethetaskwasrepeatedregardlessofwhichdaysitwasrepeated

on).ThesedatawerethenanalysedusingarepeatedmeasuresANOVAinSPSS.Oneofthe

shared supervision tasks (plugging cables) was not analysed due to inconstancies in task

execution.


https://doi.org/10.1101/2020.06.16.151944


Numericalcognition

ToassessthecognitiveloadrelatedtoThumbuse,anumericalcognitiontaskwasperformed

twice, on both the first and the last training sessions (40-42). Participantswere asked to

performacooperationtask,buildingaJengatower(describedabove),whilesimultaneously

presentedwithasetoflowandhighpitchauditorytonesplayedfromalaptop.Thetones

werepresentedevery1-6sinarandomisedorder,foratotaldurationof1minuteperblock.

Startingwithanumber10,participantswereinstructedtoadd1tothecurrentnumberafter

hearingahightone,andsubtract1fromthecurrentnumberafterhearingalowtoneAfter

each mathematical operation, participants were instructed to verbally respond with the

resultingnumber.Participantsperformed5blocksofthenumericalcognitiontaskduringeach

session. Numerical cognition blockswere always preceded and followedwith 5 blocks of

normal(baseline)buildingaJengatowertask(5baselineblocks,5numericalcognitionblocks,

5baselineblocks).Notethatthefirst3participantsdidnotcompletethenumericalcognition

task.

Foreachparticipant, theaveragenumberof Jengafloorsbuiltwascalculatedfromall the

numericalcognitionblocksinwhichthecorrectmathematicaloperationswereperformed.

Trialsinwhichawrongnumberwasgivenwerediscarded(onaverage15-19%oftrialswere

discarded per participant). To determinewhether the extra cognitive load caused by the

numericalcognitiontaskhadanyimpactonparticipants’motorperformancewhenusingthe

extra Thumb, the average score from the numerical cognition taskwas compared to the

baselinescore.Thiswasdoneseparatelyforthefirstandthelastdayoftraining.Thebaseline

scorewascalculatedastheaveragenumberofJengafloorsbuiltacrossthetwoblocks(10

trials)thatproceededandfollowedthenumericalcognitiontask.

Trackinghandmovements

To assess the changes in finger coordination across all training tasks, we tracked the

kinematics of the augmented (right) hand using flex sensors embedded in a dataglove

(CyberGlove, Virtual Technologies, Palo Alto, CA, USA). Finger kinematics have been

previouslyshownto reflect thebrainorganisationbetter thanEMG-derivedmeasures (9).

Here,thesensorsofthedataglovewereassociatedwith19degreesoffreedomofthehand


https://doi.org/10.1101/2020.06.16.151944


and included themetacarpal-phalangeal (MCP), proximal interphalangeal (PIP) and distal

interphalangeal(DIP)jointanglesforthefourfingers(index,middle,ringandlittlefinger),the

carpometacarpal,(CMC)metacarpal-phalangeal(MCP)andinterphalangealjoint(IP)angles

forthebiologicalthumb,thethreerelativeabductionanglesbetweenthefourfingersand

theabductionanglebetweenthebiologicalthumbandthepalmofthehand.Sensorswere

sampled continuously at 100 Hz using Shadow Robot’s (https://www.shadowrobot.com)

CyberGloveinterfacefortheRobotOperatingSystem(ROS,https://www.ros.org).

ParticipantsworetheCyberGloveunderneaththeextraThumbthroughoutallofthetraining

sessions. Kinematics associatedwith each of the training tasks performed during a given

session were recorded onto a separate file. Due to initial equipment issues, the first 4

augmentationparticipantsdidnotweartheCyberGloveduringtraining.

Cyberglovecalibration

TheCyberGlovewascalibratedforeachparticipantatthebeginningofeachtrainingsession,

using amin-max pose calibration procedure providedwith the ROS CyberGlove package.

Duringthecalibration,participantswerepresentedwithasetofcarefullyselectedhandposes

andgivena fewsecondstoshapetheirhandaccordingly.Foreachhandpose, thesensor

valuesweresampledandaveragedoveronesecondofrecording.Averagedsensorvalues

were then saved alongside the actual joint angles, determined based on the hand

configuration. Once all hand poses have been recorded, a linear regression was used to

calculatethemappingfromthesensor-valuestothejointangles.

Handkinematicsanalysis

Wefocusedthehandkinematicsanalysisonthedatarecordedduringthefirstandlastdays

oftraining.Recordeddatafromthe19sensorswerecalibratedusingtheestablishedmapping

fromthesensor-valuestothejointangles.Thejointanglesweresmoothedusinga3rdorder

Savitzky-Golay filter, with a window length of 151 samples. Angular velocities were then

calculatedfromthefirstdifferenceofthefilteredjointangledatadividedbythetimestep.

Sincemostofthefingermovementsemployedduringthetrainingtaskswereexecutedusing

thePIPjoints,tosimplifytheanalysis(9),onlydatafromthesefivejointswereanalysed.Due


https://doi.org/10.1101/2020.06.16.151944


to acquisitionerrors, for2 augmentationparticipants and2 controlparticipants, thedata

recordedduringthefirstdayoftrainingwasunavailable.Therefore,fortheseparticipants,

thedatafromtheseconddayoftrainingwasusedinstead.Similarly,whenthedatafromthe

lastdayoftrainingwasunavailable(3augmentationparticipants,2controlparticipants),the

datafromthepenultimatedayoftrainingwasusedinstead.Duetocalibration issues,the

hand kinematics data of 1 augmentation and 2 control participantswere discarded from

subsequentanalysis.

Toquantifythecomplexityofthehandmovementsacrossbothgroups,wefirstconducteda

PrincipalComponentsAnalysis(PCA)oftheangularvelocitiesofthePIPjoints.PriortoPCA,

theangularvelocitieswerez-normalised(43)notehoweverthatsimilarresultswereobtained

with not normalised data (44). The extracted principal components (PCs) were ordered

according to the amount of variance explained by each component. Consistent with the

literature(44,45),wefoundthatthefirstPCaccountedformorethan40%oftotalvariance

andreflectedacoordinatedmovementofallthefingers(Figure2AandSupplementaryFigure

S3 for all the PCs). To quantify the dimensionality of the hand movements, for each

participantandday,werecordedthenumberofPCsneededtoexplain80%oftotalvariance

(46).Thesewerethencomparedacrossgroupsinarepeated-measuresANOVAwithtime(day

1, day 5) as awithin-subjects factor and group (augmented, control) as between-subject

factor.Next,toquantifythecontributionoftheall-digitmovementstothecomplexityofthe

handkinematics,wecomparedtheamountofvarianceexplainedbythefirstPCacrossboth

groupsusingthesamerepeated-measuresANOVAdesign.

Next,tointerrogatemoredetailedchangestothefingercooperationpatterncausedbythe

handaugmentation,welookedatthedegreeofcouplingbetweendigitpairs,adaptingthe

methodsusedin(44).Weusedlinearregressiontofittheangularvelocitydataofagivendigit

asa functionoftheangularvelocityofeachoftheotherdigits individually.Thisyieldeda

singledeterminationcoefficient (R2) foreachdigitpair,expressing theproportionof total

varianceofeachdigit’sangularvelocitythatcouldbeexplainedbyalinearreconstruction,

based on its paired regressionwith each of the other four digits. Qualitative comparison

betweentheresultsobtainedinthecontrolgroup(whodidnotusetheextraThumbduring

the training) and outcomes of previous hand kinematics research conducted during free


https://doi.org/10.1101/2020.06.16.151944


movement (44), confirmed that the conducted analysis resulted in a typical finger

coordinationprofile.

To assess the effect of Thumb use on the finger coordination profile across groups, we

performed a linear mixed model analysis (LMM) with fixed factors of time, group

(augmentation vs controls) and digit pair, a random effect of participant and a random

participant-specific slope of time. The LMM was evaluated in R (version 3.5.2) under

restricted maximum likelihood (REML) conditions with Satterthwaite adjustment for the

degreesoffreedom.

Pre-posttestingprotocol

Toassesstheneuralcorrelatesofhandaugmentationweusedasetofpre-topost-training

comparison measures, consisting of both behavioural and neuroimaging tasks. To

characterisetheemergingrepresentationoftheextraThumb,weprobedtheproprioception

andmotorcontroloftheThumbusingasequentialvariationofthehand-Thumbcoordination

task. We also assessed perceived (phenomenological) embodiment of the Thumb using

questionnaires. To interrogate changes to the natural hand representation,wemeasured

biological finger co-dependencies using finger kinematics and force-enslavement task.

Changes tobody imagewereprobedusing tactiledistanceandhand laterality judgement

tasks.Finally,fMRIwasusedtotrackthehandrepresentationinthesensorimotorcortexof

the brain and to interrogate changes to the relationship between the hand and feet

representations.Withtheexceptionofthehand-Thumbcoordinationtask,participantswere

notwearingtheThumbduringtesting.

Hand-Thumbcoordination(sequential)

ToprobechangestoimplicitmotorcontroloftheThumb,asequentialvariationofthehand-

Thumb coordination (finger to Thumb opposition) task has been used. In this task,

participantssequentiallyopposedtheThumbtothetipofeachof thefive fingersof their

augmentedhand,startingwiththe little finger.Participantswere instructedtorepeatthis

movementcycleasmanytimesaspossiblewithina1-minuteblock,whilemaintaininghigh

accuracy. The task consisted of 5 blocks. To assess the proprioception of the Thumb,


https://doi.org/10.1101/2020.06.16.151944


participantswerefurtheraskedtoperform5blocksofthesametaskwhileblindfolded.The

experimenter recorded the number of successful hits per block. For each participant, an

average score (numberofhits)was calculated separately foreach session (pre,post) and

vision condition (with vision, blindfolded). Due to a data acquisition mistake, 1 control

participantwasnotincludedintheanalysis.

Embodimentquestionnaires

Toassesschanges in thephenomenologicalembodimentof theThumb,participantswere

askedtocompleteanembodimentquestionnairesbeforethefirsttrainingsessionandagain

afterthelasttrainingsession.Duetodatacollectionissues,5augmentationparticipantsand

1 control participant only completed the post-training embodiment questionnaire.

Participantswereaskedtoratetheiragreementwith12statements(basedon(19))ona7-

pointLikert-typescalerangingfrom-3(stronglydisagree)to+3(stronglyagree).Statements

were clustered into4main categories,probingdifferentaspectsofembodiment,namely:

bodyownership,agency,bodyimageandsomatosensation.

Table1.Embodimentquestionsdividedinto4separateembodimentcategoriesBodyownership

1. “Itseemsliketheroboticfingerbelongstome”2. “Itseemsliketheroboticfingerisapartofmyhand”3. “Itseemsliketheroboticfingerisapartofmybody”4. “Itseemsliketheroboticfingerisaforeignbody”(negative)5. “Itseemsliketheroboticfingerisfusedwithmybody”6. “ItseemslikeIhavesixfingers”

Agency1. “ItseemslikeIcanmovetheroboticfingerifIwant”2. “ItseemslikeIamincontroloftheroboticfinger”

BodyImage1. “ItseemslikeIamlookingdirectlyatmyownfinger,ratherthanaprosthesis”2. “Itseemsliketheroboticfingerbeginstoresemblemyotherfingers”

Somatosensation1. “Icanfeeltemperatureintheroboticfinger”2. “Icanfeelthepostureoftheroboticfinger”


https://doi.org/10.1101/2020.06.16.151944


Foreachparticipant,questionnairescoreswereaveragedwithineachembodimentcategory.

Notethatinthebodyownershipcategory,theopposite(negative)valueofthe‘foreignbody’

statement has been used while computing the average. 1 augmentation participant was

discarded from this analysis, as their averaged agency scorewas classified as a statistical

outlier(differentfromthemeanscorebymorethan3standarddeviations).

Forceenslavement

To estimate the degree of co-dependency across the biological fingers, a custom-built

keyboard (previously used in (9))was used tomeasure isometric finger forces generated

duringindividuatedfingerpresses.

Duringthepreandposttestingsessions,participantsperformedindividuatedforcepresses

with instructed fingers, while forces produced with all of the fingers were recorded.

Participantswereinstructedtoalwayskeepalloftheirfingersonthekeysofthekeyboard.A

visualrepresentationoftheforceexertedwithall5fingerswaspresentedonascreen(Figure

3C).Followingthemethodsdescribedin(9),theexperimentbeganbyestimatingthestrength

ofeachfinger,bymeasuringtworepetitionsofthemaximumvoluntaryforce(MVF)ofeach

digit.Allsubsequenttrialsrequiredtheproductionofisometricfingertipforcesata75%of

themaximumvoluntaryforcefortheinstructeddigit(9).Atthestartofeverytrial,aforce

target-zone (target-force ± 25%)was highlighted in green in the visual display. This cued

participants tomakea short forcepresswith the instructed finger inorder tomatchand

maintainthetarget-forcefor1swhilekeepingtheuninstructedfingersasstableaspossible.

Thetrialwasstoppedifforceoftheinstructeddigitdidnotreachthetarget-zonewithinthe

2sfollowingthestimulusonset.Singletrials,presentedinarandomisedorderweregrouped

inblocks,witheachblockconsistingof40trials(8trialsperfinger).Participantsperformed4

force-enslavementblocksduringeachsession.Thedataof twoaugmentationparticipants

wereexcludedfromthesubsequentanalysis,asthoseparticipantsproducedextraordinarily

lowMVFforces(below2N).

Foreachparticipant,therecordedforcedatawasfirstfilteredwitha3rdorderSavitzky-Golay

filterwithawindowlengthof51samples.Thedatawasthenseparatedintoindividualtrials


https://doi.org/10.1101/2020.06.16.151944


(fingerpresses).Trialsininwhichtheforceproducedwiththeinstructedfingerdidnotreach

thetargetforcewerediscardedfromtheanalysis.Withineachtrial,alinearregressionwas

usedtofittheforcegeneratedbytheinstructeddigitasafunctionoftheforcegeneratedby

eachoftheotherdigits.Theresultingdeterminationcoefficients(R2)wereaveragedacross

trialstoyielda5x5matrixforceenslavementmatrix,expressingtheinvoluntaryforcechanges

acrossnon-instructedfingersduringthepressesoftheinstructedfinger.

To estimate gross changes to finger co-dependency,we performed a linearmixedmodel

analysis(LMM)withfixedfactorsoftime,group(augmentationvscontrols)anddigitpair,a

randomeffectofparticipantandarandomparticipant-specificslopeoftime.TheLMMwas

evaluatedusingthesameparametersastheonesusedforfingerco-usemodelling(seeHand

kinematicsanalysis).TotesttheassumptionthathavinganextraroboticThumbwouldimpact

theindependenceofthebiologicalthumb,asimilar linearmixedmodelwascreatedusing

onlytheforceenslavementcausedbythethumb(4values,oneforeachenslavedfinger).

Tactiledistanceperception

To examine whether Thumb usage would lead to incorporating it into the body

representation, we tested participants’ ability to discriminate between tactile distances

applied over their wrist and forearm. The design was inspired by a previous study (47)

showinggreaterbiasesinspatialtactileperceptionwhentestedacrossajoint.Priortothe

experiment,theexperimentermarkedthemidpointoftheparticipant’sforearm,aswellas

thebasepointoftheextraThumbwithapen.

Duringtheexperiment,participantswereseatedinanarmchair,withtheirrightelbowrested

onanelevatedfoampaddingwiththeforearmatfullflexionandtheirlefthandplacedona

mouse connected to the experimental computer. The tactile stimuli (hereafter distances)

comprisedofcustom-madecalliperswithacrylicpinsfixedatdistancesof50,60and70mm.

Ineachtrial,twodistanceswerepresentedsequentially–oneoverthemarkedbasepointof

the Thumb, one over themidpoint of the ventral side of the forearm (both in the same

orientation). The experimenter presented the distances manually ensuring that the two

points of each calliper touched the skin simultaneously. Participants were instructed to


https://doi.org/10.1101/2020.06.16.151944


indicatewhichofthedistancestheyperceivedaslargerbypressingtheleft(distanceoverthe

Thumb’sbaseperceivedas larger)orright(distanceovertheforearmperceivedas larger)

mouse buttonwith their left hand. The task consisted of 3 blocks. Each block included 5

presentationsofeachofthefollowingratiosofdistancesinarandomisedorder:50/70mm,

50/60mm,60/70mm,60/60mm,70/60mm,60/50mm,and70/50mm).

Followingthemethodsdescribedin(47),wemeasuredtheproportionofresponsesinwhich

thestimuliappliedoverthebasepointoftheThumbwasjudgedaslarger,asafunctionofthe

ratioofthelengthofthestimuli.CumulativeGaussiancurveswerefittothedatausingMatlab

(v.2017b).Pointofsubjectiveequality(PSE)wascalculatedseparatelyforeachparticipant

and session, as the ratio of stimuli atwhich the psychometric function crossed the 50%.

Additionally,theinterquartilerange(IQR)–thatisthedifferencebetweenthepointsonthe

x-axiswherethepsychometricfunctioncrosses25%and75%-wascalculatedasameasure

oftheprecisionofparticipant’sjudgements.Oneaugmentationparticipantandtwocontrol

participants were excluded from further analysis due to poor goodness of fit coefficient

(R2<0.15)forthepsychometricfunctioninthepre-session.TheremainingR2scores,averaged

acrossparticipants,showedanexcellentfittothedata(R2=0.88±0.15

HandLateralityJudgement

To examine whether Thumb usage leads to changes in body image, participants were

presentedwithdrawingsofhandoutlines,adaptedfrom(48)andaskedtodecidewhether

thehanddrawingscorresponded toa rightora lefthand.Participantswere instructed to

respondverballybyindicatingthehandlaterality(leftorright)ofeachpresentedimageas

fastaspossible,whilemaintaininghighaccuracy.Thestimuli includeddrawingsofleftand

righthands,presentedinfourdifferentpostures(dorsalview,palmview,sidefromthumb

view,andpalmfromwristview)andat7differentangles(upright0°,30°,90°,150°,210°,

270°,and330° inaclockwisedirection).Participantscompleted fourexperimentalblocks,

each includingallofthe56hand images.Hand imageswerepresented inarandomorder

usingPsychopysoftware.Participantswereseatedcomfortablyinfrontofalaptopcomputer

withtheirhandsobstructedbyablackcape.Eachhanddrawingwasprecededby1second

presentationofafixationcrossanddisappearedeitherafteraverbalresponsewasprovided


https://doi.org/10.1101/2020.06.16.151944


orafter10secondsofnoresponse.Timefromthestartoftheimagedisplaytovoiceonset

wasrecordedastheparticipants’reactiontime(RT).Audiofileswithparticipant’sresponses

wererecordedforoff-lineaccuracyanalysis.Duetoequipmentmalfunction,2augmentation

participantsand1controlparticipantdidnotcompletethehandlateralityjudgementtask.

Allaudiorecordingsandtheappropriateclassificationofreaction-timeswereverifiedoffline

byanaiveexperimenter.Trialswithnoisyrecordingswereexcludedfromfurtheranalysis.

Accuracywascomputedastheproportionofcorrectresponseofallvalidtrials.Onlytrials

withcorrectresponseswereincludedintheRTanalysis.RTswerelogarithmicallytransformed

inordertocorrectfortheskewedRTdistributionandtosatisfytheconditionsforparametric

statistical testing. Transformed RTs deviating more than 3 standard deviations from the

participants’means(separatelyforeachsession)werediscarded.

Statisticalanalysis

AllstatisticalanalysiswasperformedusingIBMSPSSStatisticsforMacintosh(Version24),R

(forlinearmixedmodels)andJASP(Version0.11.1).Testsfornormalitywerecarriedoutusing

aShapiro-Wilktest.Trainingdatathatwerenotnormallydistributedwerelog-transformed

prior to further statistical analysis. With the exception of hand kinematics and force

enslavementdatasets,thatwereanalysedusinglinearmixedmodels(LMM),allthewithin

group comparisons were carried out using paired t-tests or repeated measures ANOVAs

(training tasks data). Between group comparisons were carried out using ANCOVAs with

group(augmentation,controls)asafixedeffectandthepre-scoreusedasacovariate(49).

Allnon-significantresultswerefurtherexaminedusingcorrespondingBayesiantests.

Scanningprocedures

Bothpre-andpost-neuroimagingsessionswerecomprisedofthefollowingfunctionalscans:

(i) a resting state scan, (ii) a motor localiser scan and (iii) four finger-mapping scans.

Additionally,astructuralscanandfieldmapswereobtainedduringeachscanningsession.


https://doi.org/10.1101/2020.06.16.151944


Restingstatescan

Participantswereinstructedtolettheirmindwanderwhilekeepingtheireyeslooselyfocused

onafixationcrossforthedurationofthescan(5min).

Motorlocaliserscan

Participantswereinstructedtomovetheright/lefthand(allfingersflexionandextension),

right/left foot (unilateral toemovements),or lips (blowingkisses)aspacedbyvisual cues

projected into the scanner bore. Each condition was repeated four times in a semi-

counterbalancedprotocolalternating12sofmovementwith12sofrest.Participantswere

trained before the scan on the degree and form of the movements. To confirm that

appropriatemovementsweremadeattheinstructedtimes,taskperformancewasvisually

monitored online. Due to data acquisition issue, motor localiser data of 1 augmentation

participantwasdiscardedfromtheanalysis.

Finger-mappingscans

Participantswereinstructedtoperformvisuallycuedmovementsofindividualdigitsofeither

hand,bilateraltoemovementsandlipsmovements.Thedifferentmovementconditions,as

well as rest periods were presented in 9s blocks. The individual digit movements were

performedintheformofbuttonpressesonMRI-compatiblebutton-boxes(4buttonsperbox)

securedontheparticipant’sthighs.Themovementsofeitherofthe(biological)thumbswere

performedbytappingthemagainstthewallofthebuttonbox.Instructionsweredelivered

viaavisualdisplayprojectedintothescannerbore.Tenverticalbars,representingthefingers

flashedindividuallyingreenatafrequencyof1Hz,instructingmovementsofaspecificdigit

atthatrate.Toeandlipsmovementswerecuedbyflashingthewords“Feet”or“Lips”atthe

same rate of 1 Hz. Each condition was repeated 4 times within each run in a semi-

counterbalancedorder.Participantsperformed4 runsof this task.Due to timing issues3

augmentationparticipantsand1controlparticipantscompletedonly3 runsof the finger-

mapping task. Additionally, due to a data acquisition issue, the finger-mapping data of 1

controlparticipantwasdiscarded.


https://doi.org/10.1101/2020.06.16.151944


MRIdataacquisition

MRIimageswereacquiredusinga3TPrismaMRIscanner(Siemens,Erlangen,Germany)with

a32-channelheadcoil.Functional imageswerecollectedusingamultibandT2*-weighted

pulsesequencewithabetween-sliceaccelerationfactorof4andnoin-sliceacceleration.This

providedtheopportunitytoacquiredatawithhighspatial(2mmisotropic)andtemporal(TR:

1450ms) resolution,covering theentirebrain.The followingacquisitionparameterswere

used: TE: 35ms; flip angle: 70°, 72 transversal slices. Field maps were acquired for field

unwarping.AT1-weightedsequence(MPRAGE)wasusedtoacquireananatomicalimage(TR:

2530ms,TE:3.34ms,flipangle:7°,spatialresolution:1mmisotropic).

MRIanalysis

MRIanalysiswas implementedusingtools fromFSL (50,51)andConnectomeWorkbench

(humanconnectome.org)software,incombinationwithMatlabscripts(versionR2016a),both

developed in-house (including FSL-compatible RSA toolbox (52)) and as part of the RSA

Toolbox (21). Cortical surface reconstructions were produced using FreeSurfer ((53, 54),

freesurfer.net).

fMRIpre-processing

Functional datawas first pre-pre-processed using FSL-FEAT (version 6.00). Pre-processing

includedmotioncorrectionusingMCFLIRT(55),brainextractionusingBET(56),temporalhigh

passfiltering,withacutoffof150sforthefinger-mappingscansand100sforresting-state

andmotor localiserscans,andspatialsmoothingusingaGaussiankernelwithaFWHMof

3mmforthefinger-mappingand5mmforresting-stateandmotorlocaliserscans.

Tomake sure that the scans from the two scanning sessionswerewell aligned, for each

participantwecalculatedamidspacebetween theirpre-andpost- scans, i.e. theaverage

spaceinwhichtheimagesareminimallyreoriented.Eachscanwasthenalignedtothispre-

postmidspaceusingFMRIB’sLinearImageRegistrationTool(FLIRT,(55,57)).Thisregistration

wasperformedseparatelyforthestructural,motorlocaliser,restingstateandfingermapping

scans,resultingin4separatemidspacesperparticipant.Allthewithin-subjectanalyseswere


https://doi.org/10.1101/2020.06.16.151944


done in the corresponding functional midspace. To allow for the co-registration of the

functional data and the anatomical ROIs (seebelow) the functionalmidspaceswere then

aligned to each participant’s structural midspace using FLIRT, optimised using Boundary-

BasedRegistration(BBR,(58)).Finally,wheninterrogatingfinger-specificactivationsonthe

group-level, the individual structural midspaces were transformed into MNI space using

FMRIB’sNonlinearImageRegistrationTool(FNIRT,(57))

Low-leveltask-basedanalysis

For task-based datasets, voxel-wiseGeneral LinearModel (GLM) analysiswas carried out

usingFEAT,toidentifyactivitypatternsrelatedtothemovementofeachdigit/bodypart.The

designwasconvolvedwithadouble-gammahemodynamicresponsefunction(HRF)andits

temporalderivative.Thesixmotionparameterswereincludedasregressorsofnointerest.In

caseoflargemovementbetweenvolumes(>1mm)additionalregressorsofnointerestwere

includedintheGLMtoaccountforeachoftheseinstancesindividually.

Forthefinger-mappingscans,12contrastsweresetup:eachdigitversusrest,feetagainst

rest and lips against rest. The estimates from the four finger mapping scans were then

averagedvoxel-wiseusingfixedeffectsmodelwithaclusterformingz-thresholdof3.1and

family-wiseerrorcorrectedclustersignificancethresholdofp<0.05,creating12mainactivity

patternsforeachsessionandparticipant.

Forthemotorlocaliserscans,4maincontrastsweresetup:right/lefthandagainstlips,rand

right/leftfootagainstlips.Theactivitypatternsassociatedwiththose6contrastswerethen

usedtodefinefunctionalregionsofinterest(ROIs,seebelow).

RegionsofInterest(ROI)definition

ChangestorepresentationalstructureofthehandwerestudiedusinganatomicalROIs,as

previouslypracticed inrelatedstudies (13,59,60).StructuralT1 images,registeredtothe

structuralmidspace,wereusedtoreconstructthepialandwhite-greymattersurfacesusing

Freesurfer. Surface co-registration across hemispheres and participants was done using

sphericalalignment.Individualsurfaceswerenonlinearlyfittedtoatemplatesurface,firstin


https://doi.org/10.1101/2020.06.16.151944


termsofthesulcaldepthmap,andthenintermsofthelocalcurvature,resultinginanoverlap

ofthefundusofthecentralsulcusacrossparticipants(61).TheanatomicalsensorimotorROI,

used for the multivariate analysis were defined on the group surface using probabilistic

cytotectonicmapsalignedtotheaveragesurface(62).TheseROIwasthenprojectedintothe

individual brains via the reconstructed individual anatomical surfaces. Since we were

primarilyinterestedinthemotorrepresentationofthehand,wehavefocusedouranatomical

ROIonM1,selectingallsurfacenodeswiththehighestprobabilityforBA4spanninga2cm

stripmedial/lateraltotheanatomicalhandknob(13,63).However,wenotethat,giventhe

probabilisticnatureofthesemasks,thedissociationbetweenS1andM1isonlyanestimate,

andthusourROIshouldbetreatedasasensorimotorone.

For our univariate analyses, we also defined functional ROIs based on the sensorimotor

representationsoftheleft/righthandandfeetforeachparticipant.Unlikethecross-validated

RSA analysis, these analyses require more spatially-restricted ROIs. We therefore used

condition-specificcontrastsfromthemotorlocaliserscans(aspreviouslypracticedine.g.(12,

64-66)To thisextend, therelevant (right/lefthandvs lips, right/left footvs lips) low-level

contrastswerefirstaveragedacrossboth(preandpost)scans.Tocreateleft/righthandROIs,

foreachparticipant,the200mostactivevoxelswereselectedfromtheaveragedleft/right

handvslipscontrast.FortheindividualfeetROIs,asimilarprocedurewasemployed,selecting

the100mostactivevoxelsfromtheaveragedleft/rightfootvslipscontrast.Voxelselection

wasrestrictedtotheprimarysomatosensory(S1)andmotor(M1)corticesasderivedfrom

themaximumprobabilisticmaps(thresholdedat25%)oftheJuelichHistologicalAtlas(67).

Voxels from both feet ROIs were combined into a single region of interest used in the

subsequentanalyses.

Restingstate

Toaccountfornon-neuronalnoisethatmightbiasfunctionalconnectivityanalyses(68,69),

weregressedoutthesixmotionparameters,aswellastheBOLDtimeseriesofwhite-matter

andcerebrospinalfluid(CSF)fromthepre-processedrestingstatedata.Forthispurpose,the

T1-weightedstructuralscans,registeredtoanatomicalmidspace,weresegmentedintogrey

matter,whitematter,andCSF,usingFSLFAST (70).Toavoid the inclusionofgreymatter


https://doi.org/10.1101/2020.06.16.151944


voxels in thenuisancemasks, the resultingmasks includedonlyvoxels identifiedaswhite

matter/CSFwithprobabilityof1,andwereerodedbyonevoxelineachdirection.Forboth

thewhitematterandCSFmaps,thefirst fiveeigenvectorswerethencalculatedusingthe

unsmoothedrestingstatetimeseries(68,69).Togetherwiththemotionparameters,these

16regressorsofnointerestwereregressedoutfromthepre-processedrestingstatetime

series. The resulting time series (residuals)were used in all the subsequent resting state

analyses.

Thelevelofresting-statecoupling(functionalconnectivity)betweentheaugmented(right)

handROIandthefeetROIswasexaminedbycorrelatingthetime-courseoftheaugmented

handROIwiththetime-courseofthefeetROI,whilepartiallingoutthetime-courseofthe

left-handROI.TheresultingsetsofpartialcorrelationswereFisherz-trasformed,andgroup-

levelstatisticalcomparisonswereconductedusingtwo-tailedpairedt-tests.

Multivariaterepresentationalstructure(RSA)

We used RSA (71) to assess the multivariate relationships between the activity patterns

generatedacrossdigitsandsessions.Thedissimilaritybetweenactivitypatternswithinthe

M1anatomicalhandROIwasmeasuredforeachdigitpairusingthecross-validatedsquared

Mahalanobisdistance(21).Wecalculatedthedistancesusingeachpossiblepairofimaging

runswithinasinglescanningsession(pre,post)andthenaveragedtheresultingdistances

acrossrunpairs.Beforeestimatingthedissimilarityforeachpatternpair,theactivitypatterns

werepre-whitenedusingtheresidualsfromtheGLM.Duetothecross-validationprocedure,

the expected value of the distance is zero (or below) if two patterns are not statistically

different from each other, and larger than zero if the two representational patterns are

different.Theresulting10uniqueinter-digitrepresentationaldistanceswereputtogetherin

arepresentationaldissimilaritymatrix(RDM).

To assess the effect of 5-day Thumb usage on the overall representation structure

(dissimilarity),weperformedalinearmixedmodelanalysis(LMM)withfixedfactorsoftime,

handanddigitpair,arandomeffectofparticipantandarandomparticipant-specificslopeof

time. The LMM was evaluated in R (version 3.5.2) under restricted maximum likelihood


https://doi.org/10.1101/2020.06.16.151944


(REML) conditionswithSatterthwaiteadjustment for thedegreesof freedom.We further

assessedthetypicalityoftherepresentationalstructureofthepostscanRDM,bycalculating

the Spearman’s rho correlation between each participant’s RDM and the group average

computedfromthepre-scandatausingaleave-one-outprocedure.Thetypicalityvalueswere

thenz-normalisedandthetypicalityoftherepresentationalstructureoftherighthand(post-

scan)wasthencomparedtothetypicalityofthelefthand’srepresentationalstructure(post-

scan) using paired t-test. Because the representational structure can be related to

behavioural aspects of hand use and is highly invariant in healthy individuals (average

correlation r = 0.9, (9)), this measure serves as a proxy for how ‘normal’ the hand

representationis.

AsanaidtovisualisingtheRDMs,wealsousedclassicalmultidimensionalscaling(MDS).MDS

projectsthehigher-dimensionalRDMintoalower-dimensionalspace,whilepreservingthe

inter-digitdissimilarityvaluesaswellaspossible.MDSwasperformedondatafromindividual

participantsandaveragedafterProcrustesalignment(withoutscaling)toremovearbitrary

rotationinducedbyMDS.NotethatMDSispresentedforintuitivevisualisationpurposesonly,

andwasnotusedforstatisticalanalysis.

Numericalmodellingofinter-digitdissimilarity

Toaidtheinterpretationoftheneuroimagingfindings,wehavecreatedasimplenumerical

simulation,modellingthepotentialeffectsof:(i)corticalmagnification,(ii)inhibitionand(iii)

addinganewdigitrepresentation;ontothecanonicalhandrepresentationalstructure.

WeaspiredtosimulateactivitypatternswithinthehandROIbycreatinganabstractstructure

(comprisedof3000“voxels”),dividedinto5equisizedfinger-selectiveregions,andsimulating

activity elicited by individuated movements of each of the fingers. During each trial

(simulationrun),themovingfingeractivatedthe“voxels”withintheassignedfinger-selective

region(withinherentnoise),whilealsopartiallyactivatingthe“voxels”assignedtotheother

4fingers.Morespecifically,withinatrial,theactivityofeach“voxel”wasrandomlysampled

fromaGaussiandistribution,withmu=1fortrialsinvolvingthe“voxel’s”preferredfingerand

rangingfrom0.1408to0.3846fortrialsinvolvingamovementofadifferentfinger.Thevalues


https://doi.org/10.1101/2020.06.16.151944


of mu were chosen based on the inter-finger relationship derived from the average

representationalstructure(RDM)ofthedominanthandofanindependentlyacquiredcohort

ofparticipants(13).Thenoiselevel(sigma=0.5)wasconstantacrossalltrialsandactivations.

Thesimulationwasrun10,000times,separatelyforeachofthe5fingers,resultingin50,000

patternsoffingerspecific“activations”.

Tomodel thecorticalmagnificationphenomenon,we increased themuvaluesassociated

withallthenon-movingfingersby10%(activationmodifier,seeTable2).Thisintroducedthe

ideathatincreasedindividuationofthefingermovements(basedonFigure2above)results

inincreasedrepresentationofthefingers(37)Similarly,modellingthehomeostaticinhibition,

wedecreasedall themuvaluesby10% (seeTable2). This interrogates the idea thatany

changestriggeredbythechangedfingerco-usewouldbeoffsetbytonicinhibition,thatwill

impacttheentirehandmap(36).Finally,toinvestigatethetheoreticaleffectofaccounting

for a new (6th) digit representation added into the hand representational structure, we

decreasedthenumberof“voxels”assignedtoeachofthe5fingers,inordertocreateanew

equisized digit-specific region. Since we didn’t have any a priori assumptions on the

representationalrelationshipbetweentheextraThumbandtherestofthefingers,thisarea

wassettobeactivatedequallyinalltrials,withmusettoanaverageactivationvalueforthe

not-movingfingers(sigma=0.5,mu=0.2193).

Foreachsimulationrunandeachmodel,wecomputedtheEuclideandistancesbetweenthe

activity patterns associatedwith eachdigit’smovement.We then averaged thedistances

acrossall10digitpairstoobtainanaveragedissimilaritymeasure.Finally,wecomparedthe

averagedistancescomputedforeachofthemodelstotheonescalculatedforthecanonical

handmodel,usingindependentsamplest-tests.


https://doi.org/10.1101/2020.06.16.151944


Table 2. Parameters used for the numerical simulation of cortical magnification, homeostaticinhibitionandadditionofanewdigit.

AcknowledgementsThisworkwassupportedbyanERCStartingGrant(715022EmbodiedTech),awardedtoTRM,

whowasfurtherfundedbyaWellcomeTrustSeniorResearchFellowship(215575/Z/19/Z)

andbySirHalleyStewartCharitableTrust(580).WethankDominicStirling,SamuelCousins,

LydiaMardell,MariaKrommandMathewKollamkulamfortheirhelpwithdatacollection;

EkaterinaTupitsynafordevelopingthescriptforthekinematicdataanalysis;JamesKilnerfor

providing us access to the Cyberglove; Joern Diedrichsen for the custom-made force

keyboards; Howard Bowman for introducing us to information theorymeasures; Gionata

Salvietti for invaluabletechnicalhelpduringpiloting;SilvestroMicera, JuanAlvaroGallego

andDaanWesselinkforhelpfulcommentsonthemanuscript;HunterShoneforproof-reading

themanuscript;DomenicoPrattichizzoforinspiringdiscussionsaboutsupernumeraryfingers;

andourparticipantsfortakingpartinthisstudy.

Parameters/simulation

CanonicalCortical

magnificationHomeostaticinhibition

Newdigit

Numberofvoxelsperdigit

600 600 600 500

Noiselevel(sigma) 0.5 0.5 0.5 0.5Meanactivationofthemovingfinger

1 1 0.9 1

Activationmodifierforthenon-moving

fingers

1 1.1 0.9 1


https://doi.org/10.1101/2020.06.16.151944


References1. G.Salviettietal.,CompensatingHandFunctioninChronicStrokePatientsThroughthe

RoboticSixthFinger.IEEETransNeuralSystRehabilEng25,142-150(2017).2. F.Y.Wu,H.H.Asada,inASMEDynamicSystemsandControlConference.(SanAntonio,

TX,2014).3. T.Sasaki,M.Y.Saraiji,C.L.Fernando,K.Minamizawa,M.Inami,paperpresentedat

theACMSIGGRAPH2017EmergingTechnologies,2017.4. N.S.Meraz,M.Sobajima,T.Aoyama,Y.Hasegawa,Modificationofbodyschemaby

useofextraroboticthumb.RobomechJ5,(2018).5. C.I.Penaloza,S.Nishio,BMIcontrolofathirdarmformultitasking.ScienceRobotics

3,(2018).6. M.J.Arcaro,P.F.Schade,M.S.Livingstone,Bodymapproto-organizationinnewborn

macaques.ProcNatlAcadSciUSA116,24861-24871(2019).7. S.Dall'Orsoetal.,SomatotopicMappingoftheDevelopingSensorimotorCortexinthe

PretermHumanBrain.CerebCortex28,2507-2515(2018).8. J.Kolasinskietal.,Perceptuallyrelevantremappingofhumansomatotopyin24hours.

Elife5,(2016).9. N.Ejaz,M.Hamada,J.Diedrichsen,Handusepredictsthestructureofrepresentations

insensorimotorcortex.NatureNeuroscience18,(2015).10. S.N.Flesheretal.,Intracorticalmicrostimulationofhumansomatosensorycortex.Sci

TranslMed8,361ra141(2016).11. F.Mancinietal.,Fine-GrainedMappingofCorticalSomatotopiesinChronicComplex

RegionalPainSyndrome.JNeurosci39,9185-9196(2019).12. S.Kikkertetal.,Revealingtheneuralfingerprintsofamissinghand.Elife5,(2016).13. D. B. Wesselink et al., Obtaining and maintaining cortical hand representation as

evidencedfromacquiredandcongenitalhandlessness.Elife8,(2019).14. M.Bruurmijn,I.P.L.Pereboom,M.J.Vansteensel,M.A.H.Raemaekers,N.F.Ramsey,

Preservation of hand movement representation in the sensorimotor areas ofamputees.Brain140,3166-3178(2017).

15. T.Elbertetal.,Alterationofdigitalrepresentationsinsomatosensorycortexinfocalhanddystonia.Neuroreport9,3571-3575(1998).

16. N.Ejazetal.,paperpresentedattheSocietyforNeuroscienceAnnualMeeting,2016.17. D.Clode,TheThirdThumb.2018(https://www.daniclodedesign.com/thethirdthumb).18. R.O.Maimon-Mor et al., Communicativehand gestures as an implicitmeasureof

artificial limb embodiment and daily usage. medRxiv, 2020.2003.2011.20033928(2020).

19. M.R.Longo,F.Schuur,M.P.Kammers,M.Tsakiris,P.Haggard,Whatisembodiment?Apsychometricapproach.Cognition107,978-998(2008).

20. F.deVignemont,Bodyschemaandbodyimage--prosandcons.Neuropsychologia48,669-680(2010).

21. H.Nilietal.,Atoolboxforrepresentationalsimilarityanalysis.PLoSComputBiol10,e1003553(2014).

22. T.R.Makin,F.deVignemont,A.A.Faisal,Neurocognitivebarrierstotheembodimentoftechnology.NatureBiomedicalEngineering1,0014(2017).

23. F.Y.Wu,H.H.Asada,inRobotics:ScienceandSystemsX.(Berkley,USA,2014).


https://doi.org/10.1101/2020.06.16.151944


24. J. Cunningham, A. Hapsari, P. Guilleminot, A. Shafti, A. A. Faisal, in2018 7th IEEEInternational Conference on Biomedical Robotics and Biomechatronics (Biorob).(2018),pp.665-670.

25. Y.Zhu,T.Ito,T.Aoyama,Y.Hasegawa,Developmentofsenseofself-locationbasedon somatosensory feedback from finger tips for extra robotic thumb control.ROBOMECHJournal6,7(2019).

26. M.M.Merzenichetal.,Topographicreorganizationofsomatosensorycorticalareas3band1inadultmonkeysfollowingrestricteddeafferentation.Neuroscience8,33-55(1983).

27. R.J.Nudo,G.W.Milliken,W.M.Jenkins,M.M.Merzenich,Use-dependentalterationsofmovementrepresentations inprimarymotorcortexofadultsquirrelmonkeys. JNeurosci16,785-807(1996).

28. A.Iriki,M.Tanaka,Y.Iwamura,Codingofmodifiedbodyschemaduringtoolusebymacaquepostcentralneurones.Neuroreport7,2325-2330(1996).

29. A.Maravita,A.Iriki,Toolsforthebody(schema).TrendsCognSci8,79-86(2004).30. L.E.Milleretal.,Sensingwithtoolsextendssomatosensoryprocessingbeyondthe

body.Nature561,239-242(2018).31. K.Ogawa,K.Mitsui,F.Imai,S.Nishida,Long-termtraining-dependentrepresentation

ofindividualfingermovementsintheprimarymotorcortex.Neuroimage202,116051(2019).

32. M.S.Graziano,T.N.Aflalo,Mappingbehavioralrepertoireontothecortex.Neuron56,239-251(2007).

33. H. Dempsey-Jones et al., Transfer of tactile perceptual learning to untrainedneighboringfingersreflectsnaturaluserelationships.JNeurophysiol115,1088-1097(2016).

34. N.M.Timme,C.Lapish,ATutorialforInformationTheoryinNeuroscience.eneuro5,ENEURO.0052-0018.2018(2018).

35. R. Alonso, I. Brocas, J. D. Carrillo, Resource Allocation in the Brain.The Review ofEconomicStudies81,501-534(2013).

36. V. Lee, J. Maguire, The impact of tonic GABAA receptor-mediated inhibition onneuronalexcitabilityvariesacrossbrainregionandcelltype.FrontNeuralCircuits8,3(2014).

37. G. H. Recanzone, M. M. Merzenich, W. M. Jenkins, K. A. Grajski, H. R. Dinse,Topographic reorganization of the hand representation in cortical area 3b owlmonkeys trained in a frequency-discrimination task. J Neurophysiol67, 1031-1056(1992).

38. C.Mehringetal.,Augmentedmanipulationabilityinhumanswithsix-fingeredhands.NatCommun10,2401(2019).

39. E.R.Obyetal.,Newneuralactivitypatternsemergewithlong-termlearning.ProcNatlAcadSciUSA116,15210-15215(2019).

40. F.M.Petrinietal.,Enhancingfunctionalabilitiesandcognitiveintegrationofthelowerlimbprosthesis.SciTranslMed11,(2019).

41. L. Resnik et al., Evaluation of EMG pattern recognition for upper limb prosthesiscontrol: a case study in comparison with direct myoelectric control. J NeuroengRehabil15,23(2018).


https://doi.org/10.1101/2020.06.16.151944


42. C.Wickens,A.Kramer, L.Vanasse,E.Donchin,Performanceof concurrent tasks:apsychophysiologicalanalysisof thereciprocityof information-processingresources.Science221,1080-1082(1983).

43. N.J.Jarque-Bou,A.Scano,M.Atzori,H.Muller,Kinematicsynergiesofhandgrasps:acomprehensivestudyonalargepubliclyavailabledataset.JNeuroengRehabil16,63(2019).

44. J.N.Ingram,K.P.Kording,I.S.Howard,D.M.Wolpert,Thestatisticsofnaturalhandmovements.ExpBrainRes188,223-236(2008).

45. M.Santello,M.Flanders,J.F.Soechting,PosturalHandSynergiesforToolUse.TheJournalofNeuroscience18,10105-10115(1998).

46. N.Lambert-Shirzad,H.F.VanderLoos,Onidentifyingkinematicandmusclesynergies:a comparison of matrix factorization methods using experimental data from thehealthypopulation.JNeurophysiol117,290-302(2017).

47. C.KnightFle,M.R.Longo,A.J.Bremner,Categoricalperceptionoftactiledistance.Cognition131,254-262(2014).

48. L.M.Parsons,Imaginedspatialtransformationsofone'shandsandfeet.CognPsychol19,178-241(1987).

49. G.J.VanBreukelen,ANCOVAversuschangefrombaseline:morepowerinrandomizedstudies,morebiasinnonrandomizedstudies[corrected].JClinEpidemiol59,920-925(2006).

50. M. Jenkinson, C. F. Beckmann, T. E. Behrens, M. W. Woolrich, S. M. Smith, Fsl.Neuroimage62,782-790(2012).

51. S. M. Smith et al., Advances in functional and structural MR image analysis andimplementationasFSL.Neuroimage23Suppl1,S208-219(2004).

52. D. Wesselink, R. Maimon-Mor, rsatoolbox, version 20fbe05. 2017(https://github.com/ronimaimon/rsatoolbox).

53. B.Fischl,A.Liu,A.M.Dale,Automatedmanifoldsurgery:constructinggeometricallyaccurateandtopologicallycorrectmodelsofthehumancerebralcortex.IEEETransMedImaging20,70-80(2001).

54. A.M.Dale,B.Fischl,M.I.Sereno,Corticalsurface-basedanalysis.I.Segmentationandsurfacereconstruction.Neuroimage9,179-194(1999).

55. M.Jenkinson,P.Bannister,M.Brady,S.Smith,Improvedoptimizationfortherobustandaccuratelinearregistrationandmotioncorrectionofbrainimages.Neuroimage17,825-841(2002).

56. S.M.Smith,Fastrobustautomatedbrainextraction.HumBrainMapp17,143-155(2002).

57. M.Jenkinson,S.Smith,Aglobaloptimisationmethodforrobustaffineregistrationofbrainimages.MedImageAnal5,143-156(2001).

58. D.N.Greve,B.Fischl,Accurateand robustbrain imagealignmentusingboundary-basedregistration.Neuroimage48,63-72(2009).

59. S.A.Arbuckle,A.Yokoi,J.A.Pruszynski,J.Diedrichsen,Stabilityofrepresentationalgeometry across a wide range of fMRI activity levels. Neuroimage 186, 155-163(2019).

60. A.Yokoi,S.A.Arbuckle,J.Diedrichsen,TheRoleofHumanPrimaryMotorCortexintheProductionofSkilledFingerSequences.TheJournalofNeuroscience38,1430-1442(2018).


https://doi.org/10.1101/2020.06.16.151944


61. B.Fischletal.,Corticalfoldingpatternsandpredictingcytoarchitecture.CerebCortex18,1973-1980(2008).

62. T.Wiestler,J.Diedrichsen,Skilllearningstrengthenscorticalrepresentationsofmotorsequences.Elife2,e00801(2013).

63. T.A.Yousryetal.,Localizationofthemotorhandareatoaknobontheprecentralgyrus.Anewlandmark.Brain120(Pt1),141-157(1997).

64. A.Hahamyetal.,RepresentationofMultipleBodyPartsintheMissing-HandTerritoryofCongenitalOne-Handers.CurrBiol27,1350-1355(2017).

65. F.M.Z.vandenHeiligenbergetal.,Artificiallimbrepresentationinamputees.Brain141,1422-1433(2018).

66. A. Hahamy, T. R. Makin, Remapping in Cerebral and Cerebellar Cortices Is NotRestrictedbySomatotopy.TheJournalofNeuroscience39,9328(2019).

67. S.B.Eickhoffetal.,AnewSPMtoolboxforcombiningprobabilisticcytoarchitectonicmapsandfunctionalimagingdata.Neuroimage25,1325-1335(2005).

68. Y.Behzadi,K.Restom,J.Liau,T.T.Liu,Acomponentbasednoisecorrectionmethod(CompCor)forBOLDandperfusionbasedfMRI.Neuroimage37,90-101(2007).

69. S.Whitfield-Gabrieli,A.Nieto-Castanon,Conn:afunctionalconnectivitytoolboxforcorrelatedandanticorrelatedbrainnetworks.BrainConnect2,125-141(2012).

70. Y. Zhang,M. Brady, S. Smith, Segmentation of brainMR images through a hiddenMarkovrandomfieldmodelandtheexpectation-maximizationalgorithm.IEEETransMedImaging20,45-57(2001).

71. N. Kriegeskorte, M. Mur, P. Bandettini, Representational similarity analysis -connectingthebranchesofsystemsneuroscience.FrontSystNeurosci2,4(2008).


https://doi.org/10.1101/2020.06.16.151944


Neurocognitive consequences of hand augmentation · 1 day ago · Neurocognitive consequences of...

Documents

Transcript of Neurocognitive consequences of hand augmentation · 1 day ago · Neurocognitive consequences of...