Image-Based Visual Servo Control of the Translation Kinematics of a Quadrotor Aerial Vehicle

[3]Y.Kawauchi,I.Makoto,andT.Fukuda,“Aprincipleofdistributedde-cisionmakingofcellularroboticsystem(CEBOT),”inProc.IEEEInt.Conf.Robot.Autom.,Piscataway,NJ:IEEEPress,1993,pp.833–838.[4]M.Yim,K.Roufas,D.Duff,Y.Zhang,C.Eldershaw,andS.B.Homans,

“Modularreconfigurablerobotsinspaceapplications,”Auton.Robots,vol.14,no.2/3,pp.225–237,2003.

[5]M.Yim,D.G.Duff,andK.D.Roufas,“PolyBot:Amodularreconfig-urablerobot,”inProc.IEEEInt.Conf.Robot.Autom.,vol.1,Piscataway,NJ:IEEEPress,2000,pp.514–520.

[6]A.Castano,W.M.Shen,andP.Will,“CONRO:Towardsdeployable

robotswithinter-robotsmetamorphiccapabilities,”Auton.Robots,vol.8,no.3,pp.309–324,2000.

[7]S.Hirose,T.Shirasu,andE.F.Fukushima,“Proposalforcooperative

robot“Gunryu”composedofautonomoussegments,”Robot.Auton.Syst.,vol.17,pp.107–118,1996.

[8]H.B.Brown,J.M.V.Weghe,C.A.Bererton,andP.K.Khosla,“Millibot

trainsforenhancedmobility,”IEEE/ASMETrans.Mechatron.,vol.7,no.4,pp.452–461,Dec.2002.

[9]R.Damoto,A.Kawakami,andS.Hirose,“Studyofsuper-mechano

colony:Conceptandbasicexperimentalset-up,”Adv.Robot.,vol.15,no.4,pp.391–408,2001.

[10]K.Motomura,A.Kawakami,andS.Hirose,“Developmentofarm

equippedsinglewheelrover:Effectivearm-posture-basedsteeringmethod,”Auton.Robots,vol.18,no.2,pp.215–229,2005.

[11]S.Murata,E.Yoshida,A.Kamimura,H.Kurokawa,K.Tomita,

andS.Kokaji,“M-tran:Self-reconfigurablemodularroboticsystem,”IEEE/ASMETrans.Mechatron.,vol.7,no.4,pp.431–441,Dec.2002.[12]W.M.Shen,M.Krivokon,H.Chiu,J.Everist,M.Rubenstein,and

J.Venkatesh,“Multimodelocomotionforreconfigurablerobots,”Auton.Robots,vol.20,no.2,pp.165–177,2006.

[13]M.Yim,W.M.Shen,B.Salemi,D.Rus,M.Moll,H.Lipson,E.Klavins,

andG.S.Chirikjian,“Modularself-reconfigurablerobotsystems,”IEEERobot.Autom.Mag.,vol.14,no.1,pp.43–52,Mar.2007.

[14]D.RusandM.Vona,“Crystallinerobots:Self-reconfigurationwithcom-pressibleunitmodules,”Auton.Robots,vol.10,no.1,pp.107–124,2001.

[15]P.White,V.Zykov,J.Bongard,andH.Lipson,“Threedimensional

stochasticreconfigurationofmodularrobots,”inProc.Robot.Sci.Syst.,Cambridge,MA:MITPress,2005,pp.161–168.[16]C.JonesandM.J.Matari´c,“Fromlocaltoglobalbehaviorinintelli-gentself-assembly,”inProc.IEEEInt.Conf.Robot.Autom.,vol.1,Los

Alamitos,CA:IEEEComput.Soc.Press,2003,pp.721–726.

[17]E.Klavins,R.Ghrist,andD.Lipsky,“Agrammaticalapproachtoself-organizingroboticsystems,”IEEETrans.Autom.Control,vol.51,no.6,pp.949–962,Jun.2006.

[18]W.M.Shen,P.Will,A.Galstyan,andC.M.Chuong,“Hormone-inspired

self-organizationanddistributedcontrolofroboticswarms,”Auton.Robots,vol.17,no.1,pp.93–105,2004.

[19]A.L.Christensen,R.O’Grady,andM.Dorigo,“Morphologycontrolina

multirobotsystem,”IEEERobot.Autom.Mag.,vol.14,no.4,pp.18–25,Dec.2007.

[20]F.Mondada,L.M.Gambardella,D.Floreano,S.Nolfi,J.-L.Deneubourg,

andM.Dorigo,“Thecooperationofswarm-bots:Physicalinteractionsincollectiverobotics,”IEEERobot.Autom.Mag.,vol.12,no.2,pp.21–28,Jun.2005.

[21]A.L.Christensen,“Efficientneuro-evolutionofhole-avoidanceandphoto-taxisforaswarm-bot,”D.E.A.thesis,Inst.RecherchesInterdisciplinairesD´ev.Intell.Artif.(IRIDIA),Univ.LibredeBruxelles,Brussels,Belgium,Tech.Rep.TR/IRIDIA/2005-14,Oct.2005.

Image-BasedVisualServoControloftheTranslation

KinematicsofaQuadrotorAerialVehicle

OdileBourquardez,RobertMahony,NicolasGuenard,Franc¸oisChaumette,TarekHamel,andLaurentEck

Abstract—Inthispaper,weinvestigatearangeofimage-basedvisual

servocontrolalgorithmsforregulationofthepositionofaquadrotoraerialvehicle.Themostpromisingcontrolalgorithmshavebeensuccessfullyimplementedonanautonomousaerialvehicleanddemonstrateexcellentperformance.

IndexTerms—Aerialroboticvehicle,visualservoing.

I.INTRODUCTION

Visualservoalgorithmshavebeenextensivelydevelopedintheroboticsfieldoverthelasttenyears[7],[10],[19],[23].Visualservocontroltechniqueshavealsobeenappliedrecentlytoalargevarietyofreduced-scaleaerialvehicles,suchasquadrotors[1],[25],helicopters[2],[22],[26],[29],airships[4],[30],andairplanes[5],[24].Inthispaper,weconsidervisualservocontrolofaquadrotoraerialvehicle.Muchoftheexistingresearchinvisualservocontrolofaerialrobots(andparticularly,autonomoushelicopters)hasusedposition-basedvi-sualservotechniques[1],[2],[22],[25]–[27],[29].Theestimatedposecanbeuseddirectlyinthecontrollaw[1],oraspartofaschemefusingvisualdataandinertialmeasurements[29].Inthispaper,wedonotdealwithposeestimation,butconsiderimage-basedvisualservo(IBVS),similartotheapproachconsideredin[4],[17],and[30].

ThesystemdynamicsissometimesexplicitlytakenintoaccountinIBVS.Thisstrategyhasbeenappliedforroboticmanipulators[9],[12],[20]andforaerialvehicles[15],[30].Anotherpopularapproach(asusuallydoneformostroboticsystemssuchasrobotarms,mobilerobots,etc.)isbasedonseparatingthecontrolproblemintoaninnerloopandanouterpositioncontrolloop.Asforhelicopters,theinnerat-titudeloopisrunathighgainusinginputsfrominertialsensors,rategy-rometers,andaccelerometersacquiredathighdatarate,whiletheouterloopisrunatlowgainusingvideoinputfromthecamera[26],[27].Theouter(visualservo)loopprovidessetpointsfortheinnerattitudeloopandclassicaltime-scaleseparationandhigh-gainargumentscanbeusedtoensurestabilityoftheclosed-loopsystem[1],[11],[15],[27].

ManuscriptreceivedJune17,2008;revisedDecember4,2008.Firstpub-lishedFebruary2,2009;currentversionpublishedJune5,2009.ThispaperwasrecommendedforpublicationbyAssociateEditorP.RivesandEditorW.K.Chunguponevaluationofthereviewers’comments.ThisworkwassupportedbytheCentreNationaldelaRechercheScientifique(CNRS)undertheProjectRobotiqueetEntit´esArtificielles(ROBEA)–RobvolintandtheInternationalProgramsforScientificCooperation(PICS)betweenFranceandAustraliaonvisualservo-controlofunmannedaerialvehicles.

O.BourquardezandF.ChaumettearewiththeInstitutdeRechercheenIn-formatiqueetSyst`emesAl´eatoires(IRISA)–CentreNationaldelaRechercheScientifique(CNRS)andl’InstitutNationaldeRechercheenInformatiqueetenAutomatique(INRIA),35042Rennes,France(e-mail:odile.bourquardez@voila.fr;francois.chaumette@irisa.fr).

R.MahonyiswiththeDepartmentofEngineering,AustralianNationalUni-versity,Canberra,A.C.T.0200,Australia(e-mail:robert.mahony@anu.edu.au).

`l’EnergieAtomiqueN.GuenardandL.EckarewiththeCommissariata

(CEA)/List,92265Fontenay-aux-Roses,France(e-mail:nicolas.guenard@cea.fr;laurent.eck@cea.fr).

T.HameliswiththeLaboratoired’Informatique,SignauxetSyst`emesdeSophiaantipolis(I3S),Universit´edeNiceSophia-Antipolis(UNSA)–CentreNationaldelaRechercheScientifique(CNRS),06903SophiaAntipolis,France(e-mail:thamel@i3s.unice.fr).

Colorversionsofoneormoreofthefiguresinthispaperareavailableonlineathttp://ieeexplore.ieee.org.

DigitalObjectIdentifier10.1109/TRO.2008.2011419

1552-3098/$25.00©2009IEEE

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

744IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009

Inthispaper,wetaketheinner/outerloopstabilityforgranted(see[14]fordetails)andconcentrateonthespecificpropertiesoftheouterIBVScontroldesign.Itallowsdesigningkinematiccontrollers,whichgivemanyadvantagesinpractice.Forexample,usinganembeddedcamerathatsendstheimagestoagroundstationimpliestimedelaysandthenaslowimage-basedcontrolloop.Itisthusinterestingtohavealowerlevellooptoensurethestabilizationofthesystem.Then,anotherad-vantagetoconsiderkinematiccontrolistoenableeasierreuseoftheIBVSscheme,sinceitisnotclosetothematerialequipmentoftheaerialvehicle.Inthispaper,severalcontrolschemesareproposed,compared,andthemostpromisingonesareshowntobestableinpracticeandtoprovidesatisfactorybehavior.

Followingearlierwork[15],[17],[28],wehavechosentousezeroandfirst-orderimagemomentsasprimaryvisualfeaturesforthecontroldesign.PerspectiveprojectionmomentswithsuitablescalingalongwithaclassicalIBVScontroldesignleadtosatisfactorytransientsandasymptoticstabilityoftheclosed-loopsystemwhentheimageplaneremainsparalleltoaplanartarget.However,thesystemresponsemaylackrobustnessforaggressivemaneuvers.Inordertoovercomethisproblem,severalcontrolschemes,basedonsphericalfirst-orderimagemoments,aredesignedandtheirperformanceisanalyzed.Themostpromisingcontrolalgorithmshavebeensuccessfullyimplementedonanautonomousaerialvehicleshowingexcellentperformance.

Thepaperisorganizedasfollows.SectionIIdevelopsaclassicalIBVScontrolschemeusingperspectiveimagemoments.SectionIIIintroducesthedefinitionandpropertiesoffirst-ordersphericalimagemomentsandpresentsarangeofcontrollawsforthetranslationalmotionofthecamerausingthisvisualfeature.SectionIVprovidesananalysisandacomparisonofthecontrollawsproposed.ExperimentalresultsarepresentedinSectionV.

II.PERSPECTIVEPROJECTION

Inthissection,anIBVScontrolforregulationofthetranslationkinematicsofanaerialvehicleispresented.

Inordertoobtainaquasi-linearanddecoupledlinkbetweentheimagespaceandthetaskspace,theimagefeaturesusedareper-spectiveprojectionimagemoments[28].Thevisualfeaturevectors=(xn,yn,an)isdefinedsuchthat[28]

󰀋a∗∗

,xn=anxg,an=Zyn=anygawhereaistheareaoftheobjectintheimage,xg,ygitscentroidcoordinates,a∗thedesiredarea,andZ∗thedesireddepthbetweenthecameraandthetarget.Thetimederivativeofsandtherelativemotionbetweenthecameraandtheobjectcanberelatedbytheclassicalequation

˙=Lυυ+Lωω(1)swhereυandωare,respectively,thelinearandangularvelocityofthe

camerabothexpressedinthecameraframe,andwhereLυandLωare,respectively,thepartsoftheinteractionmatrixrelatedtothetransla-tionalandrotationalmotions.Thedesiredimagefeatureisdenotedbys∗,andthevisualerrorisdefinedbye=s−s∗.

ClassicalIBVScontroldesignaimstoimposelinearexponentialstabilityontheimageerrorkinematics[10],[21],[28]toensurean

˙=−λe,withλapositiveexponentialdecoupleddecreasefore(e

gain).Usingetocontrolthetranslationaldegreesoffreedom,theclassicalIBVScontrolinputis

υ=−(Lυ)−1(λe+Lωω),

theobservedvisualdata.Thevisualfeatures=(xn,yn,an)isofparticularinterestsinceLυ=−I3inthecasewherethecameraimageplaneisparalleltothetargetplane[28].Inthatcase,sincethelinkbetweenimagespaceandtaskspaceislinearanddecoupled,thecontrolscheme(2)isknowntoleadtosatisfactoryclosed-loopbehaviorforholonomicrobot[28].Itis,infact,equivalenttoaposition-basedvisualservo,butwithoutanyposeestimationrequired.

Intheapplicationconsidered,thecameraismountedtopointdirectlydownwardinthequadrotorandtheimageandtargetplanearenevermorethanacoupleofdegreesoffset.Asaconsequence,theapproxi-mationLυ≈−I3isvalid.Furthermore,themotionofthequadrotorissmoothandslowandthevalueofLωωissmallcomparedwiththeer-rorλein(2).Thus,areasonableapproximationof(2)forthepurposesofthispaperis

λ>0.(3)υ=λe,Equation(3)doesnotrequiretheestimationofany3-Dparameters

andcanbeimplementedbasedonlyontheobservedimagefeaturess.ThiscontrolwasimplementedontheexperimentalplatformandtheresultsarediscussedinSectionV-B.Thelimitationofthisapproach,however,liesinitsdependenceontheparticulargeometryoftheap-plicationconsideredandtherequirementtoconsideronlysmoothslowtrajectoriesofthevehicle.Ifthevehicleundertakesaggressivema-noeuvres,ortheparalleltargetplaneassumptionisinvalidatedforaparticularapplication,theapproximationLυ≈−I3willfail,andmoreimportantly,theapproximationLωω≈0mayalsofail.Thissecondissueintroducesasignificantdynamicdisturbanceinthesystemre-sponsethatcannotbecancelleddirectlywithouttheriskofintroducingzerodynamiceffectsintotheclosed-loopresponsesimilartothosestudiedinrecentresearch[11],[18].ThepotentiallimitationsoftheclassicalIBVScontroldesignbasedonperspectiveprojectionfeaturesmotivateustoconsideraclassofsphericalprojectionfeaturesandnonlinearcontroldesigntechniques.

III.SPHERICALPROJECTION

A.Modeling

Inthissection,weuseanunnormalizedfirst-ordersphericalimagemomentalongwithaninertialgoalvectortogenerateanimageerror[17].ConsiderapointtargetconsistingofnpointsPicorresponding

sphericalimagesurface.Thetoimagepointspi(i∈(1,...,n))onthe󰀇

centroidofatargetisdefinedtobeq=ni=1pi.Thecentroidqisa3-Dvector.Thankstothesphericalcamerageometry,thethirdentryofthecentroidisnonlinearlyrelatedtothedepthofthecamerafromtheobservedtargetconstellation.

Forapointtargetcomprisingafinitenumberofimagepoints,the

˙=kinematicsoftheimagecentroidareeasilyverifiedtobe[17]q󰀇i=nπpi󰀄

−ω×q−Qυ,whereQ=i=1|Pi|andπp=(I3−pp).Aslongasthereareatleasttwopointspiinimagespace,thematrixQispositivedefinite[17].

Letbdenotethevectorthatdefinesthedirectionofthefixeddesiredsetpointforthevisualfeatureq,expressedinafixedinertialframeFA.Theimage-basederrorconsideredis

δ=q−q∗

(4)

λ>0.(2)

Generally,theinteractiontermsLυandLωdependnonlinearlyonthestateofthesystemandcannotbereconstructedexactlyfrom

whereq∗=R󰀄b,andtherotationmatrixRbetweenthecameraframeFCandthefixedinertialframeFA(seeFig.1)isassumedtobeknown,acommonassumptionwhendealingwiththecontrolofunder-actuatedsystemssuchashelicopters[17].

Thereasonforchoosingtheimageerrorinthismanneristhatitensuresthepassivityproperty.Theimageerrorkinematicsare[17]

˙=δ×ω−Qυ.δ

(5)

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009745

imagefeaturesandthetranslationaldegreesoffreedom.Furthermore,satisfactorybehavioroftheimagefeatureswillautomaticallyinduceanacceptablebehaviorinthetaskspace.

Weproposetoconsideranewimagefeature

f=F(|q|)q0,

withF(|q|)=󰀉R|q|n2−|q|2(9)

Fig.1.

CameraframeFC,fixedframeFA,andvisualfeaturesqandq∗.

Itcanbeshownthat|δ|andδ0=Rδareafunctionofpositiononly[6].Thispropertycanbeexploitedtocontrolthetranslationaldynamicsindependentlyoftherotations.B.ProportionalControl

Apureproportionalfeedbackoftheunnormalizedcentroid[17]ensuresglobalasymptoticstability(GAS)property,butinpracticeithasbeenshownin[6]thattaskspaceandimagespacebehaviorarenotacceptable.ThisisduetothefactthattheconvergenceratesaregivenbyQ,andthismatrixisnotwell-conditioned.Thesimplecontrollaw

υ=kδδ,

isthusnotsuitableinpractice.C.PartitionedControl

Asolutionforcompensatingthepoorsensitivityinthepreviouscontroldesignistouseapartitionedapproachbysinglingouttheproblematiccomponentforspecialtreatment[8],[15].

Theideaistoseparatethevisualerrortermintotwocriteriawithdifferentsensitivity.Thenewvisualfeature

δA=δ11+λq∗0δ12

(7)

kδ>0

(6)

whereq0=|qisthenormalizedfirst-ordermomentandF(|q|)repre-q|sentsaroughapproximationoftheactualdepthZfromthegeometriccenterofthetarget.nisthenumberofpointsobservedandRistheapproximateradiusofthetarget.Theerrorδfisdefinedasfollows

δf=f−f∗=F(|q|)q0−F(|q∗|)q∗0.

(10)

˙f=−ω×δf−MQυwhereM(q)=Itcanbeshownthatδ

󰀐󰀑∂F(|q|)F(|q|)󰀄󰀄I[6].qq+−qq03000∂|q||q|Itcanbeshownthatf󰀂−ξ(whereξrepresentsthecamerapositionwithrespecttothetarget,expressedinthecameraframe)andMQM󰀂Q−1[6].Sincef󰀂−ξ,anintuitiveideaistochoose

υ=kfδf,

kf>0.

(11)

SinceMQ󰀂I3,weobtainapproximatelythesameconvergencerateforthecomponentsoftheerror[6].

AswewillseeinSectionV-C2,theexperimentalresultsusingthiscontrollawshowexcellentperformance.Itsadvantageisalsothatitiseasilyimplemented,sincethecontrollawisadirectfunctionofthevisualerrorδf.Furthermore,since(10)hastheadditionalpassivityproperty,itisexpectedtobewell-adaptedforawiderangeofaerialvehiclesandexperimentalconditions.

However,similartotheperspectivemomentscontroldesign,theglobalasymptoticstabilityhasnotbeendemonstrated.E.GASControlLawWithModifiedRescaledImageFeatureInthissection,weattempttodefineanewimagefeatureandcontrollawthatcombinethepropertiesofgoodtransientbehavior,goodlocalexponentialstability,andglobalasymptoticstability.TheapproachtakenistodefineanewscalingfunctionG(|q|)andscaledimagefeature

g=G(|q|)q0,

˙=−ω×g−HQυwithg

(12)

isdefinedbyusingtheconstantλ(chosenasshownin[6]),andthe

followingtwonewerrorterms

δ11=

q∗0

×q,

δ12=

q∗󰀄0δ,

with

q∗0

q∗

=∗.|q|Itcanbeshownthatthecontrollaw

󰀄

υ=kAA(q∗0)δA,

kA>0(8)

∗∗∗󰀄

withA(q∗0)=sk(q0)+λq0q0ensuresthatthesystemisGAS[6].

∗

Notethatsk(q∗0)istheskew-symmetricmatrixsuchthatsk(q0)w=q∗0×wforanyvectorw.

Thispartitionedcontrolschemehasbeenusedin[15]bydesigningandexperimentingadynamiccontrolofaquadrotor.AsshowninSectionV-C1,althoughitenablestoensurethedesirableGASpropertyinpractice,thepartitionedcontrolschemecanleadtopoorbehaviorofthesystemassoonasthedistancebetweeninitialanddesiredpositionincreases[6],[15].Inordertoensuregoodbehaviorinpractice,weproposethefollowingcontrollaws.

whereG(|q|)canbechosensothatHinducesgoodpropertiesforasymptoticstabilityoftheresultingcontrollaw.SimilartoSectionIII-DforthederivationofM,wehavetherelationshipbetweenmatrixH󰀑∂G(|q|)G(|q|)󰀐󰀄

andfunctionG(|q|)[6]:H(q)=∂|q|q0q0+|q|I3−q0q󰀄0.Theerrorδgisdefinedasfollows

δg=g−g∗=G(|q|)q0−G(|q∗|)q∗0.

Recalling(12),thedynamicsofthiserrorfunctionisgivenby

˙δg=−ω×δg−HQυ,andwecannotethatδgensuresthepas-sivityproperty,asexpectedfromthechoiceofg.󰀉

ChoosingthescalefactorG(|q|)=α(|q|)|q|F(|q|),andcontrollaw

H(q)

δg,kg>0(13)υ=kg

α(|q|)2whereα(|q|)issuchthatα(|q∗|)=1[6],ensuresGASandgoodlo-calexponentialstabilityoftheclosed-loopsystem[6].Thenewimagefeatureg=G(|q|)q0,andthepreviousfeaturef=F(|q|)q0arede-signedinthesamemanner:thedirectionofthefeatureisgivenbyq0,andthenormisgivenbythescalingfactorG(|q|)andF(|q|),

D.RescaledImageFeature

Toimprovetherelationshipbetweentaskspacebehaviorandimagespacebehavior,itisnaturaltotrytodetermineanimagefeaturethatisasclosetothe3-Dtranslationbetweenthecameraandthetargetaspossible[28].Suchachoiceleadstoaninteractionmatrixclosetotheidentity,leadingtoalinearanddecoupledlinkbetweenthe

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

746IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009

TABLEI

PROPERTIESOFTHEDIFFERENTCONTROLSCHEMESCONSIDERED

respectively.G(|q|)providesalessaggressivescalingcorrectionthanF(|q|)[6].Thisimprovesthesensitivityoftheimagefeaturetopixelnoiseandimprovesrobustnessoftheclosed-loopsystem.Adisad-vantageofthenewimagefeaturegisthatitisnotascloselylinkedtotheactualtaskspacecoordinatesasthefeaturef(orthe2-Dper-spectivemomentsusedinSectionII).SinceF(|q|)isanapproxi-mationofthedepth,thefeaturef=F(|q|)q0isdirectlyrelatedtothe3-Dposition.󰀉Incaseofthefeatureg,usingthescalefactorG(|q|)=α(|q|)|q|F(|q|),therelationshipbetweenimagespaceandtaskspaceisnonlinear.Thisleadstosomedegradationoftheglobaltransientbehaviorforcertaininitialconditions.However,thisissuehaslimitedeffectontheobservedperformanceoftheclosed-loopsysteminpractice.AsshowninSectionV-C3,thepracticalresultsareexcellent.

IV.ANALYSIS

ArangeofIBVSschemeshasbeenpresentedinSectionsIIandIII.TableIgivessummaryofthepropertiesforeachcontrolschemeintermsofstability,transientbehavior,linearity,andpassivity.

Inpractice,twoofthemostimportantpropertiesaregoodtran-sientconditioning(directconvergenceofallelementsofpositionintaskspacewithoutanyobserveddivergenceorpeakingtransients),andbalancedlocalexponentialstability(equalasymptoticrateofconver-genceinallaxesofthepositionintaskspace).Threecontrolschemespresentinterestingproperties:theperspectiveimagemoments(controlscheme1),therescaledproportionalfeedback(controlscheme4),andthemodifiedrescaledcontrol(controlscheme5).Amongthesethreebestcontrollaws,eachonehasadvantagesanddrawbacks,andnooneisgloballybetterthantheothers.Inthenextsection,thesethreecontrollawsarevalidatedandcomparedthroughexperimentalresults.

V.EXPERIMENTALRESULTSANDCOMPARISON

OFSOMECONTROLLAWS

Inthissection,weprovideexperimentalverificationoftheperfor-manceoftheproposedcontrolschemesonanaerialroboticvehicle.Theexperimentswereundertakenonaquadrotoraerialvehicle.Thetaskconsideredistostabilizethevehiclewithrespecttoaspecifiedtarget.A.ExperimentalConditions

1)PrototypeDescription:Theunmannedaerialvehicleusedfortheexperimentationisaquadrotor,whichisanomnidirectionalverti-caltakeoffandlanding(VTOL)vehicleideallysuitedforstationaryandquasi-stationaryflightconditions.Itconsistsoffourfixed-pitchpropellerslinkedtoanelectricalmotorateachextremityofacrossframe(seeFig.2).Thevehicleisequippedwithanavionicsstackincludinganinertialmeasurementunit(IMU)supplyingthevehicleattitudeandacontrollerboard[15].Theembeddedloopallowingtheattitudestabilizationrunsat166Hzandthetimetoreachanattitudeor-derisabout300ms.Anumericalwirelesslinkallowsthetransmission

Fig.2.Experimentalsystem.

Fig.3.Low-andhigh-levelcontrolloops.

oftheattitudecommandbetweenthequadrotorandagroundstation(Pentium4)withatimetransmissionof110ms.Acamerasituatedbelowthequadrotorisembeddedandobservesatargetontheground,consistingoffourblackmarksontheverticesofaplanarrectangle(30cm×40cm)(seeFig.2).Awirelessanaloglinktransmitscam-eraimagestothegroundstation.Allthevisualservocontrolstestedareimplementedonthegroundstationatthesampletimeof60ms.Consequently,consideringthehighsamplingratelowlevelandthelowsamplingratehighlevel,wecanassumethatthelowlevelandthehighlevelcontrolareentirelydecoupled.Ademonstrationbasedonsingularperturbationsandsimilarargumentsasin[13]canshowthestabilityoftheentireclosed-loopsystem.A3-DestimationofthevehiclepositionwithrespecttothetargetisalsoobtainedbyfusingthedataoftheembeddedIMUandthevisualdatainaparticlefilter[3].Thisestimateisusedtoprovideanestimateofgroundtruthforthe3-Dbehaviorofthevehicleandtoprovideanestimateofthelinearvelocityofthevehiclethatisusedbytheinnerloopcontrolleroftheairframedynamics[14](seeFig.3).Inthispaper,only2-DvisualinformationisusedintheouterIBVScontrolloopforpositionregulation.

2)ExperimentalProtocol:Inordertocomparetheproposeddif-ferentkinematicvisualservocontrollers,theinitialconditionsoftheexperimentswerechosenidentically.Foreachexperiment,thequadro-torwasservo-controlledtoaspecificinitialpositionusingastandardstate-spacecontrollerderivinginformationfromthetaskspaceposi-tionestimate.Whenthevehicleisstabilizedatthisposition,thevisualcontrolisinitiatedandthe3-Dposition,obtainedfromtheparticlefilter,isrecorded.Thisprotocolensuresthattheflightconditionsare

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009747

Fig.4.Resultsobtainedforυ=0.4e:timeevolution(inseconds)oftherealpositioninthetaskspace(inmeters),(a)ofthemvelocityoutputofthevisualservocontrolυ(inmetersperseconds)(b).Theevolutionofthevisualerrorisplottedon(c),andthetrajectoryofthefourblackmarksintheimageplaneareplottedin(d).

Fig.5.

󰀄Resultsobtainedforυ=kAA(q∗0)δA,configuredasFig.4.

thesameandallowsthecomparisonbetweenthedifferentcontrollers.

Thevelocitydemandwasalsosaturatedat20cm/stoensuretheve-hicleremainsinquasi-stationaryflightregime[16].Consideringtimeslatencyandthehighsamplingofthehigh-levelcontroller,onlylowgainshavebeenused.Thetechniqueusedtotunethesegainsconsistsinincreasingthegaintoincreasethebandwidthandstoppingjustbe-foretheUAVbecomesunstable.Then,thesegainshavebeenreducedinordertohaveanexponentialconvergenceinabout10s.

TheinitialpositionofthevehicleisX󰀂0.7m,Y󰀂−0.65m,Z󰀂2m,anditsdesiredpositionisX󰀂0m,Y󰀂0m,Z󰀂1.4m(whichisabovethecenterofthetargetat1.4mheightoftheground).TheasymptoticvalueforthematrixQisQ∗=diag(2.35,2.36,0.057)andwehaveb󰀂(0,0,3.96).

Inthefollowingsubsections,fourkinematicimage-basedcontrolschemesforthetranslationalmotionofthequadrotorareconsidered.Foreachexperiment,the3-Dpositionofthecamerainthetaskspacereferenceframeisdepicted,alongwiththevelocityoutputofthevisualservocontrollaw.Theevolutionofthevisualerrorconsideredisalsodepicted,aswellasthetrajectoryofthefourblackmarksintheimageplane.

Fig.6.Resultsobtainedforυ=0.47δf,configuredasFig.4.

C.SphericalImageMoments

1)PartitionedControl:Withthepartitionedcontrollawusingde-compositionatthesetpointandsphericalimagemoments(controllaw3),thevisualerrorcomponentsarequiteperturbedbutconverge[seeFig.5(c)].Theproblemisthatthecontrollawisnotadequatelyfarfromthedesiredposition.Consequently,wecanseethattheconver-gencerateisnotthesameonthethreecomponentsoftheposition,andZcomponentisnotsuitable[seeFig.5(a)].Moreover,thevelocityout-putofthevisualservocontrolisverydisturbed,evenafterconvergence[t>25s,seeFig.5(b)].

2)ProportionalFeedback:Therescaledproportionalfeedbackus-ingsphericalimagemoments(controllaw4)hasthesamedesirableimagefeaturepropertiesascontrollaw1.Thepracticalresultsareverysatisfactory(seeFig.6)andsimilartotheresultsobtainedwithcontrollaw1.

B.PerspectiveImageMoments

Theclassicalperspectiveimagemomentscontroller(controllaw1)providesalinearcorrespondencebetweentheimagespaceandtaskspaceaslongastherelativerotationbetweenimageandtargetplaneissmall.Theresultingclosed-loopsystemresponseisexpectedtobesatisfactorybothintransientperformanceandasymptoticconvergenceandinbothimageandtaskspace.Thepracticalresultsusingthequadro-torareverysatisfactory(seeFig.4)undertheconsideredexperimentalconditions.However,asaconsequenceofthelimitingassumptionsontherotation,thesystemisneitherGASnorpassive.Moreover,itisexpectedthatstrongrotationalmotionwillsignificantlydisturbtheperformanceofthesystem.

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

748IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009

TABLEII

RMSEOFTHEVELOCITIESFOREACHCONTROLLAW

συ=

󰀊

222συ+συ+συalongwithσυK=XYZ

󰀉󰀇i(υKi−υ¯K)2,forK∈{X,Y,Z}andwhereυ¯KistheaverageofυKbetween10

and25s.

AscanbeseeninTableII,thenoisemeasuredattheoutputofallcontrollawsisverysimilar.Thethreecontrollawshaveverysimilarbehaviorwithrespecttonoise.

VI.CONCLUSION

Thispaperhasinvestigatedasuiteofimage-basedkinematicvisualservocontrolschemestocontrolaquadrotor.Usingthewell-knownperspectiveimagemomentstodesignaclassicalIBVStranslationalcontrollawleadstogoodsystembehaviorintheexperimentalstud-iesundertaken.However,thiscontrolschemedoesnotensureglobalasymptoticstabilityorpassivityoftheclosed-loopsystem,bothprop-ertiesthatwebelievewillbeimportantforthedevelopmentoffullydynamicIBVScontrolschemesinthefuture.First-ordersphericalimagemomentsalongwithaninertialgoalvectorallowustode-signtranslationalcontrollawsindependentfromtherotationmotion.Globalasymptoticstabilityisobtainedbyusingthesevisualfeaturesandasimpleproportionalfeedback,butthebehaviorontheZ-axisisnotacceptable.Arangeofcontrollawshasbeenproposedinordertoimprovethebehaviorofthesystem.Themostpromisingapproachinvestigatedinvolvesrescalingthesphericalimagemomentstoobtainanimagefeaturethatminimizesthesensitivityinthedepthaxis.Theperspectiveimagemomentscontroldesign,aswellasthreeofthecon-trollawsusingsphericalimagemomentswereimplementedonthequadrotor.Inpracticeandasexpected,threecontrolalgorithmsleadtoacceptablebehaviorofthesystem.

REFERENCES

[1]E.Altu˘g,J.P.Ostrowski,andC.J.Taylor,“Controlofaquadrotorheli-copterusingdualcameravisualfeedback,”Int.J.Robot.Res.,vol.24,

no.5,pp.329–341,2005.

[2]O.Amidi,T.Kanade,andK.Fujita,“Avisualodometerforautonomous

helicopterflight,”J.Robot.Auton.Syst.,vol.28,pp.185–193,Aug.1999.

[3]S.Arulampalam,S.Maskell,N.J.Gordon,andT.Clapp,“Atutorial

onparticlefiltersforon-linenon-linear/non-gaussianBayesiantracking,”IEEETrans.SignalProcess.,vol.50,no.2,pp.174–188,Feb.2002.[4]J.R.Azinheira,P.Rives,J.R.H.Carvalho,G.F.Silveira,E.C.dePaiva,

andS.S.Bueno,“Visualservocontrolforthehoveringofanoutdoorroboticairship,”inProc.IEEEInt.Conf.Robot.Autom.,Washington,DC,May2002,vol.3,pp.2787–2792.

[5]O.BourquardezandF.Chaumette,“Visualservoingofanairplanefor

auto-landing,”inIEEE/RSJInt.Conf.Intell.RobotsSyst.,IROS2007,SanDiego,CA,Oct.,pp.1314–1319.

[6]O.Bourquardez,R.Mahony,N.Guenard,F.Chaumette,T.Hamel,and

L.Eck.(2007,Jul.).Kinematicvisualservocontrolofaquadrotoraerialvehicle.IRISA,Rennes,France,Tech.Rep.1858[Online].Available:http://www.irisa.fr/lagadic/publi/year/2007-fra.html

[7]F.ChaumetteandS.Hutchinson,“Visualservocontrol,PartI:Basic

approaches,”IEEERobot.Autom.Mag.,vol.13,no.4,pp.82–90,Dec.2006.

[8]P.CorkeandS.A.Hutchinson,“Anewpartitionedapproachtoimage-basedvisualservocontrol,”IEEETrans.Robot.Autom.,vol.17,no.4,pp.507–515,Aug.2001.

Fig.7.

Resultsobtainedforυ=0.3α(|q|)2δg,configuredasFig.4.

H(q)

Infact,therescaledvisualfeaturef=F(|q|)q0isveryclosetothe3-Dposition,analogouslytothevisualfeaturesusedincontrollaw1.Thecontrollawsareasimpleproportionalfeedbackinthetwocases.Theadvantageofthesphericalimagemomentsisthattheyensurethepassivityproperty,andshouldbemorerobusttoaggressivemaneuversofanaerialvehicleaswellasleadingmorenaturallytoafulldynamicIBVScontroldesign.Apotentialproblem,however,istherequirementtoestimatethecameraattitudeinordertoreconstructtheimage-basederrorterm.ThereisnoformalproofofGASforcontrollaw4;however,duetothenaturalstructureoftheimagefeature,weexpectthatthedomainofstabilityforthiscontrollawwillbesufficientlylargesothatunstablebehaviorwillnotbeencounteredinpractice.

3)GASControlLaw:Thelastsuitablecontrollaw(5)isbasedonamodifiedrescaledvisualfeature,inordertoensureGAS.ThiscontrollawprovidestheguaranteeofGASthatismissingincontrollaw4.Itsonlydrawbackisthatthevisualfeatureisnolongerlinearlyrelatedtothe3-Dpositionandthismayleadtoslightlydegradedtransientresponseintaskspace.

AscanbeseeninFig.7,thiscontrolschemeleadstoverysatisfactorybehavior:equalconvergenceratesofthevisualerrorcomponents,andequalconvergenceratesinthetaskspace.Moreover,intheconsideredexperiment,thetransientbehaviorisacceptable.D.NoiseSensitivity

Atfirstglance,theresults(seeFigs.4,6,and7)forthethreesuitablecontrolschemesareverysimilar.

Apotentialproblemwiththecontrollaws4and5isthattherotationmatrixRbetweenthecameraframeandtheinertialframehastobeestimated.However,thisestimationdoesnotseemtointroducenoise,delay,oranysignificantperturbationsinpractice(compareFig.4withFigs.6and7).

Tobetterunderstandthenoisesensitivityofeachcontrolscheme,wehavecomputedtheroot-mean-squareerror(RMSE)oftheveloc-itydemandovertheperiodbetween10and25s,duringwhichpe-riodallthreeclosed-loopsystemsarestabilizedinaneighborhoodofthesetpoint.Notethattogetridoftheeffectofthegains,wecon-siderthevelocityoutputwithoutthegainsλ,kf,andkg.Wecompute

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.

IEEETRANSACTIONSONROBOTICS,VOL.25,NO.3,JUNE2009749

[9]P.I.CorkeandM.C.Good,“Dynamiceffectsinvisualclosed-loop

systems,”IEEETrans.Robot.Autom.,vol.12,no.5,pp.671–683,Oct.1996.

[10]B.Espiau,F.Chaumette,andP.Rives,“Anewapproachtovisualservoing

inrobotics,”IEEETrans.Robot.Autom.,vol.8,no.3,pp.313–326,Jun.1992.

[11]E.Frazzoli,M.A.Dahleh,andE.Feron,“Real-timemotionplanning

foragileautonomousvehicles,”J.Guid.ControlDyn.,vol.25,no.1,pp.116–129,2002.

[12]J.GangloffandM.deMathelin,“Visualservoingofa6DOFmanipulator

forunknown3Dprofilefollowing,”IEEETrans.Robot.Autom.,vol.18,no.4,pp.511–520,Aug.2002.[13]N.Guenard,“Optimisationetimpl´ementationdeloisdecommandeem-barqu´eespourlat´el´eop´erationintuitivedemicrodronesa´eriensX4-Flyer,”

Ph.D.dissertation,Universit´edeNiceSophiaAntipolis,Nice,France,2007.

[14]N.Guenard,T.Hamel,andL.Eck,“Controllawfortheteleoperation

ofanunmannedaerialvehicleknownasanX4-flyer,”inProc.IEEE/RSJInt.Conf.Intell.RobotsSyst.,Beijing,China,Oct.2006,pp.3249–3254.[15]N.Guenard,T.Hamel,andR.Mahony,“Apracticalvisualservocontrol

foraunmannedaerialvehicle,”IEEETrans.Robot.,vol.24,no.2,pp.331–340,Apr.2008.

[16]N.Guenard,T.Hamel,V.Moreau,andR.Mahony,“Designofacontroller

allowedtheintuitivecontrolofanX4-flyer,”presentedattheInt.IFACSymp.RobotControl,Unive.Bologna,Bologna,Italy,Sep.2006.

[17]T.HamelandR.Mahony,“Visualservoingofanunderactuateddynamic

rigid-bodysystem:Animagebasedapproach,”IEEETrans.Robot.Au-tom.,vol.18,no.2,pp.187–198,Apr.2002.

[18]T.HamelandR.Mahony,“Imagebasedvisualservo-controlforaclass

ofaerialroboticsystems,”Automatica,vol.43,no.11,pp.1975–1983,Nov.2007.

[19]K.HatanoandK.Hashimoto,“Image-basedvisualservousingzoom

mechanism,”inProc.SICE2003Annu.Conf.,Aug.4–6,vol.3,pp.2443–2446.

[20]R.Kelly,R.Carelli,O.Nasisi,B.Kuchen,andF.Reyes,“Stablevi-sualservoingofcamera-in-handroboticsystems,”IEEE/ASMETrans.Mechatronics,vol.5,no.1,pp.39–48,Mar.2000.

[21]E.Malis,F.Chaumette,andS.Boudet,“21/2Dvisualservoing,”IEEE

Trans.Robot.Autom.,vol.15,no.2,pp.238–250,Apr.1999.[22]K.Nordberg,P.Doherty,G.Farneb¨ack,P.-E.Forss´en,G.Granlund,

A.Moe,andJ.Wiklund,“VisionforaUAVhelicopter,”presentedattheIEEE/RSJInt.Conf.Intell.RobotsSyst.,WorkshopAerialRobot.,Lausanne,Switzerland,Oct.2002,pp.xxxx.

[23]R.Pissard-GibolletandP.Rives,“Applyingvisualservoingtechniques

tocontrolofamobilehand-eyesystem,”inProc.IEEEInt.Conf.Robot.Autom.,Nagasaki,Japan,1995,pp.166–171.

[24]P.RivesandJ.R.Azinheira,“Visualauto-landingofanautonomous

aircraft,”INRIA,SophiaAntipolis,France,Tech.Rep.4606,Nov.2002.[25]H.Romero,R.Benosman,andR.Lozano,“Stabilizationandlocationofa

fourrotorhelicopterapplyingvision,”inProc.Amer.ControlConf.,ACC2006,Minneapolis,MN,Jun.,pp.3930–3936.

[26]S.Saripalli,J.F.Montgomery,andG.S.Sukhatme,“Visually-guided

landingofanunmannedaerialvehicle,”IEEETrans.Robot.Autom.,vol.19,no.3,pp.371–381,Jun.2003.

[27]O.Shakernia,Y.Ma,T.Koo,andS.Sastry,“Landinganunmannedair

vehicle:Visionbasedmotionestimationandnonlinearcontrol,”AsianJ.Control,vol.1,no.3,pp.128–145,Sep.1999.

[28]O.TahriandF.Chaumette,“Point-basedandregion-basedimagemoments

forvisualservoingofplanarobjects,”IEEETrans.Robot.,vol.1,no.6,pp.1116–1127,Dec.2005.

[29]A.D.Wu,E.N.Johnson,andA.A.Proctor,“Vision-aidedinertialnavi-gationforflightcontrol,”presentedattheAIAAGuid.,Navigat.,ControlConf.Exhib.,SanFrancisco,CA,Aug.2005.

[30]H.ZhangandJ.P.Ostrowski,“Visualservoingwithdynamics:Control

ofanunmannedblimp,”inProc.IEEEInt.Conf.Robot.Autom.,ICRA1999,Detroit,MI,May,vol.1999,pp.618–623.

QualitativeVision-BasedPathFollowing

ZhichaoChenandStanleyT.Birchfield,SeniorMember,IEEE

Abstract—Wepresentasimpleapproachforvision-basedpathfollowingforamobilerobot.Baseduponanovelconceptcalledthefunnellane,thecoordinatesoffeaturepointsduringthereplayphasearecomparedwiththoseobtainedduringtheteachingphaseinordertodeterminetheturningdirection.Increasedrobustnessisachievedbycouplingthefeaturecoordi-nateswithodometryinformation.Thesystemrequiresasingleoff-the-shelf,forward-lookingcamerawithnocalibration(eitherexternalorinternal,in-cludinglensdistortion).Implicitcalibrationofthesystemisneededonlyintheformofasinglecontrollergain.Thealgorithmisqualitativeinnature,requiringnomapoftheenvironment,noimageJacobian,nohomogra-phy,nofundamentalmatrix,andnoassumptionaboutaflatgroundplane.Experimentalresultsdemonstratethecapabilityofreal-timeautonomousnavigationinbothindoorandoutdoorenvironmentsandonflat,slanted,androughterrainwithdynamicoccludingobjectsfordistancesofhun-dredsofmeters.Wealsodemonstratethatthesameapproachworkswithwide-angleandomnidirectionalcameraswithonlyslightmodification.IndexTerms—Control,featuretracking,mobilerobotnavigation,vision-basednavigation.

I.INTRODUCTION

Route-basedknowledge,inwhichthespatiallayoutofanenviron-mentisrecordedfromtheperspectiveofaground-levelobserver,isanimportantcomponentofhumanandanimalnavigationsystems[31].Inthisrepresentation,navigatingfromonelocationtoanotherinvolvescomparingcurrentvisualinputswithasequenceofviewscapturedalongthepathinapreviousinstance.Applicationsthatwouldbene-fitfromsuchapath-followingcapabilityincludecourieranddeliveryrobots[4],robotictourguides[32],orreconnaissancerobotsfollowingascout[7].

Oneapproachtopathfollowingisvisualservoing,inwhichtherobotiscontrolledtoalignthecurrentimagewithareferenceim-age,bothtakenbyanonboardcamera[14].Suchanapproachgener-allyemploysaJacobiantorelatethecoordinatesofworldpointstotheirprojectedimagecoordinates[5],ahomographyorfundamentalmatrixtorelatethecoordinatesbetweenimages[20],[27],[29],[36],orbundleadjustmenttominimizethereprojectionerrorovermulti-pleimageframes[28].Asaresult,thecamerausuallymustbecali-brated[5],[27],[28],[36],andevenuncalibratedsystemsrequirelens

ManuscriptreceivedJanuary14,2008;revisedSeptember20,2008.FirstpublishedApril14,2009;currentversionpublishedJune5,2009.ThispaperwasrecommendedforpublicationbyAssociateEditorJ.D.TardosandEditorL.Parkeruponevaluationofthereviewers’comments.Thisworkwassup-portedinpartbyaPh.D.FellowshipfromtheNationalInstituteforMedicalInformatics.

TheauthorsarewiththeDepartmentofElectricalandComputerEngineering,ClemsonUniversity,Clemson,SC29634USA(e-mail:zhichac@clemson.edu;stb@clemson.edu).

Thispaperhassupplementarydownloadablemultimediamaterialavailableathttp://ieeexplore.ieee.org,providedbytheauthor.Thematerialincludesthevideo(“ramp.avi”)thatshowsthemobilerobotfollowingapredeterminedpath.Thefourquadrantsareasfollows:(Topleft):Livevideofromtheonboardcam-eraduringthereplayphase.Redsquaresindicatefeatures,withgreenoutlinesindicatingfeaturesthatguidetherobottotheright,andyellowoutlinesindicatingfeaturesthatguidetherobottotheleft.(Topright):Themilestoneimagecapturedduringtheteachingphase.(Bottomleft):Therobotascapturedbyanoffboardvideocamera.(Bottomright):Top-downviewofthepathtraveledbytherobot(white:teaching,red:replay).ThevideoisanAVIfilethatplaysunderWin-dowsMediaPlayer11(2006),aswellasearlierversionssuchasWindowsMediaPlayer6.4(1999),onWindowsXP.Thesizeis8.35MB.Additionalvideosmaybefoundathttp://www.ces.clemson.edu/∼stb/research/mobilerobot.Contactstb@clemson.eduforfurtherquestionsaboutthiswork.DigitalObjectIdentifier10.1109/TRO.2009.2017140

1552-3098/$25.00©2009IEEE

Authorized licensed use limited to: UR Rennes. Downloaded on June 16, 2009 at 05:06 from IEEE Xplore. Restrictions apply.