* model of an activity. We provide both quantitative and qualitative results of our zero-shot framework using these “on the fly” models on the challenging DIVA dataset[^2], which contains fine-grained human-object interactions under a real world video surveillance setting.

In summary, the main contributions of the paper are:

-   A zero-shot classification of complex action sequences with dynamic action signatures which establishes a new state-of-the-art on Olympic Sports [@olympic-sports-2010] dataset. We outperform all other methods for the ZSL evaluation regardless of training assumptions (inductive/transductive).

-   To the best of our knowledge, we are the first to demonstrate zero-shot decoding of complex action sequences. We present our results on a surgical dataset, JIGSAWS [@jigsaws-2014], to jointly segment and classify fine-grained surgical gestures where we establish an impressive baseline.

-   A demonstration of zero-shot classification of fine-grained human-object interactions that requires no supervised training of attributes by leveraging off-the-shelf object detectors in video surveillance.

Related Work
============

Methodology {#sec:methodology}
===========

We first establish a basic hierarchy of concepts. At the highest level we have the *activity*—for example, suturing in robotic surgery. Each activity can be decomposed into a sequence of actions $(y_1, \ldots, y_N)$. Possible examples of actions are “Pushing needle through tissue" in a suturing activity or “throwing javelin" in a sporting event. Zero-shot learning approaches further decompose each action $y$ into a set of $K$ elementary attributes (usually taken to be binary-valued) $y = \{a_1, \ldots, a_K\}$.

Given a video recording of an activity (represented as a sequence of frames $X = (x_1, \ldots, x_T)$), our goal is to map each frame $x_t$ to its corresponding action $y_t$ by detecting the presence or absence of each attribute $\hat{a}(x_t)$ in the video, then choosing the action whose signature $a(y_t)$ best fits those attributes. In other words, we choose the action with highest score: $$\label{eq:decode_cost}
    \hat{y}(x) = \arg
* model of an activity. We provide both quantitative and qualitative results of our zero - scene model using these “ on the fly ” model on the ambitious DIVA dataset[^2 ], which contains fine - grained human - aim interaction under a real universe video surveillance setting. 

 In summary, the independent contributions of the paper are: 

 -    A zero - guess categorization of complex action sequences with active action touch which establishes a new state - of - the - art on Olympic Sports [ @olympic - sports-2010 ] dataset. We surpass all other methods for the ZSL evaluation regardless of education assumptions (inductive / transductive). 

 -    To the good of our knowledge, we are the beginning to demonstrate zero - shot decoding of complex action sequence. We present our results on a surgical dataset, JIGSAWS [ @jigsaws-2014 ], to jointly segment and classify fine - granulate surgical gestures where we establish an impressive baseline. 

 -    A demonstration of zero - shot classification of fine - grained human - object interactions that necessitate no supervised training of attributes by leverage off - the - ledge aim detectors in video surveillance. 

 Related Work 
 = = = = = = = = = = = = 

 Methodology { # sec: methodology } 
 = = = = = = = = = = = 

 We first establish a basic hierarchy of concepts. At the highest level we have the * activity*—for case, suturing in robotic surgery. Each activity can be decomposed into a sequence of military action $ (y_1, \ldots, y_N)$. Possible examples of actions are “ Pushing needle through tissue " in a suturing action or “ throwing javelin " in a sporting event. Zero - shot learning approaches far decompose each action $ y$ into a set of $ K$ elementary attributes (usually taken to be binary - valued) $ y = \{a_1, \ldots, a_K\}$. 

 give a video recording of an bodily process (represented as a sequence of skeleton $ X = (x_1, \ldots, x_T)$), our goal is to map each human body $ x_t$ to its corresponding military action $ y_t$ by detecting the bearing or absence of each attribute $ \hat{a}(x_t)$ in the video, then choosing the legal action whose key signature $ a(y_t)$ best fits those attributes. In other words, we choose the action with highest score: $ $ \label{eq: decode_cost } 
     \hat{y}(x) = \arg
* mofel of an activity. We pruvide both quanjirative and qhalitatixe results of our zero-shot fcameqork ysing these “on the fly” models oj the chqlleiging DIVA datasxf[^2], which contajks fiue-jrained human-obmect interawtions under a rdap world video surveillance setting.

Ig summaty, the main conttibutpogs or the paper are:

-   A zero-shot classjficatimn of complex action sequences with dynwmic action signatures which estavlisrws a new stage-of-the-art on Olympic Aports [@olympic-sports-2010] dataset. We outpzrform all ityer kethods for the DSL evaluatiok regargless og training assmmptimns (inductive/transductivx).

-   To the best of oor knowledce, we are the first ti demmnstsate zeru-shkt dscodinh oh complex adtion sequebces. We present our rqwults on a sudgical dwtaset, JIGSAWS [@jigsaws-2014], to jointly segmett znd classify fine-grainee surgical gestures wjere we eftablish an impressive baseline.

-   A demonstration of zxru-shit zoadsification of fine-grained human-object interasfipnx that requiref no supervosfd jraining of atjributes bg leveraging off-thf-shelf jbjecr detectows im video surveillance.

Related Work
============

Methodojigy {#sec:methodology}
===========

Ce first estcblish a baxic hierarchy of concepcs. At fhe highest level we fave the *activith*—fog exdmple, suujring in robotic furgery. Eech aetivity zan ne decjmposed inho a sequence of actions $(y_1, \ldltv, y_N)$. Possihle examples of actions are “Pusimng needle thtouch nissue" in a suburing activity or “throwing jcvelin" iu a spurting eveht. Zero-vhot learnigg approaches vurther decokpose eash axtiob $y$ intu a set of $K$ elrmentary cutributes (ysually taken to bc binxdy-valued) $y = \{a_1, \lbuus, a_K\}$.

Given a vidro fecjrfiig of dn activity (seprdsevyed ar a sequencc ow frsmes $X = (x_1, \ldots, x_T)$), ogr gkal is to map each fvame $x_t$ ti its cowresponding avtion $y_t$ by detectlng tie prerence or absence of each attribute $\hat{z}(x_t)$ in thf vldeo, then chojsinn thg action whpse signature $a(y_t)$ best fits those attriuutes. In other words, wg choose the action wntm highest scpre: $$\lwbel{eq:decmde_cost}
    \hat{y}(x) = \art
* model of an activity. We provide and results of zero-shot framework using on challenging DIVA dataset[^2], contains fine-grained human-object under a real world video surveillance In summary, the main contributions of the paper are: - A zero-shot classification complex action sequences with dynamic action signatures which establishes a new state-of-the-art on Sports dataset. outperform other methods for the ZSL evaluation regardless of training assumptions (inductive/transductive). - To the best of knowledge, we are the first to demonstrate zero-shot of complex action sequences. present our results on a dataset, [@jigsaws-2014], to segment classify surgical gestures where establish an impressive baseline. - A demonstration of zero-shot classification of fine-grained human-object interactions that requires no training of leveraging off-the-shelf detectors video Related Work ============ =========== We first establish a basic At the highest level we have the *activity*—for suturing in surgery. Each activity can be decomposed a sequence of actions $(y_1, \ldots, y_N)$. Possible of actions are “Pushing needle through tissue" in a suturing activity or “throwing javelin" in event. Zero-shot learning approaches decompose each action into set $K$ attributes (usually to be binary-valued) $y = \{a_1, \ldots, a_K\}$. Given a video of an activity (represented as a sequence of frames $X \ldots, our goal is map each frame $x_t$ its action $y_t$ by detecting or of in video, choosing the action whose $a(y_t)$ best fits those attributes. other words, we choose $$\label{eq:decode_cost} \hat{y}(x) = \arg
* model of an activity. We providE both quantItatiVe aNd qUaLitaTive Results of our zeRO-shoT framework using these “on The flY” mODels ON tHe chaLlenginG dIva DatAsEt[^2], WhiCh COnTains FinE-graineD human-objeCt iNtEractions undER a Real world vIdeO surveillancE seTting.

IN sUmmARy, the MaiN contRibutiONs of thE paper are:

-   a zERo-shot CLassifiCATiOn of Complex action sequENcES with dynamic acTion siGnATuRES whIch EstablisheS a New stATe-of-the-ARt ON oLymPIc Sports [@olympIc-sports-2010] datASet. we outpErForM All othEr metHoDS foR the ZSL evalUatiOn regardlEss of tRAining aSSumptioNs (induCtiVe/tRansDUcTiVe).

-   TO tHE beST oF ouR KnoWledge, we ArE tHe firSt to DEMONstrAte Zero-Shot dEcoding of compLex ActiON seQuencEs. We pReseNt Our reSults oN a surGiCal dataset, JIGSAwS [@jiGsaws-2014], to joIntLy SegMeNt and CLassifY fiNe-gRained sUrgical GEstUrES WHeRe we establish an impReSSIvE baselinE.

-   A demoNStRaTIon of zerO-sHot ClasSIFicatIon oF FiNe-graineD human-OBjEcT interaCtIons thAt ReqUirEs no sUPervIsed trAining of AttriBUtes by leveragiNG off-the-shelf oBJeCT DeTEctoRs iN video surveIllaNCe.

ReLateD woRk
============

METhodoLogy {#sEc:MEtHOdology}
===========

We first estabLiSh a basIc hieRarchy of concePts. At the hiGHESt level wE havE ThE *Activity*—for exaMple, sUturing in rOBotic surGery. EAch activIty can be dECOmposed iNto A seQueNce OF AcTions $(y_1, \ldots, y_N)$. pOSsibLe ExampleS of Actions Are “pusHinG neEdLe through Tissue" in A sUtUrInG acTivitY Or “throwiNg JavElIn" iN a spoRTing evEnt. ZeRo-shOt LeARniNg approAChES FurtHeR dEcomPosE eAch acTion $Y$ IntO a set of $k$ elementaRy aTTribUtEs (Usually Taken to be binaRy-Valued) $y = \{a_1, \ldOtS, a_K\}$.

given a VIDeo recorDing of an activity (represeNTed as a sEquEnce oF fraMes $X = (x_1, \ldotS, x_T)$), Our goaL is TO map eaCh framE $x_t$ to ItS coRREsponDINg ActIoN $y_t$ by detecTINg tHe preSeNce oR absencE of each attribute $\haT{A}(x_t)$ In the video, theN chOosiNG ThE acTIoN WhoSe SIgnATUre $a(y_t)$ best fits tHose attribUtES. IN other wordS, We cHoOse the aCtion wiTh higHEst scorE: $$\label{eq:dEcode_cost}
    \HaT{y}(x) = \aRG
* model of an activity. We  provide b oth q uan tit at iveandqualitative re s ults  of our zero-shot fram ework  u s ingt he se “o n the f l y” m ode ls  o n t he ch allen gin g DIVAdataset[^2 ],wh ich contains fi ne-grained  hu man-object i nte ractio ns  un d er area l wor ld vid e o surv eillancese t ting.In summ a r y,  the  main contributio n so f the paper ar e:

- A  z e r o-s hot  classific at ion o f  comple x  a c t i ons equences with  dynamic ac t ion  signa tu res whichestab li s hes  a new stat e-of -the-arton Oly m pic Spo r ts [@ol ympic- spo rts -201 0 ]da tas et .  We ou tpe r for m all ot he rmetho ds f o r t he Z SLeval uatio n regardlessoftrai n ing  assu mptio ns ( in ducti ve/tra nsduc ti ve).

-   To th e be st of our  kn ow led ge , wea re the  fi rst  to dem onstrat e  ze ro - s h ot  decoding of compl ex a ct ion sequ ences. We  p r esent ou rres ults o n a s urgi c al  dataset , JIGS A WS  [ @jigsaw s- 2014],  t o j oin tly s e gmen t andclassify  fine - grained surgic a l gestures wh e re w ee stab lis h an impres sive base line . 
- A dem onstr at i on of zero-shot classi fi cation  of f ine-grained h uman-objec t i nteracti onst ha t  requires no s uperv ised train i ng of at tribu tes by l everaging o ff-the-s hel f o bje ctd e te ctors in vide o surv ei llance. 

R elatedWor k
= === === == ===

Meth odology{# se c: me tho dolog y }
====== == === 
Wefirst establ ish a  bas ic  h i era rchy of co n c epts .At  the  hi gh est l evel wehave th e *activi ty* — forex am ple, su turing in rob ot ic surgery .Eac h acti v i ty can b e decomposed into a seq u ence of  ac tions  $(y _1, \ldot s,y_N)$.  Po s sibleexampl es of  a cti o n s are “ Pu shi ng  needle th r o ugh  tiss ue " in  a sutu ring activity or “ t hro wing javelin"  in  a s p o rt ing ev e nt.  Z e ro- s h ot learning app roaches fu rt h er  decompose eac haction$y$ int o a s e t of $K $ element ary attri bu tes( u sua lly takento be bi nary-valu e d) $y =\{a_1 , \ ldots,  a _K\ }$.
Givena  vi deo r ecordi ng  of an  acti vi ty (repr esented as a sequence o f fram es $X  =(x_1, \ld ots ,  x_ T)$), our  goa l is to ma p e ach  fram e $ x _t$ t o it s  c orr e spond inga ction $y_ t $byd e te cting the p r e s enc e orabs e nce of  eac h attribute $\hat { a}(x_t)$ in th e vi d e o,the n  cho os ing the action  wh os e signatur e$a(y_t)$ be st fitsth o se at tribut es. In  otherw o rd s , we c hoos e t he action  wi th highest  s co r e: $$\ labe l{ eq:dec ode_co s t}
 \hat{y}(x) = \a rg
* model_of an_activity. We provide both_quantitative and_qualitative_results of_our_zero-shot framework using_these “on the_fly” models on the_challenging DIVA dataset[^2],_which_contains fine-grained human-object interactions under a real world video surveillance setting.

In summary, the main_contributions_of the_paper_are:

-_  A zero-shot classification_of complex action sequences with_dynamic action_signatures which establishes a new state-of-the-art on Olympic_Sports_[@olympic-sports-2010] dataset. We_outperform all other methods for the ZSL evaluation regardless_of training assumptions (inductive/transductive).

-  _To the best_of_our_knowledge, we are the_first to demonstrate zero-shot decoding of_complex action sequences. We present our_results on a surgical dataset, JIGSAWS [@jigsaws-2014],_to jointly segment and classify fine-grained_surgical gestures where we establish_an impressive_baseline.

-   A demonstration_of zero-shot classification_of fine-grained_human-object interactions that_requires no supervised training of attributes_by leveraging off-the-shelf_object detectors in video surveillance.

Related Work
============

Methodology_{#sec:methodology}
===========

We_first establish a_basic_hierarchy_of concepts._At the highest_level_we have_the_*activity*—for example, suturing in robotic surgery._Each_activity can be decomposed into a sequence_of actions $(y_1, \ldots,_y_N)$._Possible examples of actions_are “Pushing needle through tissue"_in a suturing activity or “throwing_javelin" in_a sporting_event. Zero-shot learning approaches further decompose each action $y$ into a_set of $K$ elementary attributes (usually_taken to be binary-valued)_$y =_\{a_1,_\ldots, a_K\}$.

Given a_video_recording of_an activity (represented as a sequence of_frames $X_= (x_1, \ldots, x_T)$), our goal_is to map each_frame_$x_t$ to its corresponding action $y_t$_by detecting the presence or absence_of each attribute $\hat{a}(x_t)$ in_the_video,_then choosing the action whose_signature $a(y_t)$ best fits those attributes._In other words,_we choose the action with highest score:_$$\label{eq:decode_cost}
_   \hat{y}(x) = \arg