}^{\textrm{QA}}$ is referred as to the loss for active QA. Given the current AOG, we use ${Loss}^{\textrm{QA}}$ to actively determine a sequence of questions about objects that cannot be explained by the current AOG, and require people to annotate bounding boxes of new object parts for supervision.

${Loss}^{\textrm{AOG}}$ is designed to learn an AOG for the CNN. ${Loss}^{\textrm{AOG}}$ penalizes 1) the incompatibility between the AOG and CNN feature maps of unannotated objects and 2) part-location errors *w.r.t.* the annotated ground-truth part locations.

It is essential to determine the optimization sequence for the three losses in the above equation. We propose to first learn the CNN by minimizing ${Loss}^{\textrm{CNN}}$ and then build an AOG based on the learned CNN. We use the active QA to obtain new part annotations and use new part annotations to grow the AOG by optimizing ${Loss}^{\textrm{QA}}$ and ${Loss}^{\textrm{AOG}}$ alternatively.

We introduce details of the three losses in the following subsections.

Learning convolutional neural networks
--------------------------------------

To simplify the story, in this research, we just consider a CNN for single-category classification, *i.e.* identifying object images of a specific category from random images. We use the log logistic loss to learn the CNN. $${Loss}^{\textrm{CNN}}=\mathbb{E}_{I\in{\bf I}}\big[{Loss}(\hat{y}_{I},y^{*}_{I})\big]$$ where $\hat{y}_{I}$ and $y^{*}_{I}$ denote the predicted and ground-truth labels of an image $I$. If the image $I$ belongs to the target category, then $y^{*}_{I}=+1$; otherwise $y^{*}_{I}=-1$.

Learning And-Or graphs
----------------------

We are given a pre-trained CNN and its training images without part annotations. We use an active QA process to obtain a small number of annotations of object-part bounding boxes, which will be introduced in Section \[sec:QA\]. Based on these inputs, in this subsection, we focus on the approach for learning an AOG to represent the object part.

### And-Or graph representations

Before the introduction of learning AOGs, we first briefly overview the structure of the AOG and the part parsing
} ^{\textrm{QA}}$ is referred as to the loss for active QA. Given the current AOG, we use $ { Loss}^{\textrm{QA}}$ to actively settle a succession of questions about objects that cannot be excuse by the current AOG, and ask people to annotate bounce box of new object parts for supervision. 

 $ { Loss}^{\textrm{AOG}}$ is designed to determine an AOG for the CNN. $ { Loss}^{\textrm{AOG}}$ penalizes 1) the incompatibility between the AOG and CNN feature maps of unannotated object and 2) part - placement errors * w.r.t. * the annotated ground - accuracy part locations. 

 It is essential to decide the optimization sequence for the three losses in the above equality. We propose to foremost learn the CNN by minimizing $ { Loss}^{\textrm{CNN}}$ and then build an AOG based on the learned CNN. We use the active QA to receive new part annotations and use new part annotations to grow the AOG by optimizing $ { Loss}^{\textrm{QA}}$ and $ { Loss}^{\textrm{AOG}}$ alternatively. 

 We introduce details of the three losses in the following subsections. 

 Learning convolutional neural networks 
 -------------------------------------- 

 To simplify the narrative, in this research, we just think a CNN for individual - category classification, * i.e. * identifying object images of a specific category from random images. We practice the log logistic loss to learn the CNN. $ $ { Loss}^{\textrm{CNN}}=\mathbb{E}_{I\in{\bf I}}\big[{Loss}(\hat{y}_{I},y^{*}_{I})\big]$$ where $ \hat{y}_{I}$ and $ y^{*}_{I}$ denote the predicted and ground - truth labels of an image $ I$. If the image $ I$ belongs to the target class, then $ y^{*}_{I}=+1 $; otherwise $ y^{*}_{I}=-1$. 

 Learning And - Or graph 
 ---------------------- 

 We are given a pre - trained CNN and its training images without part annotations. We practice an active QA process to obtain a small number of annotations of object - part bounding boxes, which will be introduced in Section   \[sec: QA\ ]. base on these inputs, in this subsection, we focus on the access for learning an AOG to typify the object contribution. 

 # # # And - Or graph representations 

 Before the initiation of learning AOGs, we first concisely overview the structure of the AOG and the part parsing
}^{\texhrm{QA}}$ is referred as to uhe loss for actirw QA. Gmven ths curreng AOG, we use ${Loss}^{\textrm{QA}}$ to axtiveoy determine a sequencd of quesnions aboyt oujects that cannot be exijainsf by vhe current AOG, and requise people to atnutcte bounding boxes of new object pares for xuoervision.

${Loss}^{\tgxtrm{SJG}}$ ia designed to learn an AOG for the CNN. ${Lovs}^{\textrm{AOG}}$ prnalizes 1) the incompatibillty hetween the AOG anf CNN featuts mwps of unannogated objebcs and 2) parj-location errors *w.r.t.* the annotatdd grpund-truth kcet pmcations.

It ms essvntial to detcgmine tve optikization sequekce fmr rhe three losses in tie above equation. We propose do first learn the XNB by kinikizivt ${Luss}^{\uexvrm{DNN}}$ anf tien build ah AOG based on the learned CNN. Wq use the actibe QA eo obtain new part annotations and use nef pzrt annotations to grow the AOG by optimizinh ${Loss}^{\texerm{QA}}$ and ${Loss}^{\textrm{AOG}}$ alternatively.

We introduce detamlr oy the gyrfe losses in the following subsections.

Learnind cpnnolutional neural networks
--------------------------------------

To slmljify the stori, in thna desearch, we just clnsider a CNB for sindle-cstegory classification, *i.e.* ieentifying oyjext images of a speeific categoxy frok rancom images. We use the lug lkgistic losd to learh the CNN. $${Loss}^{\texgrm{BNN}}=\mdthbb{E}_{I\in{\bf I}}\big[{Loss}(\hat{y}_{I},y^{*}_{Y})\big]$$ wherx $\hat{v}_{I}$ and $y^{*}_{K}$ demote tre predictfd and ground-truth labeld of cn imdge $I$. If tje image $I$ belongs to the targev category, thgn $f^{*}_{I}=+1$; mtherwisz $y^{*}_{I}=-1$.

Lesrning And-Or draphs
----------------------

We are gnven a pxe-traived CNN ans its tcaining imagqs without pasj annotations. We use wn axtivw QA pruzess to obtain a small ubmber of qnnotations of objcct-patt bounding boxes, whuch will be inttodjceq pn Xesdion \[sec:QA\]. Baved un gnese knputs, in tmis subxection, we focus on dhe zpproach for learnonn an AOG jo represqnt the objecy part.

### And-Or graph reprxsentavions

Brfote the introduction of learning AOGs, we virft briefly odervlew jhe structuxe of the AOG and the part parsing
}^{\textrm{QA}}$ is referred as to the loss QA. the current we use ${Loss}^{\textrm{QA}}$ of about objects that be explained by current AOG, and require people to bounding boxes of new object parts for supervision. ${Loss}^{\textrm{AOG}}$ is designed to learn AOG for the CNN. ${Loss}^{\textrm{AOG}}$ penalizes 1) the incompatibility between the AOG and feature of objects 2) part-location errors *w.r.t.* the annotated ground-truth part locations. It is essential to determine the optimization for the three losses in the above equation. propose to first learn CNN by minimizing ${Loss}^{\textrm{CNN}}$ and build AOG based the CNN. use the active to obtain new part annotations and use new part annotations to grow the AOG by optimizing ${Loss}^{\textrm{QA}}$ ${Loss}^{\textrm{AOG}}$ alternatively. details of three in following subsections. Learning networks -------------------------------------- To simplify the story, we just consider a CNN for single-category classification, identifying object of a specific category from random We use the log logistic loss to learn CNN. $${Loss}^{\textrm{CNN}}=\mathbb{E}_{I\in{\bf I}}\big[{Loss}(\hat{y}_{I},y^{*}_{I})\big]$$ where $\hat{y}_{I}$ and $y^{*}_{I}$ denote the predicted and ground-truth labels of an If the image $I$ to the target then otherwise Learning graphs ---------------------- are given a pre-trained CNN and its training images without part We use an active QA process to obtain a small annotations object-part bounding boxes, will be introduced in \[sec:QA\]. on these inputs, in we on learning AOG represent the object part. And-Or graph representations Before the of learning AOGs, we of the AOG and the part parsing
}^{\textrm{QA}}$ is referred as to the Loss for actIve QA. givEn tHe CurrEnt AoG, we use ${Loss}^{\texTRm{QA}}$ To actively determine a seQuencE oF QuesTIoNs aboUt objecTS tHAT caNnOt Be eXpLAiNed by The Current aOG, and requIre PeOple to annotaTE bOunding boxEs oF new object paRts For supErVisIOn.

${LosS}^{\teXtrm{AoG}}$ is deSIgned tO learn an AoG FOr the Cnn. ${Loss}^{\teXTRm{aOG}}$ pEnalizes 1) the incompATiBIlity between thE AOG anD Cnn fEATurE maPs of unannoTaTed obJEcts and 2) PArT-LOCatIOn errors *w.r.t.* thE annotated gROunD-truth PaRt lOCationS.

It is EsSEntIal to determIne tHe optimizAtion sEQuence fOR the thrEe lossEs iN thE aboVE eQuAtiOn. wE prOPoSe tO FirSt learn tHe cNn by miNimiZING ${loss}^{\TexTrm{CnN}}$ and Then build an AOg baSed oN The LearnEd CNN. we usE tHe actIve QA tO obtaIn New part annotatiOns aNd use new pArt AnNotAtIons tO Grow thE AOg by OptimizIng ${Loss}^{\TExtRm{qa}}$ ANd ${loss}^{\textrm{AOG}}$ alterNaTIVeLy.

We intrOduce dETaIlS Of the thrEe LosSes iN THe folLowiNG sUbsectioNs.

LearNInG cOnvolutIoNal neuRaL neTwoRks
--------------------------------------

To SImplIfy the Story, in tHis reSEarch, we just conSIder a CNN for siNGlE-CAtEGory ClaSsification, *I.e.* idENtifYing OBjEct IMages Of a spEcIFiC Category from random iMaGes. We uSe the Log logistic loSs to learn tHE cnN. $${Loss}^{\teXtrm{cnN}}=\MAthbb{E}_{I\in{\bf I}}\biG[{Loss}(\Hat{y}_{I},y^{*}_{I})\big]$$ WHere $\hat{y}_{i}$ and $y^{*}_{i}$ denote tHe predictED And grounD-trUth LabEls OF An Image $I$. If the imAGE $I$ beLoNgs to thE taRget catEgoRy, tHen $Y^{*}_{I}=+1$; oThErwise $y^{*}_{I}=-1$.

LEarning ANd-or GrApHs
----------------------

WE are gIVen a pre-tRaIneD CnN aNd its TRaininG imagEs wiThOuT ParT annotaTIoNS. we usE aN aCtivE QA PrOcess To obTAin A small nUmber of anNotATionS oF oBject-paRt bounding boxEs, Which will bE iNtrOduced IN section \[sEc:QA\]. Based on these inputs, iN This subSecTion, wE focUs on the apProAch for LeaRNing an aOG to rEpresEnT thE OBject PARt.

### and-or Graph repreSENtaTions

beFore The intrOduction of learning aoGs, We first brieflY ovErviEW ThE stRUcTUre Of THe Aog And the part parsiNg
}^{\textrm{QA}}$ is referr ed as to t he lo ssfor  a ctiv e QA . Given the cu r rent  AOG, we use ${Loss}^{ \text rm { QA}} $  t o act ively d e te r m ine  a  s equ en c eof qu est ions ab out object s t ha t cannot bee xp lained bythe  current AOG , a nd req ui rep eople  to  anno tate b o unding  boxes of  n e w obje c t parts f or  sup ervision.

${Loss } ^{ \ textrm{AOG}}$is des ig n ed t o l ear n an AOG f or  theC NN. ${L o ss } ^ { \te x trm{AOG}}$ pe nalizes 1)t heincomp at ibi l ity be tween  t h e A OG and CNNfeat ure mapsof una n notated objects  and 2 ) p art -loc a ti on  er ro r s * w .r .t. *  th e annota te dgroun d-tr u t h part  lo cati ons.
It is essent ial  tod ete rmine  theopti mi zatio n sequ encefo r the three los sesin the ab ove  e qua ti on. W e  propo setofirst l earn th e  CN Nb y mi nimizing ${Loss}^{ \t e x tr m{CNN}}$  and t h en  b u ild an A OG  ba sedo n  thelear n ed  CNN. We  use t h eac tive QA  t o obta in  ne w p art a n nota tionsand usenew p a rt annotations to grow the A O Gb y  o p timi zin g ${Loss}^{ \tex t rm{Q A}}$ an d $ { Loss} ^{\te xt r m{ A OG}}$ alternatively .
We in trodu ce details of  the three l o sses inthef ol l owing subsecti ons.
Learningc onvoluti onalneural n etworks
- - - -------- --- --- --- --- - - -- -----------
T o  sim pl ify the  st ory, in  th isres ear ch , we just  conside raCN Nfor  sing l e-catego ry  cl as sif icati o n, *i. e.* i dent if yi n g o bject i m ag e s  ofasp ecif icca tegor y fr o m r andom i mages. We  us e  the  l og  logist ic loss to le ar n the CNN.  $ ${L oss}^{ \ t extrm{CN N}}=\mathbb{E}_{I\in{\b f  I}}\bi g[{ Loss} (\ha t{y}_{I}, y^{ *}_{I} )\b i g]$$ w here $ \hat{ y} _{I } $  and$ y ^{ *}_ {I }$ denotet h e p redic te d an d groun d-truth labels ofa n i mage $I$. Ifthe  ima g e  $ I$b el o ngs  t o  th e target category , then $y^ {* } _{ I}=+1$; ot h erw is e $y^{* }_{I}=- 1$.
L earning  And-Or g raphs
--- -- ---- - - --- --------
We are g iven a pr e -trai n ed  CNNand  its t ra ini ng im ages w i tho ut pa rt ann ot ations . Weus e an act ive QA process to obtai n a sm all n umb er of ann ota t ion s of obje ct-p art boundi ngbox es, w hic h  will  bei nt rod u ced i n Se c tion \[se c :Q A\] . Ba sed on thes e i npu ts, i n t h is sub sect ion, we focus ont he approach fo r le a r nin g a n  AOG  t o represent th e o bj e c t part.
# ## And-Or g raph rep re s entat ions
Before  the in t r od u ctionof l ear ning AOGs , w ef irst br ie fl y  overv iewth e stru ctureo f th e AOG and the part  pars i n g
}^{\textrm{QA}}$ is_referred as_to the loss for_active QA._Given_the current_AOG,_we use ${Loss}^{\textrm{QA}}$_to actively determine_a sequence of questions_about objects that_cannot_be explained by the current AOG, and require people to annotate bounding boxes of_new_object parts_for_supervision.

${Loss}^{\textrm{AOG}}$_is designed to learn an_AOG for the CNN. ${Loss}^{\textrm{AOG}}$_penalizes 1)_the incompatibility between the AOG and CNN feature_maps_of unannotated objects_and 2) part-location errors *w.r.t.* the annotated ground-truth part_locations.

It is essential to determine the_optimization sequence for_the_three_losses in the above_equation. We propose to first learn_the CNN by minimizing ${Loss}^{\textrm{CNN}}$ and_then build an AOG based on the_learned CNN. We use the active_QA to obtain new part_annotations and_use new part annotations to_grow the AOG_by optimizing_${Loss}^{\textrm{QA}}$ and ${Loss}^{\textrm{AOG}}$_alternatively.

We introduce details of the three_losses in the_following subsections.

Learning convolutional neural networks
--------------------------------------

To simplify_the_story, in this_research,_we_just consider_a CNN for_single-category_classification, *i.e.*_identifying_object images of a specific category_from_random images. We use the log logistic_loss to learn the_CNN._$${Loss}^{\textrm{CNN}}=\mathbb{E}_{I\in{\bf I}}\big[{Loss}(\hat{y}_{I},y^{*}_{I})\big]$$ where $\hat{y}_{I}$_and $y^{*}_{I}$ denote the predicted_and ground-truth labels of an image_$I$. If_the image_$I$ belongs to the target category, then $y^{*}_{I}=+1$; otherwise $y^{*}_{I}=-1$.

Learning And-Or_graphs
----------------------

We are given a pre-trained CNN_and its training images_without part_annotations._We use an_active_QA process_to obtain a small number of annotations_of object-part_bounding boxes, which will be introduced_in Section \[sec:QA\]. Based on_these_inputs, in this subsection, we focus_on the approach for learning an_AOG to represent the object_part.

###_And-Or_graph representations

Before the introduction of_learning AOGs, we first briefly overview_the structure of_the AOG and the part parsing