Towards Domain Adaptive Neural Contextual Bandits
Domain Adaptation;Deep Learning;Adversarial Learning
Yes
No ethics review needed.
See weakness.
1. The design of the new algorithm originates from an observation that leveraging data across domains leads to sub-linear regrets, which makes the whole method simple, yet elegant.

2. Theoretical proof has been provided to support the performance of the method.

3. The algorithm is extensively tested on three datasets.
The manuscript introduces a domain adaption algorithm for contextual bandit. It further proves that the proposed method achieves a sub-linear regret bound. Empirical experiments conducted on real-world data shows the the proposed method outperforms existing state-of-the-art contextual bandit methods for cross domains.
It would be helpful to elaborate on why the data divergence term is sub-linear in the proof.
Yes
No ethics review needed.
See weakness.
(1) The proposed problem setting of contextual bandits in a domain adaptation scenario is interesting and challenging.

(2) The method utilizes unlabeled data from both source and target domains for effective representation learning and alignment across different domains.

(3) The algorithm is capable of attaining a sub-linear regret bound in the target domain by solving an online network lasso problem with time-dependent regularization.
The paper introduces an algorithm for domain adaptation in contextual bandits. This method aims to adapt a neural contextual bandit algorithm from a low-cost source domain to a high-cost target domain, to address the challenge of distribution shifts between the two domains. The theoretical analysis conducted demonstrates that DABand achieves a sub-linear regret bound in the target domain, which is interesting. Empirical results are presented using real-world datasets, which show that DABand outperforms existing contextual bandit methods. The proposed work has broader implications, including potential insights into reinforcement learning with human feedback algorithms.
The method leverages unlabeled data from both source and target domains to learn robust representations and aligns them effectively across different domains, enabling efficient domain adaptation. While the proposed algorithm builds upon the NeuralLinUCB framework, it introduces an adaptation in the loss function specifically tailored for updating the neural network. This loss function integrates insights from classic domain adaptation techniques, and thereby has some similarity with existing methods.
Yes
No ethics review needed.
- Besides the presentation in Theorem 3.1, are there any regret scale for linear contextual bandits to help me better understand the performance of the algorithm?
- The authors provide a theoretical justification on the decomposition for the domain-transfer issue for contextual bandits. 
- The authors provide empirical evaluation demonstrating the performance of the algorithm.
This paper studies the domain transfer in (neural) contextual bandits. The authors provide a decomposition of the domain transfer in contextual bandits, and proposed an algorithm leveraging the adversarial training in domain-transfer. Theoretical results are presented understanding this decomposition and empirical evaluation are performed to validate the performance of the algorithm
- (Minor) Definition 3.6 is hard to understand for me. In particular, are $\mathbf x$ and $x$ referring to the same thing? Are the authors implicitly assumes $h \in \mathcal H$ and $x \in \mathbb R^n$ in this case? It would be helpful if the authors could rephrase the mathematical formulation on this. 
- In Theorem 3.1 needs further justification. For example, I wonder why the sublinear in $R_S$ will lead to the sublinear result in $R_T$. I checked the proof for Theorem 3.1 in appendix but found nowhere explicitly discuss this part. In addition, why the data-divergence term is sublinear? 
- The adversarial training paradigm adapted in controlling the data divergence term in the proposed algorithm requires better justification: why the hypothesis class here is the same with the hypothesis class used in contextual bandits?
- It lacks some discussions on the performance analysis in linear contextual bandits or discussions in related literautre. The authors might want to provide more discussion on the theoretical insight of this. 

- (Minor) Section 4.1 title: eq (16) -> eq (5)
We introduce the first general domain adaptation method for contextual bandits.
@inproceedings{
anonymous2024towards,
title={Towards Domain Adaptive Neural Contextual Bandits},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=LNkMWCEssX},
note={under review}
}
Contextual bandit algorithms are essential for solving real-world decision making problems. In practice, collecting a contextual bandit's feedback from different domains may involve different costs. For example, measuring drug reaction from mice (as a source domain) and humans (as a target domain). Unfortunately, adapting a contextual bandit algorithm from a source domain to a target domain with distribution shift still remains a major challenge and largely unexplored. In this paper, we introduce the first general domain adaptation method for contextual bandits. Our approach learns a bandit model for the target domain by collecting feedback from the source domain. Our theoretical analysis shows that our algorithm maintains a sub-linear regret bound even adapting across domains. Empirical results show that our approach outperforms the state-of-the-art contextual bandit algorithms on real-world datasets.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Domain Adaptation Deep Learning Adversarial Learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/1644bb974c5327db835fd987795858b54a85a70f.pdf
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Towards Domain Adaptive Neural Contextual Bandits
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission