How to choose a correct prior
The question this response was trying to answer, roughly: "What is a way to reveal information about the outcome of $n$ i.i.d. coin flips that helps you learn the bias of the coin, but wihout revealing too much about the individual bits?"

Since the bits are i.i.d., the numbers $h$ and $t$ (number each of heads and tails) are completely sufficient to describe what the sequence tells you about the bias of the coin. (Or if you know $n$, either number alone.) This follows because, for each possible parameter $p$, all sequences resulting in a particular count $(h,t)$ are equally likely. So revealing the individual bits adds no information about the bias of the coin once $h$ and $t$ are known. On the other hand, presumably you don't mind releasing $h$ and $t$ as they do not give too much information about any one bit flip.

As a sidenote, one way to approach this general sort of problem is differential privacy. I'm not sure that it's a good fit for your setting, but it formalizes the notion of how much one learns about data from computing some function on the data. The idea is that, if I have two sequences $x$ and $x'$ that differ only on a single bit (say the $i$th bit), then my function, which is randomized, should produce approximately the same distribution over outputs:

\begin{equation}
\Pr[f(x) = a] \leq e^{\varepsilon} \Pr[f(x') = a]
\end{equation}
for all $a$.