How are SetCPU frequencies formulated? Are the frequencies suggested by SetCPU for each device based on a certain algorithm or was it worked out device specifically or are they just arbitrarily formed upon extensive testing?

I have an old HTC Wildfire which has a maximum frequency at 768MHz

The steps of frequencies show by SetCPU itself is 176,264,352,518,576,614,633,652,672,691,710,729,748,768 all in MHz.

I eventually created a setcpu.txt file which I put in the root of the SD-Card. The thing is when I had a frequency of 800MHz SetCPU still read it but the max frequency it could obviously touch was 768MHz.

I repeat, my question is how are the frequencies listed by SetCPU formulated?
The answer lies in the kernel source for that HTC Wildfire - look in arch/arm/mach-msm/acpuclock.c to see the table of the accepted frequencies. Since the WildFire is based on MSM-7x25 chipset here's the excerpt of the frequencies table hardcoded, assuming its a GSM handset:

/* 7x01/7x25 normal with GSM capable modem */
static struct clkctl_acpu_speed pll0_245_pll1_768_pll2_1056[] = {
    { 0, 19200, ACPU_PLL_TCXO, 0, 0, 19200, 0, 0, 30720 },
    { 1, 122880, ACPU_PLL_0, 4, 1,  61440, 1, 3,  61440 },
    { 0, 128000, ACPU_PLL_1, 1, 5,  64000, 1, 3,  61440 },
    { 0, 176000, ACPU_PLL_2, 2, 5,  88000, 1, 3,  61440 },
    { 1, 245760, ACPU_PLL_0, 4, 0,  81920, 2, 4,  61440 },
    { 1, 256000, ACPU_PLL_1, 1, 2, 128000, 1, 5, 128000 },
    { 0, 352000, ACPU_PLL_2, 2, 2,  88000, 3, 5, 128000 },
    { 1, 384000, ACPU_PLL_1, 1, 1, 128000, 2, 6, 128000 },
    { 1, 528000, ACPU_PLL_2, 2, 1, 132000, 3, 7, 128000 },
    { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, {0, 0, 0}, {0, 0, 0} }
};



First column, indicates that the frequency can be scaled, if it's a 1 in there, that chipset frequency can be scaled up or down, think of ramping up the CPU when there's a processor-intensive app running and that is done by the kernel dynamically within the confines of a safe threshold without burning down the chipset - please note:  This is only for the kernel to use, further on in the answer, that shows the overclocking table, that is "end-user" responsibility - not the kernel!!!!!!
Second column, the normal frequency in KHz as dictated by the what I think is called the a11 clock (somewhere within the bowels of the chipset itself)
Third column, not exactly sure but is specifically related to chipset itself
Fourth column, the selector used for the normal frequency setting
Fifth column, the divisor used for normal frequency setting
Sixth column, the overclocking frequency in KHz
Seventh column, the overclocking frequency selector
Eighth column, the divisor used for overclocking frequency setting
Ninth column, not sure either what that signifies


There's actually in fact, 12 fields used, the remaining fields are filled in at kernel boot-up, perhaps adding guards and ensuring that the conserved values are not explicitly over-ridden, a "gate-keeper" if you like, read on...

Further on in the source, there is usually a conservative over-clocking frequencies (the common MSM-7x 01 / 25 / 27 ) chipsets supports it), again same layout as the "conservative values" as highlighted above, notice every single one of the frequencies can be scaled up/ramped down for each overclock frequency That is how SetCPU can adjust those values as it has a 1 in there in the first column

#ifndef CONFIG_MSM_CPU_FREQ_OVERCLOCK_AHB
/* Conservative AHB overclocking */
    { 0, 403200, ACPU_PLL_2, 2, 2, 134400, 2, 5, 122880 },
    { 1, 480000, ACPU_PLL_0, 4, 1, 160000, 2, 6, 122880 },
    { 1, 595200, ACPU_PLL_2, 2, 1, 198400, 2, 7, 122880 },
    { 1, 604800, ACPU_PLL_2, 2, 1, 201600, 2, 7, 200000 },
    { 1, 614400, ACPU_PLL_0, 4, 0, 204800, 2, 7, 200000 },
    { 1, 633600, ACPU_PLL_0, 4, 0, 211200, 2, 7, 200000 },
    { 1, 652800, ACPU_PLL_0, 4, 0, 217600, 2, 7, 200000 },
    { 1, 672000, ACPU_PLL_0, 4, 0, 224000, 2, 7, 200000 },
    { 1, 691200, ACPU_PLL_0, 4, 0, 230400, 2, 7, 200000 },
    { 1, 710400, ACPU_PLL_0, 4, 0, 236800, 2, 7, 200000 },
    { 1, 729600, ACPU_PLL_0, 4, 0, 243200, 2, 7, 200000 },
    { 1, 748800, ACPU_PLL_0, 4, 0, 249600, 2, 7, 200000 },
    { 1, 768000, ACPU_PLL_0, 4, 0, 256000, 2, 7, 200000 },
    { 1, 787200, ACPU_PLL_0, 4, 0, 262400, 2, 7, 200000 },
    { 1, 806400, ACPU_PLL_0, 4, 0, 268800, 2, 7, 200000 },
//  { 1, 825600, ACPU_PLL_0, 4, 0, 275200, 2, 7, 200000 },
//  { 1, 844800, ACPU_PLL_0, 4, 0, 281600, 2, 7, 200000 },
#else
/* Agressive AHB overclocking */
    { 0, 403200, ACPU_PLL_2, 2, 2, 201600, 1, 5, 200000 },
    { 1, 480000, ACPU_PLL_0, 4, 1, 240000, 1, 6, 200000 },
    { 1, 595200, ACPU_PLL_2, 2, 1, 297600, 1, 7, 200000 },
    { 1, 604800, ACPU_PLL_2, 2, 1, 302400, 1, 7, 200000 },
    { 1, 614400, ACPU_PLL_0, 4, 0, 307200, 1, 7, 200000 },
    { 1, 633600, ACPU_PLL_0, 4, 0, 316800, 1, 7, 200000 },
    { 1, 652800, ACPU_PLL_0, 4, 0, 326400, 1, 7, 200000 },
    { 1, 672000, ACPU_PLL_0, 4, 0, 336000, 1, 7, 200000 },
    { 1, 691200, ACPU_PLL_0, 4, 0, 345600, 1, 7, 200000 },
    { 1, 710400, ACPU_PLL_0, 4, 0, 355200, 1, 7, 200000 },
    { 1, 729600, ACPU_PLL_0, 4, 0, 364800, 1, 7, 200000 },
    { 1, 748800, ACPU_PLL_0, 4, 0, 374400, 1, 7, 200000 },
    { 1, 768000, ACPU_PLL_0, 4, 0, 384000, 1, 7, 200000 },
    { 1, 787200, ACPU_PLL_0, 4, 0, 393600, 1, 7, 200000 },
    { 1, 806400, ACPU_PLL_0, 4, 0, 403200, 1, 7, 200000 },
//  { 1, 825600, ACPU_PLL_0, 4, 0, 412800, 1, 7, 200000 },
//  { 1, 844800, ACPU_PLL_0, 4, 0, 422400, 1, 7, 200000 },
#endif /* CONFIG_MSM_CPU_FREQ_OVERCLOCK_AHB */
    { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, {0, 0, 0}, {0, 0, 0} }
};


Depending on the table lookups, and depending on the chipset, only the max is specified, and is re-inforced by the kernel (in the remainder of the fields of the record that the kernel reads from and process it - as mentioned above!)


  I eventually created a setcpu.txt file which I put in the root of the
  SD-Card. The thing is when I had a frequency of 800MHz SetCPU still
  read it but the max frequency it could obviously touch was 768MHz.


That is why you can never override the maximum as dictated, as in the OP's case, to prevent thermal damage to the chipset and clock frequency!

Random reboots, random freezes, battery overheating quicker than normal (normal as in within the regulated CPU clock frequency threshold) are all symptoms of a overclocked chipset. 

This is a big give-away clue when someone asks about random reboots but does not say that it is probably overclocked!