How do I figure out the number of unique combinations I am developing an algorithm that uses a lookup table to process the data, I want to know how many unique tables can be created. If I have a table of 256 values I know that 256! would tell me how many tables are possible, but the math of the algorithm moves forward or backward along this table based on the calculation so the tables are essentially circular with the first and last value sitting next to each other.  Because of this every table with the values in an order that would match exactly to another table except a different start/end would not be unique and not counted. 

Without generating every possible table and testing it against others how do I figure out many tables are possible?
The number of possible unique tables for your application is $255!$.  The key term you will want to look up is circular permutation. 

The number of tables you can have without wrapping around is a permutation of the elements (also called a 'linear permutation').  The number of possible tables of a permutation of the elements is $n!$, which is why you (correctly) calculated the number of possible permutations of the table as $256!$.  

Because your use is circular, you would need to calculate the number of unique circular permutations, which is equal to $(n-1)!$.  So for the usage in your application where you have 256 elements, the number of possible tables is $(256-1)! = 255!$.