How to create a new dataframe using the another dataframe
Not sure what you mean by "unique", I guess if there're n critics, m items, what you need is a dataframe with shape n*m rows and 3 columns, right? If so, try the demo:

#1. the original dataframe
df = pd.DataFrame([['a',1,5],['b',2,3],['b',3,2],['c',8,1],['a',1,5]],columns=['critic','item','rating'])

#2. create the first two columns(critic, item) by their permutation
from itertools import product
first_two_column = pd.DataFrame(list(product(set(df.critic),set(df.item))),columns=['critic','item'])

#3. merge the first two column with ratings, using left join to add nan, drop duplicate first
first_two_column.merge(right=df.drop_duplicates(),on=['critic','item'],how='left')