How can I print a custom message for unpublished nodes upon submission?
First, you should be using the current -dev version of Rules - that will make it much easier to do what you want.

Second, for the "Node is of type" condition, switch to the data selector for the first entry then select "node" from the data selector. This is the data object "node" that was created. For the content type, enter the machine id of the content type using the direct input mode (the default).