Unable to enable publishing on restored database
After some further investigation through profiling, we found the issue. 

We traced the whole process of enabling publishing on the database by running 

sp_replicationdboption @dbname = 'restoreddb', @optname = 'publish', @value = 'true'


and zeroed in on the part that creates the sys tables:

EXEC @retcode = sys.sp_MScreate_pub_tables


This is where it gets interesting:

EventClass  TextData 
44  IF not exists (select * from sys.objects where name = 'sysarticles')
45  IF not exists (select * from sys.objects where name = 'sysarticles')
44  IF not exists (select * from sys.objects where name = 'sysarticlecolumns')
45  IF not exists (select * from sys.objects where name = 'sysarticlecolumns')
44  IF not exists (select * from sys.objects where name = 'sysschemaarticles')
45  IF not exists (select * from sys.objects where name = 'sysschemaarticles')
44  IF not exists (select * from sys.objects where name = 'sysextendedarticlesview')
45  IF not exists (select * from sys.objects where name = 'sysextendedarticlesview')
44  IF NOT EXISTS (select * from sys.objects where name = 'syspublications')
45  IF NOT EXISTS (select * from sys.objects where name = 'syspublications')
44  IF not exists (select * from sys.objects where name = 'syssubscriptions')
45  IF not exists (select * from sys.objects where name = 'syssubscriptions')
44  IF not exists (select * from sys.objects where name = 'sysarticleupdates')
45  IF not exists (select * from sys.objects where name = 'sysarticleupdates')
44  IF not exists (select * from sys.objects where name = 'MSpub_identity_range')
45  IF not exists (select * from sys.objects where name = 'MSpub_identity_range')
44  IF not exists (select * from sys.objects where name = 'systranschemas' and schema_id = 1)
45  IF not exists (select * from sys.objects where name = 'systranschemas' and schema_id = 1)
44  CREATE TABLE dbo.systranschemas
            (
                tabid int not null,
                startlsn binary(10) not null,
                endlsn binary(10) not null,             typeid int not null default 52
            )     

45  CREATE TABLE dbo.systranschemas
            (
                tabid int not null,
                startlsn binary(10) not null,
                endlsn binary(10) not null,             typeid int not null default 52
            )


With eventclass 44 = SPStmtStarting and 45 = SPStmtCompleted.

So what's happening here? It looks like the 'IF NOT EXISTS' statements were returning false for all the sys tables EXCEPT systranschemas, but we checked that these tables don't exists at all under System Tables. 

So we picked a select from there and ran it

select * from sys.objects where name = 'sysarticles'


We found that there is a SYNONYM with the same object name but under a different schema in the database. This is a remnant of a migration project we did a few months ago where we moved all the tables from another database to this one under a different schema. The synonyms were created as the first step - instead of just creating them for user tables, we created them for the system tables as well.

Regardless, I am a little appalled at how lazy the coding is in this system procedure - it does not check for the 'is_ms_shipped' flag in the sys.objects, nor does it check for the schema - strangly except for 'systranschemas' where it does check for the schema and thus was able to create the table without any issues.