SQL Server - Adding non-nullable column to existing table - SSDT Publishing
I'll share how I have done this in the past.  It is designed to solve the 
specific limitation of pre-deployment scripts that you call out in your second 
point:


  Adding the column in a pre-deployment script will fail the publish when it 
  automatically tries to create the same column, a second time (even if the pre-
  deployment script is written to be idempotent)


Why pre-deployment scripts don't work for this

When you deploy an SSDT project, the way it stitches thing together is like 
this (a bit simplified, but in general):


Do the "schema compare" between the source (dacpac file) and the target 
(database)
Generate a deployment script based on the results of that compare
Process any pre-deployment scripts in the dacpac (doing token replacement, 
etc.) and insert the contents into the beginning of the deployment script
Do the same for post-deployment scripts, appending to the end of the 
deployment script


When a new column exists in the dacpac and not in the target database, step #2 
will generate code to add that column.  So if the pre-deployment script adds 
this column, the main part of the script will fail (because it assumes the 
column doesn't exist, based on the results of the schema compare in step #1)

Solution: pre-SSDT script

Martin Smith mentioned this option in a comment, and it's the solution that 
has worked best for me so far:


  We use premodel scripts in our deplyment pipeline. This is not something 
  that is part of SSDT but a step that runs before the dacfx publish. So in 
  this case the premodel script could add the column with the desired values 
  and make it not null and by the time the publish happens it is already in 
  the state expected by SSDT so it doesnt have anything to do. I'm still yet 
  to find much use for predeploy scripts. – Martin Smith 
  Jun 1 at 21:45


The steps to implement this solution in general are:


Create a script in the SSDT project to hold your "pre-SSDT" T-SQL code


Depending on how your deployment process works, code in these files 
should probably be idempotent

Make sure to set this script to "Build Action=None" and "Copy to Output 
Directory=Copy always"


the "copy always" option is especially important, as the deployment 
process needs to be able to find this script in your 
deployment artifacts

In your deployment process, locate and run this script (or scripts) 
before the SSDT schema compare occurs
Once that script has been executed successfully, you can engage DacServices 
/ DacFx / whatever to complete your deployment as usual


In the end, this allows you to add the column using whatever custom code you 
like, populated using complex business logic, in the pre-SSDT script.

You also add the column definition in the SSDT project (so source control 
still matches the real life state of the database).  But when the schema 
compare runs, it sees no changes related to that column (because you've 
already deployed it).

Other uses of pre-SSDT

I often find when testing deployments that SSDT performs a "table rebuild" 
operation* when it's completely unnecessary.  This is where a new table is a 
created with the updated schema, all data is copied to that table, the old 
table is dropped, and the new table is renamed to replace the old table.

This can lead to massive transaction log file growth and other problems if 
the table is large. If I notice that a schema change is causing this, I'll 
instead make the change myself in pre-SSDT (which is usually a simple 
ALTER TABLE statement) and avoid the table rebuild.

Is this a good idea?

I think so.  If you read Critiquing two different approaches to 
delivering databases: Migrations vs state by Alex Yates, this is 
essentially combining the two approaches a bit.  SSDT is state based, but we 
incorporate a migration step (before SSDT) to handle some of the more complex 
scenarios that SSDT just has no way of dealing with in a general way.

In doing some searching while writing this answer, this is actually a very 
common approach discussed in the SSDT user community once you know what to 
search for.  I've seen it called:


pre-compare
pre-model
pre-DAC
pre-SSDT 


Etc.  Here's a great article that covers a lot of the points that I mentoned 
above: 

Pre-Compare &amp; Pre-Deployment Scripts to SSDT

And one from Red Gate (in the #4 – Changing from system type to user 
defined type section) that also refers to this as pre-compare:

How to Fix Ten SSDT Deployment Snags, With or Without ReadyRoll

So what's the point of pre-deployment scripts?

Martin points out that he hasn't found "much use for predeploy scripts."  I 
tend to feel the same way.  But there are scenarios where they can be useful.

One example a coworker pointed out to me was storing some data in a temp table 
to be used in the post deployment script (say you're moving a column from one 
table to another).



*The table rebuild looks like this, which is horrifying, right?  

GO
PRINT N'Starting rebuilding table [dbo].[MyTable]...';


GO
BEGIN TRANSACTION;

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;

SET XACT_ABORT ON;

CREATE TABLE [dbo].[tmp_ms_xx_MyTable] (
    [Id] BIGINT IDENTITY (1, 1) NOT NULL,
    -- etc, other columns
);

IF EXISTS (SELECT TOP 1 1 
           FROM   [dbo].[MyTable])
    BEGIN
        SET IDENTITY_INSERT [dbo].[tmp_ms_xx_MyTable] ON;
        INSERT INTO [dbo].[tmp_ms_xx_MyTable] ([Id], ...)
        SELECT   [Id],
                 -- etc, other columns
        FROM     [dbo].[MyTable]
        ORDER BY [Id] ASC;
        SET IDENTITY_INSERT [dbo].[tmp_ms_xx_MyTable] OFF;
    END

DROP TABLE [dbo].[MyTable];

EXECUTE sp_rename N'[dbo].[tmp_ms_xx_MyTable]', N'MyTable';

COMMIT TRANSACTION;

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;