Prevent Modification of dbo objects while allowing other schemas to be modified
We have full control of this server and have always allowed sysadmin
  rights to any developer.


The sysadmin privilege effectively short-circuits any other permissions. If a user is a member of the sysadmin fixed server role, then any DENY permissions will be ignored.

You can work-around this by revoking sysadmin and instead using the CONTROL SERVER permission. CONTROL SERVER is similar to the sysadmin fixed server role, except that CONTROL SERVER will obey DENY permissions.

GRANT CONTROL SERVER TO dev
EXEC sp_dropsrvrolemember 'dev', 'sysadmin'
DENY ALTER ON SCHEMA::dbo TO dev


If you want to completely undo the developers' need to have high-level access on the server (which is a pretty good idea), then you could take two approaches:


Grant wide access to the database, then explicitly deny permissions on dbo.

USE MyDB
GRANT SELECT,INSERT,UPDATE,DELETE,EXECUTE,ALTER ON DATABASE::MyDB TO dev
GRANT VIEW DEFINITION TO dev
DENY ALTER ON SCHEMA::dbo TO dev
EXEC sp_dropsrvrolemember 'dev', 'sysadmin'

Explicitly grant just the permissions the users require; do not grant ALTER permission in dbo.

USE MyDB
GRANT SELECT,INSERT,UPDATE,DELETE,EXECUTE,ALTER ON SCHEMA::custom TO dev
GRANT SELECT,INSERT,UPDATE,DELETE,EXECUTE ON SCHEMA::dbo TO dev
GRANT VIEW DEFINITION TO dev
EXEC sp_dropsrvrolemember 'dev', 'sysadmin'



Ultimately, you want to follow (or be close to) the principal of least privilege. But you also have to balance complexity and manageability.

I'd suggest you create a few test scenarios to test your permissions when running as dev, so that you can make sure the user has the right permissions. You can do this using EXECUTE AS LOGIN:

EXECUTE AS LOGIN='SomeDeveloper';
CREATE TABLE custom.AM2 (foo BIT);
GO
CREATE PROCEDURE custom.GetAM2 AS SELECT * FROM custom.AM2;
GO
DROP TABLE custom.AM2;
DROP PROCEDURE custom.GetAM2;
GO
REVERT;