How to reduce fragmentation of nonclustered index on date partitioned log table
I'm wondering what the best way would be to keep a nonclustered index for looking by accountId without all the fragmentation

You've got the partition key in the nonclustered index, so it's on the partition scheme (by default), and only the head partition is going to get inserts and additional fragmentation.
So you can rebuild the older partitions rarely and they won't get fragmented, and rebuild the index for just the head partition on a more frequent basis.
But for an index with insertions spread across the sort order you'll always generate fragmentation.  And fragmentation doesn't always matter.  It really depends on the storage design and the workload.
Also you're keeping two complete copies of this table, one sorted by ChangedAt and one sorted by (AccountId,InventoryId,ChangedAt).  You can probably just make the (AccountId,InventoryId,ChangedAt) index the clustered index, and only store the table one time.