Log (ldf) file grows continuously
Conceptually this is what should be happening within your database when the end of the physical file is reached:



However, it sounds like you have a log header that doesn't want to wrap back around to the start of the physical log file and continues to fire off auto-growth events instead.  This can occur for any number of reasons, but the most common is a vlf in front of the logical log has a DBCC LOGINFO status of 2 for whatever reason, even after switching recovery models as you're doing now, but the way to fix it is the same regardless of the reason.  Simply perform the following steps:


Perform a Log Backup (e.g. BACKUP LOG [DBNAME]...) against the database
Perform a DBCC SHRINKFILE operation on the transaction log file specifying 1 MB as the target size
Perform another Log Backup against the database
Perform another DBCC SHRINKFILE operation on the transaction log file specifying 1 MB as the target size
Manually increase the size of the log file, incrementally per Kimberly Tripp's article, to grow the TLog back to an appropriate size.


So what's happening here?


The first TLog backup simply forces a CHECKPOINT operation and flushes the log to disk.
The first DBCC Shrinkfile operation shrinks the physical TLOG down to where the current logical log is located
The second Log Backup issues another checkpoint and will likely backup/clear any resident log operations located in front of the logical log.  These were never flushed in the past for whatever reason, but should be cleared at this point.  This step is critical to fixing your issue and sometimes this backup can take a bit of time depending on how much data needs to be written down to disk.
The second DBCC SHRINKFILE operation moves the logical log file to the head of the physical log file, eliminates excessive VLFs that are likely there now, and reduces the TLOG file back down to a very small size, in this case 1MB.  It is imperative to manually grow the file in step 5 so you don't run into vlf issues again down the road
Manually increasing the size of the tlog will allow you to control the size and number of VLFs within your physical log file.  This is an opportune time to configure this properly, so take advantage.


One final note on the process.  If your tlog is highly active, you may need to perform this a few times or during a window of reduced activity.  High activity may result in another errant auto-growth event occurring before you execute the second log backup.  This basically turns what you thought was step 3 back into step 1.  Generally Steps 1 and 2 go by quickly, and step 3 takes the longest to complete.  Make sure to follow-through with steps 4 and 5.