Biggest wait WriteLog
After doing a lot more investigation this is what I have found.

It fixed the issue (significant performance gain and WRITELOG has an average wait time of 0.0126 which was initially 14.681) 

Apparently the issue was with the Number of Virtual Log files in my physical log file.

There is a job scheduled to rebuild indexes every night, the job creates 36GB of logs, and until few weeks ago someone had add a job to shrink log file on weekly basis. Log file was being shrunk to 500MB. 

Since it is a very busy server the log file would grow in size and it was set to auto grow by 3 percent. Each time it grew it added more and more VLFs. As a result my 35.5GB log file had 1600 VLFs. 

To resolve the issue I did the following:  

-- To get the number of VLFs in a physical Log file.
DECLARE @count INT, @DBname SYSNAME;
------------------------------------------------------------
SET @DBname = N'TEST_DB'
------------------------------------------------------------

DBCC LOGINFO(@DBname)
     SET @count = @@ROWCOUNT;

SELECT @DBname AS DatabaseName , @count Total_VLFs;
GO


The above query returned the databasename and number of VLFs (1600)

To reduce the number of VLFs this is what I did:

USE TEST_DB;
GO

DBCC SHRINKFILE (TEST_DB_log, 0, TRUNCATEONLY)
GO

DBCC SHRINKFILE (TEST_DB_log  , 0)
GO

ALTER DATABASE TEST_DB
  MODIFY FILE ( NAME       = TEST_DB_log
              , SIZE       = 36864MB
              , FILEGROWTH = 4096MB)
GO


This reduced the number of VLFs from 1600 to 20. The end-users has seen a sudden improvement in the application performance. 

Now one thing I am not sure about is, if 20 VLFs is the right number of VLFs for a 36GB log file. But it seems to have a good impact on the performance of the database.