Seconds_Behind_Master high even though Read_Master_Log_Pos and Exec_Master_Log_Pos are very close
Regarding the seconds_behind_master oscillation.
I found that the description in seconds_behind_master in the mysql documentation is not accurate. I checked the source code. 

long time_diff= ((long)(time(0) - mi-&gt;rli-&gt;last_master_timestamp)
                            - mi-&gt;clock_diff_with_master);


Seconds behind master is calculated based on the difference between current timestamp on the slave and the timestamp on the statement which is being executed by the SQL thread and any initial difference in the clocks between the slave and the master.
And if the SQL thread is completely caught up with the IO thread (i.e. SQL thread has executed all the statements in the relay log file), it is shown as 0 in mysql5.6 (This was changed to show NULL in this case in 5.7)

In my case, the IO thread was the one that was lagging, I checked the status of the binlog file on the master and relay log file on the slave. The IO thread was waiting for the binlog_dump thread to send data to the slave. For some reason, the IO thread was not fetching/able to fetch all the data from the master(Could be a network related issue. Looking into it). The rate of data growth on the slave's relay log file was slower than the rate of the data growth on the master's binlog even when there was a lot of data that can be fetched. And before the IO thread is able to fetch more data and write to the relay log file, the SQL thread was finishing executing those statements. And seconds_behind_master becomes 0.  

Whenever SQL thread is executing these events in the relay log file, it shows a huge lag and once it has completed executing these events, the value falls to 0.