1. A method of encoding a video into a coded video bitstream with temporal motion vector prediction, the method comprising: determining a value of a flag for indicating whether temporal motion vector prediction is used or not used for the inter-picture prediction of a sub-picture unit of a picture; and writing the flag having said value into a header of the sub-picture unit or a header of the picture; wherein if the flag indicates that temporal motion vector prediction is used, the method further comprises: creating a first list of motion vector predictors comprising a plurality of motion vector predictors including at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; selecting a motion vector predictor out of the first list for a prediction unit in the sub-picture unit; and writing a first parameter into the coded video bitstream for indicating the selected motion vector predictor out of the first list; and wherein if a temporal layer of the picture is determined to be the lowest or base layer, the value of the flag is set to indicate that temporal motion vector prediction is not used; otherwise, the value of the flag is set to indicate that temporal motion vector prediction is used.