Predict task duration
I think the analysis which you have done was good.

Regarding the Survival Analysis procedure, I think using it in your scenario is good enough. Even it might take time but the results from that are good and very insightful.

Since you have applied survival analysis on the data, you need to make sure that these assumptions are met:


There are several different ways to estimate a survival function or
a survival curve. There are a number of popular parametric methods
that are used to model survival data, and they differ in terms of
the assumptions that are made about the distribution of survival
times in the population. Some popular distributions include the
exponential, Weibull, Gompertz and log-normal distributions.
Perhaps the most popular is the exponential distribution, which
    assumes that a participant's likelihood of suffering the event of
    interest is independent of how long that person has been event-free.
    Other distributions make different assumptions about the probability
    of an individual developing an event (i.e., it may increase,
    decrease or change over time). More details on parametric methods
    for survival analysis can be found in Hosmer and Lemeshow and Lee
    and Wang1.
Here on two nonparametric methods, which make no assumptions about
how the probability that a person develops the event changes over
time. Using nonparametric methods, we estimate and plot the survival
distribution or the survival curve. Survival curves are often
plotted as step functions, as shown in the figure below. Time is
shown on the X-axis and survival (proportion of people at risk) is
shown on the Y-axis. Note that the percentage of participants
surviving does not always represent the percentage who are alive
(which assumes that the outcome of interest is death). "Survival"
can also refer to the proportion who are free of another outcome
event (e.g., percentage free of MI or cardiovascular disease), or it
can also represent the percentage who do not experience a healthy
outcome (e.g., cancer remission).


You can go through this link for better understanding.

Regarding Poisson Distribution, Did you plot and check whether the data is following Poisson Distribution like:



The Poisson distribution is an appropriate model if the following assumptions are true.


k is the number of times an event occurs in an interval and k can
take values 0, 1, 2, ….
The occurrence of one event does not affect the probability that a
second event will occur. That is, events occur independently.
The rate at which events occur is constant. The rate cannot be higher
in some intervals and lower in other intervals.
Two events cannot occur at exactly the same instant; instead, at each
very small sub-interval exactly one event either occurs or does not
occur.
The probability of an event in a small sub-interval is proportional
to the length of the sub-interval.


Or


The actual probability distribution is given by a binomial
distribution and the number of trials is sufficiently bigger than the
number of successes one is asking about


If these conditions are met then you can use Poisson Model, go through this link

Implementation of this in R, Python.

Finally, to address your 2 questions:


Your approach is Correct there is no problem with using that method, to improve your results you need to work on feature engg(deriving new variables).Since it you are considering the duration as a continuous variable(did you perform that log transformation, which you have stated in the beginning?) 
In your scenario I think Survival and Poisson gives you better result, if you think these take more time then try getting sample of data and get your work done. If you are considering the out-come as a continuous variable then you can use Random Forest,XGBoost , all methods which are used for predicting a continuous variable(but if were you I would spend more time in fitting Survival and Poisson and then shift to other prediction techniques)


Let me know if you have any issues!