What problem does kin selection solve?
Why natural selection leads to selfishness
Imagine a population of individuals. At a given locus, some individual carry the A allele and some others can the S allele (A for altruistic and S for Selfish). For simplicity, we will assume organisms are haploid. Individuals carrying the S allele are selfish and will take care of themselves only. The A allele are altruistic meaning that they produce behaviours (and non-behavioural phenotypes) that will reduce their fitness and increase the fitness of other individuals in the population.
As the altruistic behave as to reduce their fitness, they will be at a selective disadvantage and therefore, over time, only selfish individuals will remain.
How can altruism evolve then?
The study of the the evolution (and evolutionary consequences) of behaviour (and non-behavioural phenotypes) that affect the fitness of other individuals in the population is called social evolution. Social evolution is a wide field of knowledge and I will here attempt a very short summary of it so many concepts will be left out!
There are two main (non-exclusive) ways by which altruism can evolve:

Repeated interactions (False altruism)
Kin interactions (True altruism)

Repeated interactions
The concept is grounded within game theory. A good introduction to game theory is definitely out of the scope of this question but I just want to give a little bit of an intuition. I will over-simplified the concepts of game theory but you should definitely have a look at it. Those concepts are very enlightening for questions of evolutionary biology, economy, psychology, sociology and even philosophy.
If for example, the altruistic individuals are actually only altruistic to those
that have shown altruism in the past (via a Tit-for-Tat strategy for example or through the use of reputation), then altruistic individuals may end up have a higher fitness than the selfish individuals. Imagine for example that selfish individuals give nothing and receive nothing. Let's assume that all altruistic individuals, all make one good deed and receive one good deed. Giving a good deed comes at a cost of having, say 0.8 fewer offsprings on average. Receiving a good deed comes at a benefit of 1.2 extra offsprings. Overall and on average, each altruistic individual will produce 0.4 more offsprings than the selfish individuals and therefore, over time the whole population will become altruistic.
We often refer to these concepts as false altruism because altruistic individuals actually has a fitness increase over their life time. Let's now talk about true altruism where altruistic individuals really just give without receiving but yet, the allele coding for altruism thrives.
Kin interactions
Note first that any kin interaction follows within a give game theory scenario. As such kin interaction is just an add-on on what I discussed earlier. For those used to Hamilton's rule, please note that Hamilton's rule apply to Prisoner's dilemna (a type of game theory scenario) only.
So, imagine now that altruistic individuals are being altruistic with their family members (their kins) only (either because they direct their altruism or because there is a strong population structure). Note that such altruistic behaviour can pretty indirect as helping your partner (who we will assume is not part of your close family) will have for consequence that your partner may offer better parental care and therefore by being altruistic to your partner, you are actually being altruistic to you offsprings (which are your kins).
Because the kins of altruistic individuals probably carry the A allele, eve if the altruistic individual is experiencing a lifetime cost in fitness, the A allele may still thrive and the recipients of this altruism are likely also carrying the A allele. In other words, a truly altruistic individual is an individual that sacrifice itself for helping the allele A present in its kins to thrive. In yet other words, true altruism is when an allele is ensuring that copies of itself present in other individual will manage to get passed on.
And this is called Kin Selection. I can now clearly answer your question
Your question

What problem does kin selection solve?

Kin selection explains &quot;true altruism&quot;!
Only &quot;true altruism&quot; can yield to such separation of the reproductive task as that some individuals are perfectly steriles. In your post, you say

From the perspective of the sterile worker: Since it can't reproduce, no strategy will affect the number of offspring it has (will always be 0)

What is missing from this logic is that, 1) at a time workers could still reproduce and 'gave it up' and 2) in many (maybe most?) social species (honeybees, termites, ants and also in non-hymenopteran eusocial species), workers are still able to sometimes reproduce and are also able to overthrow the queen.