How can I get an entry based on category and slug (via url segments)?
An entry is not related to itself, but that's what you added as a filter.
Use the slug parameter instead.

{% set categorySlug = craft.request.getSegment(2) %}
{% set productSlug = craft.request.getSegment(3) %}

{% set entries = craft.entries.section('products') %}

{% if categorySlug|length %}
    {% set category = craft.categories.group('toys').slug(category).first() %}
    {% if category %}
        {% set entries = entries.relatedTo({ targetElement: category }) %}
    {% endif %}
{% endif %}

{% if productSlug|length %}
    {% set entries = entries.slug(productSlug) %}
{% endif %}

{% set product = entries.first() %}