After setting $base_url, I can only access the front page
It sounds like the answer from Oleg Ivanov would work. However, if you want to tackle this with a .htaccess approach, the following works for me. Find and replace SUBDIRECTORY with your subdirectory.

Amend your Drupal's .htaccess with the following:

RewriteBase /

# homepage redirect
RewriteRule ^$ /SUBDIRECTORY/index.php?q= [L]

# if directory or regular file...
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d [OR]
RewriteCond %{REQUEST_URI} =/favicon.ico
# stop here
RewriteRule ^ - [L]

# if a regular file in SUBDIRECTORY directory
RewriteCond %{DOCUMENT_ROOT}/SUBDIRECTORY/%{REQUEST_URI} -f
RewriteRule .* SUBDIRECTORY/$0 [L]

# Redirect everything to /SUBDIRECTORY
RewriteRule .* /SUBDIRECTORY/index.php?q=$0 [QSA]