What's the proper way to refer to all the code in `core/lib/Drupal`, or the code in `core/lib/Drupal/Core' or `core/lib/Drupal/Core/Foo`?
What you describe is correct, when you install Drupal, everything in the 'core' directory is considered the core set of Drupal files, or the bare minimum installation package.

The files in 'core/lib' is essentially all of the source code for the running system.  Personally, this is what "core" means to me.  The files (for the most part1) outside of that are the scaffold (or support files).

The files in 'core/lib/Drupal/Core' are called "Core", pronounced with a big C.  The files in 'core/lib/Drupal/Component' are called "Component", pronounced with a big C.  Confusing, yes?

"Component" are the set of files that Drupal uses, but don't actually rely on anything else in "Core".  For example, DateTimePlus and Time are two Components, that don't need anything in 'core/lib/Drupal/Core'.

"Core" are the set of files that Drupal uses, but are interdependent on each other, and also require "Component".  For example, DrupalDateTime relies on DateTimePlus (in Component) and the Language subsystem (in Core).

1Ignoring tests for this discussion.