What is the mechanism used to control how Drupal pages are rendered? I'm a Java programmer trying to get started with Drupal.  Specifically, I want to be able to create custom pages.  More specifically, I want to understand the nuts and bolts of how this is done (I'm an engineer; it's just my nature)  I have a little PHP experience and I've configured Drupal installations for years.  I'm aware of many of the modules that help make this process easier (such as the Views module), but again--I'm interested in working through the nuts and bolts myself.

I'm not specifically interested in creating new Drupal modules, unless that is the only way to create new pages in Drupal.  My primary interest is in being able to create a custom page.  Let me give a few examples:


I have a db of funny quotes.  I'd like to be able to use Drupal to display a selected quote AND HAVE COMPLETE CONTROL OVER THE PAGE LAYOUT (and again--not using an existing module; I want to understand the inner workings).  Don't ask why I'd want to use Drupal for this, or suggest alternate methods of accomplishing it.  It's a made up example to illustrate what I'm wanting to learn. :-)  
This is a less made up example.  I'd like to create a "newsletter" where the site editors can select submitted content and customize layout (all in a custom UI) and then create a page with that content, one suitable for either viewing in a browser or printing.  Sort of a "desktop publishing" concept for Drupal.  Again, don't suggest ways of accomplishing this.  It's an example to illustrate what I'm wanting to learn.


Basically, I like Drupal's permissioning, modularity, and content management but I'd like to learn how to control display (as complete control over display as possible).  The problem is that I don't even know what topics to start learning.  Creating modules?  Themeing?  Something else?  What is the mechanism or system that I need to learn to be able to completely control how Drupal pages are rendered?

Corollary question: does such control (not limited to just a block or region, but including menus, headers, footers, and everything visible on the page) require modifying the Drupal code directly or can it all be done in the theme/module systems?
Aaron is right that you'd likely want to handle much of this at the theme layer. You'd want to familiarize yourself with how drupal handles templates.

As for the actual creation of a page in code, this page does a pretty good outline of a method. I've not done this myself, but it reads correctly.

You'll first want to understand how to build a module. The basics are, you need a mymodule folder, and a mymodule.module file and mymodule.info file. Anywhere you see a resource that says to use hook_some_function, you're going to replace hook with mymodule.

The nuts and bolts of the page above outline:

-- Implementing of hook_menu (in this instance, the module's name is starting_drupal_dev)

/**
* Implements hook_menu().
*/
function starting_drupal_dev_menu() {
  $items['starting_drupal_dev/custom'] = array(
    'title' =&gt; 'Custom page',
    'page callback' =&gt; 'starting_drupal_dev_custom',
    'access arguments' =&gt; array('access content'),
  );

  return $items;
}


-- Using the custom function starting_drupal_dev_custom to define how the page renders

/**
* Custom callback function.
*/
function starting_drupal_dev_custom() {
  $content['raw_markup'] = array(
    '#type' =&gt; 'markup',
    '#markup' =&gt; '&lt;p&gt;Hello world&lt;/p&gt;',
  );

  return $content;
}


You could then assign your various database queries to variables in order to render them within #markup.

Again, I've not used this particular method myself, but it seems to me that you could at least use this as a starting point for what you're trying to do. Not sure whether or not you could combine this method with theme templates for layout or not. Maybe somebody else could chime in with regard to that.

As for your final question, NEVER (usually) HACK CORE. Handle everything in modules and themes.