Is there cultural variations on what is consider offal?
Short summary: views on these various body parts are never constant, whether over time or even within communities. 

Part of the difficulty with this question is the very definition of the word offal and what it implies.  First, a little mini-history, taken partly from the Oxford English Dictionary.

Offal is literally derived from off + fall, i.e., that which "falls off" or is discarded during some sort of process; the word entered English around the year 1400.  This sense is rare in modern English, but it still occurs in some manufacturing processes, e.g., woodchips that are discarded from milling.

In the 1400s, it was applied in the same sense to butchers, who initially included only those things which were cut off/out of the carcass and truly considered inedible or useless from a culinary perspective (e.g., the contents of the intestines, which would contain the animal's waste and would rarely be consumed because of the potential for disease, parasites, etc.).  Gradually, this definition expanded over the years to include elements that butchers might discard or separate from the primary "meat" (i.e., mostly muscle).  As the OED defines it:


  The edible parts collectively which are cut off in preparing the
  carcass of an animal for food. In early use applied mainly to the
  entrails; later extended to include the head, tail, and internal
  organs such as the heart, liver, etc.


Given that many of these "discarded" or separated items were not considered as desirable for eating, the word gradually acquired a negative connotation, which the OED first finds in 1581:


  The parts of a slaughtered or dead animal considered unfit for human
  consumption; decomposing flesh, carrion.


The OED adds that this last sense is "Sometimes used contemptuously," as in the first 1581 quotation that references "dirtie tripes and offalls."

In answering this question, there seems to be a conflict between two possible interpretations about the word offal: on the one hand, there's the "offal" that's discarded because it's truly inedible or difficult to use in a culinary sense, but then there's the "offal" that's merely separated out because some people or some cultures find it repulsive.  

That latter "offal" is still generally used by somebody, whether it's


eaten by people who don't find it repulsive (and might even consider it a delicacy) 
eaten by poor people who can't afford more desirable food (as referenced in the question when oxtail was given to "laborers")
ground up, processed, and disguised in processed foods
given to animals to eat


For a recent perspective, I quote from the book Offal: A Global History (2013) by Nina Edwards:


  What might the term "offal" include?  The Chambers Dictionary's
  definition sounds a little less than enthusiastic: "waste or rejected
  parts esp. entrails, heart, liver, kidney, tongue etc.: anything
  worthless or unfit for use."  Other edible innards not specified here
  include connective tissue, bone marrow, lungs, spleen, sweetbreads,
  testicles, udders, tripe, heads and the features thereof (brains,
  eyes, cheeks, snout or muzzle and ears), skin, tails, trotters, lard
  and blood.  Offal is sometimes thought of as inner organs and viscera
  alone, but I include all edible exterior parts.


Including all edible (yet undesirable by some) parts seems reasonable in the definition, given that even as early as 1660 the OED has a quotation lumping in sheep trotters with "offal."  But in some places the word carries a distinct connotation of "organ meat" specifically.

In any case, as said at the beginning, the perspectives on these various body parts are never constant, whether over time or even within communities.  Most cultures have entire classes of animals that are considered inedible or repulsive, whether pork in Jewish or Islamic traditions or almost all insects in various Western cultures.  The concept of "offal" is just an extension of communal dietary guidelines or preferences to the avoidance of specific parts of animals.

It's also important to note that there are various reasons for rejecting various foods.  Sometimes foods have unusual textures or flavors, but just as often they are rejected because of the associations of specific body parts (e.g., feet or snouts or tails being "dirty," tongues because of their associations with animal feeding, genitals or reproductive organs for puritanical reasons, etc.).  Or, in some cases the food perhaps resembles actual animal "parts" too much for our modern culture that is often divorced from butchering: in the era of the chicken "nugget" (or "boneless skinless chicken breast") and ground beef, an actual tongue or tail or ear tends to remind the eater that the food is not from some anonymous cut of "meat" but rather from an actual animal.

In some of these cases, the actual meat may not have a flavor or texture very different from the rest of the animal, but some people still reject it on the basis of its appearance or knowledge of its source within the animal.  (An oxtail isn't very different in flavor or texture from short ribs or even some parts of the chuck after long cooking.)

A final connotation to "offal" referenced in the question is that of social class, which is also often associated with cooking techniques and cuisine types.  A steak cut from a tenderloin can be cooked fast on a hot fire/grill and be tender and tasty in a matter of minutes, but the percentage of tender meat available from a cow is quite small and thus traditionally a meal for the elite.  An oxtail generally requires hours of slow simmering in a stew or braise, traditionally a meal associated with lower classes.  Also, many types of offal are more nutritious than "regular" meat (e.g., many internal organs) and/or have higher calories due to excess fat and connective tissue, which actually historically made them essential for lower classes to survive.  (In less developed cultures in various parts of the world, these types of offal are quite frequently given to children or esteemed elders because of the recognition of their nutritional value.)

As the question points out, there has been a recent shift in some Western countries to embrace certain types of "offal" among middle and upper classes.  Most cultures seem to maintain some random animal parts as delicacies ("sweetbreads" being one example in Western culture), but the recent trend seems to be a combination of adventurous eating (popular in an era of multiculturalism) and a desire not to be "wasteful."  However, despite the term "offal" with its etymological connection to waste, most "offal" has rarely been truly discarded.  It just tends to be eaten by the people who like it or who don't have other things to eat.