What are options in reinforcement learning?
An option is a generalization of the concept of action. The concept of an option (or macro-action) was introduced in the context of reinforcement learning in the paper Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning (1998) by Richard S. Sutton, Doina Precup and Satinder Singh, so that to capture the idea that certain actions are composed of other sub-actions. Section 2 of the mentioned paper formally defines the concept of an option (which is actually composed of a policy, a termination condition, and an initiation set).

The authors of the mentioned paper give examples of options


  Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint torques.


The option picking up an object, going to lunch, and traveling to a distant city is composed of other sub-actions (e.g. picking up an object), but is itself an action (or macro-action). A primitive action (e.g. joint torques) is itself an option.

A set of options defined over an MDP constitutes a semi-Markov decision process (SMDP), which are MDPs where the time between actions is not constant but it is variable. In other words, a semi-MDP (SMDP) is an extension of the concept of MDP that is used to deal with problems where there are actions of different levels of abstraction.  For example, consider a footballer that needs to take a freekick. The action "take a freekick" involves a sequence of other actions, like "run towards the ball", "look at the wall", etc. The action "take a freekick" takes a variable number of time steps (which depends on the other sub-actions). 

Semi-MDPs are thus used to deal with such problems that involve actions of different levels of abstraction. Hierarchical reinforcement learning (HRL) is a generalization (or extension) of reinforcement learning where the environment is modeled as a semi-MDP.

Curiously, certain models that have won the RoboCup (the famous AI football) context are based on the concept of semi-MDPs, options and HRL. See e.g. WrightEagleBASE, which use the MAXQ-OP (MAXQ online planning) algorithm.

Semi-MDPs can be converted to MDPs. The picture below (which is a screenshot of figure 1 of the mentioned paper that introduces the "options framework" in RL) illustrates the relationship between semi-MDPs and MDPs.



The empty circles (in the middle) are options, while the black circles (at the top) are primitive actions (which are themselves options).

In the paper Reinforcement learning of motor skills with policy gradients mentioned in the question, apparently, the term skill is not formally defined. However, I suppose that skills can be represented as options.