title: Creating identifiers containing universal character names via token concatenation
body: <p>I wrote this code that creates identifiers containing universal character names via token concatenation. </p>
  
  <pre><code>//#include &lt;stdio.h&gt;
  int printf(const char*, ...);
  
  #define CAT(a, b) a ## b
  
  int main(void) {
      //int \u306d\u3053 = 10;
      int CAT(\u306d, \u3053) = 10;
  
      printf("%d\n", \u306d\u3053);
      //printf("%d\n", CAT(\u306d, \u3053));
  
      return 0;
  }
  </code></pre>
  
  <p>This code worked well with <a href="http://melpon.org/wandbox/permlink/v4MhEWFYKdvIXwJY" rel="nofollow">gcc 4.8.2 with <code>-fextended-identifiers</code> option</a> and <a href="http://www.tutorialspoint.com/compile_c_online.php?PID=0Bw_CjBb95KQMcDI0cFdHSGNlY0k" rel="nofollow">gcc 5.3.1</a>, but didn't work with <a href="http://melpon.org/wandbox/permlink/xdxfytlCfYuZsc6G" rel="nofollow">clang 3.3</a> with error message:</p>
  
  <pre><code>prog.c:10:17: error: use of undeclared identifier 'ねこ'
          printf("%d\n", \u306d\u3053);
                         ^
  1 error generated.
  </code></pre>
  
  <p>and local clang (Apple LLVM version 7.0.2 (clang-700.1.81)) with error message:</p>
  
  <pre><code>$ clang -std=c11 -Wall -Wextra -o uctest1 uctest1.c
  warning: format specifies type 'int' but the argument has type
        '&lt;dependent type&gt;' [-Wformat]
  uctest1.c:10:17: error: use of undeclared identifier 'ねこ'
          printf("%d\n", \u306d\u3053);
                         ^
  1 warning and 1 error generated.
  </code></pre>
  
  <p>When I used <code>-E</code> option to have the compilers output code with macro expanded,  gcc 5.3.1 emitted this:</p>
  
  <pre><code># 1 "main.c"
  # 1 "&lt;built-in&gt;"
  # 1 "&lt;command-line&gt;"
  # 1 "/usr/include/stdc-predef.h" 1 3 4
  # 1 "&lt;command-line&gt;" 2
  # 1 "main.c"
  
  int printf(const char*, ...);
  
  
  
  int main(void) {
  
   int \U0000306d\U00003053 = 10;
  
   printf("%d\n", \U0000306d\U00003053);
  
  
   return 0;
  }
  </code></pre>
  
  <p>local clang emitted this:</p>
  
  <pre><code># 1 "uctest1.c"
  # 1 "&lt;built-in&gt;" 1
  # 1 "&lt;built-in&gt;" 3
  # 326 "&lt;built-in&gt;" 3
  # 1 "&lt;command line&gt;" 1
  # 1 "&lt;built-in&gt;" 2
  # 1 "uctest1.c" 2
  
  int printf(const char*, ...);
  
  
  
  int main(void) {
  
   int \u306d\u3053 = 10;
  
   printf("%d\n", ねこ);
  
  
   return 0;
  }
  </code></pre>
  
  <p>As you see, the identifiers declared and used in <code>printf()</code> matches in gcc's output, but they don't match in clang's output.</p>
  
  <p>I know that creating universal character names via token concatenation invokes undefined behavior.</p>
  
  <p>Quote from <a href="http://chimera.roma1.infn.it/SP/COMMON/iso-iec-9899-1990.pdf" rel="nofollow">N1570</a> 5.1.1.2 Translation phases:</p>
  
  <blockquote>
    <p>If a character sequence that
    matches the syntax of a universal character name is produced by token
    concatenation (6.10.3.3), the behavior is undefined.</p>
  </blockquote>
  
  <p>I thought that this character sequence <code>\u306d\u3053</code> may "match the syntax of a universal character name" because it contains universal character names as its substring.
  I also thought that "match" may mean that the entire token produced via concatenation stands for one universal character name, and that therefore this undefined behavior isn't invoked in this code.</p>
  
  <p>Reading <a href="https://www.securecoding.cert.org/confluence/display/c/PRE30-C.+Do+not+create+a+universal+character+name+through+concatenation" rel="nofollow">PRE30-C. Do not create a universal character name through concatenation</a>, I found a comment saying this kind of concatenation is allowed:</p>
  
  <blockquote>
    <p>What is forbidden, to create a new UCN via concatenation. Like doing</p>
    
    <p>assign(\u0001,0401,a,b,4)</p>
    
    <p>just concatenating stuff that happens to contain UCNs anywhere is okay.</p>
  </blockquote>
  
  <p>And a log that shows that <a href="https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=30605322" rel="nofollow">a code example like this case (but with 4 characters)</a> is replaced with <a href="https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=32244087" rel="nofollow">another code example</a>.</p>
  
  <p>Does my code example invoke some undefined behaviors (not limited to ones invoked by producing universal character names via token concatenation)?
  Or is this a bug in clang?</p>