Why doesn't "\ w" match Unicode word characters (for example, "ğ, İ, ş, ç, ö, ü") in a Perl regular expression?




Why doesn't "\ w" match Unicode word characters (for example, "ğ, İ, ş, ç, ö, ü") in a Perl regular expression?


I tried to include these characters in the regex `m{\w+}g`
  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

. However, it does not match "ğ, İ, ş, ç, ö, ü".


How can I make this work?



```
use strict;
use warnings;
use v5.12;
use utf8;

open(MYINPUTFILE, "< $ARGV[0]");

my @strings;
my $delimiter;
my $extensions;
my $id;

while(<MYINPUTFILE>)
{
    my($line) = $_;
    chomp($line);
    print $line."\n";
    unshift(@strings,$line =~ /\w+/g);
    $delimiter = /[.\_\s]/;
    $extensions = /pdf$|doc$|docx$/;
    $id = /^200|^201/;
}

foreach(@strings){
    print $_."\n";
}

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
```

The input file looks like this:



>  Çidem\_Şener   
> Hüsnü Tağlip   
> ...
> 
> 
> 


The output looks like this:



```
H 

sn 

Ta 

lip

 

idem\_ 

ener

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
```

In code, I am trying to read a file and take each line in an array. (The delimiter can be `_`
  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

either `.`
  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

or `\s`
  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

).
Unicode can be a problem, and Perl has its own quirks. Essentially, Perl creates a firewall that surrounds all Unicode I / O capabilities. You must tell Perl if the I / O path is encoded. If so, the DECODE rule for any input and / or ENCODE for any output.


Decoding converts the data from {encoding} to Perl's internal representation, which is probably a combination of bytes and code points.


The coding is just the opposite.


Thus, it is actually possible to "decode into" and "encode" up to two different encodings. You just need to say what it is. Encoding / decoding is usually done with the file I / O layer, but you can use the encoding module (distribution part) to manually convert between encodings.


 [perldocs in Unicode](https://translate.googleusercontent.com/translate_c?depth=1&pto=aue&rurl=translate.google.com&sl=ru&sp=nmt4&tl=en&u=) does not read light.



  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

Here's an example that can help visualize it (there are many other ways).



```
use strict;
use warnings;
use Encode;


# This is an internalized string with these UTF-8 codepoints
# 
my $internal_string_1 = "\x{C7}\x{69}\x{64}\x{65}\x{6D}\x{5F}\x{15E}\x{65}\x{6E}\x{65}\x{72}\x{20}\x{48}\x{FC}\x{73}\x{6E}\x{FC}\x{20}\x{54}\x{61}\x{11F}\x{6C}\x{69}\x{70}";


# Open a temp file for writing as UTF-8.
# Output to this file will be automatically encoded from Perl internal to UTF-8 octets.
# Write the internal string.
# Check the file with a UTF-8 editor.
# 
open (my $out, '>:utf8', 'temp.txt') or die "can't open temp.txt for writing $!";
print $out $internal_string_1;
close $out;


# Open the temp file for readin as UTF-8.
# All input from this file will be automatically decoded as UTF-8 octets to Perl internal.
# Read/decode to a different internal string.
# 
open (my $in, '<:utf8', 'temp.txt') or die "can't open temp.txt for reading $!";
$/ = undef;
my $internal_string_2 = <$in>;
close $in;


# Change the binmode of STDOUT to UTF-8.
# Output to STDOUT will now be automatically encoded from Perl internal to UTF-8 octets.
# Capture STDOUT to a file then check with a UTF-8 editor.
# 
binmode STDOUT, ':utf8';
print $internal_string_2, "\n\n";


# Use encode() to convert an internal string to UTF-8 octets
# Format the UTF-8 octets to hex values
# Print to STDOUT
# 
my $octets = encode ("utf8", $internal_string_2);
print "Encoded (out) string -> UTF-8 (octets):\n";
print " length = ".length($octets)."\n";
print " octets = $octets\n";
print " HEX val = ";
for (split //, $octets) {
    printf ("0x%X ", ord($_));
}
print "\n\n";


# Use decode() to convert external UTF-8 octets to an internal string.
# Format the internal string to codepoints (hex values).
# Print to STDOUT.
# 
my $internal_string_3 = decode ("utf8", $octets);
print "Decoded (in) string <- UTF-8 (octets):\n";
print " length = ".length($internal_string_3)."\n";
print " string = $internal\_string\_3\n";
print " code points = ";
for (split //, $internal_string_3) {
    printf ("\\x{%X} ", ord($_));
}

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
```

Output



```
Çidem\_Şener Hüsnü Tağlip

Encoded (out) string -> UTF-8 (octets):
   length  =  29
   octets  =  Ãidem\_Åener HÃ¼snÃ¼ TaÄlip
   HEX val = 0xC3 0x87 0x69 0x64 0x65 0x6D 0x5F 0xC5 0x9E 0x65 0x6E 0x65 0x72 0x20 0x48 0xC3 0xBC 0x73 0x6E 0xC3 0xBC 0x20 0x54 0x61 0xC4 0x9F 0x6C 0x69 0x70

Decoded (in) string <- UTF-8 (octets):
   length      =  24
   string      =  Çidem\_Şener Hüsnü Tağlip
   code points = \x{C7} \x{69} \x{64} \x{65} \x{6D} \x{5F} \x{15E} \x{65} \x{6E} \x{65} \x{72} \x{20} \x{48} \x{FC} \x{73} \x{6E} \x{FC} \x{20} \x{54} \x{61} \x{11F} \x{6C} \x{69} \x{70}

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
```
hi, thank you for the help.

we were able to find the issue by comparing with the workstation of one of our coworker not having this issue. the character “é” from the file location was causing it to bug as it is a special character. we have had to change our computers language settings from french t… All whitespace: `/\s/g`  

All non-alphanumerics: `/[^a-zA-Z0-9]/g` >printf("%c", len\_char); //<this does not print anything too!!  

What did you expect it to print? You realize that the %c format modifier takes the integer value you supply and prints the character representation of that integer, right? In other words, it's not going to print '3'...ever. As long as the lengths are single digits, you can say len\_char + '0' and get the correct representation, but once the length exceeds 9, you're SOL with this method. Hi Shirin\_2,


Just got to see your question you asked on this forum.  

You are welcome to the world of Perl where impossible things are made possible and difficult things made easy :)


As regards your question. It actually a straight forward hash parse with no need for either of the file been compared. The reason is, the all the first column in the first file are all present in the second file. Of course, for the practice of what is it we can still compare but to what end? Except we are going to include our findings in the final compare table. 


All that been said. There are more than one ways to do this. But one of the better way is to use the module *Text::CSV\_XS* instead of spliting on each of the line of the files.


Since you have two files, you can use either a single hash variable or two hash variables, since the keys *must* always be unique you can get all you wanted and then compare to print out desired output.


Here is an one way of doing it....



```
use warnings;
use strict;
use Text::CSV_XS;
use Data::Dumper;
use Inline::Files;

my $hash = {};

my $csv = Text::CSV_XS->new( { binary => 1, sep_char => ';' } );
while ( my $row = $csv->getline(*FILE1) ) {
    push @{ $hash->{ $row->[1] }{ $row->[0] } }, $row->[2];
}

$csv = Text::CSV_XS->new( { binary => 1, sep_char => ';' } );
while ( my $row = $csv->getline(*FILE2) ) {
    push @{ $hash->{ $row->[1] }{ $row->[2] } }, $row->[0];
}

print Dumper($hash);

__FILE1__
CLAYCOUNTY;Wood;statecode=FL
CLAYCOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
NASSAUCOUNTY;Wood;statecode=UT
__FILE2__
119736;Residential;CLAYCOUNTY
448094;Residential;CLAYCOUNTY
206893;Residential;CLAYCOUNTY
333743;Residential;CLAYCOUNTY
172534;Residential;CLAYCOUNTY
785275;Residential;CLAYCOUNTY
995932;Residential;CLAYCOUNTY
223488;Residential;CLAYCOUNTY
433512;Residential;CLAYCOUNTY
640802;Residential;SUWANNEECOUNTY
403866;Residential;SUWANNEECOUNTY
828788;Residential;SUWANNEECOUNTY
751490;Residential;SUWANNEECOUNTY
972562;Residential;SUWANNEECOUNTY
367541;Residential;SUWANNEECOUNTY
481360;Residential;SUWANNEECOUNTY
920232;Residential;NASSAUCOUNTY
727659;Residential;NASSAUCOUNTY
471817;Residential;NASSAUCOUNTY
983043;Residential;NASSAUCOUNTY
578286;Residential;NASSAUCOUNTY
```

and my **raw** output will be



```
$VAR1 = {
          'Wood' => {
                      'NASSAUCOUNTY' => [
                                          'statecode=UT'
                                        ],
                      'SUWANNEECOUNTY' => [
                                            'statecode=FL',
                                            'statecode=TX',
                                            'statecode=TX',
                                            'statecode=TX'
                                          ],
                      'CLAYCOUNTY' => [
                                        'statecode=FL',
                                        'statecode=FL'
                                      ]
                    },
          'Residential' => {
                             'CLAYCOUNTY' => [
                                               '119736',
                                               '448094',
                                               '206893',
                                               '333743',
                                               '172534',
                                               '785275',
                                               '995932',
                                               '223488',
                                               '433512'
                                             ],
                             'SUWANNEECOUNTY' => [
                                                   '640802',
                                                   '403866',
                                                   '828788',
                                                   '751490',
                                                   '972562',
                                                   '367541',
                                                   '481360'
                                                 ],
                             'NASSAUCOUNTY' => [
                                                 '920232',
                                                 '727659',
                                                 '471817',
                                                 '983043',
                                                 '578286'
                                               ]
                           }
        };
```

From the output, you can see Wood => countryName => {....}, then you can also see Residential => countryName => {......}. Which can then be worked with and desired output gotten.


You might have to open and read these files, I used module *Inline::Files* to achieve that purpose in this program. The comparsion and the output of a desired output is left for the OP as practice. 


Hope this helps. hey [@skyler.sorensen](/u/skyler.sorensen), welcome to the auth0 community!

this error is appearing because of the following limitation:

[image]

[metadata best practices](https://auth0.com/docs/best-practices/metadata-best-practices#metadata-storage-and-size-limits)

best practices for metadata in auth0.

metadata is allowed 10 strings with max 500 chars. may be you can breakdown … you write an int, but try to receive a c-string, that can not work.

the radio will not be available (i.e. have a packet received) 99.9% of the time,

introducing a delay and printing something if that is the case is just crazy.

so your code seems to be copy and pasted crap without any understandin… [@cssc](/u/cssc), another option would be to take the regex way.

first, it finds the index of the 5th pipe.

then, it takes the substring until that index.

give this a try:

inputstr.substring(0, regex.matches(inputstr, "\|")(4).index + 1) Unfortunatley after many many years of asking for this, Illustrator still has not added this feature to text frame options.

The closest you can get is to draw a rectangle, click with the vertical type tool (But that will only allow you one character per line.)

![Screen Shot 2016-04-13 at 3.23.20 PM.png](/legacyfs/online/910043_Screen Shot 2016-04-13 at 3.23.20 PM.png)

You could do this with the align palette. You would need to select another element to align against (eg: a rectangle filled with none). You need to add an outline effect to the type, and in the align flyout menu turn ON use preview bounds.

![Screen Shot 2016-04-13 at 3.30.11 PM.png](/legacyfs/online/910045_Screen Shot 2016-04-13 at 3.30.11 PM.png) not [full disclosure] a linux person per se.

something similar to the following should work:

https://www.unix.com/shell-programming-and-scripting/68166-converting-integer-string.html

start simple with just enough code to test the concept.

see if it works and modify/correct as necessary.

then if it still does not work, then post your documented code accordingly along an explanation of what is or is not happening as you expect or desire. Your explanation is not consistent. You state that the user is to enter a range as in "find me all numbers between x and y that are divisible by both 3 and 4." Later you state that you "input value 120 10 times". That is inconsistent with the problem so either the problem is stated incorrectly or you are not inputting the expected data. Decide what is the correct problem then write it up in pseudo-code before you write the actual code.


If a range is expected, make sure you test that it is a valid range (x <= y).