Perl comparing two files and print output


Hi PerlGurus, 


I am really new to perl and liitle handson with the concepts here. I am trying to learn the hash & Array concepts and come up a self exercise to do practice. 


*Got two files* 


inFile01 as below:

```
CLAYCOUNTY;Wood;statecode=FL
CLAYCOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
NASSAUCOUNTY;Wood;statecode=UT
```

infile02 as below:

```
119736;Residential;CLAYCOUNTY
448094;Residential;CLAYCOUNTY
206893;Residential;CLAYCOUNTY
333743;Residential;CLAYCOUNTY
172534;Residential;CLAYCOUNTY
785275;Residential;CLAYCOUNTY
995932;Residential;CLAYCOUNTY
223488;Residential;CLAYCOUNTY
433512;Residential;CLAYCOUNTY
640802;Residential;SUWANNEECOUNTY
403866;Residential;SUWANNEECOUNTY
828788;Residential;SUWANNEECOUNTY
751490;Residential;SUWANNEECOUNTY
972562;Residential;SUWANNEECOUNTY
367541;Residential;SUWANNEECOUNTY
481360;Residential;SUWANNEECOUNTY
920232;Residential;NASSAUCOUNTY
727659;Residential;NASSAUCOUNTY
471817;Residential;NASSAUCOUNTY
983043;Residential;NASSAUCOUNTY
578286;Residential;NASSAUCOUNTY
```

Step 1: Parse file inFile01 with search key "Wood" and store the file columns into hash. I want to use 2nd and 3rd column and put it into a hash. with the 2nd column being the keys and 1st column being the distinct values.


Step 2: From above step take the column 1 distict value from inFile01 and find all the values in column 1 of my inFile02 and write them to output file.  

In inFile02 I want to use the 3rd and 1st column, with the 3rd column being key and 1st column being the values.



```
e.g. For key 
CLAYCOUNTY values are 119736, 448094, 206893, 333743, 172534, 785275, 995932, 223488, 433512 
SUWANNEECOUNTY are 640802,403866,828788,751490,972562,367541,481360 
NASSAUCOUNTY are 920232,727659, 471817, 983043, 578286
```

I have not decided the output file format how to print them but pleaseeeeeeeee help me on this.


Thanks in advance.
Hi Shirin\_2,


Just got to see your question you asked on this forum.  

You are welcome to the world of Perl where impossible things are made possible and difficult things made easy :)


As regards your question. It actually a straight forward hash parse with no need for either of the file been compared. The reason is, the all the first column in the first file are all present in the second file. Of course, for the practice of what is it we can still compare but to what end? Except we are going to include our findings in the final compare table. 


All that been said. There are more than one ways to do this. But one of the better way is to use the module *Text::CSV\_XS* instead of spliting on each of the line of the files.


Since you have two files, you can use either a single hash variable or two hash variables, since the keys *must* always be unique you can get all you wanted and then compare to print out desired output.


Here is an one way of doing it....



```
use warnings;
use strict;
use Text::CSV_XS;
use Data::Dumper;
use Inline::Files;

my $hash = {};

my $csv = Text::CSV_XS->new( { binary => 1, sep_char => ';' } );
while ( my $row = $csv->getline(*FILE1) ) {
    push @{ $hash->{ $row->[1] }{ $row->[0] } }, $row->[2];
}

$csv = Text::CSV_XS->new( { binary => 1, sep_char => ';' } );
while ( my $row = $csv->getline(*FILE2) ) {
    push @{ $hash->{ $row->[1] }{ $row->[2] } }, $row->[0];
}

print Dumper($hash);

__FILE1__
CLAYCOUNTY;Wood;statecode=FL
CLAYCOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=FL
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
SUWANNEECOUNTY;Wood;statecode=TX
NASSAUCOUNTY;Wood;statecode=UT
__FILE2__
119736;Residential;CLAYCOUNTY
448094;Residential;CLAYCOUNTY
206893;Residential;CLAYCOUNTY
333743;Residential;CLAYCOUNTY
172534;Residential;CLAYCOUNTY
785275;Residential;CLAYCOUNTY
995932;Residential;CLAYCOUNTY
223488;Residential;CLAYCOUNTY
433512;Residential;CLAYCOUNTY
640802;Residential;SUWANNEECOUNTY
403866;Residential;SUWANNEECOUNTY
828788;Residential;SUWANNEECOUNTY
751490;Residential;SUWANNEECOUNTY
972562;Residential;SUWANNEECOUNTY
367541;Residential;SUWANNEECOUNTY
481360;Residential;SUWANNEECOUNTY
920232;Residential;NASSAUCOUNTY
727659;Residential;NASSAUCOUNTY
471817;Residential;NASSAUCOUNTY
983043;Residential;NASSAUCOUNTY
578286;Residential;NASSAUCOUNTY
```

and my **raw** output will be



```
$VAR1 = {
          'Wood' => {
                      'NASSAUCOUNTY' => [
                                          'statecode=UT'
                                        ],
                      'SUWANNEECOUNTY' => [
                                            'statecode=FL',
                                            'statecode=TX',
                                            'statecode=TX',
                                            'statecode=TX'
                                          ],
                      'CLAYCOUNTY' => [
                                        'statecode=FL',
                                        'statecode=FL'
                                      ]
                    },
          'Residential' => {
                             'CLAYCOUNTY' => [
                                               '119736',
                                               '448094',
                                               '206893',
                                               '333743',
                                               '172534',
                                               '785275',
                                               '995932',
                                               '223488',
                                               '433512'
                                             ],
                             'SUWANNEECOUNTY' => [
                                                   '640802',
                                                   '403866',
                                                   '828788',
                                                   '751490',
                                                   '972562',
                                                   '367541',
                                                   '481360'
                                                 ],
                             'NASSAUCOUNTY' => [
                                                 '920232',
                                                 '727659',
                                                 '471817',
                                                 '983043',
                                                 '578286'
                                               ]
                           }
        };
```

From the output, you can see Wood => countryName => {....}, then you can also see Residential => countryName => {......}. Which can then be worked with and desired output gotten.


You might have to open and read these files, I used module *Inline::Files* to achieve that purpose in this program. The comparsion and the output of a desired output is left for the OP as practice. 


Hope this helps.
Unicode can be a problem, and Perl has its own quirks. Essentially, Perl creates a firewall that surrounds all Unicode I / O capabilities. You must tell Perl if the I / O path is encoded. If so, the DECODE rule for any input and / or ENCODE for any output.


Decoding converts the data from {encoding} to Perl's internal representation, which is probably a combination of bytes and code points.


The coding is just the opposite.


Thus, it is actually possible to "decode into" and "encode" up to two different encodings. You just need to say what it is. Encoding / decoding is usually done with the file I / O layer, but you can use the encoding module (distribution part) to manually convert between encodings.


 [perldocs in Unicode](https://translate.googleusercontent.com/translate_c?depth=1&pto=aue&rurl=translate.google.com&sl=ru&sp=nmt4&tl=en&u=) does not read light.



  




 (adsbygoogle = window.adsbygoogle || []).push({});
 
  

Here's an example that can help visualize it (there are many other ways).



```
use strict;
use warnings;
use Encode;


# This is an internalized string with these UTF-8 codepoints
# 
my $internal_string_1 = "\x{C7}\x{69}\x{64}\x{65}\x{6D}\x{5F}\x{15E}\x{65}\x{6E}\x{65}\x{72}\x{20}\x{48}\x{FC}\x{73}\x{6E}\x{FC}\x{20}\x{54}\x{61}\x{11F}\x{6C}\x{69}\x{70}";


# Open a temp file for writing as UTF-8.
# Output to this file will be automatically encoded from Perl internal to UTF-8 octets.
# Write the internal string.
# Check the file with a UTF-8 editor.
# 
open (my $out, '>:utf8', 'temp.txt') or die "can't open temp.txt for writing $!";
print $out $internal_string_1;
close $out;


# Open the temp file for readin as UTF-8.
# All input from this file will be automatically decoded as UTF-8 octets to Perl internal.
# Read/decode to a different internal string.
# 
open (my $in, '<:utf8', 'temp.txt') or die "can't open temp.txt for reading $!";
$/ = undef;
my $internal_string_2 = <$in>;
close $in;


# Change the binmode of STDOUT to UTF-8.
# Output to STDOUT will now be automatically encoded from Perl internal to UTF-8 octets.
# Capture STDOUT to a file then check with a UTF-8 editor.
# 
binmode STDOUT, ':utf8';
print $internal_string_2, "\n\n";


# Use encode() to convert an internal string to UTF-8 octets
# Format the UTF-8 octets to hex values
# Print to STDOUT
# 
my $octets = encode ("utf8", $internal_string_2);
print "Encoded (out) string -> UTF-8 (octets):\n";
print " length = ".length($octets)."\n";
print " octets = $octets\n";
print " HEX val = ";
for (split //, $octets) {
    printf ("0x%X ", ord($_));
}
print "\n\n";


# Use decode() to convert external UTF-8 octets to an internal string.
# Format the internal string to codepoints (hex values).
# Print to STDOUT.
# 
my $internal_string_3 = decode ("utf8", $octets);
print "Decoded (in) string <- UTF-8 (octets):\n";
print " length = ".length($internal_string_3)."\n";
print " string = $internal\_string\_3\n";
print " code points = ";
for (split //, $internal_string_3) {
    printf ("\\x{%X} ", ord($_));
}

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
```

Output



```
Çidem\_Şener Hüsnü Tağlip

Encoded (out) string -> UTF-8 (octets):
   length  =  29
   octets  =  Ãidem\_Åener HÃ¼snÃ¼ TaÄlip
   HEX val = 0xC3 0x87 0x69 0x64 0x65 0x6D 0x5F 0xC5 0x9E 0x65 0x6E 0x65 0x72 0x20 0x48 0xC3 0xBC 0x73 0x6E 0xC3 0xBC 0x20 0x54 0x61 0xC4 0x9F 0x6C 0x69 0x70

Decoded (in) string <- UTF-8 (octets):
   length      =  24
   string      =  Çidem\_Şener Hüsnü Tağlip
   code points = \x{C7} \x{69} \x{64} \x{65} \x{6D} \x{5F} \x{15E} \x{65} \x{6E} \x{65} \x{72} \x{20} \x{48} \x{FC} \x{73} \x{6E} \x{FC} \x{20} \x{54} \x{61} \x{11F} \x{6C} \x{69} \x{70}

        

        
        
        
 (adsbygoogle = window.adsbygoogle || []).push({});
 
        

    
``` How you do this will depend on a number of factors: since you haven't tried to explain in any detail, I'll assume you know nothing, and just give you the basics.  

  

The easiest way to do this is to read the text file into a single string, and then use the .NET framework to find the match:  


```
Dim text As String = File.ReadAllText("D:\Temp\MyFile.txt")
Dim index As Integer = text.IndexOf("hello")
If index >= 0 Then
   ' String is in file, starting at character "index"
End If
```
  

There are other ways, but that is pretty much the simplest. Hi,  

  

check the following Code  

SQLCopy Code

```
-- Question 1 Answer
SELECT E.EmployeeID,E.Day, (E.TotalTime/60) 'TotalHours', (E.TotalTime%60) 'Minites'
FROM (SELECT EmployeeID,CONVERT(VARCHAR(10),FromTime,102) 'Day',SUM(DATEDIFF(mi,FromTime,ToTime)) 'TotalTime'
      FROM tblEmployeeTimeCard
      GROUP BY EmployeeID,CONVERT(VARCHAR(10),FromTime,102))E

-- Question 1&2 Answer
SELECT E.EmployeeID,E.Day, (E.TotalTime/60) 'TotalHours', (E.TotalTime%60) 'Minites',
CASE WHEN (E.TotalTime/60)>=6 THEN E.TotalTime - (30* ROUND((E.TotalTime/60*6),1))
     ELSE (E.TotalTime/60) END 'TotalWorkHours'
FROM (SELECT EmployeeID,CONVERT(VARCHAR(10),FromTime,102) 'Day',SUM(DATEDIFF(mi,FromTime,ToTime)) 'TotalTime'
      FROM tblEmployeeTimeCard
      GROUP BY EmployeeID,CONVERT(VARCHAR(10),FromTime,102))E
```
  

  

For ur Question 2, I am not getting. give some more details about that. if possible send some sample data. if access over lap like (From date is 2013 -04-05 11:55:00PM to 2013-04-05 19:10:00PM)  

then we can't able to fine the time difference. So check the code for Question No 1 and give the feedback.  

  

Regards,  

GVPrabu The behaviour must be because you are appending the same array to both of the arrays.  

Try appending a different array for cell*code and cell*type separately. c = input()  

t = len(c)  

temp = ""  

for i in range(0,len(c)):  

    if t<3:  

        break  

    else:  

        if i in (0,1,t-2,t-1):  

            temp = temp+c[i]  

        else:  

            continue  

print(temp) PF1:  

Name 25 A  

Code 10 S 0  

Location 30 A


PF2:  

LCode 10 S 0  

LLocation 30 A  

LName 25 A


we can’t copy pf1 to pf2, because pf1 and pf2 field are interchanged. so we can’t copy..  

if you try to copy your records, it ll show like this ‘+++++++’ 


best you try to change field order like this.. (must change the field order PF1 or PF2) 


PF2:  

LName 25 A  

LCode 10 S 0  

LLocation 30 A


after using cpyf command… don’t forget the below step to fill….  

<b>Record format field mapping . . > *NOCHK</b> A couple of notes to get started:



> 
> .apply(literal\_eval)
> 
> 
> 


Wouldn't this be better as `map`?



> 
> I then re-create the Dask DF:
> 
> 
> ddf = dd.from\_pandas(ddf\_out, npartitions=nCores)
> 
> 
> 


`ddf_out` was already a dask dataframe, I don't know why you had to do this.



> 
> The columns in result seem to match those in metadir.
> 
> 
> 


The value of `result.columns` is taken from the meta you provided, no computing happens until you ask for it (dask is lazy in most operations). Does the ValueError exception not provide further information?


Here is a full example



```
x = ({'F9_07_PZ_COMP_DIRECT': '0',
  'F9_07_PZ_DIRTRSTKEY_NAME': 'DEBRA MEALY',
  'F9_07_PZ_COMP_OTHER': '0',
  'F9_07_PZ_COMP_RELATED': '0',
  'F9_07_PZ_TITLE': 'CHAIR PERSON',
  'F9_07_PZ_AVE_HOURS_WEEK': '1.00',
  'F9_07_PC_TRUSTEE_INDIVIDUAL': 'X'},
 {'F9_07_PZ_COMP_DIRECT': '0',
  'F9_07_PZ_DIRTRSTKEY_NAME': 'HELEN GORDON',
  'F9_07_PZ_COMP_OTHER': '0',
  'F9_07_PZ_COMP_RELATED': '0',
  'F9_07_PZ_TITLE': 'VICE CHAIR',
  'F9_07_PZ_AVE_HOURS_WEEK': '1.00',
  'F9_07_PC_TRUSTEE_INDIVIDUAL': 'X'})
df = pd.DataFrame({'a': x})
d = dd.from_pandas(df, 1)
meta = pd.DataFrame(columns=['F9_07_PZ_COMP_DIRECT', 
       'F9_07_PZ_DIRTRSTKEY_NAME',
       'F9_07_PZ_COMP_OTHER', 'F9_07_PZ_COMP_RELATED', 'F9_07_PZ_TITLE',
       'F9_07_PZ_AVE_HOURS_WEEK', 'F9_07_PC_TRUSTEE_INDIVIDUAL'], dtype="O")
d.map_partitions(lambda df: df.a.apply(pd.Series), meta=meta).compute()

```

How did I know what `meta` to use? I applied the function to the pandas dataframe - you might use a small piece of the dataframe to do this.


Some additional notes:


* it is an anti-pattern to load the data with pandas, pass to dask workers and then gather the whole result back to a pandas (in memory) dataframe, you are very unlikely to see a speedup this way and may incur a lot of overhead. You are better off loading with something like `dd.read_csv`, and aggregating or writing with dask functions too. Only `compute()` on something that will be small or not return anything (because it involves writing the output). The official examples do not use from\_pandas.
* string and dict processing are python methods, and so hold the interpreter lock (GIL) of any python function: threads will not actually run in parallel. To get parallelism, you need to run in processes, which is most easily achieved using <https://docs.dask.org/en/latest/setup/single-distributed.html>
* the distributed scheduler also gives you access to the dashboard, which has a lot of useful information to diagnose how your system is running. You can also configure a lot about its behaviour, in case you have firewall rules you need to obey. not sure i understood. are you having problems with this code?

if yes, i think you’re using the wrong comparison operator (not = but ==). [@manish\_patel](/u/manish_patel)

check as below

[[image]](https://aws1.discourse-cdn.com/uipath/original/3x/2/e/2e77ce85497a8ce450a3dde9b288abe7fc380f24.png "image")

attached workflow for your reference

[timedifferenceinexcel.zip](/uploads/short-url/jkrk9luusffty75tmi9bdijp9g0.zip) (17.1 kb)

mark as solution if this helps

thanks The code is as shown below 

**clc;**

**close** **all;**

**clear** **all;**

**% define n**

**n = 0:18;**

**N = 19;** **% N tap filter**

**M = (N-1)/2;**

**% define window**

**% define the filter**

**h = 1./((n-M)*pi).*sin((n-M)*pi/5).*blackman(19)'.*((n>=0)-(n>=N));**

**h(M+1) = 2/15;** **% this corresponds to n=1 where the denominator is zero**

**% plot the filter**

**figure;**

**stem(n,h,'filled');grid** **on;title('h[n]');**

**xlabel('n');ylabel('stem height');**

**% plot the frequency response of the filter**

**figure;**

**freqz(h,1);grid** **on;**

  


  


![](/image?k=f31b88da6411d9811f191c8b336bed5e)![](/image?k=d351b5006ccd60c46820c64292935f2f)