问:
Understanding the usage of braces
I'm learning Tcl/Tk and am confused on the usage of curly braces in tcl.

To me it seems to be used to both indicate scope and declare strings! Is this a bug (or feature)?

Is my interpretation correct?

答:
In reply to Toddius Zho's comment, I've seen a few questions (and had them my self) why curly braces are used when using variables e.g. ${var}

In TCL a standard scalar variable can be comprised of any character.
However the variable substitution operator "$" assumes you are using alphanumeric characters and underscores (plus name space separators "::")

So if a variable is set with a non alphanumeric character, such as "!", TCL will error when attempting to substitute the variable.

```<code>% set myvar! 123
123
% puts $myvar!
can't read "myvar": no such variable
</code>```

Using curly brackets the substitution operator does not error:

```<code>% set myvar! 123
123
%puts ${myvar!}
123
</code>```

This is because TCL does not perform substitution/interpolation on anything defined within curly braces.

http://www.tcl.tk/man/tcl8.5/TclCmd/Tcl.htm

Title: Understanding the usage of braces

Body:
In reply to Toddius Zho's comment, I've seen a few questions (and had them my self) why curly braces are used when using variables e.g. ${var}

In TCL a standard scalar variable can be comprised of any character.
However the variable substitution operator "$" assumes you are using alphanumeric characters and underscores (plus name space separators "::")

So if a variable is set with a non alphanumeric character, such as "!", TCL will error when attempting to substitute the variable.

% set myvar! 123
123
% puts $myvar!
can't read "myvar": no such variable

Using curly brackets the substitution operator does not error:

% set myvar! 123
123
%puts ${myvar!}
123

This is because TCL does not perform substitution/interpolation on anything defined within curly braces.

http://www.tcl.tk/man/tcl8.5/TclCmd/Tcl.htm


来源: stackexchange

问:
Whitespace Matching Regex - Java
The Java API for regular expressions states that \s will match whitespace. So the regex \\s\\s should match two spaces.

```<code>Pattern whitespace = Pattern.compile("\\s\\s");
matcher = whitespace.matcher(modLine);
while (matcher.find()) matcher.replaceAll(" ");
</code>```

The aim of this is to replace all instances of two consecutive whitespace with a single space. However this does not actually work.

Am I having a grave misunderstanding of regexes or the term "whitespace"?

答:
You can’t use \s in Java to match white space on its own native character set, because Java doesn’t support the Unicode white space property — even though doing so is strictly required to meet UTS#18’s RL1.2!  What it does have is not standards-conforming, alas.

Unicode defines 26 code points as \p{White_Space}: 20 of them are various sorts of \pZ GeneralCategory=Separator, and the remaining 6 are \p{Cc} GeneralCategory=Control.

White space is a pretty stable property, and those same ones have been around virtually forever. Even so,  Java has no property that conforms to The Unicode Standard for these, so you instead have to use code like this:

```<code>String whitespace_chars =  ""       /* dummy empty string for homogeneity */
                        + "\\u0009" // CHARACTER TABULATION
                        + "\\u000A" // LINE FEED (LF)
                        + "\\u000B" // LINE TABULATION
                        + "\\u000C" // FORM FEED (FF)
                        + "\\u000D" // CARRIAGE RETURN (CR)
                        + "\\u0020" // SPACE
                        + "\\u0085" // NEXT LINE (NEL)
                        + "\\u00A0" // NO-BREAK SPACE
                        + "\\u1680" // OGHAM SPACE MARK
                        + "\\u180E" // MONGOLIAN VOWEL SEPARATOR
                        + "\\u2000" // EN QUAD
                        + "\\u2001" // EM QUAD
                        + "\\u2002" // EN SPACE
                        + "\\u2003" // EM SPACE
                        + "\\u2004" // THREE-PER-EM SPACE
                        + "\\u2005" // FOUR-PER-EM SPACE
                        + "\\u2006" // SIX-PER-EM SPACE
                        + "\\u2007" // FIGURE SPACE
                        + "\\u2008" // PUNCTUATION SPACE
                        + "\\u2009" // THIN SPACE
                        + "\\u200A" // HAIR SPACE
                        + "\\u2028" // LINE SEPARATOR
                        + "\\u2029" // PARAGRAPH SEPARATOR
                        + "\\u202F" // NARROW NO-BREAK SPACE
                        + "\\u205F" // MEDIUM MATHEMATICAL SPACE
                        + "\\u3000" // IDEOGRAPHIC SPACE
                        ;
/* A \s that actually works for Java’s native character set: Unicode */
String     whitespace_charclass = "["  + whitespace_chars + "]";
/* A \S that actually works for  Java’s native character set: Unicode */
String not_whitespace_charclass = "[^" + whitespace_chars + "]";
</code>```

Now you can use whitespace_charclass + "+" as the pattern in your replaceAll.

Sorry ’bout all that. Java’s regexes just don’t work very well on its own native character set, and so you really have to jump through exotic hoops to make them work.

And if you think white space is bad, you should see what you have to do to get \w and \b to finally behave properly!

Yes, it’s possible, and yes, it’s a mindnumbing mess.  That’s being charitable, even. The easiest way to get a standards-comforming regex library for Java is to JNI over to ICU’s stuff. That’s what Google does for Android, because OraSun’s doesn’t measure up.

If you don’t want to do that but still want to stick with Java, I have a front-end regex rewriting library I wrote that “fixes” Java’s patterns, at least to get them conform to the requirements of RL1.2a in UTS#18, Unicode Regular Expressions.

ContentLicense: CC BY-SA 3.0

Title: Whitespace Matching Regex - Java

Body:
The Java API for regular expressions states that \s will match whitespace. So the regex \\s\\s should match two spaces.

Pattern whitespace = Pattern.compile("\\s\\s");
matcher = whitespace.matcher(modLine);
while (matcher.find()) matcher.replaceAll(" ");

The aim of this is to replace all instances of two consecutive whitespace with a single space. However this does not actually work.

Am I having a grave misunderstanding of regexes or the term "whitespace"?

Body:
You can’t use \s in Java to match white space on its own native character set, because Java doesn’t support the Unicode white space property — even though doing so is strictly required to meet UTS#18’s RL1.2!  What it does have is not standards-conforming, alas.

Unicode defines 26 code points as \p{White_Space}: 20 of them are various sorts of \pZ GeneralCategory=Separator, and the remaining 6 are \p{Cc} GeneralCategory=Control.

White space is a pretty stable property, and those same ones have been around virtually forever. Even so,  Java has no property that conforms to The Unicode Standard for these, so you instead have to use code like this:

String whitespace_chars =  ""       /* dummy empty string for homogeneity */
                        + "\\u0009" // CHARACTER TABULATION
                        + "\\u000A" // LINE FEED (LF)
                        + "\\u000B" // LINE TABULATION
                        + "\\u000C" // FORM FEED (FF)
                        + "\\u000D" // CARRIAGE RETURN (CR)
                        + "\\u0020" // SPACE
                        + "\\u0085" // NEXT LINE (NEL)
                        + "\\u00A0" // NO-BREAK SPACE
                        + "\\u1680" // OGHAM SPACE MARK
                        + "\\u180E" // MONGOLIAN VOWEL SEPARATOR
                        + "\\u2000" // EN QUAD
                        + "\\u2001" // EM QUAD
                        + "\\u2002" // EN SPACE
                        + "\\u2003" // EM SPACE
                        + "\\u2004" // THREE-PER-EM SPACE
                        + "\\u2005" // FOUR-PER-EM SPACE
                        + "\\u2006" // SIX-PER-EM SPACE
                        + "\\u2007" // FIGURE SPACE
                        + "\\u2008" // PUNCTUATION SPACE
                        + "\\u2009" // THIN SPACE
                        + "\\u200A" // HAIR SPACE
                        + "\\u2028" // LINE SEPARATOR
                        + "\\u2029" // PARAGRAPH SEPARATOR
                        + "\\u202F" // NARROW NO-BREAK SPACE
                        + "\\u205F" // MEDIUM MATHEMATICAL SPACE
                        + "\\u3000" // IDEOGRAPHIC SPACE
                        ;
/* A \s that actually works for Java’s native character set: Unicode */
String     whitespace_charclass = "["  + whitespace_chars + "]";
/* A \S that actually works for  Java’s native character set: Unicode */
String not_whitespace_charclass = "[^" + whitespace_chars + "]";

Now you can use whitespace_charclass + "+" as the pattern in your replaceAll.

Sorry ’bout all that. Java’s regexes just don’t work very well on its own native character set, and so you really have to jump through exotic hoops to make them work.

And if you think white space is bad, you should see what you have to do to get \w and \b to finally behave properly!

Yes, it’s possible, and yes, it’s a mindnumbing mess.  That’s being charitable, even. The easiest way to get a standards-comforming regex library for Java is to JNI over to ICU’s stuff. That’s what Google does for Android, because OraSun’s doesn’t measure up.

If you don’t want to do that but still want to stick with Java, I have a front-end regex rewriting library I wrote that “fixes” Java’s patterns, at least to get them conform to the requirements of RL1.2a in UTS#18, Unicode Regular Expressions.

ContentLicense: CC BY-SA 4.0


问:
"IOException was unhandled" that error keeps appearing
Im having this problem and it doesnt seem to go away. Theres a code ive done in vb.net, in which i to to stream files but i keep on getting that error

"IOException was unhandled" The process cannot access the file
  'C:\Users\Lenovo\documents\visual studio 2010\Projects\File
  streaming\File Streaming\bin\Debug\banking.xls' because it is being
  used by another process.

Heres the code:

```<code>Imports System.IO

Imports System.Windows.Forms.Form
Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

    End Sub

    Private Sub btnDisplay_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnDisplay.Click

        Dim FileStr As New FileStream("banking.xls", FileMode.Create, FileAccess.Write)

        Dim a As New StreamWriter(FileStr)

        'a.WriteLine("File should be displayed in the RTB............")
        'a.Close()

       **ERROR APPEARS HERE**
        FileStr = New FileStream("banking.xls", FileMode.Open, FileAccess.Read)

        Dim i As New StreamReader(FileStr)

        i.BaseStream.Seek(0, SeekOrigin.Begin)

        If i.Peek() > -1 Then

            rtbDisplay.Text &= i.ReadLine()

        End If
        i.Close()

    End Sub

    Private Sub btnClear_Click(sender As System.Object, e As System.EventArgs) Handles btnClear.Click

        rtbDisplay.Text = ""

    End Sub

    Private Sub btnExit_Click(sender As System.Object, e As System.EventArgs) Handles btnExit.Click

        Me.Close()

    End Sub

    Private Function FileStream() As Object
        Throw New NotImplementedException
    End Function

End Class
</code>```

答:
FileStream and StreamWriter are IDisposable objects, and should be disposed when you are done with them. when IDisposable have local scope, it is good practice to create them with a Using block.

```<code>Sub Main()
    Using FileStr As New FileStream("banking.xls", FileMode.Create, FileAccess.Write)
        Using a As New StreamWriter(FileStr)
            a.WriteLine("File should be displayed in the RTB............")
        End Using
    End Using

    Using FileStr = New FileStream("banking.xls", FileMode.Open, FileAccess.Read)
        Dim i As New StreamReader(FileStr)
    End Using
End Sub
</code>```

Still, I don't know why you are writing text to an xls file like this. If you are manipulating Excel files, you should look at Open Xml or Microsoft Office Interop

Body:
Im having this problem and it doesnt seem to go away. Theres a code ive done in vb.net, in which i to to stream files but i keep on getting that error

"IOException was unhandled" The process cannot access the file
  'C:\Users\Lenovo\documents\visual studio 2010\Projects\File
  streaming\File Streaming\bin\Debug\banking.xls' because it is being
  used by another process.

Heres the code:

Imports System.IO

Imports System.Windows.Forms.Form
Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

    End Sub

    Private Sub btnDisplay_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnDisplay.Click

        Dim FileStr As New FileStream("banking.xls", FileMode.Create, FileAccess.Write)

        Dim a As New StreamWriter(FileStr)

        'a.WriteLine("File should be displayed in the RTB............")
        'a.Close()

       **ERROR APPEARS HERE**
        FileStr = New FileStream("banking.xls", FileMode.Open, FileAccess.Read)

        Dim i As New StreamReader(FileStr)

        i.BaseStream.Seek(0, SeekOrigin.Begin)

        If i.Peek() > -1 Then

            rtbDisplay.Text &= i.ReadLine()

        End If
        i.Close()

    End Sub

    Private Sub btnClear_Click(sender As System.Object, e As System.EventArgs) Handles btnClear.Click

        rtbDisplay.Text = ""

    End Sub

    Private Sub btnExit_Click(sender As System.Object, e As System.EventArgs) Handles btnExit.Click

        Me.Close()

    End Sub

    Private Function FileStream() As Object
        Throw New NotImplementedException
    End Function

End Class

Body:
FileStream and StreamWriter are IDisposable objects, and should be disposed when you are done with them. when IDisposable have local scope, it is good practice to create them with a Using block.

Sub Main()
    Using FileStr As New FileStream("banking.xls", FileMode.Create, FileAccess.Write)
        Using a As New StreamWriter(FileStr)
            a.WriteLine("File should be displayed in the RTB............")
        End Using
    End Using

    Using FileStr = New FileStream("banking.xls", FileMode.Open, FileAccess.Read)
        Dim i As New StreamReader(FileStr)
    End Using
End Sub

Still, I don't know why you are writing text to an xls file like this. If you are manipulating Excel files, you should look at Open Xml or Microsoft Office Interop


问:
How to intercept request in Puppeteer before current page is left?

Usecase:

We need to capture all outbound routes from a page. Some of them may not be implemented using link elements src="..."> but via some javascript code or as GET/POST forms.

PhantomJS:

In Phantom we did this using onNavigationRequested callback. We simply clicked at all the elements defined by some selector and used onNavigationRequested to capture the target url and possibly method or POST data in a case of form and then canceled that navigation event.

Puppeteer:

I tried request interception but at the moment request gets intercepted the current page is already lost so I would have to go back.

Is there a way how to capture the navigation event when the browser is still at the page that triggered the event and to stop it?

Thank you.

答:
page.setRequestInterception(true); The documentation has a really thorough example here: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetrequestinterceptionvalue.
Make sure to add some logic like in the example (and below) they avoid image requests. You would capture it and then abort each request.

```<code>page.on('request', interceptedRequest => {
     if (interceptedRequest.url.endsWith('.png') ||
                              interceptedRequest.url.endsWith('.jpg'))
         interceptedRequest.abort();
     else
         interceptedRequest.continue();
});
</code>```

Body:
Usecase:

We need to capture all outbound routes from a page. Some of them may not be implemented using link elements  but via some javascript code or as GET/POST forms.

PhantomJS:

In Phantom we did this using onNavigationRequested callback. We simply clicked at all the elements defined by some selector and used onNavigationRequested to capture the target url and possibly method or POST data in a case of form and then canceled that navigation event.

Puppeteer:

I tried request interception but at the moment request gets intercepted the current page is already lost so I would have to go back.

Is there a way how to capture the navigation event when the browser is still at the page that triggered the event and to stop it?

Thank you.

Body:
page.setRequestInterception(true); The documentation has a really thorough example here: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetrequestinterceptionvalue.
Make sure to add some logic like in the example (and below) they avoid image requests. You would capture it and then abort each request.

page.on('request', interceptedRequest => {
     if (interceptedRequest.url.endsWith('.png') ||
                              interceptedRequest.url.endsWith('.jpg'))
         interceptedRequest.abort();
     else
         interceptedRequest.continue();
});


问:
Hadoop Streaming Job Failed (Not Successful) in Python
I'm trying to run a Map-Reduce job on Hadoop Streaming with Python scripts and getting the same errors as Hadoop Streaming Job failed error in python but those solutions didn't work for me.

My scripts work fine when I run "cat sample.txt | ./p1mapper.py | sort | ./p1reducer.py"

But when I run the following:

```<code>./bin/hadoop jar contrib/streaming/hadoop-0.20.2-streaming.jar \
    -input "p1input/*" \
    -output p1output \
    -mapper "python p1mapper.py" \
    -reducer "python p1reducer.py" \
    -file /Users/Tish/Desktop/HW1/p1mapper.py \
    -file /Users/Tish/Desktop/HW1/p1reducer.py
</code>```

(NB: Even if I remove the "python" or type the full pathname for -mapper and -reducer, the result is the same)

This is the output I get:

```<code>packageJobJar: [/Users/Tish/Desktop/HW1/p1mapper.py, /Users/Tish/Desktop/CS246/HW1/p1reducer.py, /Users/Tish/Documents/workspace/hadoop-0.20.2/tmp/hadoop-unjar4363616744311424878/] [] /var/folders/Mk/MkDxFxURFZmLg+gkCGdO9U+++TM/-Tmp-/streamjob3714058030803466665.jar tmpDir=null
11/01/18 03:02:52 INFO mapred.FileInputFormat: Total input paths to process : 1
11/01/18 03:02:52 INFO streaming.StreamJob: getLocalDirs(): [tmp/mapred/local]
11/01/18 03:02:52 INFO streaming.StreamJob: Running job: job_201101180237_0005
11/01/18 03:02:52 INFO streaming.StreamJob: To kill this job, run:
11/01/18 03:02:52 INFO streaming.StreamJob: /Users/Tish/Documents/workspace/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:54311 -kill job_201101180237_0005
11/01/18 03:02:52 INFO streaming.StreamJob: Tracking URL: http://www.glassdoor.com:50030/jobdetails.jsp?jobid=job_201101180237_0005
11/01/18 03:02:53 INFO streaming.StreamJob:  map 0%  reduce 0%
11/01/18 03:03:05 INFO streaming.StreamJob:  map 100%  reduce 0%
11/01/18 03:03:44 INFO streaming.StreamJob:  map 50%  reduce 0%
11/01/18 03:03:47 INFO streaming.StreamJob:  map 100%  reduce 100%
11/01/18 03:03:47 INFO streaming.StreamJob: To kill this job, run:
11/01/18 03:03:47 INFO streaming.StreamJob: /Users/Tish/Documents/workspace/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:54311 -kill job_201101180237_0005
11/01/18 03:03:47 INFO streaming.StreamJob: Tracking URL: http://www.glassdoor.com:50030/jobdetails.jsp?jobid=job_201101180237_0005
11/01/18 03:03:47 ERROR streaming.StreamJob: Job not Successful!
11/01/18 03:03:47 INFO streaming.StreamJob: killJob...
Streaming Job Failed!
</code>```

For each Failed/Killed Task Attempt:

```<code>Map output lost, rescheduling: getMapOutput(attempt_201101181225_0001_m_000000_0,0) failed :
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_201101181225_0001/attempt_201101181225_0001_m_000000_0/output/file.out.index in any of the configured local directories
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
    at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:2887)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:324)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
</code>```

Here are my Python scripts:
p1mapper.py

```<code>#!/usr/bin/env python

import sys
import re

SEQ_LEN = 4

eos = re.compile('(?<=[a-zA-Z])\.')   # period preceded by an alphabet
ignore = re.compile('[\W\d]')

for line in sys.stdin:
    array = re.split(eos, line)
    for sent in array:
        sent = ignore.sub('', sent)
        sent = sent.lower()
        if len(sent) >= SEQ_LEN:
            for i in range(len(sent)-SEQ_LEN + 1):
                print '%s 1' % sent[i:i+SEQ_LEN]
</code>```

p1reducer.py

```<code>#!/usr/bin/env python

from operator import itemgetter
import sys

word2count = {}

for line in sys.stdin:
    word, count = line.split(' ', 1)
    try:
        count = int(count)
        word2count[word] = word2count.get(word, 0) + count
    except ValueError:    # count was not a number
        pass

# sort
sorted_word2count = sorted(word2count.items(), key=itemgetter(1), reverse=True)

# write the top 3 sequences
for word, count in sorted_word2count[0:3]:
    print '%s\t%s'% (word, count)
</code>```

Would really appreciate any help, thanks!

UPDATE:

hdfs-site.xml:

```<code><?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>

          <name>dfs.replication</name>

          <value>1</value>

</property>

</configuration>
</code>```

mapred-site.xml:

```<code><?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>

          <name>mapred.job.tracker</name>

          <value>localhost:54311</value>

</property>

</configuration>
</code>```

答:
You are missing a lot of configurations and you need to define directories and such.  See here:

http://wiki.apache.org/hadoop/QuickStart

Distributed operation is just like the pseudo-distributed operation described above, except:

Specify hostname or IP address of the master server in the values for fs.default.name and mapred.job.tracker in conf/hadoop-site.xml. These are specified as host:port pairs.

Specify directories for dfs.name.dir and dfs.data.dir in conf/hadoop-site.xml. These are used to hold distributed filesystem data on the master node and slave nodes respectively. Note that dfs.data.dir may contain a space- or comma-separated list of directory names, so that data may be stored on multiple devices.

Specify mapred.local.dir in conf/hadoop-site.xml. This determines where temporary MapReduce data is written. It also may be a list of directories.

Specify mapred.map.tasks and mapred.reduce.tasks in conf/mapred-default.xml. As a rule of thumb, use 10x the number of slave processors for mapred.map.tasks, and 2x the number of slave processors for mapred.reduce.tasks.

List all slave hostnames or IP addresses in your conf/slaves file, one per line and make sure jobtracker is in your /etc/hosts file pointing to your jobtracker node

ContentLicense: CC BY-SA 2.5

Body:
I'm trying to run a Map-Reduce job on Hadoop Streaming with Python scripts and getting the same errors as Hadoop Streaming Job failed error in python but those solutions didn't work for me.

My scripts work fine when I run "cat sample.txt | ./p1mapper.py | sort | ./p1reducer.py"

But when I run the following:

./bin/hadoop jar contrib/streaming/hadoop-0.20.2-streaming.jar \
    -input "p1input/*" \
    -output p1output \
    -mapper "python p1mapper.py" \
    -reducer "python p1reducer.py" \
    -file /Users/Tish/Desktop/HW1/p1mapper.py \
    -file /Users/Tish/Desktop/HW1/p1reducer.py

(NB: Even if I remove the "python" or type the full pathname for -mapper and -reducer, the result is the same)

This is the output I get:

packageJobJar: [/Users/Tish/Desktop/HW1/p1mapper.py, /Users/Tish/Desktop/CS246/HW1/p1reducer.py, /Users/Tish/Documents/workspace/hadoop-0.20.2/tmp/hadoop-unjar4363616744311424878/] [] /var/folders/Mk/MkDxFxURFZmLg+gkCGdO9U+++TM/-Tmp-/streamjob3714058030803466665.jar tmpDir=null
11/01/18 03:02:52 INFO mapred.FileInputFormat: Total input paths to process : 1
11/01/18 03:02:52 INFO streaming.StreamJob: getLocalDirs(): [tmp/mapred/local]
11/01/18 03:02:52 INFO streaming.StreamJob: Running job: job_201101180237_0005
11/01/18 03:02:52 INFO streaming.StreamJob: To kill this job, run:
11/01/18 03:02:52 INFO streaming.StreamJob: /Users/Tish/Documents/workspace/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:54311 -kill job_201101180237_0005
11/01/18 03:02:52 INFO streaming.StreamJob: Tracking URL: http://www.glassdoor.com:50030/jobdetails.jsp?jobid=job_201101180237_0005
11/01/18 03:02:53 INFO streaming.StreamJob:  map 0%  reduce 0%
11/01/18 03:03:05 INFO streaming.StreamJob:  map 100%  reduce 0%
11/01/18 03:03:44 INFO streaming.StreamJob:  map 50%  reduce 0%
11/01/18 03:03:47 INFO streaming.StreamJob:  map 100%  reduce 100%
11/01/18 03:03:47 INFO streaming.StreamJob: To kill this job, run:
11/01/18 03:03:47 INFO streaming.StreamJob: /Users/Tish/Documents/workspace/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:54311 -kill job_201101180237_0005
11/01/18 03:03:47 INFO streaming.StreamJob: Tracking URL: http://www.glassdoor.com:50030/jobdetails.jsp?jobid=job_201101180237_0005
11/01/18 03:03:47 ERROR streaming.StreamJob: Job not Successful!
11/01/18 03:03:47 INFO streaming.StreamJob: killJob...
Streaming Job Failed!

For each Failed/Killed Task Attempt:

Map output lost, rescheduling: getMapOutput(attempt_201101181225_0001_m_000000_0,0) failed :
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_201101181225_0001/attempt_201101181225_0001_m_000000_0/output/file.out.index in any of the configured local directories
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
    at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:2887)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:324)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

Here are my Python scripts:
p1mapper.py

#!/usr/bin/env python

import sys
import re

SEQ_LEN = 4

eos = re.compile('(?<=[a-zA-Z])\.')   # period preceded by an alphabet
ignore = re.compile('[\W\d]')

for line in sys.stdin:
    array = re.split(eos, line)
    for sent in array:
        sent = ignore.sub('', sent)
        sent = sent.lower()
        if len(sent) >= SEQ_LEN:
            for i in range(len(sent)-SEQ_LEN + 1):
                print '%s 1' % sent[i:i+SEQ_LEN]

p1reducer.py

#!/usr/bin/env python

from operator import itemgetter
import sys

word2count = {}

for line in sys.stdin:
    word, count = line.split(' ', 1)
    try:
        count = int(count)
        word2count[word] = word2count.get(word, 0) + count
    except ValueError:    # count was not a number
        pass

# sort
sorted_word2count = sorted(word2count.items(), key=itemgetter(1), reverse=True)

# write the top 3 sequences
for word, count in sorted_word2count[0:3]:
    print '%s\t%s'% (word, count)

Would really appreciate any help, thanks!

UPDATE:

hdfs-site.xml:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

<property>

          <name>dfs.replication</name>

          <value>1</value>

</property>

</configuration>

mapred-site.xml:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

<property>

          <name>mapred.job.tracker</name>

          <value>localhost:54311</value>

</property>

</configuration>


问:
MySQL SELECT MIN price with priority of field
uI have a database with 3 price fields.
Now I want to query the smallest price in the following order

If MIN price 1 is greater 0 then price 1

ELSE If MIN price 2 is greater 0 then price 2

ELSE If MIN price 3 is greater 0 then price 3

But I always get the lowest price, no matter in which field this is

```<code>SELECT f.user,
(SELECT
    (
        CASE
        WHEN MIN(s.price2) > '0' THEN CONCAT(MIN(s.price2), '*Week*')
        WHEN MIN(s.price1) > '0' THEN CONCAT(MIN(s.price1), '*Week*')
        WHEN MIN(s.price3) > '0' THEN CONCAT(MIN(s.price3), '*Week*')
        ELSE 0
        END
    )
 FROM prices s WHERE f.user = s.user
) as price FROM table f WHERE f.user = 84

+------+--------+--------+--------+--------+
| id   |   user | price1 | price2 | price3 |
+------+--------+--------+--------+--------+
|    1 |     84 |    785 |      0 |      0 |
|    2 |     84 |    550 |    200 |    100 |
|    3 |     84 |      0 |     90 |     45 |
|    4 |     84 |      0 |      0 |     80 |
|    5 |     84 |      0 |    300 |     80 |
|    6 |     84 |    100 |     95 |      0 |
+------+--------+--------+--------+--------+
</code>```

What i want: Priority has MIN price2, then MIN price1, then MIN price4.

```<code>+------+-------+-------+
| id   |  user | price |
+------+-------+-------+
|    1 |    84 |    90 |
+------+-------+-------+
</code>```

答:
I am still not sure I understand your question correctly. You take the minumum prices:

MIN(price1) = 100

MIN(price2) = 90

MIN(price3) = 45

SUM(price2) has highest priority. As it is greater than zero we return this. (Otherwise we'd check SUM(price1) and if that were zero or less too, we'd check SUM(price3).)

```<code>select
  case
    when sum(price2) > 0 then sum(price2)
    when sum(price1) > 0 then sum(price1) end
    when sum(price3) > 0 then sum(price3) end
    else 0
  end as price
from prices
where user = 84;
</code>```

Body:
uI have a database with 3 price fields.
Now I want to query the smallest price in the following order

If MIN price 1 is greater 0 then price 1

ELSE If MIN price 2 is greater 0 then price 2

ELSE If MIN price 3 is greater 0 then price 3

But I always get the lowest price, no matter in which field this is

SELECT f.user,
(SELECT
    (
        CASE
        WHEN MIN(s.price2) > '0' THEN CONCAT(MIN(s.price2), '*Week*')
        WHEN MIN(s.price1) > '0' THEN CONCAT(MIN(s.price1), '*Week*')
        WHEN MIN(s.price3) > '0' THEN CONCAT(MIN(s.price3), '*Week*')
        ELSE 0
        END
    )
 FROM prices s WHERE f.user = s.user
) as price FROM table f WHERE f.user = 84

+------+--------+--------+--------+--------+
| id   |   user | price1 | price2 | price3 |
+------+--------+--------+--------+--------+
|    1 |     84 |    785 |      0 |      0 |
|    2 |     84 |    550 |    200 |    100 |
|    3 |     84 |      0 |     90 |     45 |
|    4 |     84 |      0 |      0 |     80 |
|    5 |     84 |      0 |    300 |     80 |
|    6 |     84 |    100 |     95 |      0 |
+------+--------+--------+--------+--------+

What i want: Priority has MIN price2, then MIN price1, then MIN price4.

+------+-------+-------+
| id   |  user | price |
+------+-------+-------+
|    1 |    84 |    90 |
+------+-------+-------+

Body:
I am still not sure I understand your question correctly. You take the minumum prices:

MIN(price1) = 100

MIN(price2) = 90

MIN(price3) = 45

SUM(price2) has highest priority. As it is greater than zero we return this. (Otherwise we'd check SUM(price1) and if that were zero or less too, we'd check SUM(price3).)

select
  case
    when sum(price2) > 0 then sum(price2)
    when sum(price1) > 0 then sum(price1) end
    when sum(price3) > 0 then sum(price3) end
    else 0
  end as price
from prices
where user = 84;


问:
kafka Synchronization :"java.io.IOException: Too many open files"
we are having problem with Kafka. sometimes Suddenly, without warning we go out of Synchronization  and start to get exceptions when emitting events.

the exception we are getting is

```<code>java.io.IOException: Too many open files
</code>```

it seems this is a generic exception thrown by Kafka in many cases. We investigated it a little and we think the root cause is when trying to emit events to some topic, it fails because kafka dosen't have a leader partition for this topic

can someone help ?

答:
One of the case this happens is that when you have big partition number, because each partition maps to a directory in file system in broker that consist of two files. one of them is for the index and another is for data. broker opens both files. so more partition numbers there is more open files. as Doomy said you can increase the open files in linux but this config is not permanent and when you close the session this config will disappear. and in next loggin if you check with this command

```<code>ulimit -a | grep "open files"
</code>```

you can see the older number. but with this config you can make it permanent:

open this file:

```<code>sudo nano /etc/pam.d/common-session
</code>```

and add this line:

```<code>session required pam_limits.so
</code>```

after that you can set the limit in limits.config as this:

```<code>sudo nano /etc/security/limits.conf
</code>```

and then you can set limit in this file forexample

```<code>* soft nofile 80000
</code>```

or any hard config. after that close your session and check again the limit of open files

Body:
we are having problem with Kafka. sometimes Suddenly, without warning we go out of Synchronization  and start to get exceptions when emitting events.

the exception we are getting is

java.io.IOException: Too many open files

it seems this is a generic exception thrown by Kafka in many cases. We investigated it a little and we think the root cause is when trying to emit events to some topic, it fails because kafka dosen't have a leader partition for this topic

can someone help ?

Body:
One of the case this happens is that when you have big partition number, because each partition maps to a directory in file system in broker that consist of two files. one of them is for the index and another is for data. broker opens both files. so more partition numbers there is more open files. as Doomy said you can increase the open files in linux but this config is not permanent and when you close the session this config will disappear. and in next loggin if you check with this command

ulimit -a | grep "open files"

you can see the older number. but with this config you can make it permanent:

open this file:

sudo nano /etc/pam.d/common-session

and add this line:

session required pam_limits.so

after that you can set the limit in limits.config as this:

sudo nano /etc/security/limits.conf

and then you can set limit in this file forexample

* soft nofile 80000

or any hard config. after that close your session and check again the limit of open files


问:
Spring Validation: Requiring an exact length on an optional field
In Spring MVC, I'm attempting to use annotations to perform validation of form input.  I have some fields that are optional, but when input is provided, it must meet certain criteria.  For example, I have one field that can be left blank, but when it is filled in, the answer must be exactly five characters long.

I've tried things like

```<code>@Size(min=5, max=5)
private String value;
</code>```

but that gives the user an error if the user doesn't fill in the field.

How can I provide some "rules" for an optional field?

答:
You could use the @Pattern(regexp="") annotation to allow customized patterns. Afterwards the error can shown with a Pattern.<bean>.value=... in your i12n property files.

Example:

In bean "Person":

```<code>Pattern(regexp="^[A-ZÑa-zñÁÉÍÓúáéíóú' -]{3,20}$")
private String names;
</code>```

In messages.properties:

```<code>Pattern.person.names=Please insert a valid name
</code>```

Body:
In Spring MVC, I'm attempting to use annotations to perform validation of form input.  I have some fields that are optional, but when input is provided, it must meet certain criteria.  For example, I have one field that can be left blank, but when it is filled in, the answer must be exactly five characters long.

I've tried things like

@Size(min=5, max=5)
private String value;

but that gives the user an error if the user doesn't fill in the field.

How can I provide some "rules" for an optional field?

Body:
You could use the @Pattern(regexp="") annotation to allow customized patterns. Afterwards the error can shown with a Pattern.<bean>.value=... in your i12n property files.

Example:

In bean "Person":

Pattern(regexp="^[A-ZÑa-zñÁÉÍÓúáéíóú' -]{3,20}$")
private String names;

In messages.properties:

Pattern.person.names=Please insert a valid name


问:
Invalid query on a HQL join query with proper syntax
I have a HQL join query as below

```<code>SELECT first FROM FirstTable AS first, SecondTable AS second WHERE second.firstTable = first;
</code>```

This above query returns as Invalid query. But if I use the same above query without the SELECT first it works perfectly fine. I am using the proper syntax as in the documentation. Any idea on what might be the mistake which I am missing or any fix? And I am using the Grails Dynamic finders to run the HQL i.e.,

```<code>FirstTable.findAll(hql.toString(),params)
</code>```

Where hql is a string builder which I am usign to build the query with all the required parameters(param).

The complete error which I get is as below

org.codehaus.groovy.grails.orm.hibernate.exceptions.GrailsQueryException: Invalid query [SELECT first FROM FirstTable AS first, SecondTable AS second WHERE second.firstTable = first group by first.id order by first.id] for domain class [class com.model.FirstTable]

FirstTable has a OneToMany mapping to SecondTable and Secondtable has a OneToOne mapping to FirstTable

答:
Like you said, it works when you remove SELECT first. Because findAll method is only responsible for queries like "from FirstTable".
It is recommended to use executeQuery instead.

In your case it would be something like :

```<code>FirstTable.executeQuery("from FirstTable AS first, SecondTable AS second WHERE second.firstTable = first");
</code>```

See these links for more detail : issues1428 or duplicate so

Body:
I have a HQL join query as below

SELECT first FROM FirstTable AS first, SecondTable AS second WHERE second.firstTable = first;

This above query returns as Invalid query. But if I use the same above query without the SELECT first it works perfectly fine. I am using the proper syntax as in the documentation. Any idea on what might be the mistake which I am missing or any fix? And I am using the Grails Dynamic finders to run the HQL i.e.,

FirstTable.findAll(hql.toString(),params)

Where hql is a string builder which I am usign to build the query with all the required parameters(param).

The complete error which I get is as below

org.codehaus.groovy.grails.orm.hibernate.exceptions.GrailsQueryException: Invalid query [SELECT first FROM FirstTable AS first, SecondTable AS second WHERE second.firstTable = first group by first.id order by first.id] for domain class [class com.model.FirstTable]

FirstTable has a OneToMany mapping to SecondTable and Secondtable has a OneToOne mapping to FirstTable

Body:
Like you said, it works when you remove SELECT first. Because findAll method is only responsible for queries like "from FirstTable".
It is recommended to use executeQuery instead.

In your case it would be something like :

FirstTable.executeQuery("from FirstTable AS first, SecondTable AS second WHERE second.firstTable = first");

See these links for more detail : issues1428 or duplicate so


问:
Vue components on demand - [Vue warn]: Cannot find element
I am working on a php/laravel project that requires using some vue components in some areas. Currently, my vue component looks as follows:

```<code>import Vue from 'vue';

import Notification from "./components/Notification.vue";

window.onload = function () {
   var main = new Vue({
      el: '#my-app',
      components: { Notification }
   });
}
</code>```

However, on pages where I do not have a #my-app element I am getting the following error:

```<code>[Vue warn]: Cannot find element: #my-app
</code>```

Is there a way to prevent this warning? Another way to ask this is, is there way to use my notification component on pages where I need it by creating a #my-app element and not have a #my-app element altogether on pages were I do not need it?

Or if I understand this correctly, I need a root #my-app element regardless of if I'm using the Notification component?

答:
You can dynamically create an element with id my-app, before mount Vue.</p>

```<code>import Vue from 'vue';

import Notification from "./components/Notification.vue";

var myAppElement = document.getElementById('my-app');

if (!myAppElement) {
   var newMyAppElement = document.createElement('div');
   newMyAppElement.setAttribute('id', 'my-app');
}

window.onload = function () {
    var main = new Vue({
    el: '#my-app',
    components: { Notification }
});
</code>```

Body:
I am working on a php/laravel project that requires using some vue components in some areas. Currently, my vue component looks as follows:

import Vue from 'vue';

import Notification from "./components/Notification.vue";

window.onload = function () {
   var main = new Vue({
      el: '#my-app',
      components: { Notification }
   });
}

However, on pages where I do not have a #my-app element I am getting the following error:

[Vue warn]: Cannot find element: #my-app

Is there a way to prevent this warning? Another way to ask this is, is there way to use my notification component on pages where I need it by creating a #my-app element and not have a #my-app element altogether on pages were I do not need it?

Or if I understand this correctly, I need a root #my-app element regardless of if I'm using the Notification component?

Body:
You can dynamically create an element with id my-app, before mount Vue.

import Vue from 'vue';

import Notification from "./components/Notification.vue";

var myAppElement = document.getElementById('my-app');

if (!myAppElement) {
   var newMyAppElement = document.createElement('div');
   newMyAppElement.setAttribute('id', 'my-app');
}

window.onload = function () {
    var main = new Vue({
    el: '#my-app',
    components: { Notification }
});


问:
How do I use AppleScript to time program execution
I would like to use AppleScript to batch run a bunch of programs and get the time that each one ran.

Example: From the command line I would type time ls like this:

```<code>$ time ls
Applications
bin
net
Library
... etc

real    0m0.013s
user    0m0.002s
sys     0m0.006s
</code>```

but when I try to use apple script and put the results into a variable, it completely ignores the time output.

Example:

```<code>set result to do shell script "time ls"
</code>```

returns

```<code>Applications
bin
net
Library
... etc
</code>```

with no mention of the time execution time anywhere.

Even making this more simple:

```<code>set result to do shell script "time"
</code>```

returns

```<code>""
</code>```

How do I get the time of execution back from AppleScript?

p.s. I'm not actually trying to time ls.  I want to time a bunch of custom created programs, but ls makes simple example.

答:
Use the exec 2>&1 command to redirect the STDERR to STDOUT, like this:

```<code>do shell script "exec 2>&1; time  ls"
</code>```

Body:
I would like to use AppleScript to batch run a bunch of programs and get the time that each one ran.

Example: From the command line I would type time ls like this:

$ time ls
Applications
bin
net
Library
... etc

real    0m0.013s
user    0m0.002s
sys     0m0.006s

but when I try to use apple script and put the results into a variable, it completely ignores the time output.

Example:

set result to do shell script "time ls"

returns

Applications
bin
net
Library
... etc

with no mention of the time execution time anywhere.

Even making this more simple:

set result to do shell script "time"

returns

""

How do I get the time of execution back from AppleScript?

p.s. I'm not actually trying to time ls.  I want to time a bunch of custom created programs, but ls makes simple example.

Body:
Use the exec 2>&1 command to redirect the STDERR to STDOUT, like this:

do shell script "exec 2>&1; time  ls"


问:
How to filter a typed Spark Dataset using pattern matching
I try to jump on the typed Dataset API but I'm stuck with filtering:

```<code>val ds: Dataset[(Int, Int)] = Seq((1,1)).toDS

ds.filter(ij => ij._1 > ij._2) // does work, but is not readable
ds.filter{case (i,j) => i<j} // does not work
</code>```

Error:(36, 14) missing parameter type for expanded function The
  argument types of an anonymous function must be fully known. (SLS 8.5)
  Expected type was: ?

I don't understand why pattern matching does not work with filter, while it works fine with map:

```<code>ds.map{case (i,j) => i+j}
</code>```

答:
Make it explicit:</p>

```<code>ds.filter{x => x match { case (i,j) => i < j}}
</code>```

Body:
I try to jump on the typed Dataset API but I'm stuck with filtering:

val ds: Dataset[(Int, Int)] = Seq((1,1)).toDS

ds.filter(ij => ij._1 > ij._2) // does work, but is not readable
ds.filter{case (i,j) => i<j} // does not work

Error:(36, 14) missing parameter type for expanded function The
  argument types of an anonymous function must be fully known. (SLS 8.5)
  Expected type was: ?

I don't understand why pattern matching does not work with filter, while it works fine with map:

ds.map{case (i,j) => i+j}

Body:
Make it explicit:

ds.filter{x => x match { case (i,j) => i < j}}


问:
My macro compares and pastes data to the specified column. I want it to compare and paste the data in the next empty column
[The sheet from where the data is to be copied, columnB2:B]

As columnB is filled I want the data in Sheet1 ColumnB to be copied to the next empty column in Sheet3

My macro compares and pastes data to the specified column. I want it to compare and paste the data in the next empty column.In the line "If FR <> 0 Then w2.Range("B" & FR).Value = c.Offset(0, 1)" I want "B" to be emptyColumn but I am not able to do that.

```<code>Sub UpdateW2()

    Dim w1 As Worksheet, w2 As Worksheet
    Dim c As Range, FR As Long
    Dim emptyColumn As Long

    Application.ScreenUpdating = False

    Set w1 = Sheet1
    Set w2 = Sheet2

    emptyColumn = w2.Cells(2, w2.Columns.Count).End(xlToLeft).Column
    If emptyColumn > 1 Then
        emptyColumn = emptyColumn + 1
    End If

    For Each c In w1.Range("B2", w1.Range("A" & Rows.Count).End(xlUp))
        FR = 0
        On Error Resume Next
        FR = Application.Match(c, w2.Columns("A"), 0)
        On Error GoTo 0
        If FR <> 0 Then w2.Range("B" & FR).Value = c.Offset(0, 1)
    Next c

    Application.ScreenUpdating = True

End Sub
</code>```

答:
```<code> emptyColumn = w2.Cells(2, w2.Columns.Count).End(xlToLeft).Column
 If emptyColumn > 1 Then
       emptyColumn = emptyColumn + 1
 End If
</code>```

This returns the  column in row 2 for which there are no entries to the right. What you want (I think) is the first column to not hold any data at all)

```<code> emptycolumn = w2.usedrange.columns.count +1
</code>```

Body:
[The sheet from where the data is to be copied, columnB2:B]

As columnB is filled I want the data in Sheet1 ColumnB to be copied to the next empty column in Sheet3

My macro compares and pastes data to the specified column. I want it to compare and paste the data in the next empty column.In the line "If FR <> 0 Then w2.Range("B" & FR).Value = c.Offset(0, 1)" I want "B" to be emptyColumn but I am not able to do that.

Sub UpdateW2()

    Dim w1 As Worksheet, w2 As Worksheet
    Dim c As Range, FR As Long
    Dim emptyColumn As Long

    Application.ScreenUpdating = False

    Set w1 = Sheet1
    Set w2 = Sheet2

    emptyColumn = w2.Cells(2, w2.Columns.Count).End(xlToLeft).Column
    If emptyColumn > 1 Then
        emptyColumn = emptyColumn + 1
    End If

    For Each c In w1.Range("B2", w1.Range("A" & Rows.Count).End(xlUp))
        FR = 0
        On Error Resume Next
        FR = Application.Match(c, w2.Columns("A"), 0)
        On Error GoTo 0
        If FR <> 0 Then w2.Range("B" & FR).Value = c.Offset(0, 1)
    Next c

    Application.ScreenUpdating = True

End Sub

Body:
 emptyColumn = w2.Cells(2, w2.Columns.Count).End(xlToLeft).Column
 If emptyColumn > 1 Then
       emptyColumn = emptyColumn + 1
 End If

This returns the  column in row 2 for which there are no entries to the right. What you want (I think) is the first column to not hold any data at all)

 emptycolumn = w2.usedrange.columns.count +1


问:
Is it possible to reuse a task in two different spots in a Luigi pipeline?
I am developing a Luigi pipeline and am running into a major problem. There doesn't seem to be any way to reuse tasks in the same pipeline.
To illustrate what I am after, consider the following workflow:

Collect_Data → Clean_data → Task_on_data_A → Task_on_data_B
                     ↳ Sample_data → Task_on_data_A → Task_on_data_B

I perform the same operations on Clean_data and Sample_data. I want to do something like this in Luigi while following DRY principles. Right now it seems like I have to literally copy and paste my code for tasks A and B to do this.

Airflow has a way to deal with situations like this. Does Luigi?

答:
I think you should draw the arrows as dependency arrows going the other way. It helps to think about it this way since the execution path is determined by the requires methods. In your scenario, it is very simple because you just have a TaskA that has a conditional dependency.

You just need to add one more task at the start that depends on two TaskBs and an indicator of whether the taskb is to be run on the sample

Pseudocode:

```<code>ParentTask
  def requires():
     return [TaskB(sample_type=True),
             TaskB(sample_type=False)]

TaskB
  def requires():
    return TaskA(sample_type)

TaskA
  def requires():
    if sample_type:
      return SampleTask()
    else:
      return CleanTask()

SampleTask
  def requires():
    return CleanTask()

CleanTask
  def requires()
    return CollectData()
</code>```

Body:
I think you should draw the arrows as dependency arrows going the other way. It helps to think about it this way since the execution path is determined by the requires methods. In your scenario, it is very simple because you just have a TaskA that has a conditional dependency.

You just need to add one more task at the start that depends on two TaskBs and an indicator of whether the taskb is to be run on the sample

Pseudocode:

ParentTask
  def requires():
     return [TaskB(sample_type=True),
             TaskB(sample_type=False)]

TaskB
  def requires():
    return TaskA(sample_type)

TaskA
  def requires():
    if sample_type:
      return SampleTask()
    else:
      return CleanTask()

SampleTask
  def requires():
    return CleanTask()

CleanTask
  def requires()
    return CollectData()


问:
Modify certain values in a data frame by indirect reference to the columns
I'm wrangling some data where we sort fails into bins and compute limited yields for each sort bin by lot.

I have a meta table that describes the sort bins. The rows are arranged in ascending test order and some of the sort labels come in with non-syntactic names.

```<code>sort_tbl <- tibble::tribble(~weight,   ~label,
                                  0, "fail A",
                                  0, "fail B",
                                  0, "fail C",
                                100,   "pass")
> sort_tbl
# A tibble: 4 x 2
  weight  label
   <dbl>  <chr>
1      0 fail A
2      0 fail B
3      0 fail C
4    100   pass
</code>```

I have a data table of limited yield by sort bin with one row per lot and one col for each sort bin. Because this table was constructed from a transposition we get instances where a particular sort never occurred for a lot and the resulting value is NA. Note that the columns in this table are arranged in descending test order.

```<code>yld_tbl <- tibble::tribble(  ~lot, ~pass, ~`fail C`, ~`fail B`, ~`fail A`,
                           "lot1",    NA,        NA,      0.00,        NA,
                           "lot2",    NA,      0.00,      0.80,        NA,
                           "lot3",  0.49,        NA,      0.50,      0.98,
                           "lot4",  0.70,      0.95,      0.74,      0.99)
> yld_tbl
# A tibble: 4 x 5
    lot  pass `fail C` `fail B` `fail A`
  <chr> <dbl>    <dbl>    <dbl>    <dbl>
1  lot1    NA       NA     0.00       NA
2  lot2    NA     0.00     0.80       NA
3  lot3  0.49       NA     0.50     0.98
4  lot4  0.70     0.95     0.74     0.99
</code>```

Some of the missing values imply a limited yield of 100% while others reflect an undefined value because we are zero yield earlier in the flow. My task is to replace the former group of NA's with 1.00 as appropriate.

One algorithm to accomplish this works left to right (descending test order) replacing NA with 1.00 if the subsequent limited yield is not NA. In the first row of the example data set, we don't change fail C since pass is missing. But we do replace fail A with 1.00 since fail B is not missing.

The correct example output would be:

```<code>> fill_ones(yld_tbl, sort_tbl)
# A tibble: 4 x 5
    lot  pass `fail C` `fail B` `fail A`
  <chr> <dbl>    <dbl>    <dbl>    <dbl>
1  lot1    NA       NA     0.00     1.00
2  lot2    NA     0.00     0.80     1.00
3  lot3  0.49     1.00     0.50     0.98
4  lot4  0.70     0.95     0.74     0.99
</code>```

答:
Following OP's approach to fill in missing 1.00 from left to right, this can be implemented using melt(), dcast() and rleid():

```<code>library(data.table)
mDT <- melt(setDT(yld_tbl), id.var = "lot")
mDT[
  mDT[, grp := rleid(is.na(value)), by = lot][, .I[is.na(value) & grp > 1]]
  , value := 1][
    , dcast(.SD, lot ~ variable)]
</code>```

```<code>    lot pass fail C fail B fail A
1: lot1   NA     NA   0.00   1.00
2: lot2   NA   0.00   0.80   1.00
3: lot3 0.49   1.00   0.50   0.98
4: lot4 0.70   0.95   0.74   0.99
5: lot5 0.95   0.95   1.00   1.00
</code>```

Data

```<code>yld_tbl <- tibble::tribble(  ~lot, ~pass, ~`fail C`, ~`fail B`, ~`fail A`,
                             "lot1",    NA,        NA,      0.00,        NA,
                             "lot2",    NA,      0.00,      0.80,        NA,
                             "lot3",  0.49,        NA,      0.50,      0.98,
                             "lot4",  0.70,      0.95,      0.74,      0.99,
                             "lot5",  0.95,      0.95,        NA,        NA)
</code>```

Note the additional "lot5" row.

Body:
I'm wrangling some data where we sort fails into bins and compute limited yields for each sort bin by lot.

I have a meta table that describes the sort bins. The rows are arranged in ascending test order and some of the sort labels come in with non-syntactic names.

sort_tbl <- tibble::tribble(~weight,   ~label,
                                  0, "fail A",
                                  0, "fail B",
                                  0, "fail C",
                                100,   "pass")
> sort_tbl
# A tibble: 4 x 2
  weight  label
   <dbl>  <chr>
1      0 fail A
2      0 fail B
3      0 fail C
4    100   pass

I have a data table of limited yield by sort bin with one row per lot and one col for each sort bin. Because this table was constructed from a transposition we get instances where a particular sort never occurred for a lot and the resulting value is NA. Note that the columns in this table are arranged in descending test order.

yld_tbl <- tibble::tribble(  ~lot, ~pass, ~`fail C`, ~`fail B`, ~`fail A`,
                           "lot1",    NA,        NA,      0.00,        NA,
                           "lot2",    NA,      0.00,      0.80,        NA,
                           "lot3",  0.49,        NA,      0.50,      0.98,
                           "lot4",  0.70,      0.95,      0.74,      0.99)
> yld_tbl
# A tibble: 4 x 5
    lot  pass `fail C` `fail B` `fail A`
  <chr> <dbl>    <dbl>    <dbl>    <dbl>
1  lot1    NA       NA     0.00       NA
2  lot2    NA     0.00     0.80       NA
3  lot3  0.49       NA     0.50     0.98
4  lot4  0.70     0.95     0.74     0.99

Some of the missing values imply a limited yield of 100% while others reflect an undefined value because we are zero yield earlier in the flow. My task is to replace the former group of NA's with 1.00 as appropriate.

One algorithm to accomplish this works left to right (descending test order) replacing NA with 1.00 if the subsequent limited yield is not NA. In the first row of the example data set, we don't change fail C since pass is missing. But we do replace fail A with 1.00 since fail B is not missing.

The correct example output would be:

> fill_ones(yld_tbl, sort_tbl)
# A tibble: 4 x 5
    lot  pass `fail C` `fail B` `fail A`
  <chr> <dbl>    <dbl>    <dbl>    <dbl>
1  lot1    NA       NA     0.00     1.00
2  lot2    NA     0.00     0.80     1.00
3  lot3  0.49     1.00     0.50     0.98
4  lot4  0.70     0.95     0.74     0.99

Body:
Following OP's approach to fill in missing 1.00 from left to right, this can be implemented using melt(), dcast() and rleid():

library(data.table)
mDT <- melt(setDT(yld_tbl), id.var = "lot")
mDT[
  mDT[, grp := rleid(is.na(value)), by = lot][, .I[is.na(value) & grp > 1]]
  , value := 1][
    , dcast(.SD, lot ~ variable)]

    lot pass fail C fail B fail A
1: lot1   NA     NA   0.00   1.00
2: lot2   NA   0.00   0.80   1.00
3: lot3 0.49   1.00   0.50   0.98
4: lot4 0.70   0.95   0.74   0.99
5: lot5 0.95   0.95   1.00   1.00

Data

yld_tbl <- tibble::tribble(  ~lot, ~pass, ~`fail C`, ~`fail B`, ~`fail A`,
                             "lot1",    NA,        NA,      0.00,        NA,
                             "lot2",    NA,      0.00,      0.80,        NA,
                             "lot3",  0.49,        NA,      0.50,      0.98,
                             "lot4",  0.70,      0.95,      0.74,      0.99,
                             "lot5",  0.95,      0.95,        NA,        NA)

Note the additional "lot5" row.


问:
How to do Dependency Injection without the tramp data?
After years of people trying to convince me that Singletons Are Evil, I'm finding I have to use Dependency Injection because it's the latest design pattern (like singletons once were). But I can't get away from the code smell: passing around dependencies, sometimes deep down into a view hierarchy, is just another anti-pattern that we used to call tramp data (data that only gets passed into an entity just so it can be passed on to another entity).

I'm not asking this to provoke anyone, I genuinely want to know what people think about this and, more importantly, how are they dealing with it? I have seen first-hand insanely long chains of DI that took maybe half a day of solid hacking to implement, simply to remove a singleton. And then someone decided to add another dependency, and we had to go through the same process again. I do not believe this was worth the effort, and all those complex interfaces create fragility and future problems.

答:
Singletons are not evil. But they can be, when used inappropriate.

They are perfectly fine if and only when:

You only need one of something.

You need that one thing to be passed around in a large number of method calls.

For example, managing external resources, like hardware or state full API's.

But they are evil if one uses them just to create global accessible variables.


问:
MySQL "create trigger tr_emp before insert or update"
```<code>delimiter //
create trigger tr_emp2 before insert or update
on employee for each row
begin
  if new.ID<>10100 then
      if new.Mgr not in (select ID from employee)
      then
      SIGNAL SQLSTATE '45000'
        SET MESSAGE_TEXT = 'Invalid Mgr_id!';
      end if;
  end if;
end;//
delimiter ;
</code>```

How to express "create trigger tr_emp before insert or update" ,is it a wrong exoression ?

答: You have to create two triggers, but you can move the common code into a procedure and have them both call the procedure.</p>

Body:
delimiter //
create trigger tr_emp2 before insert or update
on employee for each row
begin
  if new.ID<>10100 then
      if new.Mgr not in (select ID from employee)
      then
      SIGNAL SQLSTATE '45000'
        SET MESSAGE_TEXT = 'Invalid Mgr_id!';
      end if;
  end if;
end;//
delimiter ;

How to express "create trigger tr_emp before insert or update" ,is it a wrong exoression ?


问:
Insert elements in a BST
I get an error with this code, where i try insert elements of a list into a BST.

The error is tree-value: expects a tree, given 4 but i have no clue why this happens.

```<code>(define-struct tree (left right value))
(define example (make-tree
             (make-tree empty empty 3)
              (make-tree
               (make-tree 4 empty 7)
               (make-tree empty empty 10) 8)5))
(define app (list 1 2 3 4 6 11 12 13))

(define (insert tree val)
(cond [(empty? tree) (make-tree empty empty val)]
    [(= val (tree-value tree)) tree]
    [(< val (tree-value tree))
     (make-tree (insert (tree-left tree) val) (tree-right tree) (tree-value tree))]
    [(> val (tree-value tree))
     (make-tree  (tree-left tree) (insert (tree-right tree) val) (tree-value tree))]))

(define (insert-list tree lst)
(cond [(empty? lst) tree]
    [else (insert-list (insert tree (first lst)) (rest lst))]))

> (insert-list example app)
tree-value: expects a tree, given 4
>
</code>```

答:
Think about what defines a tree. In your example a tree is one of

What defines a tree.

My interpretation of your code is that a tree is defined as follows:

```<code>A tree is one of:
 | empty
 | (make-tree tree tree number)
</code>```

If that is the definition you mean, you are not creating a valid tree. Look at the definition of example: (make-tree 4 empty 7) is not valid. You can fix that by changing the definition of example:

```<code>(define example (make-tree
  (make-tree empty empty 3)
    (make-tree
       (make-tree empty empty 7)
       (make-tree empty empty 10) 8) 5))
</code>```

Title: Insert elements in a BST

Body:
I get an error with this code, where i try insert elements of a list into a BST.

The error is tree-value: expects a tree, given 4 but i have no clue why this happens.

(define-struct tree (left right value))
(define example (make-tree
             (make-tree empty empty 3)
              (make-tree
               (make-tree 4 empty 7)
               (make-tree empty empty 10) 8)5))
(define app (list 1 2 3 4 6 11 12 13))

(define (insert tree val)
(cond [(empty? tree) (make-tree empty empty val)]
    [(= val (tree-value tree)) tree]
    [(< val (tree-value tree))
     (make-tree (insert (tree-left tree) val) (tree-right tree) (tree-value tree))]
    [(> val (tree-value tree))
     (make-tree  (tree-left tree) (insert (tree-right tree) val) (tree-value tree))]))

(define (insert-list tree lst)
(cond [(empty? lst) tree]
    [else (insert-list (insert tree (first lst)) (rest lst))]))

> (insert-list example app)
tree-value: expects a tree, given 4
>

Body:
Think about what defines a tree. In your example a tree is one of

What defines a tree.

My interpretation of your code is that a tree is defined as follows:

A tree is one of:
 | empty
 | (make-tree tree tree number)

If that is the definition you mean, you are not creating a valid tree. Look at the definition of example: (make-tree 4 empty 7) is not valid. You can fix that by changing the definition of example:

(define example (make-tree
  (make-tree empty empty 3)
    (make-tree
       (make-tree empty empty 7)
       (make-tree empty empty 10) 8) 5))


问:
How does one clear user input in Bash when using 'read'?
I have this code in an infinite loop:

```<code>read -a output
echo ${output[@]}
</code>```

When the code runs the screen might look like this:

```<code>Hello, World # line where value was given to output (I want this gone)
Hello, World # echoed output
[] # <-- cursor waiting for next input
</code>```

Is there a way to clear the line where the user inputs the value? Like, as in get rid of it totally (not just changing the line to whitespace that will still leave a gap in the desired output)?

答: Why not just remove the echo ${output[@]}? Since you're not quoting your array variable, this seems to be the same result as if you were to use tput hax.

Body:
I have this code in an infinite loop:

read -a output
echo ${output[@]}

When the code runs the screen might look like this:

Hello, World # line where value was given to output (I want this gone)
Hello, World # echoed output
[] # <-- cursor waiting for next input

Is there a way to clear the line where the user inputs the value? Like, as in get rid of it totally (not just changing the line to whitespace that will still leave a gap in the desired output)?


问:
What is the smallest possible Windows (PE) executable?
As a precursor to writing a compiler I'm trying to understand the Windows (32-bit) Portable Executable format. In particular I'd like to see an example of a bare-bones executable which does nothing except load correctly, run and exit.

I've tried writing and compiling a simple C main function which does nothing but the resulting .exe is ~22KB and contains many imports from KERNEL32.DLL (presumably used by LIBC to set up environment, heaps etc.). Even the DOS Header could probably be smaller (it currently prints the default 'This program cannot be run in DOS mode').

What is the structure of the smallest possible Windows 32-bit executable?

答:
As quoted from source (Creating the smallest possible PE executable): 1

 Smallest possible PE file: 97 bytes

Smallest possible PE file on Windows 2000: 133 bytes

Smallest PE file that downloads a file over WebDAV and executes it: 133 bytes

The files above are the smallest possible PE files due to requirements of the PE file format and cannot be improved further.

This result was achieved with some clever NASM tricks, such as removing the step that links to C stdlib and removing a number of header fields and data directories.

The full source code is below. It is effectively the same as the article with these modification:

Removal of blank lines

sectalign label renamed to sect_align. Since the time this assembly code was written sectalign became a NASM keyword. Rename it to avoid warnings and errors.

The code is as follows:

```<code>; tiny97.asm, copyright Alexander Sotirov

BITS 32
;
; MZ header
; The only two fields that matter are e_magic and e_lfanew

mzhdr:
    dw "MZ"       ; e_magic
    dw 0          ; e_cblp UNUSED

; PE signature
pesig:
    dd "PE"       ; e_cp, e_crlc UNUSED       ; PE signature

; PE header
pehdr:
    dw 0x014C     ; e_cparhdr UNUSED          ; Machine (Intel 386)
    dw 1          ; e_minalloc UNUSED         ; NumberOfSections

;   dd 0xC3582A6A ; e_maxalloc, e_ss UNUSED   ; TimeDateStamp UNUSED

; Entry point
start:
    push byte 42
    pop eax
    ret

codesize equ $ - start

    dd 0          ; e_sp, e_csum UNUSED       ; PointerToSymbolTable UNUSED
    dd 0          ; e_ip, e_cs UNUSED         ; NumberOfSymbols UNUSED
    dw sections-opthdr ; e_lsarlc UNUSED      ; SizeOfOptionalHeader
    dw 0x103      ; e_ovno UNUSED             ; Characteristics

; PE optional header
; The debug directory size at offset 0x94 from here must be 0

filealign equ 4
sect_align equ 4  ; must be 4 because of e_lfanew

%define round(n, r) (((n+(r-1))/r)*r)

opthdr:
    dw 0x10B      ; e_res UNUSED              ; Magic (PE32)
    db 8                                      ; MajorLinkerVersion UNUSED
    db 0                                      ; MinorLinkerVersion UNUSED

; PE code section
sections:
    dd round(codesize, filealign)  ; SizeOfCode UNUSED  ; Name UNUSED
    dd 0  ; e_oemid, e_oeminfo UNUSED ; SizeOfInitializedData UNUSED
    dd codesize  ; e_res2 UNUSED  ; SizeOfUninitializedData UNUSED  ; VirtualSize
    dd start  ; AddressOfEntryPoint  ; VirtualAddress
    dd codesize  ; BaseOfCode UNUSED  ; SizeOfRawData
    dd start  ; BaseOfData UNUSED  ; PointerToRawData
    dd 0x400000  ; ImageBase  ; PointerToRelocations UNUSED
    dd sect_align ; e_lfanew  ; SectionAlignment  ; PointerToLinenumbers UNUSED
    dd filealign  ; FileAlignment  ; NumberOfRelocations, NumberOfLinenumbers UNUSED
    dw 4  ; MajorOperatingSystemVersion UNUSED ; Characteristics UNUSED
    dw 0  ; MinorOperatingSystemVersion UNUSED
    dw 0  ; MajorImageVersion UNUSED
    dw 0  ; MinorImageVersion UNUSED
    dw 4  ; MajorSubsystemVersion
    dw 0  ; MinorSubsystemVersion UNUSED
    dd 0  ; Win32VersionValue UNUSED
    dd round(hdrsize, sect_align)+round(codesize,sect_align) ; SizeOfImage
    dd round(hdrsize, filealign)  ; SizeOfHeaders
    dd 0  ; CheckSum UNUSED
    db 2  ; Subsystem (Win32 GUI)

hdrsize equ $ - $$
filesize equ $ - $$
</code>```

To build into an executable use:

```<code>nasm -f bin tiny97.asm -o tiny97.exe
</code>```

For GNU/Linux ELF executables, See the article "Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux". TL;DR: 1340 bytes, using NASM

Note: This answer is an expansion of J...'s comment on Dec 3 '16 at 17:31, in order to preserve the information found in the link (in case that too goes dead).

Tiny PE; Alexander Sotirov; viewed 15/11/2017 @ 17:50 SAST

OwnerDisplayName: Matthew Murdoch

LastEditorDisplayName: Matthew Murdoch

Body:
As quoted from source (Creating the smallest possible PE executable): 1

 Smallest possible PE file: 97 bytes

Smallest possible PE file on Windows 2000: 133 bytes

Smallest PE file that downloads a file over WebDAV and executes it: 133 bytes

The files above are the smallest possible PE files due to requirements of the PE file format and cannot be improved further.

This result was achieved with some clever NASM tricks, such as removing the step that links to C stdlib and removing a number of header fields and data directories.

The full source code is below. It is effectively the same as the article with these modification:

Removal of blank lines

sectalign label renamed to sect_align. Since the time this assembly code was written sectalign became a NASM keyword. Rename it to avoid warnings and errors.

The code is as follows:

; tiny97.asm, copyright Alexander Sotirov

BITS 32
;
; MZ header
; The only two fields that matter are e_magic and e_lfanew

mzhdr:
    dw "MZ"       ; e_magic
    dw 0          ; e_cblp UNUSED

; PE signature
pesig:
    dd "PE"       ; e_cp, e_crlc UNUSED       ; PE signature

; PE header
pehdr:
    dw 0x014C     ; e_cparhdr UNUSED          ; Machine (Intel 386)
    dw 1          ; e_minalloc UNUSED         ; NumberOfSections

;   dd 0xC3582A6A ; e_maxalloc, e_ss UNUSED   ; TimeDateStamp UNUSED

; Entry point
start:
    push byte 42
    pop eax
    ret

codesize equ $ - start

    dd 0          ; e_sp, e_csum UNUSED       ; PointerToSymbolTable UNUSED
    dd 0          ; e_ip, e_cs UNUSED         ; NumberOfSymbols UNUSED
    dw sections-opthdr ; e_lsarlc UNUSED      ; SizeOfOptionalHeader
    dw 0x103      ; e_ovno UNUSED             ; Characteristics

; PE optional header
; The debug directory size at offset 0x94 from here must be 0

filealign equ 4
sect_align equ 4  ; must be 4 because of e_lfanew

%define round(n, r) (((n+(r-1))/r)*r)

opthdr:
    dw 0x10B      ; e_res UNUSED              ; Magic (PE32)
    db 8                                      ; MajorLinkerVersion UNUSED
    db 0                                      ; MinorLinkerVersion UNUSED

; PE code section
sections:
    dd round(codesize, filealign)  ; SizeOfCode UNUSED  ; Name UNUSED
    dd 0  ; e_oemid, e_oeminfo UNUSED ; SizeOfInitializedData UNUSED
    dd codesize  ; e_res2 UNUSED  ; SizeOfUninitializedData UNUSED  ; VirtualSize
    dd start  ; AddressOfEntryPoint  ; VirtualAddress
    dd codesize  ; BaseOfCode UNUSED  ; SizeOfRawData
    dd start  ; BaseOfData UNUSED  ; PointerToRawData
    dd 0x400000  ; ImageBase  ; PointerToRelocations UNUSED
    dd sect_align ; e_lfanew  ; SectionAlignment  ; PointerToLinenumbers UNUSED
    dd filealign  ; FileAlignment  ; NumberOfRelocations, NumberOfLinenumbers UNUSED
    dw 4  ; MajorOperatingSystemVersion UNUSED ; Characteristics UNUSED
    dw 0  ; MinorOperatingSystemVersion UNUSED
    dw 0  ; MajorImageVersion UNUSED
    dw 0  ; MinorImageVersion UNUSED
    dw 4  ; MajorSubsystemVersion
    dw 0  ; MinorSubsystemVersion UNUSED
    dd 0  ; Win32VersionValue UNUSED
    dd round(hdrsize, sect_align)+round(codesize,sect_align) ; SizeOfImage
    dd round(hdrsize, filealign)  ; SizeOfHeaders
    dd 0  ; CheckSum UNUSED
    db 2  ; Subsystem (Win32 GUI)

hdrsize equ $ - $$
filesize equ $ - $$

To build into an executable use:

nasm -f bin tiny97.asm -o tiny97.exe

For GNU/Linux ELF executables, See the article "Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux". TL;DR: 1340 bytes, using NASM

Note: This answer is an expansion of J...'s comment on Dec 3 '16 at 17:31, in order to preserve the information found in the link (in case that too goes dead).

Tiny PE; Alexander Sotirov; viewed 15/11/2017 @ 17:50 SAST


问:
Gradle exec commandLine does not work as expected
I'm currently working on a Gradle project in OSX
My function in a .gradle file looks like this

```<code>ext.MyFunction = {

     def fastlaneCommand = [
             'fastlane',
             '-version'
     ]

     def stdout = new ByteArrayOutputStream()

     exec {
         ignoreExitValue true
         standardOutput stdout
         workingDir PathsModel.instance.GetDeployerRoot()
         commandLine fastlaneCommand
         LOG.WARN("YOUR CLI COMMAND: " + commandLine)
     }
     println "Output:\n$stdout"
}
</code>```

And then in 'build.gradle'

```<code>task jenkins_deploy() {
    doFirst {
        MyFunction()
    }
}
</code>```

When it comes time for commandLine to be executed
This outputs:

```<code>W A R N I N G:  YOUR CLI COMMAND: [fastlane, -version]

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jenkins_deploy'.
> A problem occurred starting process 'command 'fastlane''
</code>```

I know for a fact that fastlane is in my path as '$HOME/.fastlane/bin' which is where the executable is located. And if I simply open Terminal and type

'fastlane -version'

from any directory, fastlane tools start-up and do what they're supposed to be doing.

I suppose my question is:
What are the possible differences between me opening a terminal and inputting the command manually, and me asking Gradle to do the exact same thing using 'exec'?
Am I misunderstanding what 'exec' and 'commandLine' actually do?

Some info on 'fastlane' is that it's using Ruby, which i don't know a lot about. This may prove relevant.

EDIT: I have attempted 'version' the 2nd element in the fastlaneCommand array, as both 'version' and '-version'

EDIT 2 (ACTUAL SOLUTION): Although the marked answer below is a definite workaround, the solution Actual solution has the full reason as to why this happens and why it works.

答:
TL;DR

I suppose it should be:

```<code>['sh', 'fastlane', '-version']
</code>```

Explanation:

Have not the link under my arm, but if you omit sh it would be executed as a script located in current directory (or directory configured as the second argument). If you prefix it with sh it will be executed with shell and $PATH variable.

Body:
I'm currently working on a Gradle project in OSX
My function in a .gradle file looks like this

ext.MyFunction = {

     def fastlaneCommand = [
             'fastlane',
             '-version'
     ]

     def stdout = new ByteArrayOutputStream()

     exec {
         ignoreExitValue true
         standardOutput stdout
         workingDir PathsModel.instance.GetDeployerRoot()
         commandLine fastlaneCommand
         LOG.WARN("YOUR CLI COMMAND: " + commandLine)
     }
     println "Output:\n$stdout"
}

And then in 'build.gradle'

task jenkins_deploy() {
    doFirst {
        MyFunction()
    }
}

When it comes time for commandLine to be executed
This outputs:

W A R N I N G:  YOUR CLI COMMAND: [fastlane, -version]

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jenkins_deploy'.
> A problem occurred starting process 'command 'fastlane''

I know for a fact that fastlane is in my path as '$HOME/.fastlane/bin' which is where the executable is located. And if I simply open Terminal and type

'fastlane -version'

from any directory, fastlane tools start-up and do what they're supposed to be doing.

I suppose my question is:
What are the possible differences between me opening a terminal and inputting the command manually, and me asking Gradle to do the exact same thing using 'exec'?
Am I misunderstanding what 'exec' and 'commandLine' actually do?

Some info on 'fastlane' is that it's using Ruby, which i don't know a lot about. This may prove relevant.

EDIT: I have attempted 'version' the 2nd element in the fastlaneCommand array, as both 'version' and '-version'

EDIT 2 (ACTUAL SOLUTION): Although the marked answer below is a definite workaround, the solution Actual solution has the full reason as to why this happens and why it works.

Body:
TL;DR

I suppose it should be:

['sh', 'fastlane', '-version']

Explanation:

Have not the link under my arm, but if you omit sh it would be executed as a script located in current directory (or directory configured as the second argument). If you prefix it with sh it will be executed with shell and $PATH variable.


问:
EF Code First: Include not working on optional relationship
I have a specific query in my code which needs to eager load all related entities (both ->1 FKs and ->N FKs) because the context will be disposed right after that.

I made a generic "Query" method that takes params Expression<Func<MyItem, object>>[] includes and then internally chains them. That part works fine.

The query looks like this:

```<code>var item = facade.Query<MyItem>(
                c => c.Childs.Select(x => x.Parent),
                c => c.Childs.Select(x => x.SubChild1),
                c => c.Childs.Select(x => x.SubChildNotWorking),
                c => c.Childs.Select(x => x.SubChild2),
                c => c.Childs.Select(x => x.SubChild3),
                c => c.Childs.Select(x => x.SubChildrens)
                ).FirstOrDefault(c => c.Name == name);
</code>```

The mapping for the not working property (placed in the configuration of SubChildNotWorking):

```<code>this.HasMany(scnw => scnw.Childs).WithOptional(c => c.SubChildNotWorking).HasForeignKey(c => c.MyForeignKey);
</code>```

Of all the includes, only the SubChildNotWorking doesn't actually work. When inspecting with the debugger the returned object, I see the proxies on all the properties. Opening the proxies gives me the correct data for all other relations, and a "The objectcontext has already been disposed exception" for the SubChildNotWorking property.

The only difference I was able to spot is that the SubChildNotWorking is a nullable FK (with nullable column on the DB and WithOptional configuration in the dbcontext) while all the others are non nullable FKs configured with WithRequired.

The database also is a legacy DB not created with Code First and not following its conventions, I just made the mappings in the DbContext. Everything else works fine.

I am trying to figure out if eager loading doesn't work on nullable FKs but I couldn't find any documentation about that.

Is that a bug, or intended behavior? But most of all, how do I solve this?

Thanks.

答:
When you are working with the include method, you are talking about the eager load pattern. Entities with relationship will be loaded as collections. For any not-nullable foreign key, there's a trivial collection starting empty, but for nullable ones, there is not. On the Code First you could make the navigation property virtual, but it's not your case. You can try load it directly on Context, for example:

```<code>var someEntity = context.someEntities.Find(1);
context.Entry(someEntity ).Reference(e => e.EntityWithFKNullable).Load();
</code>```

or the shorter version:

```<code>context.EntitiesWithFKNullable.Load();
</code>```

Body:
I have a specific query in my code which needs to eager load all related entities (both ->1 FKs and ->N FKs) because the context will be disposed right after that.

I made a generic "Query" method that takes params Expression<Func<MyItem, object>>[] includes and then internally chains them. That part works fine.

The query looks like this:

var item = facade.Query<MyItem>(
                c => c.Childs.Select(x => x.Parent),
                c => c.Childs.Select(x => x.SubChild1),
                c => c.Childs.Select(x => x.SubChildNotWorking),
                c => c.Childs.Select(x => x.SubChild2),
                c => c.Childs.Select(x => x.SubChild3),
                c => c.Childs.Select(x => x.SubChildrens)
                ).FirstOrDefault(c => c.Name == name);

The mapping for the not working property (placed in the configuration of SubChildNotWorking):

this.HasMany(scnw => scnw.Childs).WithOptional(c => c.SubChildNotWorking).HasForeignKey(c => c.MyForeignKey);

Of all the includes, only the SubChildNotWorking doesn't actually work. When inspecting with the debugger the returned object, I see the proxies on all the properties. Opening the proxies gives me the correct data for all other relations, and a "The objectcontext has already been disposed exception" for the SubChildNotWorking property.

The only difference I was able to spot is that the SubChildNotWorking is a nullable FK (with nullable column on the DB and WithOptional configuration in the dbcontext) while all the others are non nullable FKs configured with WithRequired.

The database also is a legacy DB not created with Code First and not following its conventions, I just made the mappings in the DbContext. Everything else works fine.

I am trying to figure out if eager loading doesn't work on nullable FKs but I couldn't find any documentation about that.

Is that a bug, or intended behavior? But most of all, how do I solve this?

Thanks.

Body:
When you are working with the include method, you are talking about the eager load pattern. Entities with relationship will be loaded as collections. For any not-nullable foreign key, there's a trivial collection starting empty, but for nullable ones, there is not. On the Code First you could make the navigation property virtual, but it's not your case. You can try load it directly on Context, for example:

var someEntity = context.someEntities.Find(1);
context.Entry(someEntity ).Reference(e => e.EntityWithFKNullable).Load();

or the shorter version:

context.EntitiesWithFKNullable.Load();


问:
Unable to use variable passed from C# into XSLT in a template match
I'm trying to pass in a variable / parameter to an XSLT stylesheet. The variable is entered by the user via a Windows Form.

The XSLT stylesheet is quite simple and it processes lots of XML documents to filter out unwanted data, but leave specific data which has been selected by the user.

The XSLT works perfectly well when I 'hard code' the variable into the template match (i.e. not trying to programatically pass it in) so the parameter is clearly not being passed to the XSLT prior to transformation.

XSLT:

```<code><?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method = "xml" indent="yes"/>
   <xsl:strip-space elements = "*" />

      <xsl:param name="applic"/>

        <xsl:template match = "@*|node()" >
          <xsl:copy>
          <xsl:apply-templates select = "@*|node()" />
         </xsl:copy>
        </xsl:template>

        <xsl:template match = "dmodule/content//*[applic/model[@*]/version[@version != '$applic']]" />
        <xsl:template match = "version[@version != '$applic']" />

        <xsl:template match = "dmodule/content//applic" />

 </xsl:stylesheet>
</code>```

This is how i'm passing the variable to the XSLT in C#

```<code> XsltArgumentList args = new XsltArgumentList();
 args.AddParam("applic", "", applicToSearch); //applicToSearch contains the user variable

 var myXslTrans = new XslCompiledTransform(true);
 myXslTrans.Load(stylesheet);

 using (StreamWriter sw = new StreamWriter(Path.Combine(outputFolder, fileNameNoExt + ".xml")))
  {
       myXslTrans.Transform(outputFolder + "\\temp.xml", args, sw);
  }
</code>```

As I say, clearly the variable / parameter isn't being passed successfully to the XSLT stylesheet.

Any idea why? I've exhausted Google and other posts.

答:
Just to confirm what Martin said - In XSLT 1.0 you can neither use a variable nor a param in a template match.

By example, here's how you could replace a pair of parameterized templates with a template which provides branches for the desired parameter / variable values:

Xslt

```<code><?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method = "xml" indent="yes"/>
  <xsl:strip-space elements = "*" />

  <xsl:param name="applic" select="1"/>

  <xsl:template match = "@*|node()" >
    <xsl:copy>
      <xsl:apply-templates select = "@*|node()" />
    </xsl:copy>
  </xsl:template>

  <xsl:template match = "version">
    <xsl:choose>
      <xsl:when test="@version=$applic">
        <Version match='true'/>
      </xsl:when>
      <xsl:otherwise>
        <Version match='false'/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
</xsl:stylesheet>
</code>```

Sample Xml

```<code><xml>
  <version version='1'></version>
  <version version='2'></version>
</xml>
</code>```

Output

```<code><xml>
  <Version match="true" />
  <Version match="false" />
</xml>
</code>```

Edit, Re: But, I need to pass a parameter into the stylesheet?

Any value embedded in the XSLT param value is simply a default value in the event that a value doesn't get passed in, i.e. 1 is just a default value - this makes it easier to test a new xslt under tools like Visual Studio.

```<code>  .... 1 is a default value here
  <xsl:param name="applic" select="1"/>
</code>```

Your code for passing the parameter to the stylesheet works just fine as is, and will override the above default value:

```<code>var args = new XsltArgumentList();
args.AddParam("applic", "", "2"); // This value overrides anything defaulted in the xslt

myXslTrans.Transform("outfile.xml", args, streamWriter);
</code>```

Body:
I'm trying to pass in a variable / parameter to an XSLT stylesheet. The variable is entered by the user via a Windows Form.

The XSLT stylesheet is quite simple and it processes lots of XML documents to filter out unwanted data, but leave specific data which has been selected by the user.

The XSLT works perfectly well when I 'hard code' the variable into the template match (i.e. not trying to programatically pass it in) so the parameter is clearly not being passed to the XSLT prior to transformation.

XSLT:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method = "xml" indent="yes"/>
   <xsl:strip-space elements = "*" />

      <xsl:param name="applic"/>

        <xsl:template match = "@*|node()" >
          <xsl:copy>
          <xsl:apply-templates select = "@*|node()" />
         </xsl:copy>
        </xsl:template>

        <xsl:template match = "dmodule/content//*[applic/model[@*]/version[@version != '$applic']]" />
        <xsl:template match = "version[@version != '$applic']" />

        <xsl:template match = "dmodule/content//applic" />

 </xsl:stylesheet>

This is how i'm passing the variable to the XSLT in C#

 XsltArgumentList args = new XsltArgumentList();
 args.AddParam("applic", "", applicToSearch); //applicToSearch contains the user variable

 var myXslTrans = new XslCompiledTransform(true);
 myXslTrans.Load(stylesheet);

 using (StreamWriter sw = new StreamWriter(Path.Combine(outputFolder, fileNameNoExt + ".xml")))
  {
       myXslTrans.Transform(outputFolder + "\\temp.xml", args, sw);
  }

As I say, clearly the variable / parameter isn't being passed successfully to the XSLT stylesheet.

Any idea why? I've exhausted Google and other posts.

Body:
Just to confirm what Martin said - In XSLT 1.0 you can neither use a variable nor a param in a template match.

By example, here's how you could replace a pair of parameterized templates with a template which provides branches for the desired parameter / variable values:

Xslt

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method = "xml" indent="yes"/>
  <xsl:strip-space elements = "*" />

  <xsl:param name="applic" select="1"/>

  <xsl:template match = "@*|node()" >
    <xsl:copy>
      <xsl:apply-templates select = "@*|node()" />
    </xsl:copy>
  </xsl:template>

  <xsl:template match = "version">
    <xsl:choose>
      <xsl:when test="@version=$applic">
        <Version match='true'/>
      </xsl:when>
      <xsl:otherwise>
        <Version match='false'/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
</xsl:stylesheet>

Sample Xml

<xml>
  <version version='1'></version>
  <version version='2'></version>
</xml>

Output

<xml>
  <Version match="true" />
  <Version match="false" />
</xml>

Edit, Re: But, I need to pass a parameter into the stylesheet?

Any value embedded in the XSLT param value is simply a default value in the event that a value doesn't get passed in, i.e. 1 is just a default value - this makes it easier to test a new xslt under tools like Visual Studio.

  .... 1 is a default value here
  <xsl:param name="applic" select="1"/>

Your code for passing the parameter to the stylesheet works just fine as is, and will override the above default value:

var args = new XsltArgumentList();
args.AddParam("applic", "", "2"); // This value overrides anything defaulted in the xslt

myXslTrans.Transform("outfile.xml", args, streamWriter);


问:
The mysql use of addslashes()

Possible Duplicate:

  What does mysql_real_escape_string() do that addslashes() doesn't?

If you are trying to prevent sql injection, the first thing you would do is use mysql_real_escape_string.  Is it possible to inject a database using addslashes()?

答:
And still it's possible if you add unquoted data to a table, i.e.

```<code>SELECT * FROM tbl WHERE id = 10
</code>```

Here you want to make sure that this id is exactly a digit

```<code>$id = intval( $_GET[ 'id' ] ) ;
$query = "SELECT * FROM tbl WHERE id = {$id}" ;
$result = mysql_query( $query ) ;
// ... bla-bla
</code>```

Title: The mysql use of addslashes()

Body:
And still it's possible if you add unquoted data to a table, i.e.

SELECT * FROM tbl WHERE id = 10

Here you want to make sure that this id is exactly a digit

$id = intval( $_GET[ 'id' ] ) ;
$query = "SELECT * FROM tbl WHERE id = {$id}" ;
$result = mysql_query( $query ) ;
// ... bla-bla


问:
With which apply function can I loop a clustering function over a grouped dataframe?
I have a data frame that contains ETAs for vehicles. The data contains many ETAs for one (actual) arrival, and I want to group the ETAs that should correspond to one arrival together.

My data looks something like this, with cluster as my desired result:

```<code>  vehicle      eta       ...   cluster
 --------- ------------ ----- ---------
  car_1     01-11-2017   ...         1
  car_1     02-11-2017   ...         1
  car_1     15-11-2017   ...         2
  horse_3   05-11-2017   ...         1
  horse_3   12-11-2017   ...         2
  horse_3   13-11-2017   ...         2
</code>```

I've written a custom function that performs this clustering. The input of the function are the rows for one vehicle, and it returns an array with cluster indices. For example, if I pass it rows 1-3 of my example data, it would output c(1,1,2). So now all I need to do is cycle over all the vehicles in the dataset and apply the function, but that's where I'm stuck.

With cluster_etas as my custom clustering function, I know I could do this:

```<code># Initialize an empty column
data$cluster <- 0

# Loop over all vehicles and perform clustering
for (i in unique(data$vehicle)) {
    filter_vector <- data$vehicle == i
    data$cluster[filter_vector] <- cluster_etas(data[filter_vector,])
}
</code>```

But I suspect there is a smarter solution that would look something like this, with an apply-kind of function filling the blanks:

```<code>data$cluster <- _____(data = data, by = vehicle, function = cluster_etas)
</code>```

Is there a function that performs this kind of operation? What would it be?

答:
tapply:

-used to apply a function over subsets of vector

```<code>str(tapply)
#function (X, INDEX, FUN = NULL, ..., simplify = TRUE)
</code>```

Explaning the arguments:

X is a vector

INDEX is a factor or a lists of factors (or something coercible to factors)

FUN is a function to be applied

... contains other arguments to be passed FUN

Therefore, I believe the solution should be something like
tapply(data$eta, data$vehicle, FUN = ..your function..)

Body:
I have a data frame that contains ETAs for vehicles. The data contains many ETAs for one (actual) arrival, and I want to group the ETAs that should correspond to one arrival together.

My data looks something like this, with cluster as my desired result:

  vehicle      eta       ...   cluster
 --------- ------------ ----- ---------
  car_1     01-11-2017   ...         1
  car_1     02-11-2017   ...         1
  car_1     15-11-2017   ...         2
  horse_3   05-11-2017   ...         1
  horse_3   12-11-2017   ...         2
  horse_3   13-11-2017   ...         2

I've written a custom function that performs this clustering. The input of the function are the rows for one vehicle, and it returns an array with cluster indices. For example, if I pass it rows 1-3 of my example data, it would output c(1,1,2). So now all I need to do is cycle over all the vehicles in the dataset and apply the function, but that's where I'm stuck.

With cluster_etas as my custom clustering function, I know I could do this:

# Initialize an empty column
data$cluster <- 0

# Loop over all vehicles and perform clustering
for (i in unique(data$vehicle)) {
    filter_vector <- data$vehicle == i
    data$cluster[filter_vector] <- cluster_etas(data[filter_vector,])
}

But I suspect there is a smarter solution that would look something like this, with an apply-kind of function filling the blanks:

data$cluster <- _____(data = data, by = vehicle, function = cluster_etas)

Is there a function that performs this kind of operation? What would it be?

Body:
tapply:

-used to apply a function over subsets of vector

str(tapply)
#function (X, INDEX, FUN = NULL, ..., simplify = TRUE)

Explaning the arguments:

X is a vector

INDEX is a factor or a lists of factors (or something coercible to factors)

FUN is a function to be applied

... contains other arguments to be passed FUN

Therefore, I believe the solution should be something like
tapply(data$eta, data$vehicle, FUN = ..your function..)