问:
Nifi: Need clarification on the Merge Content processor
Because I don't think it works the way my supervisor think it works.

We're taking in a series of about 8 csv files from an FTP and these files are rather small (under 1MB). He's (rightfully, I think) concerned that cluster size on HDFS is going to be wasted. So he wants to use the Merge Content processor to resolve this. He seems to believe that the Merge Content processor will 'collate' files with the same name, making a bigger single file.

To clarify: The way he wants it to work is if today's "sales_report.csv" comes in and there's already a "sales_report.csv" existing in the directory, he wants the new data from today's "sales_report.csv" to be added as new rows to the existing file. I hope that makes sense.

Instead, I'm getting very different results. I have the flow set up so that it picks the files up from the FTP, creates a directory on HDFS based on the folder, and then a subfolder based on the year. When I leave the MC processor out, this all works perfectly. When I put the MC processor in, I get three files - one of them has its original name and two of them have a long string of random characters. We're using the default settings for the Merge Content processor.

Based on what I've described above it does it sound like the MC processor is what we're looking for?

答:
The MergeContent processor works by combining multiple flowfiles into a single flowfile. This is not the same as appending new data to an existing file stored in HDFS (what your manager wants).

To accomplish this, you have a few options:

Keep your current flow without the MergeContent processor; you will still have the "small files" problem with HDFS.

Use a SQL-like interface to HDFS, such as Hive (and optionally HBase (why)). You can then consume the new data (today's sales_report.csv), and treat the rows in that file as NiFi records and persist them to the appropriate Hive table (effectively accomplishing an append operation).

Retrieve the existing sales_report.csv from HDFS, combine the contents with the new content using MergeContent, and persist the new merged content back to HDFS. This is a fairly wasteful operation, and not recommended. (See Iterative Processing in Alan Gates' Pig and Hive at Yahoo!).

Which option you pursue is dependent on your specific requirements:

Does the data need to be stored in the same file in HDFS, or just be accessible in the same directory?

Does the data need to be stored in the original CSV file format, or is tabular storage acceptable?

How large is the "existing" data stored in HDFS vs. the new incoming data?


来源: stackexchange

问:
Issues with R's plot: expr did not evaluate to an object of length n
I want to plot this function in R:

My code:

```<code>   lambda <- function (i) {
    m <- 50
    j=max(1,i+m)
    n <- 27090
    ((2*m+1)^(-1))*
    for (i in j:min(n,i+m)) {
    sum (fires_2009_2015$Wi)[j]
    }
    }
    plot(lambda)
</code>```

However when I run the code, I get an error:

"Error in curve(expr = x, from = from, to = to, xlim = xlim, ylab = ylab,  :
    'expr' did not evaluate to an object of length 'n' "

答:
The problem is that plot(lambda) will call the function lambda() with a vector argument for i, and your definition of lambda() won't handle that properly.  Because of your for loop, it looks a little tricky to make it handle vector arguments efficiently, but there's always the brute force method:

```<code>lambda <- Vectorize(lambda)
</code>```

After this your plot(lambda) should do something.  It's probably not what you want, because it will default to a range from 0 to 1, but you can do things like

```<code>plot(lambda, from = 0, to = 20)
</code>```

to change the default range.

ContentLicense: CC BY-SA 4.0

Body:
I want to plot this function in R:

My code:

   lambda <- function (i) {
    m <- 50
    j=max(1,i+m)
    n <- 27090
    ((2*m+1)^(-1))*
    for (i in j:min(n,i+m)) {
    sum (fires_2009_2015$Wi)[j]
    }
    }
    plot(lambda)

However when I run the code, I get an error:

"Error in curve(expr = x, from = from, to = to, xlim = xlim, ylab = ylab,  :
    'expr' did not evaluate to an object of length 'n' "

Body:
The problem is that plot(lambda) will call the function lambda() with a vector argument for i, and your definition of lambda() won't handle that properly.  Because of your for loop, it looks a little tricky to make it handle vector arguments efficiently, but there's always the brute force method:

lambda <- Vectorize(lambda)

After this your plot(lambda) should do something.  It's probably not what you want, because it will default to a range from 0 to 1, but you can do things like

plot(lambda, from = 0, to = 20)

to change the default range.


问:
Get xml web service request
I'm submitting a request to a web service, but I'm receiving some errors.  They've asked to see an example of the xml request and response.  I used Visual Studio to consume the web service, so I'm just calling a method in my code - I don't actually see any xml.

Is there a way to grab the XML request and response as XML or at least a text string?

Thanks

答:
I had to debug what was going across the line recently and tools like wireshark and fiddler are great tools for debugging the request and response unless you are using HTTPS or you are debugging on your local machine and executing the client and the web service locally.

I found a method that allows you to see the details of both the request and response without having to modify a single line of your code.

.NET has a feature built in called tracing. By enabling tracing for the System.NET namespace you can capture everything.

Here are the steps to enable.

Add the following code to your app.config in your client applications.

```<code><system.diagnostics>
<trace autoflush="true"  />
<sources>
    <source name="System.Net">
        <listeners>
            <add name="System.Net"/>
        </listeners>
    </source>
</sources>
<sharedListeners>
    <add name="System.Net"
        type="System.Diagnostics.TextWriterTraceListener"
        initializeData="System.Net.trace.log" />
</sharedListeners>
<switches>
    <add name="System.Net" value="Verbose" />
</switches>
</system.diagnostics>
</code>```

Now when you execute your client application you can go into the folder that your executable was run from and find the file System.Net.trace.log

You will then find in the log file your request and the servers response. The great thing about this solution is you do not have to install or run anything extra. However the solution is probably only a solution for developing or diagnosing something in test or stage environment rather than production. However I am assuming because you mention creating the solution in Visual Studio and it not working that you are clearly in the development stage.

ContentLicense: CC BY-SA 2.5

Title: Get xml web service request

Body:
I had to debug what was going across the line recently and tools like wireshark and fiddler are great tools for debugging the request and response unless you are using HTTPS or you are debugging on your local machine and executing the client and the web service locally.

I found a method that allows you to see the details of both the request and response without having to modify a single line of your code.

.NET has a feature built in called tracing. By enabling tracing for the System.NET namespace you can capture everything.

Here are the steps to enable.

Add the following code to your app.config in your client applications.

<system.diagnostics>
<trace autoflush="true"  />
<sources>
    <source name="System.Net">
        <listeners>
            <add name="System.Net"/>
        </listeners>
    </source>
</sources>
<sharedListeners>
    <add name="System.Net"
        type="System.Diagnostics.TextWriterTraceListener"
        initializeData="System.Net.trace.log" />
</sharedListeners>
<switches>
    <add name="System.Net" value="Verbose" />
</switches>
</system.diagnostics>

Now when you execute your client application you can go into the folder that your executable was run from and find the file System.Net.trace.log

You will then find in the log file your request and the servers response. The great thing about this solution is you do not have to install or run anything extra. However the solution is probably only a solution for developing or diagnosing something in test or stage environment rather than production. However I am assuming because you mention creating the solution in Visual Studio and it not working that you are clearly in the development stage.

ContentLicense: CC BY-SA 3.0


问:
Login page doesn't show in django all-auth
I am having a problem with django allauth login page. I was able to complete the tutorial django all-auth, however I encountered a problem with the login page when I tried http://localhost/accounts/login/ nothing is showing but only a blank page.

And the tutorial does not show how to create the login page, so I guess that is the last step I should be working on.

What I did until now is this,

settings.py

```<code>INSTALLED_APPS = (
   'django.contrib.sites',
   'allauth',
   'allauth.account',
   'allauth.socialaccount',
   'allauth.socialaccount.providers.google',
)

SOCIALACCOUNT_PROVIDERS = {'google': {'SCOPE': ['email'],
     'AUTH_PARAMS': {'access_type': 'online'}
 }
}

# Django all auth settings
AUTHENTICATION_BACKENDS = (
    # Needed to login by username in Django admin, regardless of `allauth`
    'django.contrib.auth.backends.ModelBackend',

    # `allauth` specific authentication methods, such as login by e-mail
    'allauth.account.auth_backends.AuthenticationBackend',
)

SITE_ID = 1
</code>```

urls.py

```<code>url(r'^accounts/', include('allauth.urls')),
</code>```

Also I already added a social application in the Django admin and it's required fields.

Thank you soo much, your help is much appreciated.

答:
Piggybacking off of @bharat's post and for others stumbling across this post trying to figure out why nothing is being displayed...

django-allauth comes with templates that have all of the content that you already need and even a few templates that you may have not added yourself. But, as comes with the templates, comes the project-owners project structure.

If you want all of your django-allauth pages to use the content that it comes with, you will need to copy and paste the templates into your template folder and then update them to work with your project structure...

So, you will need to do to the typical installation process and then add all of the templates as well. I just can't find this documented anywhere. You can see all of the templates here.

EDIT: If you look closely at the top of the templates page it says 'Overridable templates' which clearly answers the issue that I had for hours. But, if you missed that as I did. There's your issue.

If you're not planning on using the django-allauth views though, you don't have to. You can totally implement social-authentication without using all of the views. So, it's up to you... But, once you implement them you will realize that it does save you an incredible amount of time in setting your future projects up. It's just a matter of getting it working the first time.

There are likely a couple of things you will have to change in the templates for everything to work.

Update the <title> block that is {% block head_title %}.

Update the {% block content %} to whatever you call your body block.

Import django-crispy-forms with {% load crispy_forms_tags %}.

Make all of your forms crispy.

If you're using all of the django-allauth views. Chances are you want an account view so you will need to create that as well because that does not come in the views.

But, you can totally just use the same pattern that django-allauth does and create something like accounts/<username>/

Body:
I am having a problem with django allauth login page. I was able to complete the tutorial django all-auth, however I encountered a problem with the login page when I tried http://localhost/accounts/login/ nothing is showing but only a blank page.

And the tutorial does not show how to create the login page, so I guess that is the last step I should be working on.

What I did until now is this,

settings.py

INSTALLED_APPS = (
   'django.contrib.sites',
   'allauth',
   'allauth.account',
   'allauth.socialaccount',
   'allauth.socialaccount.providers.google',
)

SOCIALACCOUNT_PROVIDERS = {'google': {'SCOPE': ['email'],
     'AUTH_PARAMS': {'access_type': 'online'}
 }
}

# Django all auth settings
AUTHENTICATION_BACKENDS = (
    # Needed to login by username in Django admin, regardless of `allauth`
    'django.contrib.auth.backends.ModelBackend',

    # `allauth` specific authentication methods, such as login by e-mail
    'allauth.account.auth_backends.AuthenticationBackend',
)

SITE_ID = 1

urls.py

url(r'^accounts/', include('allauth.urls')),

Also I already added a social application in the Django admin and it's required fields.

Thank you soo much, your help is much appreciated.


问:
CD in to a directory path with illegal chars that is stored in a variable
I have situation. It is about applying a cd command to a path of a directory that I will receive as an argument from a command line and pass it in to variable inside a script. I will however explain the gist of my problem below in a few steps. This is really silly as there is something here that I am not seeing even though my eyes are wide open.

~> cd /home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/
```

<p>Result:</p>

<pre class="lang-none prettyprint-override"><code>~/Exercise_Files/FOO - 2005 _ CONT _ BAR _ 20190527; This - is - crap>
</code>```

The previous execution works with that directory name that has all sorts of stuff to mess up the shell while it expands the directory name in to all sorts of god awful interpretations.

The problem is explained below:

when I do:

~> c="/home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/"
```

<p>and I do an echo:</p>

<pre class="lang-none prettyprint-override"><code>~> echo $c
/home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/
</code>```

I get everything as it should with the illegal chars properly escaped and stored in to the variable c;

but then if I do this:

~> cd "$c"
-bash: cd: /home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/: No such file or directory
```

I get an error.

Can anyone please shed some light in to what I can do to resolve this problem? In fact I also have to use this procedure inside an rsync too. When i use this variable passing method in there, rsync also throws an error when it tries to change the directory for the purpose of syncing files but the problem that I explained above is the core one that is messing up my script.

答: The problem is that you're essentially double-escaping. If you need to cd to a folder called foo bar, then either cd foo\ bar or cd "foo bar" will work, but cd "foo\ bar" (the equivalent of what you're doing) will not, as it's actually looking for a folder called foo\ bar.

Body:
I have situation. It is about applying a cd command to a path of a directory that I will receive as an argument from a command line and pass it in to variable inside a script. I will however explain the gist of my problem below in a few steps. This is really silly as there is something here that I am not seeing even though my eyes are wide open.

~> cd /home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/

Result:

~/Exercise_Files/FOO - 2005 _ CONT _ BAR _ 20190527; This - is - crap>

The previous execution works with that directory name that has all sorts of stuff to mess up the shell while it expands the directory name in to all sorts of god awful interpretations.

The problem is explained below:

when I do:

~> c="/home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/"

and I do an echo:

~> echo $c
/home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/

I get everything as it should with the illegal chars properly escaped and stored in to the variable c;

but then if I do this:

~> cd "$c"
-bash: cd: /home/av/Exercise_Files/FOO\ -\ 2005\ _\ CONT\ _\ BAR\ _\ 20190527\;\ This\ -\ is\ -\ crap/: No such file or directory

I get an error.

Can anyone please shed some light in to what I can do to resolve this problem? In fact I also have to use this procedure inside an rsync too. When i use this variable passing method in there, rsync also throws an error when it tries to change the directory for the purpose of syncing files but the problem that I explained above is the core one that is messing up my script.


问:
Plotting scatterplots from a dataframe on a grid with matplotlib
Is there a way to do a grid with scatterplots from all columns from a dataframe, where Y is one of the dataframe columns?

I can do a for loop on either matplotlib or seabornfor this (see codes below), but I can't make them show on a grid.

I want them to be displayed in grid visualization to make it easier to compare them.

This is what I CAN do:

```<code>for col in boston_df:
    plt.scatter(boston_df[col], boston_df["MEDV"], c="red", label=col)
    plt.ylabel("medv")
    plt.legend()
    plt.show()
</code>```

or

```<code>for col in boston_df:
    sns.regplot(x=boston_df[col], y=boston_df["MEDV"])
    plt.show()
</code>```

Now if I try to create a subplot for example and use ax.scatter() in my loop like this

```<code>fig, ax = plt.subplots(3, 5,figsize=(16,6))
for col in boston_df:
    ax.scatter(boston_df[col], boston_df["MEDV"], c="red", label=col)
    plt.ylabel("medv")
    plt.legend()
    plt.show()
</code>```

it gives me the error AttributeError: 'numpy.ndarray' object has no attribute 'scatter'

It would be beautiful to find some solution simple like this:

```<code>df.hist(figsize=(18,10), density=True, label=df.columns)
plt.show()
</code>```

答:
Consider using the ax argument of pandas DataFrame.plot and seaborn's regplot:

```<code>fig, ax = plt.subplots(1, 5, figsize=(16,6))

for i,col in enumerate(boston_df.columns[1:]):
     #boston_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     sns.regplot(x=boston_df[col], y=boston_df["MEDV"], ax=ax[i])

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)      # TO ACCOMMODATE TITLE

plt.show()
</code>```

To demonstrate with random data:

```<code>import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

### DATA BUILD
np.random.seed(6012019)
random_df = pd.DataFrame(np.random.randn(50,6),
                         columns = ['MEDV', 'COL1', 'COL2', 'COL3', 'COL4', 'COL5'])

### PLOT BUILD
fig, ax = plt.subplots(1, 5, figsize=(16,6))

for i,col in enumerate(random_df.columns[1:]):
     #random_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[i])

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)

plt.show()
plt.clf()
plt.close()
</code>```

For multiple rows across multiple columns, adjust the assignment to ax which is a numpy array using indexes: ax[row_idx, col_idx].

```<code>import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

### DATA BUILD
np.random.seed(6012019)
random_df = pd.DataFrame(np.random.randn(50,14),
                         columns = ['MEDV', 'COL1', 'COL2', 'COL3', 'COL4',
                                    'COL5', 'COL6', 'COL7', 'COL8', 'COl9',
                                    'COL10', 'COL11', 'COL12', 'COL13'])

### PLOT BUILD
fig, ax = plt.subplots(2, 7, figsize=(16,6))

for i,col in enumerate(random_df.columns[1:]):
     #random_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     if i <= 6:
        sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[0,i])
     else:
        sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[1,i-7])

ax[1,6].axis('off')                  # HIDES AXES ON LAST ROW AND COL

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)

plt.show()
plt.clf()
plt.close()
</code>```

Body:
Is there a way to do a grid with scatterplots from all columns from a dataframe, where Y is one of the dataframe columns?

I can do a for loop on either matplotlib or seabornfor this (see codes below), but I can't make them show on a grid.

I want them to be displayed in grid visualization to make it easier to compare them.

This is what I CAN do:

for col in boston_df:
    plt.scatter(boston_df[col], boston_df["MEDV"], c="red", label=col)
    plt.ylabel("medv")
    plt.legend()
    plt.show()

or

for col in boston_df:
    sns.regplot(x=boston_df[col], y=boston_df["MEDV"])
    plt.show()

Now if I try to create a subplot for example and use ax.scatter() in my loop like this

fig, ax = plt.subplots(3, 5,figsize=(16,6))
for col in boston_df:
    ax.scatter(boston_df[col], boston_df["MEDV"], c="red", label=col)
    plt.ylabel("medv")
    plt.legend()
    plt.show()

it gives me the error AttributeError: 'numpy.ndarray' object has no attribute 'scatter'

It would be beautiful to find some solution simple like this:

df.hist(figsize=(18,10), density=True, label=df.columns)
plt.show()

Body:
Consider using the ax argument of pandas DataFrame.plot and seaborn's regplot:

fig, ax = plt.subplots(1, 5, figsize=(16,6))

for i,col in enumerate(boston_df.columns[1:]):
     #boston_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     sns.regplot(x=boston_df[col], y=boston_df["MEDV"], ax=ax[i])

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)      # TO ACCOMMODATE TITLE

plt.show()

To demonstrate with random data:

import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

### DATA BUILD
np.random.seed(6012019)
random_df = pd.DataFrame(np.random.randn(50,6),
                         columns = ['MEDV', 'COL1', 'COL2', 'COL3', 'COL4', 'COL5'])

### PLOT BUILD
fig, ax = plt.subplots(1, 5, figsize=(16,6))

for i,col in enumerate(random_df.columns[1:]):
     #random_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[i])

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)

plt.show()
plt.clf()
plt.close()

For multiple rows across multiple columns, adjust the assignment to ax which is a numpy array using indexes: ax[row_idx, col_idx].

import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

### DATA BUILD
np.random.seed(6012019)
random_df = pd.DataFrame(np.random.randn(50,14),
                         columns = ['MEDV', 'COL1', 'COL2', 'COL3', 'COL4',
                                    'COL5', 'COL6', 'COL7', 'COL8', 'COl9',
                                    'COL10', 'COL11', 'COL12', 'COL13'])

### PLOT BUILD
fig, ax = plt.subplots(2, 7, figsize=(16,6))

for i,col in enumerate(random_df.columns[1:]):
     #random_df.plot(kind='scatter', x=col, y='MEDV', ax=ax[i])
     if i <= 6:
        sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[0,i])
     else:
        sns.regplot(x=random_df[col], y=random_df["MEDV"], ax=ax[1,i-7])

ax[1,6].axis('off')                  # HIDES AXES ON LAST ROW AND COL

fig.suptitle('My Scatter Plots')
fig.tight_layout()
fig.subplots_adjust(top=0.95)

plt.show()
plt.clf()
plt.close()


问:
Is it possible to provide type safety to method decorator options
I decided to write some utility decorators such as memoize, rateLimiter. I want to achieve as much type safety as possible without unnecessary boilerplate code.

Is it possible to ensure full type safety in decorators like that without manually specified generics?

type GET_FUNCTION_SIGNATURE<
  T extends TypedPropertyDescriptor<any>
> = T extends TypedPropertyDescriptor<infer U> ? U : never;

interface ITestDecoratorOptions<DECORATED_FUNCTION_ARGUMENTS_TYPE, DECORATED_FUNCTION_RETURN_TYPE> {
  getKeyFromArgs: (args: DECORATED_FUNCTION_ARGUMENTS_TYPE) => string;
  getDefaultValue: (args: DECORATED_FUNCTION_ARGUMENTS_TYPE) => DECORATED_FUNCTION_RETURN_TYPE;
}

const testDecorator = <TYPED_PROPERTY_DESCRIPTOR extends TypedPropertyDescriptor<any>>(
  options: ITestDecoratorOptions<
    Parameters<GET_FUNCTION_SIGNATURE<TYPED_PROPERTY_DESCRIPTOR>>,
    ReturnType<GET_FUNCTION_SIGNATURE<TYPED_PROPERTY_DESCRIPTOR>>
  >
) => {
  return (
    target: Object,
    key: string,
    descriptor = Object.getOwnPropertyDescriptor(target, key) as PropertyDescriptor
  ): TYPED_PROPERTY_DESCRIPTOR => {
    return null as any;
  };
};

class Test {
  //             \/ Is it possible to remove that generic and keep full type safety here?
  @testDecorator<TypedPropertyDescriptor<(a: number, b: string) => boolean>>({
    getKeyFromArgs: args => {
          // number               string
      return args[0].toString() + args[1]; // full type checking
    },
    getDefaultValue: args => {
      // full type checking: on args(number, string) and return type(boolean)
      if (args[0] === 1) {
        return true;
      }

      return false;
    }
  })
  public someMethod(a: number, b: string): boolean {
    return true;
  }
}
```

答:
This is a known issue in TypeScript, with no obvious solution (other than manually specifying generic type parameters).

The problem with implementing this, as explained in this comment by @DanielRosenwasser is that using a decorators is like calling a curried function, and the kind of generic inference you want would be like this:

```<code>declare let f: <T>(callback: (x: T) => void) => (y: T) => void;
f(x => x.a)({ a: 100 }); // error!
//     ~ <-- T is inferred as {} or unknown,
</code>```

which doesn't work because TypeScript infers the generic type when the function f is invoked on its callback argument, and does not wait until the returned function is itself called.  So by the time the T type would actually be known be the compiler, it's too late and it's already failed to infer properly.

I don't know if I have any advice other than to continue to manually specify arguments, and maybe go to that issue in TypeScript and give it a  or describe your use case if you think it's more compelling than the other ones mentioned.  Good luck!

Body:
This is a known issue in TypeScript, with no obvious solution (other than manually specifying generic type parameters).

The problem with implementing this, as explained in this comment by @DanielRosenwasser is that using a decorators is like calling a curried function, and the kind of generic inference you want would be like this:

declare let f: <T>(callback: (x: T) => void) => (y: T) => void;
f(x => x.a)({ a: 100 }); // error!
//     ~ <-- T is inferred as {} or unknown,

which doesn't work because TypeScript infers the generic type when the function f is invoked on its callback argument, and does not wait until the returned function is itself called.  So by the time the T type would actually be known be the compiler, it's too late and it's already failed to infer properly.

I don't know if I have any advice other than to continue to manually specify arguments, and maybe go to that issue in TypeScript and give it a  or describe your use case if you think it's more compelling than the other ones mentioned.  Good luck!


问:
How to force web page to have same scale in all devices?
My web page need to be shown in an app. When it shows in the Android version, it is like 3 times larger than in iOS version.

It is probably due to the settings in each app's webview, but I don't made the app so I can't change anything about it. The only thing I can change is my web page, which is a javascript Google Map, the map marker became super large in the Android app.

So is there a thing that I can force my web page to have same scale regardless of the webview settings or app settings?

答:
As far as I know you cannot, your web page must be developed in the way to fit all screens by codes like -

vw/vh, % etc

media screens configurations

Your devices cannot control the contents of web page from outside unless they are designed so.


问:
The configuration file 'appsettings.json' was not found and is not optional
The Azure error is:

.Net Core: Application startup exception:
  System.IO.FileNotFoundException: The configuration file
  'appsettings.json' was not found and is not optional.

So this is a bit vague. I can't seem to nail this down. I'm trying to deploy a .Net Core Web API project to Azure, and I'm getting this error:

:(  Oops. 500 Internal Server Error
  An error occurred while starting the application.

I've deployed plain old .Net WebAPI's and they have worked. I've followed online tutorials and they have worked. But somehow my project is broke. Enabling stdoutLogEnabled on Web.config and looking at the Azure Streaming Logs gives me this:

```<code>2016-08-26T02:55:12  Welcome, you are now connected to log-streaming service.
Application startup exception: System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional.
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Quanta.API.Startup..ctor(IHostingEnvironment env) in D:\Source\Workspaces\Quanta\src\Quanta.API\Startup.cs:line 50
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
   at Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
   at Microsoft.Extensions.Internal.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type)
   at Microsoft.AspNetCore.Hosting.Internal.StartupLoader.LoadMethods(IServiceProvider services, Type startupType, String environmentName)
   at Microsoft.AspNetCore.Hosting.WebHostBuilderExtensions.<>c__DisplayClass1_0.<UseStartup>b__1(IServiceProvider sp)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.FactoryService.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.SingletonCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureStartup()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
Hosting environment: Production
Content root path: D:\home\site\wwwroot
Now listening on: http://localhost:30261
Application started. Press Ctrl+C to shut down.
</code>```

Ok, that seems simple. It can't find appsettings.json. Looking at my config ( startup.cs ) it seems very well defined. My Startup looks like this:

public class Startup
{
    private static string _applicationPath = string.Empty;
    private static string _contentRootPath = string.Empty;
    public IConfigurationRoot Configuration { get; set; }
    public Startup(IHostingEnvironment env)
    {
        _applicationPath = env.WebRootPath;
        _contentRootPath = env.ContentRootPath;
        // Setup configuration sources.

        var builder = new ConfigurationBuilder()
            .SetBasePath(_contentRootPath)
            .AddJsonFile("appsettings.json")
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);

        if (env.IsDevelopment())
        {
            // This reads the configuration keys from the secret store.
            // For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
            builder.AddUserSecrets();
        }

        builder.AddEnvironmentVariables();
        Configuration = builder.Build();
    }
    private string GetXmlCommentsPath()
    {
        var app = PlatformServices.Default.Application;
        return System.IO.Path.Combine(app.ApplicationBasePath, "Quanta.API.xml");
    }

    // This method gets called by the runtime. Use this method to add services to the container.
    // For more information on how to configure your application, visit http://go.microsoft.com/fwlink/?LinkID=398940
    public void ConfigureServices(IServiceCollection services)
    {
        var pathToDoc = GetXmlCommentsPath();

        services.AddDbContext<QuantaContext>(options =>
            options.UseSqlServer(Configuration["Data:DefaultConnection:ConnectionString"],
            b => b.MigrationsAssembly("Quanta.API")));

        //Swagger
        services.AddSwaggerGen();
        services.ConfigureSwaggerGen(options =>
        {
            options.SingleApiVersion(new Info
            {
                Version = "v1",
                Title = "Project Quanta API",
                Description = "Quant.API",
                TermsOfService = "None"
            });
            options.IncludeXmlComments(pathToDoc);
            options.DescribeAllEnumsAsStrings();
        });

        // Repositories
        services.AddScoped<ICheckListRepository, CheckListRepository>();
        services.AddScoped<ICheckListItemRepository, CheckListItemRepository>();
        services.AddScoped<IClientRepository, ClientRepository>();
        services.AddScoped<IDocumentRepository, DocumentRepository>();
        services.AddScoped<IDocumentTypeRepository, DocumentTypeRepository>();
        services.AddScoped<IProjectRepository, ProjectRepository>();
        services.AddScoped<IProtocolRepository, ProtocolRepository>();
        services.AddScoped<IReviewRecordRepository, ReviewRecordRepository>();
        services.AddScoped<IReviewSetRepository, ReviewSetRepository>();
        services.AddScoped<ISiteRepository, SiteRepository>();

        // Automapper Configuration
        AutoMapperConfiguration.Configure();

        // Enable Cors
        services.AddCors();

        // Add MVC services to the services container.
        services.AddMvc()
            .AddJsonOptions(opts =>
            {
                // Force Camel Case to JSON
                opts.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
            });
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app)
    {
        app.UseStaticFiles();
        // Add MVC to the request pipeline.
        app.UseCors(builder =>
            builder.AllowAnyOrigin()
            .AllowAnyHeader()
            .AllowAnyMethod());

        app.UseExceptionHandler(
          builder =>
          {
              builder.Run(
                async context =>
                {
                    context.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
                    context.Response.Headers.Add("Access-Control-Allow-Origin", "*");

                    var error = context.Features.Get<IExceptionHandlerFeature>();
                    if (error != null)
                    {
                        context.Response.AddApplicationError(error.Error.Message);
                        await context.Response.WriteAsync(error.Error.Message).ConfigureAwait(false);
                    }
                });
          });

        app.UseMvc(routes =>
        {
            routes.MapRoute(
                name: "default",
                template: "{controller=Home}/{action=Index}/{id?}");

            // Uncomment the following line to add a route for porting Web API 2 controllers.
            //routes.MapWebApiRoute("DefaultApi", "api/{controller}/{id?}");
        });

        //Ensure DB is created, and latest migration applied. Then seed.
        using (var serviceScope = app.ApplicationServices
          .GetRequiredService<IServiceScopeFactory>()
          .CreateScope())
        {
            QuantaContext dbContext = serviceScope.ServiceProvider.GetService<QuantaContext>();
            dbContext.Database.Migrate();
            QuantaDbInitializer.Initialize(dbContext);
        }

        app.UseSwagger();
        app.UseSwaggerUi();

    }
}
```

<p>This works fine locally. But once we publish to Azure, this fails. I'm at a loss. I've created new .Net core project that deploy to Azure just find. But this one project, that I put all my time into to, seems to fail. I'm about ready to copy and paste code out of the project that fails to run and into a new project, but i'm really curious on what's breaking this.</p>

<p>Any ideas?</p>

<p>EDIT:
So my Program.cs was:</p>

<pre class="lang-cs prettyprint-override"><code>using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;

namespace Quanta.API
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var host = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseIISIntegration()
                .UseStartup<Startup>()
                .Build();

            host.Run();
        }
    }
}
</code>```

Edit2:
Per Frans, I checked the publishOptions. It was:

```<code>"publishOptions": {
"include": [
  "wwwroot",
  "web.config"
]
</code>```

I took a publishOptions from a working project and changed it to:

```<code> "publishOptions": {
  "include": [
    "wwwroot",
    "Views",
    "Areas/**/Views",
    "appsettings.json",
    "web.config"
  ]
  },
</code>```

It still gave a 500 error, but it didn't give a stack trace saying it coulding load appsettings.json. Now it was complaining about a connection to SQL. I noticed that my SQL connection string code is mentioned in a lot of RC1 blog posts. RC2 of .Net Core changed it. So I updated it to:

```<code>  "Data": {
    "ConnectionStrings": {
      "DefaultConnection": "Server=(localdb)\\MSSQLLocalDB;Database=QuantaDb;Trusted_Connection=True;MultipleActiveResultSets=true"
    }
  },
</code>```

And changed my startup to:

 services.AddDbContext<QuantaContext>(options =>
        options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"),
        b => b.MigrationsAssembly("Quanta.API")));
```

Finally, it worked.

I must have followed an older RC1 example and not realized it.

答:
Update:

From @bunjeeb in comments. Use System.AppContext.BaseDirectory instead of Assembly.GetEntryAssembly().Location for .NET 5 <= with <PublishSingleFile>true</PublishSingleFile>.

Original:

For me the error was using Directory.GetCurrentDirectory(). This worked fine running locally but on a production server it failed when the program was started from Powershell. Replaced with Assembly.GetEntryAssembly().Location and everything worked.

Complete code:

```<code>var builder = new ConfigurationBuilder()
        .SetBasePath(Path.GetDirectoryName(Assembly.GetEntryAssembly().Location))
        .AddJsonFile("appsettings.json");

var configuration = builder.Build();
</code>```

Body:
The Azure error is:

.Net Core: Application startup exception:
  System.IO.FileNotFoundException: The configuration file
  'appsettings.json' was not found and is not optional.

So this is a bit vague. I can't seem to nail this down. I'm trying to deploy a .Net Core Web API project to Azure, and I'm getting this error:

:(  Oops. 500 Internal Server Error
  An error occurred while starting the application.

I've deployed plain old .Net WebAPI's and they have worked. I've followed online tutorials and they have worked. But somehow my project is broke. Enabling stdoutLogEnabled on Web.config and looking at the Azure Streaming Logs gives me this:

2016-08-26T02:55:12  Welcome, you are now connected to log-streaming service.
Application startup exception: System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional.
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Quanta.API.Startup..ctor(IHostingEnvironment env) in D:\Source\Workspaces\Quanta\src\Quanta.API\Startup.cs:line 50
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
   at Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
   at Microsoft.Extensions.Internal.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type)
   at Microsoft.AspNetCore.Hosting.Internal.StartupLoader.LoadMethods(IServiceProvider services, Type startupType, String environmentName)
   at Microsoft.AspNetCore.Hosting.WebHostBuilderExtensions.<>c__DisplayClass1_0.<UseStartup>b__1(IServiceProvider sp)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.FactoryService.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.SingletonCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureStartup()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
Hosting environment: Production
Content root path: D:\home\site\wwwroot
Now listening on: http://localhost:30261
Application started. Press Ctrl+C to shut down.

Ok, that seems simple. It can't find appsettings.json. Looking at my config ( startup.cs ) it seems very well defined. My Startup looks like this:

public class Startup
{
    private static string _applicationPath = string.Empty;
    private static string _contentRootPath = string.Empty;
    public IConfigurationRoot Configuration { get; set; }
    public Startup(IHostingEnvironment env)
    {
        _applicationPath = env.WebRootPath;
        _contentRootPath = env.ContentRootPath;
        // Setup configuration sources.

        var builder = new ConfigurationBuilder()
            .SetBasePath(_contentRootPath)
            .AddJsonFile("appsettings.json")
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);

        if (env.IsDevelopment())
        {
            // This reads the configuration keys from the secret store.
            // For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
            builder.AddUserSecrets();
        }

        builder.AddEnvironmentVariables();
        Configuration = builder.Build();
    }
    private string GetXmlCommentsPath()
    {
        var app = PlatformServices.Default.Application;
        return System.IO.Path.Combine(app.ApplicationBasePath, "Quanta.API.xml");
    }

    // This method gets called by the runtime. Use this method to add services to the container.
    // For more information on how to configure your application, visit http://go.microsoft.com/fwlink/?LinkID=398940
    public void ConfigureServices(IServiceCollection services)
    {
        var pathToDoc = GetXmlCommentsPath();

        services.AddDbContext<QuantaContext>(options =>
            options.UseSqlServer(Configuration["Data:DefaultConnection:ConnectionString"],
            b => b.MigrationsAssembly("Quanta.API")));

        //Swagger
        services.AddSwaggerGen();
        services.ConfigureSwaggerGen(options =>
        {
            options.SingleApiVersion(new Info
            {
                Version = "v1",
                Title = "Project Quanta API",
                Description = "Quant.API",
                TermsOfService = "None"
            });
            options.IncludeXmlComments(pathToDoc);
            options.DescribeAllEnumsAsStrings();
        });

        // Repositories
        services.AddScoped<ICheckListRepository, CheckListRepository>();
        services.AddScoped<ICheckListItemRepository, CheckListItemRepository>();
        services.AddScoped<IClientRepository, ClientRepository>();
        services.AddScoped<IDocumentRepository, DocumentRepository>();
        services.AddScoped<IDocumentTypeRepository, DocumentTypeRepository>();
        services.AddScoped<IProjectRepository, ProjectRepository>();
        services.AddScoped<IProtocolRepository, ProtocolRepository>();
        services.AddScoped<IReviewRecordRepository, ReviewRecordRepository>();
        services.AddScoped<IReviewSetRepository, ReviewSetRepository>();
        services.AddScoped<ISiteRepository, SiteRepository>();

        // Automapper Configuration
        AutoMapperConfiguration.Configure();

        // Enable Cors
        services.AddCors();

        // Add MVC services to the services container.
        services.AddMvc()
            .AddJsonOptions(opts =>
            {
                // Force Camel Case to JSON
                opts.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
            });
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app)
    {
        app.UseStaticFiles();
        // Add MVC to the request pipeline.
        app.UseCors(builder =>
            builder.AllowAnyOrigin()
            .AllowAnyHeader()
            .AllowAnyMethod());

        app.UseExceptionHandler(
          builder =>
          {
              builder.Run(
                async context =>
                {
                    context.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
                    context.Response.Headers.Add("Access-Control-Allow-Origin", "*");

                    var error = context.Features.Get<IExceptionHandlerFeature>();
                    if (error != null)
                    {
                        context.Response.AddApplicationError(error.Error.Message);
                        await context.Response.WriteAsync(error.Error.Message).ConfigureAwait(false);
                    }
                });
          });

        app.UseMvc(routes =>
        {
            routes.MapRoute(
                name: "default",
                template: "{controller=Home}/{action=Index}/{id?}");

            // Uncomment the following line to add a route for porting Web API 2 controllers.
            //routes.MapWebApiRoute("DefaultApi", "api/{controller}/{id?}");
        });

        //Ensure DB is created, and latest migration applied. Then seed.
        using (var serviceScope = app.ApplicationServices
          .GetRequiredService<IServiceScopeFactory>()
          .CreateScope())
        {
            QuantaContext dbContext = serviceScope.ServiceProvider.GetService<QuantaContext>();
            dbContext.Database.Migrate();
            QuantaDbInitializer.Initialize(dbContext);
        }

        app.UseSwagger();
        app.UseSwaggerUi();

    }
}

This works fine locally. But once we publish to Azure, this fails. I'm at a loss. I've created new .Net core project that deploy to Azure just find. But this one project, that I put all my time into to, seems to fail. I'm about ready to copy and paste code out of the project that fails to run and into a new project, but i'm really curious on what's breaking this.

Any ideas?

EDIT:
So my Program.cs was:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;

namespace Quanta.API
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var host = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseIISIntegration()
                .UseStartup<Startup>()
                .Build();

            host.Run();
        }
    }
}

Edit2:
Per Frans, I checked the publishOptions. It was:

"publishOptions": {
"include": [
  "wwwroot",
  "web.config"
]

I took a publishOptions from a working project and changed it to:

 "publishOptions": {
  "include": [
    "wwwroot",
    "Views",
    "Areas/**/Views",
    "appsettings.json",
    "web.config"
  ]
  },

It still gave a 500 error, but it didn't give a stack trace saying it coulding load appsettings.json. Now it was complaining about a connection to SQL. I noticed that my SQL connection string code is mentioned in a lot of RC1 blog posts. RC2 of .Net Core changed it. So I updated it to:

  "Data": {
    "ConnectionStrings": {
      "DefaultConnection": "Server=(localdb)\\MSSQLLocalDB;Database=QuantaDb;Trusted_Connection=True;MultipleActiveResultSets=true"
    }
  },

And changed my startup to:

 services.AddDbContext<QuantaContext>(options =>
        options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"),
        b => b.MigrationsAssembly("Quanta.API")));

Finally, it worked.

I must have followed an older RC1 example and not realized it.

Body:
Update:

From @bunjeeb in comments. Use System.AppContext.BaseDirectory instead of Assembly.GetEntryAssembly().Location for .NET 5 <= with <PublishSingleFile>true</PublishSingleFile>.

Original:

For me the error was using Directory.GetCurrentDirectory(). This worked fine running locally but on a production server it failed when the program was started from Powershell. Replaced with Assembly.GetEntryAssembly().Location and everything worked.

Complete code:

var builder = new ConfigurationBuilder()
        .SetBasePath(Path.GetDirectoryName(Assembly.GetEntryAssembly().Location))
        .AddJsonFile("appsettings.json");

var configuration = builder.Build();


问:
Casting a column to JSON/dict and flattening JSON values in a column in pyspark
I am new to Pyspark and I am figuring out how to cast a column type to dict type and then flatten that column to multiple columns using explode.

Here's how my dataframe looks like:

```<code>   col1    | col2        |
    -----------------------
    test:1  | {"test1":[{"Id":"17","cName":"c1"},{"Id":"01","cName":"c2","pScore":0.003609}],
{"test8":[{"Id":"1","cName":"c11","pScore":0.0},{"Id":"012","cName":"c2","pScore":0.003609}]
    test:2  | {"test1:subtest2":[{"Id":"18","cName":"c13","pScore":0.00203}]}
</code>```

Right now, the schema of this dataframe is

```<code>root
 |-- col1: string (nullable = true)
 |-- col2: string (nullable = true)
</code>```

The output I am looking to have is like this:

```<code>col1   | col2           | Id | cName | pScore  |
------------------------------------------------
test:1 | test1          | 17 | c1    | null    |
test:1 | test1          | 01 | c2    | 0.003609|
test:1 | test8          | 1  | c11   | 0.0     |
test:1 | test8          | 012| c2    | 0.003609|
test:2 | test1:subtest2 | 18 | c13   | 0.00203 |
</code>```

I am having trouble defining the right schema for col2 to cast its type from String to json or dict. And then, I would like to be able to explode the values to multiple columns as shown above. Any help would be greatly appreciated. I am using Spark 2.0 + .

Thank you!

答:
Because of the different key name for each row in the JSON, defining a general schema for the json is not going to work well, I believe it's better to handle this via UDFs:

```<code>import pyspark.sql.functions as f
import pyspark.sql.types as t
from pyspark.sql import Row
import json

def extract_key(dumped_json):
    """
    Extracts the single key from the dumped json (as a string).
    """
    if dumped_json is None:
        return None

    d = json.loads(dumped_json)

    try:
        return list(d.keys())[0]
    except IndexError:
        return None

def extract_values(dumped_json):
    """
    Extracts the single array value from the dumped json and parses each element
    of the array as a spark Row.
    """
    if dumped_json is None:
        return None

    d = json.loads(dumped_json)
    try:
        return [Row(**_d) for _d in list(d.values())[0]]
    except IndexError:
        return None

# Definition of the output type of the `extract_values` function
output_values_type = t.ArrayType(t.StructType(
 [t.StructField("Id", t.StringType()),
  t.StructField("cName", t.StringType()),
  t.StructField("pScore", t.DoubleType())]
))

# Define UDFs
extract_key_udf = f.udf(extract_key, t.StringType())
extract_values_udf = f.udf(extract_values, output_values_type)

# Extract values and keys
extracted_df = df.withColumn("values", extract_values_udf("col2")). \
 withColumn("col2", extract_key_udf("col2"))
# Explode the array
exploded_df = extracted_df.withColumn("values", f.explode("values"))

# Select the wanted columns
final_df = exploded_df.select("col1", "col2", "values.Id", "values.cName",
                              "values.pScore")
</code>```

The result is then as wanted:

```<code>+------+--------------+---+-----+--------+
|col1  |col2          |Id |cName|pScore  |
+------+--------------+---+-----+--------+
|test:1|test1:subtest1|17 |c1   |0.002034|
|test:1|test1:subtest1|01 |c2   |0.003609|
|test:2|test1:subtest2|18 |c13  |0.00203 |
+------+--------------+---+-----+--------+
</code>```

Body:
I am new to Pyspark and I am figuring out how to cast a column type to dict type and then flatten that column to multiple columns using explode.

Here's how my dataframe looks like:

   col1    | col2        |
    -----------------------
    test:1  | {"test1":[{"Id":"17","cName":"c1"},{"Id":"01","cName":"c2","pScore":0.003609}],
{"test8":[{"Id":"1","cName":"c11","pScore":0.0},{"Id":"012","cName":"c2","pScore":0.003609}]
    test:2  | {"test1:subtest2":[{"Id":"18","cName":"c13","pScore":0.00203}]}

Right now, the schema of this dataframe is

root
 |-- col1: string (nullable = true)
 |-- col2: string (nullable = true)

The output I am looking to have is like this:

col1   | col2           | Id | cName | pScore  |
------------------------------------------------
test:1 | test1          | 17 | c1    | null    |
test:1 | test1          | 01 | c2    | 0.003609|
test:1 | test8          | 1  | c11   | 0.0     |
test:1 | test8          | 012| c2    | 0.003609|
test:2 | test1:subtest2 | 18 | c13   | 0.00203 |

I am having trouble defining the right schema for col2 to cast its type from String to json or dict. And then, I would like to be able to explode the values to multiple columns as shown above. Any help would be greatly appreciated. I am using Spark 2.0 + .

Thank you!

Body:
Because of the different key name for each row in the JSON, defining a general schema for the json is not going to work well, I believe it's better to handle this via UDFs:

import pyspark.sql.functions as f
import pyspark.sql.types as t
from pyspark.sql import Row
import json

def extract_key(dumped_json):
    """
    Extracts the single key from the dumped json (as a string).
    """
    if dumped_json is None:
        return None

    d = json.loads(dumped_json)

    try:
        return list(d.keys())[0]
    except IndexError:
        return None

def extract_values(dumped_json):
    """
    Extracts the single array value from the dumped json and parses each element
    of the array as a spark Row.
    """
    if dumped_json is None:
        return None

    d = json.loads(dumped_json)
    try:
        return [Row(**_d) for _d in list(d.values())[0]]
    except IndexError:
        return None

# Definition of the output type of the `extract_values` function
output_values_type = t.ArrayType(t.StructType(
 [t.StructField("Id", t.StringType()),
  t.StructField("cName", t.StringType()),
  t.StructField("pScore", t.DoubleType())]
))

# Define UDFs
extract_key_udf = f.udf(extract_key, t.StringType())
extract_values_udf = f.udf(extract_values, output_values_type)

# Extract values and keys
extracted_df = df.withColumn("values", extract_values_udf("col2")). \
 withColumn("col2", extract_key_udf("col2"))
# Explode the array
exploded_df = extracted_df.withColumn("values", f.explode("values"))

# Select the wanted columns
final_df = exploded_df.select("col1", "col2", "values.Id", "values.cName",
                              "values.pScore")

The result is then as wanted:

+------+--------------+---+-----+--------+
|col1  |col2          |Id |cName|pScore  |
+------+--------------+---+-----+--------+
|test:1|test1:subtest1|17 |c1   |0.002034|
|test:1|test1:subtest1|01 |c2   |0.003609|
|test:2|test1:subtest2|18 |c13  |0.00203 |
+------+--------------+---+-----+--------+


问:
Passing Jquery DataTables instance to onsubmit function
I have a master details page, i have created a javascript function in an external file that is always called upon submitting any form, this function sends an ajax post call to the corresponding ASP.NET MVC controller, and upon success the controller returns a json object to the SubmitForm() function which in turn reloads the datatable instance and displays a notification, it has worked fine, but now that i need to display multiple datatables in the same page dynamically i always get an error (Cannot read property 'reload' of undefined)

I tried adding an optional parameter in the SubmitForm(form, id='') function, so in case my form returns an id, it reloads the corresponding datatable but it gave me that same error.

```<code>//JS function in file.js
function SubmitForm(form, id = '') {
$.validator.unobtrusive.parse(form);
if ($(form).valid()) {
    $.ajax({
        type: "POST",
        url: form.action,
        data: $(form).serialize(),
        success: function (data) {
            if (data.success) {
                //global variable that is always present in all pages
                datatable.ajax.reload();
                if (id) {
                    var tbl = $(id).dataTable();
                    tbl.ajax.reload();
                }
                $(form)[0].reset();
                $.notify(data.message, { globalPosition: "top center", className: "success" });
                if (data.type != "Add") {
                    $("#myModal").modal('hide');
                }
            }
        }
    });
}
return false;
}
  //MVC FORM in partial view
@using (Html.BeginForm("AddOrEditSE", "ProjectStaff", FormMethod.Post, new
{ id = "form", onsubmit = "return SubmitForm(this,'#SE')" }))
{
    // elements
}
</code>```

Expected result:

Notify.js success message, and a reload of the corresponding datatable
  (in this case #SE)

Actual Result:

Error Cannot read property 'reload' of undefined

答:
I managed to solve the problem after some digging, i defined my datatable instant in a variable and in the form onsubmit property i sent that variable name, in the if statement i accessed the datatable API by using variablename.DataTable().ajax.reload();

```<code>function SubmitForm(form, tbl = null) {
$.validator.unobtrusive.parse(form);
if ($(form).valid()) {
    $.ajax({
        type: "POST",
        url: form.action,
        data: $(form).serialize(),
        success: function (data) {
            if (data.success) {
                datatable.ajax.reload();
                if (tbl) {
                    //to access datatable instance, use varname.DataTable().
                    tbl.DataTable().ajax.reload();
                }
                $(form)[0].reset();
                $.notify(data.message, { globalPosition: "top center", className: "success" });
                if (data.type != "Add") {
                    $("#myModal").modal('hide');
                }
            }
        }
    });
}
return false;
}
</code>```

Form partial view

```<code>@using (Html.BeginForm("AddOrEditSE", "ProjectStaff", FormMethod.Post, new { id = "form", onsubmit = "return SubmitForm(this,SETable)" }))
{
      // elements
}
</code>```

index view

```<code>  var SETable;
  SETable = $("#SE").dataTable({ //parameters });
</code>```

Body:
I have a master details page, i have created a javascript function in an external file that is always called upon submitting any form, this function sends an ajax post call to the corresponding ASP.NET MVC controller, and upon success the controller returns a json object to the SubmitForm() function which in turn reloads the datatable instance and displays a notification, it has worked fine, but now that i need to display multiple datatables in the same page dynamically i always get an error (Cannot read property 'reload' of undefined)

I tried adding an optional parameter in the SubmitForm(form, id='') function, so in case my form returns an id, it reloads the corresponding datatable but it gave me that same error.

//JS function in file.js
function SubmitForm(form, id = '') {
$.validator.unobtrusive.parse(form);
if ($(form).valid()) {
    $.ajax({
        type: "POST",
        url: form.action,
        data: $(form).serialize(),
        success: function (data) {
            if (data.success) {
                //global variable that is always present in all pages
                datatable.ajax.reload();
                if (id) {
                    var tbl = $(id).dataTable();
                    tbl.ajax.reload();
                }
                $(form)[0].reset();
                $.notify(data.message, { globalPosition: "top center", className: "success" });
                if (data.type != "Add") {
                    $("#myModal").modal('hide');
                }
            }
        }
    });
}
return false;
}
  //MVC FORM in partial view
@using (Html.BeginForm("AddOrEditSE", "ProjectStaff", FormMethod.Post, new
{ id = "form", onsubmit = "return SubmitForm(this,'#SE')" }))
{
    // elements
}

Expected result:

Notify.js success message, and a reload of the corresponding datatable
  (in this case #SE)

Actual Result:

Error Cannot read property 'reload' of undefined

Body:
I managed to solve the problem after some digging, i defined my datatable instant in a variable and in the form onsubmit property i sent that variable name, in the if statement i accessed the datatable API by using variablename.DataTable().ajax.reload();

function SubmitForm(form, tbl = null) {
$.validator.unobtrusive.parse(form);
if ($(form).valid()) {
    $.ajax({
        type: "POST",
        url: form.action,
        data: $(form).serialize(),
        success: function (data) {
            if (data.success) {
                datatable.ajax.reload();
                if (tbl) {
                    //to access datatable instance, use varname.DataTable().
                    tbl.DataTable().ajax.reload();
                }
                $(form)[0].reset();
                $.notify(data.message, { globalPosition: "top center", className: "success" });
                if (data.type != "Add") {
                    $("#myModal").modal('hide');
                }
            }
        }
    });
}
return false;
}

Form partial view

@using (Html.BeginForm("AddOrEditSE", "ProjectStaff", FormMethod.Post, new { id = "form", onsubmit = "return SubmitForm(this,SETable)" }))
{
      // elements
}

index view

  var SETable;
  SETable = $("#SE").dataTable({ //parameters });


问:
Terminology for function interface/DLL used only to fix parameter types between different languages?
I have several cases where I needed to make an interface library somewhat "dummy", meaning that it is just needed to make an application written in language A call a library written in language B, in order to fix data types and other small details, without adding functionalities.
Something like this:

Application in language A => Interface library => library in language B.

For example I had a case where A was Visual Basic, B was C#, and "interface library" written in Visual Basic.
Another case where A was LabView, B was C, and "interface library" written in C.

I don't remember if there is a standard name to call such an interface library in English language, I mean something like "shell library" or "dummy shell library", etc.
Any suggestion, even if not "standard", is welcome.

答:
Here are some of the terms used for mapping between two languages or APIs

language binding

glue code

foreign function interface

wrapper library

adapter

Personally, I often use the term glue code which wikipedia defines as

...code that serves solely to "adapt" different parts
  of code that would otherwise be incompatible. Glue code does not
  contribute any functionality towards meeting program requirements.
  Instead, it often appears in code that lets existing libraries or
  programs interoperate,...


问:
Trying the find a more semantic way for writing Spring Entity in Kotlin
I have an Spring entity written in Kotlin:

```<code>@Entity
class Book(
    @Id
    var id: Long? = null,
    var title: String? = null, // cannot actually be null, immutable
    var isInStock: Boolean? = null, // cannot actually be null
    var description: String? = null,
 )
</code>```

All the fields must be nullable, because Spring needs to initialize empty objects.

However, this makes using the entity complicated, because I always have to convert the nullable types to their not nullable equivalents. This is not semantic: I would like to really see, which fields can actually be nullable and which ones cannot (or can only be null during the initialization process).

Furthermore, some fields are mutable, but some should never be altered after the entity has been created. It would be nice to use Kotlin's val and var to make this distinction.

So, I would like to work with the following class:

```<code>class BetterBook(
    val id: Long,
    val title: String,
    var isInStock: Boolean,
    var description: String? = null,
 )
</code>```

This way it is very clear which fields are mutable and which fields can be null.

I pondered upon creating a wrapper around Book. Has anybody had a similar thought? Would it be a good solution architecturally?

答:
I am using JHipster (or KHipster, its Kotlin blueprint). Because all the entities are generated by KHipster, I do not want to change these files, for all my changes would be overridden when I regenerate the entity. After experimenting with different solutions I decided to add an extra wrapper layer.

A wrapper is like a PR manager for the entity. It can be constructed from the entity, then it can be modified in the service layer and once it is time to save it, the wrapper will give you back the updated entity.

This is how a KHipster entity approximately looks like:

```<code>@Entity
class Agreement(
    @Id
    var id: Long? = null, // should be immuatble
    var description: String? = null, // cannot be null, immutable if `isSigned == true`
    var isSigned: Boolean? = null,
 )
</code>```

Using this entity directly is error prone and not very handy. The wrapper solves all these issues:

```<code>class AgreementWrapper(agreement: Agreement) {
    val id = agreement.id  // NOTICE that we can now use `val`
    var description: String = agreement.description!! // NOT NULLABLE
        set(value) {
            if (isSigned == true) throw Exception("A signed agreement cannot be changed!")
            field = value
        }
    var isSigned = agreement.isSigned!!
        set(value) {
            if (isSigned == true && value == false) throw Exception("You cannot undo it!")
            field = value
        }

    val entity: Agreement // This will give us back the updated entity
        get() {
            return Agreement(id, description, isSigned)
        }
}

</code>```

Furthermore, the wrapper can also by used to create new entities. Adding default values to entities (for instance adding a timestamp) would also require changing the file generated by KHipster, but here there is no problem with it. I just added another constructor to the wrapper, which accepts the necessary parameters and uses defaults otherwise. It can also deal with validation and logging.

Thus wrappers act like a bridge between entities and the service layer.

Body:
I have an Spring entity written in Kotlin:

@Entity
class Book(
    @Id
    var id: Long? = null,
    var title: String? = null, // cannot actually be null, immutable
    var isInStock: Boolean? = null, // cannot actually be null
    var description: String? = null,
 )

All the fields must be nullable, because Spring needs to initialize empty objects.

However, this makes using the entity complicated, because I always have to convert the nullable types to their not nullable equivalents. This is not semantic: I would like to really see, which fields can actually be nullable and which ones cannot (or can only be null during the initialization process).

Furthermore, some fields are mutable, but some should never be altered after the entity has been created. It would be nice to use Kotlin's val and var to make this distinction.

So, I would like to work with the following class:

class BetterBook(
    val id: Long,
    val title: String,
    var isInStock: Boolean,
    var description: String? = null,
 )

This way it is very clear which fields are mutable and which fields can be null.

I pondered upon creating a wrapper around Book. Has anybody had a similar thought? Would it be a good solution architecturally?

Body:
I am using JHipster (or KHipster, its Kotlin blueprint). Because all the entities are generated by KHipster, I do not want to change these files, for all my changes would be overridden when I regenerate the entity. After experimenting with different solutions I decided to add an extra wrapper layer.

A wrapper is like a PR manager for the entity. It can be constructed from the entity, then it can be modified in the service layer and once it is time to save it, the wrapper will give you back the updated entity.

This is how a KHipster entity approximately looks like:

@Entity
class Agreement(
    @Id
    var id: Long? = null, // should be immuatble
    var description: String? = null, // cannot be null, immutable if `isSigned == true`
    var isSigned: Boolean? = null,
 )

Using this entity directly is error prone and not very handy. The wrapper solves all these issues:

class AgreementWrapper(agreement: Agreement) {
    val id = agreement.id  // NOTICE that we can now use `val`
    var description: String = agreement.description!! // NOT NULLABLE
        set(value) {
            if (isSigned == true) throw Exception("A signed agreement cannot be changed!")
            field = value
        }
    var isSigned = agreement.isSigned!!
        set(value) {
            if (isSigned == true && value == false) throw Exception("You cannot undo it!")
            field = value
        }

    val entity: Agreement // This will give us back the updated entity
        get() {
            return Agreement(id, description, isSigned)
        }
}

Furthermore, the wrapper can also by used to create new entities. Adding default values to entities (for instance adding a timestamp) would also require changing the file generated by KHipster, but here there is no problem with it. I just added another constructor to the wrapper, which accepts the necessary parameters and uses defaults otherwise. It can also deal with validation and logging.

Thus wrappers act like a bridge between entities and the service layer.


问:
How to change the font type for the entire website in Typo3
This might be due to the fact that I am a beginner in Typo3 and do not have a good grasp on its structure, but I cannot seem to be able to find where to change the font used across the whole website.

I have tried adding this to the css file of my extension:

```<code>html {
    font-family: "Times New Roman", Times, serif;
}
</code>```

But the font on my website does not change to Times New Roman.

I'm also using bootstrap by including it in setup.typoscript, in case this makes any difference.

The source code of my webpage shows that it is using the following css files (where I added the code above to the file mentioned in the second line):

```<code><link rel="stylesheet" type="text/css" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" media="all">

<link rel="stylesheet" type="text/css" href="/typo3/typo3conf/ext/myextension/Resources/Public/Css/layout.min.css?1557481820" media="all">
</code>```

答:
Your problem here happens because Bootstrap overwrites your font-family definition for the  tag; it applies a different font to the  tag, and it is independent from the usage of TYPO3 as CMS.

You can check it by inspecting the DOM with Chrome or Firefox Inspector.

To avoid it, I think you have a couple of options

Use a custom version of Bootstrap instead of the one from the CDN, you can compile it from its scss files

Overwrite Bootstrap rule, for example with something like:

```<code>body {
  font-family: "Times New Roman", Times, serif;
}
</code>```

this should be sufficient, as your CSS is loaded after Bootstrap in the code.

Or even use an higher specificity rule, for example:

```<code>   html body {
      font-family: "Times New Roman", Times, serif;
    }
</code>```

Body:
This might be due to the fact that I am a beginner in Typo3 and do not have a good grasp on its structure, but I cannot seem to be able to find where to change the font used across the whole website.

I have tried adding this to the css file of my extension:

html {
    font-family: "Times New Roman", Times, serif;
}

But the font on my website does not change to Times New Roman.

I'm also using bootstrap by including it in setup.typoscript, in case this makes any difference.

The source code of my webpage shows that it is using the following css files (where I added the code above to the file mentioned in the second line):

<link rel="stylesheet" type="text/css" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" media="all">

<link rel="stylesheet" type="text/css" href="/typo3/typo3conf/ext/myextension/Resources/Public/Css/layout.min.css?1557481820" media="all">

Body:
Your problem here happens because Bootstrap overwrites your font-family definition for the <html> tag; it applies a different font to the <body> tag, and it is independent from the usage of TYPO3 as CMS.

You can check it by inspecting the DOM with Chrome or Firefox Inspector.

To avoid it, I think you have a couple of options

Use a custom version of Bootstrap instead of the one from the CDN, you can compile it from its scss files

Overwrite Bootstrap rule, for example with something like:

body {
  font-family: "Times New Roman", Times, serif;
}

this should be sufficient, as your CSS is loaded after Bootstrap in the code.

Or even use an higher specificity rule, for example:

   html body {
      font-family: "Times New Roman", Times, serif;
    }


问:
Asserting an exception thrown from a mock object constructor
Assume: VS2010, .NET 4, C#, NUnit, Moq

I am new to TDD and came across this issue while working through a project.

Given the class:

```<code>public abstract class MyFileType
{
    public MyFileType(String fullPathToFile)
    {
        if (!File.Exists(fullPathToFile))
        {
            throw new FileNotFoundException();
        }

        // method continues

    }
}
</code>```

I am trying to test it using the method:

```<code>[Test]
[ExpectedException(typeof(System.IO.FileNotFoundException))]
public void MyFileType_CreationWithNonexistingPath_ExceptionThrown()
{
    String nonexistingPath = "C:\\does\\not\\exist\\file.ext";
    var mock = new Mock<MyFileType>(nonexistingPath);
}
</code>```

The test fails and NUnit reports an exception was never thrown.

I did find a section in the NUnit docs talking about asserting with exceptions, but the examples didn't seem like what I am trying to do. I am still getting started with NUnit and Moq so I may be going about this the wrong way.

UPDATE:

To help clarify why this example uses an abstract class, it is the base class of a series file types where only the loading and disposing of the data would differ between subclassed types. My initial thought was to put the logic for open/setup into a base class since it is the same for all types.

答:
If you have to have the class an abstract one, we should then just implement it as it's meant to be (simplicity):
MSDN: an abstract class

So, agreeing (with alexanderb)that a mock is probably not needed here and also with Stecy on the .Throws NUnit Assert extension, you can create a class in the test that calls the base class as follows:

```<code>using System;
using System.IO;

namespace fileFotFoundException {
    public abstract class MyFile {

        protected MyFile(String fullPathToFile) {
            if (!File.Exists(fullPathToFile)) throw new FileNotFoundException();
        }
    }
}

namespace fileFotFoundExceptionTests {
    using fileFotFoundException;
    using NUnit.Framework;

    public class SubClass : MyFile {
        public SubClass(String fullPathToFile) : base(fullPathToFile) {
            // If we have to have it as an abstract class...
        }
    }

    [TestFixture]
    public class MyFileTests {

        [Test]
        public void MyFile_CreationWithNonexistingPath_ExceptionThrown() {
            const string nonExistingPath = "C:\\does\\not\\exist\\file.ext";

            Assert.Throws<FileNotFoundException>(() => new SubClass(nonExistingPath));
        }
    }
}
</code>```

Body:
Assume: VS2010, .NET 4, C#, NUnit, Moq

I am new to TDD and came across this issue while working through a project.

Given the class:

public abstract class MyFileType
{
    public MyFileType(String fullPathToFile)
    {
        if (!File.Exists(fullPathToFile))
        {
            throw new FileNotFoundException();
        }

        // method continues

    }
}

I am trying to test it using the method:

[Test]
[ExpectedException(typeof(System.IO.FileNotFoundException))]
public void MyFileType_CreationWithNonexistingPath_ExceptionThrown()
{
    String nonexistingPath = "C:\\does\\not\\exist\\file.ext";
    var mock = new Mock<MyFileType>(nonexistingPath);
}

The test fails and NUnit reports an exception was never thrown.

I did find a section in the NUnit docs talking about asserting with exceptions, but the examples didn't seem like what I am trying to do. I am still getting started with NUnit and Moq so I may be going about this the wrong way.

UPDATE:

To help clarify why this example uses an abstract class, it is the base class of a series file types where only the loading and disposing of the data would differ between subclassed types. My initial thought was to put the logic for open/setup into a base class since it is the same for all types.

Body:
If you have to have the class an abstract one, we should then just implement it as it's meant to be (simplicity):
MSDN: an abstract class

So, agreeing (with alexanderb)that a mock is probably not needed here and also with Stecy on the .Throws NUnit Assert extension, you can create a class in the test that calls the base class as follows:

using System;
using System.IO;

namespace fileFotFoundException {
    public abstract class MyFile {

        protected MyFile(String fullPathToFile) {
            if (!File.Exists(fullPathToFile)) throw new FileNotFoundException();
        }
    }
}

namespace fileFotFoundExceptionTests {
    using fileFotFoundException;
    using NUnit.Framework;

    public class SubClass : MyFile {
        public SubClass(String fullPathToFile) : base(fullPathToFile) {
            // If we have to have it as an abstract class...
        }
    }

    [TestFixture]
    public class MyFileTests {

        [Test]
        public void MyFile_CreationWithNonexistingPath_ExceptionThrown() {
            const string nonExistingPath = "C:\\does\\not\\exist\\file.ext";

            Assert.Throws<FileNotFoundException>(() => new SubClass(nonExistingPath));
        }
    }
}


问:
How to escape a PCRE pattern with bash (ag)

Question

How do I escape a regex (PCRE) to use in bash (with ag)?

Attempts

ag (?!.*rib)(?=.*dist)^(\w+)$ # -bash: !.*wib: event not found
ag "(?!.*rib)(?=.*dist)^(\w+)$" # -bash: !.*wib: event not found
ag '(?!.*rib)(?=.*dist)^(\w+)$' # This does not crash, but finds nothing
ag (\?!.\*rib)(\?=.\*dist)^(\\w+)$ # -bash: !.*wib: event not found
```

<p><strong>Edit</strong>: I was incorrect - the third one above finds nothing (but it does seem to be thinking hard)</p>

<p>I'm using osx</p>

```$ bash -version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.
```

答:
Let's assume you have a file, foo.txt with four lines of the following text:

```<code>distribute
distrib
distrubte
dist
</code>```

Then using ag from bash, you'd employ your regex as follows:

```<code>$ ag "(?\!.*rib)(?=.*dist)" foo.txt
3:distrubte
4:dist
</code>```

Line 3 and 4 would return as found matches. Note: I used double quotes, not single and ONLY escaped the exclamation mark -- as I didn't want it interpreted by bash, but used in the regex. If I wanted it used, I could do something like this:

```<code>$echo dist
dist
$ag "!$$" foo.txt
4:dist
</code>```

Now line four is matched. What happened here is the !$ of !$$ was expanded by the shell as the last argument to the previous command, i.e. dist. And, the final $ was left as a literal, making the expanded command ag "dist$", and this matches the fourth line in file.

For this reason, I actually prefer to use/struggle with double quotes since they afford one the ability to swap in variables, arguments, and shell capabilities into Ag's search string. YMMV.

Body:
Question

How do I escape a regex (PCRE) to use in bash (with ag)?

Attempts

ag (?!.*rib)(?=.*dist)^(\w+)$ # -bash: !.*wib: event not found
ag "(?!.*rib)(?=.*dist)^(\w+)$" # -bash: !.*wib: event not found
ag '(?!.*rib)(?=.*dist)^(\w+)$' # This does not crash, but finds nothing
ag (\?!.\*rib)(\?=.\*dist)^(\\w+)$ # -bash: !.*wib: event not found

Edit: I was incorrect - the third one above finds nothing (but it does seem to be thinking hard)

I'm using osx

$ bash -version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.

Body:
Let's assume you have a file, foo.txt with four lines of the following text:

distribute
distrib
distrubte
dist

Then using ag from bash, you'd employ your regex as follows:

$ ag "(?\!.*rib)(?=.*dist)" foo.txt
3:distrubte
4:dist

Line 3 and 4 would return as found matches. Note: I used double quotes, not single and ONLY escaped the exclamation mark -- as I didn't want it interpreted by bash, but used in the regex. If I wanted it used, I could do something like this:

$echo dist
dist
$ag "!$$" foo.txt
4:dist

Now line four is matched. What happened here is the !$ of !$$ was expanded by the shell as the last argument to the previous command, i.e. dist. And, the final $ was left as a literal, making the expanded command ag "dist$", and this matches the fourth line in file.

For this reason, I actually prefer to use/struggle with double quotes since they afford one the ability to swap in variables, arguments, and shell capabilities into Ag's search string. YMMV.


问:
Use mongodb in nuxt
I am trying to use MongoDB for long term and complex storage that Vuex isn't suited for.

I have MongoDB installed and running and I installed the mongoose package.

In my plugins folder, I have a create a script that initializes the module and exports it:

plugins/mongoose.js

```<code>const mongoose = require('mongoose');

mongoose.connect('mongodb://localhost/users', { useNewUrlParser: true });

export default({ app }, inject) => {

    inject('mongoose', mongoose);

}
</code>```

and then in my nuxt.config.js, I declare the module and set it as server side only.

nuxt.config.js

```<code>...
plugins: [
    { src: '~/plugins/mongoose.js', mode: 'server' },
],
...
</code>```

and then finally, in one of my pages I try to access it in a method.

pages/users.vue

```<code><template>
    <button @click="addUser('joe')">Joe</button>
</template>

<script>
    export default {
        methods: {
            addUser(name) {
                console.log(this.$mongoose);
            }
        }
    }
</script>
</code>```

and when I click the button in the console I get cannot stringify a function change and then cannot stringify a function name. I can't tell if this is working but I can't use any properties of the mongoose object.

There doesn't seem to be much structured information on this topic even though it seems like it would be fairly common.

答:
You cant do it like this. Nuxt is frontend. So the code will be executed on client in their browsers. Your event will be executed in browser when user click on button. So you cant access mongoose there. mongoose dont work in browser, only in node.

So you need to have backend that will do your mongoose stuff and send\recieve data from your nuxt there using REST or GraphQL or whatever

Title: Use mongodb in nuxt

Body:
I am trying to use MongoDB for long term and complex storage that Vuex isn't suited for.

I have MongoDB installed and running and I installed the mongoose package.

In my plugins folder, I have a create a script that initializes the module and exports it:

plugins/mongoose.js

const mongoose = require('mongoose');

mongoose.connect('mongodb://localhost/users', { useNewUrlParser: true });

export default({ app }, inject) => {

    inject('mongoose', mongoose);

}

and then in my nuxt.config.js, I declare the module and set it as server side only.

nuxt.config.js

...
plugins: [
    { src: '~/plugins/mongoose.js', mode: 'server' },
],
...

and then finally, in one of my pages I try to access it in a method.

pages/users.vue

<template>
    <button @click="addUser('joe')">Joe</button>
</template>

and when I click the button in the console I get cannot stringify a function change and then cannot stringify a function name. I can't tell if this is working but I can't use any properties of the mongoose object.

There doesn't seem to be much structured information on this topic even though it seems like it would be fairly common.


问:
connect to mongodb in docker from node app in AWS SAM
I am getting errors connecting to mongodb running in a docker container from my Nodejs app running in AWS SAM (used to say "in my host").

I run mongodb like:

```<code>$ docker run --name mongo-myapp --rm -d -p 27018:27017 mongo
</code>```

and I see the results:

```<code>$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
ab8248d17d2d        mongo               "docker-entrypoint.s…"   6 minutes ago       Up 6 minutes        0.0.0.0:27018->27017/tcp   mongo-myapp
</code>```

~~I can successfully connect and insert data using clients running on my host like MongoDB Compass and Robo 3T Community Edition, specifying port 27018.~~

When I attempt to connect with this code running on my host, (not in a docker container):

```<code>const { MongoClient } = require('mongodb');
const mongoConnection = 'mongodb://127.0.0.1:27018';
MongoClient.connect(mongoConnection, (err, db) => {
    if (err) {
        console.error("Failed to connect to mongodb server", err);
        return reject(err);
    }
    console.log("Connected successfully to mongodb server");
    resolve(db);
});
</code>```

I always see the error:

MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27018

I get the same error using another port with all steps, like 27017.

UPDATE

It turns out my code was not running on the host. It was running in another docker container. I did not realize AWS SAM would put my code into a docker container so I did not mention it in the original question.

Now I run my code with mocha test to make sure it will run on my host, and it connects to the mongo database with no problems.

When I launched a local server using AWS SAM's start-api, I had problems. Perhaps the solution will be to specify a network when starting the mongo container as well as the SAM environment.

答:
Since you are able to connect from other clients in the same host, I assume that the container is bound to specific external facing ip of the host.

Can you try connecting to the ip of the host instead of 127.0.0.1

Eg: mongodb://external-ip:27018

Though the mapping -p 27018:27017 should bind the port to all ips, you can enforce this by -p 0.0.0.0:27018:27017.

Body:
I am getting errors connecting to mongodb running in a docker container from my Nodejs app running in AWS SAM (used to say "in my host").

I run mongodb like:

$ docker run --name mongo-myapp --rm -d -p 27018:27017 mongo

and I see the results:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
ab8248d17d2d        mongo               "docker-entrypoint.s…"   6 minutes ago       Up 6 minutes        0.0.0.0:27018->27017/tcp   mongo-myapp

~~I can successfully connect and insert data using clients running on my host like MongoDB Compass and Robo 3T Community Edition, specifying port 27018.~~

When I attempt to connect with this code running on my host, (not in a docker container):

const { MongoClient } = require('mongodb');
const mongoConnection = 'mongodb://127.0.0.1:27018';
MongoClient.connect(mongoConnection, (err, db) => {
    if (err) {
        console.error("Failed to connect to mongodb server", err);
        return reject(err);
    }
    console.log("Connected successfully to mongodb server");
    resolve(db);
});

I always see the error:

MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27018

I get the same error using another port with all steps, like 27017.

UPDATE

It turns out my code was not running on the host. It was running in another docker container. I did not realize AWS SAM would put my code into a docker container so I did not mention it in the original question.

Now I run my code with mocha test to make sure it will run on my host, and it connects to the mongo database with no problems.

When I launched a local server using AWS SAM's start-api, I had problems. Perhaps the solution will be to specify a network when starting the mongo container as well as the SAM environment.


问:
Use line break in mPDF when data is from mysql table
I'm using mPDF to display data from mysql database. It happens the mPDF can't display the line break.

In database data is like :

one

two

three

the mPDF displays like this : one two three

I found an answer in this page :[How to put line break in mpdf when data is extracted from mysql database

But this cannot solve my problem.

My PHP CODE

```<code>$mpdf = new \Mpdf\Mpdf(['tempDir' => __DIR__ . '/custom/temp/dir/path','format' => 'A5']);

$sql ="SELECT * FROM  table";
$this->data['dataTable']= $this->db->query($sql);

$html = $this->load->view('admin/print/cirugia_toracia_report',$this->data,true);

$mpdf->WriteHTML($html);

$mpdf->Output();
</code>```

Some helps is greatly appreciated

答:
You need to convert line breaks to
 HTML tags. Call nl2br function over the data you want to convert.

```<code>$this->data['dataTable']['column_to_break'] = nl2br($this->data['dataTable']['column_to_break']);
</code>```

Body:
I'm using mPDF to display data from mysql database. It happens the mPDF can't display the line break.

In database data is like :

one

two

three

the mPDF displays like this : one two three

I found an answer in this page :[How to put line break in mpdf when data is extracted from mysql database

But this cannot solve my problem.

My PHP CODE

$mpdf = new \Mpdf\Mpdf(['tempDir' => __DIR__ . '/custom/temp/dir/path','format' => 'A5']);

$sql ="SELECT * FROM  table";
$this->data['dataTable']= $this->db->query($sql);

$html = $this->load->view('admin/print/cirugia_toracia_report',$this->data,true);

$mpdf->WriteHTML($html);

$mpdf->Output();

Some helps is greatly appreciated

Body:
You need to convert line breaks to <br> HTML tags. Call nl2br function over the data you want to convert.

$this->data['dataTable']['column_to_break'] = nl2br($this->data['dataTable']['column_to_break']);


问:
auto increment value in foreach loop, but it keeps going
Summary: Trying to add +1 each time I go through my foreach loop, but it keeps going up even though I would only expect a maximum of 11 results in the loop

I tried to move around on the $i++; but without luck, I also check that I made sure the $i = 0; was outside of the foreach loop.

```<code><?php
$facade = new NewsArticleFacade();
$news = $facade->getAll();
unset($facade);

$i = 0;
foreach($news as $new)
{
  $i++;
  if ($i = 5000000000000000)
  {
    echo "if statement: " . $i;
    break;
  }
  $content = substr($new->getContent(), 0, 69);
  echo "<div class='news'>";
    echo "<h4><a href='/news/news.php?id=".$new->getNewsID()."'>".$new->getTitle()."</a></h4>";
    echo "<span>by <a href='#'>".$new->getUser()->getUsername()."</a> - ".$new->getDate()->format("d-M-Y")."</span>";
    echo "<p>".$content." ...</p>";
    echo "<div class='gradient'></div>";
    echo "<img src='img/news/0000000001.jpg'>";
  echo "</div>";
}
?>
</code>```

What really confuses me is if I remove the if statement for the loop, and just echo $i each time, it will echo out 11 in total, what am i doing wrong with my if?

I would expect the loop to go through 11 times since I have 11 entries in the database for this selection, but it keeps going up, if I echo within the if statement i can see the value will just be whatever i limit the if statement to be.

Thanks in advance!

答:
This

```<code>if ($i = 5000000000000000)
</code>```

Is assignment and it makes little sense as such in your code. You probably wanted to have comparison operation which is expressed with ==

```<code>if ($i == 5000000000000000)
</code>```

Body:
Summary: Trying to add +1 each time I go through my foreach loop, but it keeps going up even though I would only expect a maximum of 11 results in the loop

I tried to move around on the $i++; but without luck, I also check that I made sure the $i = 0; was outside of the foreach loop.

<?php
$facade = new NewsArticleFacade();
$news = $facade->getAll();
unset($facade);

$i = 0;
foreach($news as $new)
{
  $i++;
  if ($i = 5000000000000000)
  {
    echo "if statement: " . $i;
    break;
  }
  $content = substr($new->getContent(), 0, 69);
  echo "
";
    echo "
getNewsID()."'>".$new->getTitle()."
";
    echo "by ".$new->getUser()->getUsername()." - ".$new->getDate()->format("d-M-Y")."";
    echo "
".$content." ...
";
    echo "

";
    echo "";
  echo "
";
}
?>

What really confuses me is if I remove the if statement for the loop, and just echo $i each time, it will echo out 11 in total, what am i doing wrong with my if?

I would expect the loop to go through 11 times since I have 11 entries in the database for this selection, but it keeps going up, if I echo within the if statement i can see the value will just be whatever i limit the if statement to be.

Thanks in advance!

Body:
This

if ($i = 5000000000000000)

Is assignment and it makes little sense as such in your code. You probably wanted to have comparison operation which is expressed with ==

if ($i == 5000000000000000)