问:
When are servlet static variables gced in gae?
I'm trying to have a GAE app that all it does is register users and associate contacts with them, and return their online status. If online return their IP. So i've a JPA entity with the userid and contacts list, and a transient ip field. Every time a user logs in I keep it in a map.

```<code>private static final Map<String, SSUser> connectedRegisteredUsers = new HashMap<String, SSUser>();
</code>```

Now the surprise is that it seems after a while the map is emptied, and will forget the users. I've no idea why that might be happening, how to verify it (beside from observing the behavior), and alternative exists, beside storing the IP in the datastore, too.

答:
App Engine can start up and shut down instances of your app at any time, in response to changing traffic patterns. There's two issues here: First, your app will be running multiple distinct instances, on multiple machines, each of which has separate state (including your static variables). Second, instances can be shut down and started up at any time, clearing the static variable.

If you need globally consistent data, you need to use memcache and/or the datastore.

Body:
I'm trying to have a GAE app that all it does is register users and associate contacts with them, and return their online status. If online return their IP. So i've a JPA entity with the userid and contacts list, and a transient ip field. Every time a user logs in I keep it in a map.

private static final Map<String, SSUser> connectedRegisteredUsers = new HashMap<String, SSUser>();

Now the surprise is that it seems after a while the map is emptied, and will forget the users. I've no idea why that might be happening, how to verify it (beside from observing the behavior), and alternative exists, beside storing the IP in the datastore, too.


来源: stackexchange

问:
Downloading an image being cached and showing in my imageview
I need to download an image where it saves in the cache using Imageview, somehow for me to do it in a simple way ?

I can download it as you can see below more in not saved in cache

```<code>private Bitmap GetImageBitmapFromUrl(string url)
    {
        Bitmap imageBitmap = null;

        using (var webClient = new WebClient())
        {
            try
            {
                var imageBytes = webClient.DownloadData(url);
                if (imageBytes != null && imageBytes.Length > 0)
                {
                    imageBitmap = BitmapFactory.DecodeByteArray(imageBytes, 0, imageBytes.Length);
                }
            }
            catch
            {

            }
        }

        return imageBitmap;
    }

    //-----------

        **Show image**
        var imageBitmap = GetImageBitmapFromUrl("URL");
        imageView.SetImageBitmap(imageBitmap);
</code>```

答:
Android: using imageView.SetImage(uri) find here

iOS:SDWebImage find here

ContentLicense: CC BY-SA 3.0

Body:
I need to download an image where it saves in the cache using Imageview, somehow for me to do it in a simple way ?

I can download it as you can see below more in not saved in cache

private Bitmap GetImageBitmapFromUrl(string url)
    {
        Bitmap imageBitmap = null;

        using (var webClient = new WebClient())
        {
            try
            {
                var imageBytes = webClient.DownloadData(url);
                if (imageBytes != null && imageBytes.Length > 0)
                {
                    imageBitmap = BitmapFactory.DecodeByteArray(imageBytes, 0, imageBytes.Length);
                }
            }
            catch
            {

            }
        }

        return imageBitmap;
    }

    //-----------

        **Show image**
        var imageBitmap = GetImageBitmapFromUrl("URL");
        imageView.SetImageBitmap(imageBitmap);


问:
Not able to find all groups using a pattern regex in matlab
I am trying to parse a regular expression in matlab. I am trying to extract all the number between '[]' for all the groups. Here are the details:

```<code>pat = '(\[\d,\d,\d,\d\])';
s1 = 'frame_1:[1,2,3,5],[11,22,33,44],[23,12,12,33],'

[matched_string] = regexp(s1,pat,'match');

>> matched_string{:}

ans =

    '[1,2,3,5]'
</code>```

I want to get all the boxes, i.e [1,2,3,5],[11,22,33,44] and [23,12,12,33].

Can someone help me figure out what I am doing wrong?

答:
Your pattern only matches single digits inside square brackets. To match one or more, add + after each:

```<code>'(\[\d+,\d+,\d+,\d+\])'
</code>```

If you do not care of the format inside the square brackets, and just need to extract square brackets with digits and commas inside, you may use a simpler

```<code>'\[[\d,]+]'
</code>```

Note that ] at the end of the regular expression is not a special char here, since there is no corresponding  [ that opens a character class, thus, no need escaping it.

Body:
I am trying to parse a regular expression in matlab. I am trying to extract all the number between '[]' for all the groups. Here are the details:

pat = '(\[\d,\d,\d,\d\])';
s1 = 'frame_1:[1,2,3,5],[11,22,33,44],[23,12,12,33],'

[matched_string] = regexp(s1,pat,'match');

>> matched_string{:}

ans =

    '[1,2,3,5]'

I want to get all the boxes, i.e [1,2,3,5],[11,22,33,44] and [23,12,12,33].

Can someone help me figure out what I am doing wrong?

Body:
Your pattern only matches single digits inside square brackets. To match one or more, add + after each:

'(\[\d+,\d+,\d+,\d+\])'

If you do not care of the format inside the square brackets, and just need to extract square brackets with digits and commas inside, you may use a simpler

'\[[\d,]+]'

Note that ] at the end of the regular expression is not a special char here, since there is no corresponding  [ that opens a character class, thus, no need escaping it.


问:
Looking for a solution to passing json data from asp.net mvc 4.5 to an angular 2 componet via razor @Html.Raw(Json.Encode(Model))
I am trying to display rows of data on a page using Angular 2, TypeScript and ASP.NET MVC 4.5. I'm trying to deliver the JSON into an Angular 2 component, but I've not been able to succeed, and the browser isn't showing any errors.

Does anyone know how to return JSON data from a .NET controller, embed in a view using (@Html.Raw(Json.Encode(Model)), and then pass into an Angular 2 component/template to be displayed?

Here is what I have tried so far (built from various examples on Plunkr and examples from the AngularJS site. I'll note that I had a basic Angular 2 sample working in the same solution, so I believe the framework and tools are setup correctly. Just having difficulty applying it. I've been referencing this plunk on my latest attempt: http://plnkr.co/edit/LEtEN9?p=preview

In my view:

```<code><my-app holdings="@Html.Raw(Json.Encode(Model))">Loading...</my-app>
</code>```

main.ts:

```<code>import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';
import { AppModule } from './boot';
const platform = platformBrowserDynamic();
platform.bootstrapModule(AppModule);
</code>```

boot.ts:

```<code>///<reference path="./../typings/globals/core-js/index.d.ts"/>
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { AppComponent } from './app';

@NgModule({
    imports: [BrowserModule],
    declarations: [AppComponent],
    bootstrap: [AppComponent]
})
export class AppModule { }
</code>```

app.ts:

```<code>import { Component, Input } from '@angular/core';
@Component({
    selector: 'my-app',
    template: `
    <ul>
      <li *ngFor="let report of holdings">
        {{ report.CorpName }}
      </li>
    </ul>
  `
})
export class AppComponent {
    @Input() holdings: any;
}
</code>```

Thank you for your help!

答:
Using the link provided by yurzui in the comments as well as the "hack" function by evanplaice here: https://github.com/angular/angular/issues/6392, I put together the following solution.

The changes are limited to the app.ts.

app.ts:

```<code>import { Component, Input, ElementRef } from '@angular/core';
@Component({
    selector: 'my-app',
    template: `
    <ul>
      <li *ngFor="let report of hack(holdings)">
        {{ report.CorpName }}
      </li>
    </ul>
  `
})
export class AppComponent {

    hack(val) {
        console.log('Before:');
        console.log(val);
        val = JSON.parse(val);
        console.log('After:');
        console.log(val);
        return val;
    }

    @Input() holdings: any;

    constructor(elm: ElementRef) {
        this.holdings = elm.nativeElement.getAttribute('holdings');
    }
}
</code>```

Body:
I am trying to display rows of data on a page using Angular 2, TypeScript and ASP.NET MVC 4.5. I'm trying to deliver the JSON into an Angular 2 component, but I've not been able to succeed, and the browser isn't showing any errors.

Does anyone know how to return JSON data from a .NET controller, embed in a view using (@Html.Raw(Json.Encode(Model)), and then pass into an Angular 2 component/template to be displayed?

Here is what I have tried so far (built from various examples on Plunkr and examples from the AngularJS site. I'll note that I had a basic Angular 2 sample working in the same solution, so I believe the framework and tools are setup correctly. Just having difficulty applying it. I've been referencing this plunk on my latest attempt: http://plnkr.co/edit/LEtEN9?p=preview

In my view:

<my-app holdings="@Html.Raw(Json.Encode(Model))">Loading...</my-app>

main.ts:

import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';
import { AppModule } from './boot';
const platform = platformBrowserDynamic();
platform.bootstrapModule(AppModule);

boot.ts:

///<reference path="./../typings/globals/core-js/index.d.ts"/>
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { AppComponent } from './app';

@NgModule({
    imports: [BrowserModule],
    declarations: [AppComponent],
    bootstrap: [AppComponent]
})
export class AppModule { }

app.ts:

import { Component, Input } from '@angular/core';
@Component({
    selector: 'my-app',
    template: `

        {{ report.CorpName }}

  `
})
export class AppComponent {
    @Input() holdings: any;
}

Thank you for your help!

Body:
Using the link provided by yurzui in the comments as well as the "hack" function by evanplaice here: https://github.com/angular/angular/issues/6392, I put together the following solution.

The changes are limited to the app.ts.

app.ts:

import { Component, Input, ElementRef } from '@angular/core';
@Component({
    selector: 'my-app',
    template: `
    <ul>
      <li *ngFor="let report of hack(holdings)">
        {{ report.CorpName }}
      </li>
    </ul>
  `
})
export class AppComponent {

    hack(val) {
        console.log('Before:');
        console.log(val);
        val = JSON.parse(val);
        console.log('After:');
        console.log(val);
        return val;
    }

    @Input() holdings: any;

    constructor(elm: ElementRef) {
        this.holdings = elm.nativeElement.getAttribute('holdings');
    }
}


问:
Aggregating based on "near" row values
I have a very messy dataframe (webscraped) that unfortunately has many double and even triple entries in it. Most of the dataframe looks like this:

```<code>> df1<-data.frame(var1=c("a","a","b","b","c","c","d","d"),var2=c("right.a",NA,"right.b",NA,"right.c",NA,"right.d",NA),var3=c("correct.a","correct.a","correct.b","correct.b","correct.c","correct.c","correct.d","correct.d"))
> df1
  var1    var2      var3
1    a right.a correct.a
2    a    <NA> correct.a
3    b right.b correct.b
4    b    <NA> correct.b
5    c right.c correct.c
6    c    <NA> correct.c
7    d right.d correct.d
8    d    <NA> correct.d
</code>```

"var1"  is my ID variable that I need to use to aggregate. My goal is to have a dataframe that looks like this:

```<code>  var1    var2      var3
1    a right.a correct.a
2    b right.b correct.b
3    c right.c correct.c
4    d right.d correct.d
</code>```

However, the main problem is, that not the whole dataframe looks like this. In fact, I have other parts that look like this:

```<code>> df2<-data.frame(var1=c("e","e","e","f","f","g","g","g"),var2=c(NA,NA,"right.e",NA,NA,NA,"right.g",NA),var3=c("correct.e","correct.e",NA,"correct.f",NA,"correct.g","correct.g",NA))
> df2
  var1    var2      var3
1    e    <NA> correct.e
2    e    <NA> correct.e
3    e right.e      <NA>
4    f    <NA> correct.f
5    f    <NA>      <NA>
6    g    <NA> correct.g
7    g right.g   wrong.g
8    g    <NA>      <NA>
</code>```

And other variations. In the end, every ID should have one row with the correct and right var2 and var3 in it. At this point, I get lost: My var1 is not unique. However, I know that duplicate IDs that "belong" together are grouped in the dataframe (as seen in my examples); e.g. there might be another "a" in row 4102 and 4103.

What I think would be the way to go is using aggregate with var1 as ID,but in addition telling R that aggregate should just check +-2 rows of var1 when doing so. Any ideas how to code this?

Thanks!

答:
Here is a method using data.table

```<code>library(data.table)

setDT(df1)[, .(var2[!is.na(var2)][1], var3[!is.na(var3)][1]), by=var1]
   var1      V1        V2
1:    a right.a correct.a
2:    b right.b correct.b
3:    c right.c correct.c
4:    d right.d correct.d
</code>```

and

```<code>setDT(df2)[, .(var2[!is.na(var2)][1], var3[!is.na(var3)][1]), by=var1]
   var1      V1        V2
1:    e right.e correct.e
2:    f      NA correct.f
3:    g right.g correct.g
</code>```

The idea in var2[!is.na(var2)][1] for example, to take the first non-missing value from var2. If all values are missing, then this returns NA. This operation is performed for both variables by var1.

If you have more than two variables, you might switch to lapply. For example, the following.

```<code>df1[, lapply(.SD, function(i) i[!is.na(i)][1]), by=var1]
   var1    var2      var3
1:    a right.a correct.a
2:    b right.b correct.b
3:    c right.c correct.c
4:    d right.d correct.d
</code>```

In an instance where more than one var1 has a valid value, and this is indicated by a non-missing var2, then you can reach the intended result with a join.

The data from the comment,

```<code>df1<-data.frame(var1=c("a","a","b","b","c","c","d","d","a","a"),
                var2=c("right.a",NA,"right.b",NA,"right.c",NA,"right.d",NA,"right.a1",NA),
                var3=c("correct.a","correct.a","correct.b","correct.b","correct.c","correct.c","correct.d","correct.d","correct.a1","correct.a1"))
</code>```

Then, with this data,

```<code>setDT(df1)[df1[, .(var2=var2[!is.na(var2)]), by=var1], on=.(var1, var2)]
   var1     var2       var3
1:    a  right.a  correct.a
2:    a right.a1 correct.a1
3:    b  right.b  correct.b
4:    c  right.c  correct.c
5:    d  right.d  correct.d
</code>```

Here, all non-missing var2 observations by var1 are merged onto the original data set.

Title: Aggregating based on "near" row values

Body:
I have a very messy dataframe (webscraped) that unfortunately has many double and even triple entries in it. Most of the dataframe looks like this:

> df1<-data.frame(var1=c("a","a","b","b","c","c","d","d"),var2=c("right.a",NA,"right.b",NA,"right.c",NA,"right.d",NA),var3=c("correct.a","correct.a","correct.b","correct.b","correct.c","correct.c","correct.d","correct.d"))
> df1
  var1    var2      var3
1    a right.a correct.a
2    a    <NA> correct.a
3    b right.b correct.b
4    b    <NA> correct.b
5    c right.c correct.c
6    c    <NA> correct.c
7    d right.d correct.d
8    d    <NA> correct.d

"var1"  is my ID variable that I need to use to aggregate. My goal is to have a dataframe that looks like this:

  var1    var2      var3
1    a right.a correct.a
2    b right.b correct.b
3    c right.c correct.c
4    d right.d correct.d

However, the main problem is, that not the whole dataframe looks like this. In fact, I have other parts that look like this:

> df2<-data.frame(var1=c("e","e","e","f","f","g","g","g"),var2=c(NA,NA,"right.e",NA,NA,NA,"right.g",NA),var3=c("correct.e","correct.e",NA,"correct.f",NA,"correct.g","correct.g",NA))
> df2
  var1    var2      var3
1    e    <NA> correct.e
2    e    <NA> correct.e
3    e right.e      <NA>
4    f    <NA> correct.f
5    f    <NA>      <NA>
6    g    <NA> correct.g
7    g right.g   wrong.g
8    g    <NA>      <NA>

And other variations. In the end, every ID should have one row with the correct and right var2 and var3 in it. At this point, I get lost: My var1 is not unique. However, I know that duplicate IDs that "belong" together are grouped in the dataframe (as seen in my examples); e.g. there might be another "a" in row 4102 and 4103.

What I think would be the way to go is using aggregate with var1 as ID,but in addition telling R that aggregate should just check +-2 rows of var1 when doing so. Any ideas how to code this?

Thanks!

Body:
Here is a method using data.table

library(data.table)

setDT(df1)[, .(var2[!is.na(var2)][1], var3[!is.na(var3)][1]), by=var1]
   var1      V1        V2
1:    a right.a correct.a
2:    b right.b correct.b
3:    c right.c correct.c
4:    d right.d correct.d

and

setDT(df2)[, .(var2[!is.na(var2)][1], var3[!is.na(var3)][1]), by=var1]
   var1      V1        V2
1:    e right.e correct.e
2:    f      NA correct.f
3:    g right.g correct.g

The idea in var2[!is.na(var2)][1] for example, to take the first non-missing value from var2. If all values are missing, then this returns NA. This operation is performed for both variables by var1.

If you have more than two variables, you might switch to lapply. For example, the following.

df1[, lapply(.SD, function(i) i[!is.na(i)][1]), by=var1]
   var1    var2      var3
1:    a right.a correct.a
2:    b right.b correct.b
3:    c right.c correct.c
4:    d right.d correct.d

In an instance where more than one var1 has a valid value, and this is indicated by a non-missing var2, then you can reach the intended result with a join.

The data from the comment,

df1<-data.frame(var1=c("a","a","b","b","c","c","d","d","a","a"),
                var2=c("right.a",NA,"right.b",NA,"right.c",NA,"right.d",NA,"right.a1",NA),
                var3=c("correct.a","correct.a","correct.b","correct.b","correct.c","correct.c","correct.d","correct.d","correct.a1","correct.a1"))

Then, with this data,

setDT(df1)[df1[, .(var2=var2[!is.na(var2)]), by=var1], on=.(var1, var2)]
   var1     var2       var3
1:    a  right.a  correct.a
2:    a right.a1 correct.a1
3:    b  right.b  correct.b
4:    c  right.c  correct.c
5:    d  right.d  correct.d

Here, all non-missing var2 observations by var1 are merged onto the original data set.


问:
android calendar crashes where iOS is working
My app crashes with NullPointerException, since the android calendar does not return a Collection  of calendar names from the dc.getCalendars(); function from the CN1 Calendar Lib, even though it is working to this point on the iOS iphone version.

```<code>Device Calendar dc = DeviceCalendar.getInstance();
Collection<String> calNames = dc.getCalendars();
String [] nameArray = calName.toArray(new String[calName.size()]);
</code>```

Its throwing me an NullPointerException for the 3rd line because calName is a null object and I cant use size() on it. Again, the code IS working on iPhone, so it seems there is a problem with android.

Thanks in advance.

答:
The javadocs for that method say that getCalendars() will return null if you don't have permission

https://github.com/sidiabale/codenameone-calendar/blob/master/CN1Calendar/src/com/codename1/calendar/DeviceCalendar.java#L91

You'll need to add the permissions using the android.xpermissions build hint.

```<code> codename1.arg.android.xpermissions=<uses-permission android:name="android.permission.READ_CALENDAR"/> <uses-permission android:name="android.permission.WRITE_CALENDAR"/>
</code>```

And it looks like you also need to set android.targetSDKVersion=21

See this SO thread for a related issue.

Body:
My app crashes with NullPointerException, since the android calendar does not return a Collection  of calendar names from the dc.getCalendars(); function from the CN1 Calendar Lib, even though it is working to this point on the iOS iphone version.

Device Calendar dc = DeviceCalendar.getInstance();
Collection<String> calNames = dc.getCalendars();
String [] nameArray = calName.toArray(new String[calName.size()]);

Its throwing me an NullPointerException for the 3rd line because calName is a null object and I cant use size() on it. Again, the code IS working on iPhone, so it seems there is a problem with android.

Thanks in advance.

Body:
The javadocs for that method say that getCalendars() will return null if you don't have permission

https://github.com/sidiabale/codenameone-calendar/blob/master/CN1Calendar/src/com/codename1/calendar/DeviceCalendar.java#L91

You'll need to add the permissions using the android.xpermissions build hint.

 codename1.arg.android.xpermissions=<uses-permission android:name="android.permission.READ_CALENDAR"/> <uses-permission android:name="android.permission.WRITE_CALENDAR"/>

And it looks like you also need to set android.targetSDKVersion=21

See this SO thread for a related issue.


问:
what is the fastest way to randomly select some numbers in a range that contains a given set of numbers?
Here is the function I want

```<code>random_select(contain_list, ttl_num, sample_num)
</code>```

There're ttl_num integers from 0 to ttl_num-1 to select from, I want to return a list of sample_num unique integers, where numbers provided in contain_list have to be in the list, and other numbers are selected randomly.

I have to do this query very often, every time with a different contain_list, but ttl_num, sample_num are the same for all the queries.

Currently what I'm doing is, first generate a set of ttl_num integers, subtract contain_list from the set, randomly select some numbers with no replacement from the rest, then concatenate it with the contain_list to get the result.

I believe this is not the fastest way, any better thoughts?

It's okay to use global variables if needed.

Edit:

sample_num is no less than contain_list length and I want to get contain_list plus sample_num - contain_list.length other random numbers

it's guaranteed that numbers in contain_list are all within the range 0 to ttl_num-1.

答:
Here are a couple of possibilities.  Neither is less complex than what you already have, but one or both of them could turn out to be faster, depending on the size of the parameter values.  Only benchmarking with your actual data would tell for sure.

Method 1

The logic here is essentially the same as what you are already doing.  It just replaces the set generation and operations with an integer array, which should be lighter weight.  It does, however, require a sort (descending) on contain_list, so whether or not it actually runs faster than what you already have probably depends on the size of contain_list.count and ttl_num.

```<code>1) initialize a tracking var, remaining_num = ttl_num

2) initialize an integer array with value = index

3) sort contain_list descending

4) iterate through contain_list (now in descending order); for each:
4.1) decrement remaining_num
4.2) swap the element at the selected index with the one at index = remaining_num

5) iterate (sample_num - contain_list.count) times; for each:
5.1) generate a random index between 0 and remaining_num (inclusive and exclusive, respectively)
5.2) decrement remaining_num
5.3) swap the element at the selected index with the one at index = remaining_num

6) The resultant samples will start at index reamining_num and run through the end of the array.
</code>```

Here is an example run for random_select({3, 7}, 10, 5)...

```<code>remaining_num = 10

available_num[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

contain_list = {7, 3}

select the 7
remaining_num = 9
available_num[] = {0, 1, 2, 3, 4, 5, 6, 9, 8, 7}

select the 3
remaining_num = 8
available_num[] = {0, 1, 2, 8, 4, 5, 6, 9, 3, 7}

select a random(0,8), e.g. 2
remaining_num = 7
available_num[] = {0, 1, 9, 8, 4, 5, 6, 2, 3, 7}

select a random(0,7), e.g. 3
remaining_num = 6
available_num[] = {0, 1, 9, 6, 4, 5, 8, 2, 3, 7}

select a random(0,6), e.g. 0
remaining_num = 5
available_num[] = {5, 1, 9, 6, 4, 0, 8, 2, 3, 7}

result = {0, 8, 2, 3, 7}
</code>```

Method 2

If ttl_num is sufficiently large and sample_num is sufficiently low, it might be worth turning things upside down.  That is, instead of creating and manipulating a set of available numbers, track only a list of selected numbers.  Then, when selecting each random target, "skip" previously selected numbers by iterating through the list of selected numbers and counting how are less than or equal to the target index.

```<code>1) initialize a tracking var, remaining_num = ttl_num - contain_list.count

2) declare an empty list (vector) of integers, selected_num[]

4) iterate through contain_list; for each:
4.1) insert cointain_list[i] into selected_num[]

5) iterate (sample_num - contain_list.count) times; for each:
5.1) generate a random target between 0 and remaining_num (inclusive and exclusive, respectively)
5.2) decrement remaining_num
5.3) iterate through selected_num; for each:
5.3.1) if target >= selected_list[j], increment target
5.4) insert target into selected_num[]

6) The resultant samples will be all elements in selected_num.
</code>```

Here is an example run for random_select({3, 7}, 10, 5)...

```<code>remaining_num = 8

selected_num[] = {}

select the 3
selected_num[] = {3}

select the 7
selected_num[] = {3, 7}

select a random(0,8), e.g. target = 2
remaining_num = 7
2 < 3; target still 2
2 < 7; target still 2
selected_num[] = {3, 7, 2}

select a random(0,7), e.g. target = 3
remaining_num = 6
3 >= 3; target becomes 4
4 < 7; target still 4
4 >= 2; target becomes 5
selected_num[] = {3, 7, 2, 5}

select a random(0,6), e.g. target = 0
remaining_num = 5
0 < 3; target still 0
0 < 7; target still 0
0 < 2; target still 0
0 < 5; target still 0
selected_num[] = {3, 7, 2, 5, 0}
</code>```

Obviously, iterating through selected_num[] when selecting each new number could get expensive if sample_num is large.  This could be alleviated somewhat by maintaining selected_num[] in descending sort order and breaking the inner loop as soon as you see a number smaller than the target.  Insert the target at that point in the list to maintain the sort.

Body:
Here is the function I want

random_select(contain_list, ttl_num, sample_num)

There're ttl_num integers from 0 to ttl_num-1 to select from, I want to return a list of sample_num unique integers, where numbers provided in contain_list have to be in the list, and other numbers are selected randomly.

I have to do this query very often, every time with a different contain_list, but ttl_num, sample_num are the same for all the queries.

Currently what I'm doing is, first generate a set of ttl_num integers, subtract contain_list from the set, randomly select some numbers with no replacement from the rest, then concatenate it with the contain_list to get the result.

I believe this is not the fastest way, any better thoughts?

It's okay to use global variables if needed.

Edit:

sample_num is no less than contain_list length and I want to get contain_list plus sample_num - contain_list.length other random numbers

it's guaranteed that numbers in contain_list are all within the range 0 to ttl_num-1.

Body:
Here are a couple of possibilities.  Neither is less complex than what you already have, but one or both of them could turn out to be faster, depending on the size of the parameter values.  Only benchmarking with your actual data would tell for sure.

Method 1

The logic here is essentially the same as what you are already doing.  It just replaces the set generation and operations with an integer array, which should be lighter weight.  It does, however, require a sort (descending) on contain_list, so whether or not it actually runs faster than what you already have probably depends on the size of contain_list.count and ttl_num.

1) initialize a tracking var, remaining_num = ttl_num

2) initialize an integer array with value = index

3) sort contain_list descending

4) iterate through contain_list (now in descending order); for each:
4.1) decrement remaining_num
4.2) swap the element at the selected index with the one at index = remaining_num

5) iterate (sample_num - contain_list.count) times; for each:
5.1) generate a random index between 0 and remaining_num (inclusive and exclusive, respectively)
5.2) decrement remaining_num
5.3) swap the element at the selected index with the one at index = remaining_num

6) The resultant samples will start at index reamining_num and run through the end of the array.

Here is an example run for random_select({3, 7}, 10, 5)...

remaining_num = 10

available_num[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

contain_list = {7, 3}

select the 7
remaining_num = 9
available_num[] = {0, 1, 2, 3, 4, 5, 6, 9, 8, 7}

select the 3
remaining_num = 8
available_num[] = {0, 1, 2, 8, 4, 5, 6, 9, 3, 7}

select a random(0,8), e.g. 2
remaining_num = 7
available_num[] = {0, 1, 9, 8, 4, 5, 6, 2, 3, 7}

select a random(0,7), e.g. 3
remaining_num = 6
available_num[] = {0, 1, 9, 6, 4, 5, 8, 2, 3, 7}

select a random(0,6), e.g. 0
remaining_num = 5
available_num[] = {5, 1, 9, 6, 4, 0, 8, 2, 3, 7}

result = {0, 8, 2, 3, 7}

Method 2

If ttl_num is sufficiently large and sample_num is sufficiently low, it might be worth turning things upside down.  That is, instead of creating and manipulating a set of available numbers, track only a list of selected numbers.  Then, when selecting each random target, "skip" previously selected numbers by iterating through the list of selected numbers and counting how are less than or equal to the target index.

1) initialize a tracking var, remaining_num = ttl_num - contain_list.count

2) declare an empty list (vector) of integers, selected_num[]

4) iterate through contain_list; for each:
4.1) insert cointain_list[i] into selected_num[]

5) iterate (sample_num - contain_list.count) times; for each:
5.1) generate a random target between 0 and remaining_num (inclusive and exclusive, respectively)
5.2) decrement remaining_num
5.3) iterate through selected_num; for each:
5.3.1) if target >= selected_list[j], increment target
5.4) insert target into selected_num[]

6) The resultant samples will be all elements in selected_num.

Here is an example run for random_select({3, 7}, 10, 5)...

remaining_num = 8

selected_num[] = {}

select the 3
selected_num[] = {3}

select the 7
selected_num[] = {3, 7}

select a random(0,8), e.g. target = 2
remaining_num = 7
2 < 3; target still 2
2 < 7; target still 2
selected_num[] = {3, 7, 2}

select a random(0,7), e.g. target = 3
remaining_num = 6
3 >= 3; target becomes 4
4 < 7; target still 4
4 >= 2; target becomes 5
selected_num[] = {3, 7, 2, 5}

select a random(0,6), e.g. target = 0
remaining_num = 5
0 < 3; target still 0
0 < 7; target still 0
0 < 2; target still 0
0 < 5; target still 0
selected_num[] = {3, 7, 2, 5, 0}

Obviously, iterating through selected_num[] when selecting each new number could get expensive if sample_num is large.  This could be alleviated somewhat by maintaining selected_num[] in descending sort order and breaking the inner loop as soon as you see a number smaller than the target.  Insert the target at that point in the list to maintain the sort.


问:
Linking native library in react native?
1)What does it mean by saying native library? What sort of library ? are ones will be used as graddle dependencies ?

2)How to link these?
I was facing trouble while using link or rnpm.

答:
Linking native library means that you are going to integrate already implemented module into your application or module which completes your react native's module functionality for eg: if you prepare a video player board with all the controls so in order to play the music you have to integrate react-native video (3rd party) because there is no inbuild lib. in react native. After integrating video lib you are good to go to play.

Steps to integrate lib(android):

1) Add package name to new packages()
2) Add dependencies to settings.gradle file and main application's gradle i.e app/gradle file.
3) sync the projects gradle because you made changes in the gradle and it's done.

After cloning/downloading the project:

1) Go to your project's home dir using cmd.
2) run npm install
3) Thereafter run rnpm link or react-native link
4) see ios folder in your project folder and if you find any pod file then run pod install after navigating into ios folder in cmd.

Cheers :)

Title: Linking native library in react native?


问:
Opening new activities with different combination of check boxes checked?
I want to make an app which has checkboxes, so what i basically want to do is that if certain checkboxes are check i want to open a certain activity. For ex; there is a question whats your gender the checkboxes available are male and female the other question is what do eat the check boxes available are pizza and burger so if male and pizza are chosen i want to open a certain activity, i want to open different activities with different combinations.how should i do this?   </p>

答:
Use CheckBox.isChecked() to get the current checked state(true/false) of the CheckBox. Use nested if_else condition to check the CheckBox's state and apply your logic to launch different activities.

Try this:

```<code>    if(checkBoxMale.isChecked()) {
        if(checkBoxPizza.isChecked()) {
            // Launch activity_male_pizza
        } else if(checkBoxBurger.isChecked()) {
            // Launch activity_male_burger
        } else {
            // Do something
        }
    } else if(checkBoxFemale.isChecked()){
        if(checkBoxPizza.isChecked()) {
            // Launch activity_female_pizza
        } else if(checkBoxBurger.isChecked()){
            // Launch activity_female_burger
        } else {
            // Do something
        }
    }
</code>```

Launch new activity:

```<code>Intent intent = new Intent(CurrentActivity.this, NextActivity.class);
startActivity(intent);
</code>```

Body:
Use CheckBox.isChecked() to get the current checked state(true/false) of the CheckBox. Use nested if_else condition to check the CheckBox's state and apply your logic to launch different activities.

Try this:

    if(checkBoxMale.isChecked()) {
        if(checkBoxPizza.isChecked()) {
            // Launch activity_male_pizza
        } else if(checkBoxBurger.isChecked()) {
            // Launch activity_male_burger
        } else {
            // Do something
        }
    } else if(checkBoxFemale.isChecked()){
        if(checkBoxPizza.isChecked()) {
            // Launch activity_female_pizza
        } else if(checkBoxBurger.isChecked()){
            // Launch activity_female_burger
        } else {
            // Do something
        }
    }

Launch new activity:

Intent intent = new Intent(CurrentActivity.this, NextActivity.class);
startActivity(intent);


问:
Firebase Dynamic Link not working if app is CLOSED
Firebase dynamic links are working well both after installation and whenever the app is in the background.

However, if it's coming from the inactive state (app is closed), it doesn't work. Both applicationOpenURL and applicationUserActivity is not being called.

Anybody experienced this before? I've been trying to solve this for days now, back and forth with Firebase's tutorial. I'm sure I'm just missing something here.

答:
Put this in didFinishLaunchingWithOptions

```<code>//Handle dynamic links when app is CLOSED
    let activityDic = launchOptions?[.userActivityDictionary]
    if let isActivityDic = activityDic {
        // Continue activity here
        return true
    }
</code>```

it will call continue userActivity if the app is opened from CLOSED state.

Body:
Put this in didFinishLaunchingWithOptions

//Handle dynamic links when app is CLOSED
    let activityDic = launchOptions?[.userActivityDictionary]
    if let isActivityDic = activityDic {
        // Continue activity here
        return true
    }

it will call continue userActivity if the app is opened from CLOSED state.


问:
Can you SELECT everything, but 1 or 2 fields, without writer's cramp?
Is it possible, in PLSQL, to select all of the fields in a table except for 1 or 2, without having to specify the fields you want?

Example, the employee table has the fields:

id

firstname

lastname

hobbies

Is it still possible to write a query similar to

```<code>select * from employee
</code>```

while leaving the field hobbies without with having to write something like this?

```<code>select id, firstname, lastname from employee
</code>```

答:
What the OP was looking for was something like:

```<code>SELECT * MINUS hobbies from...
</code>```

The best thing to do to avoid a lot of typing (and get all the column names correct) is to open the table description and cut and paste all the column names and delete the ones you don't want, comma separate the remaining ones and put them on a singe line or two.

It is easy, fast, accurate and you won't confuse the next person who has to work on your code.

Body:
Is it possible, in PLSQL, to select all of the fields in a table except for 1 or 2, without having to specify the fields you want?

Example, the employee table has the fields:

id

firstname

lastname

hobbies

Is it still possible to write a query similar to

select * from employee

while leaving the field hobbies without with having to write something like this?

select id, firstname, lastname from employee

Body:
What the OP was looking for was something like:

SELECT * MINUS hobbies from...

The best thing to do to avoid a lot of typing (and get all the column names correct) is to open the table description and cut and paste all the column names and delete the ones you don't want, comma separate the remaining ones and put them on a singe line or two.

It is easy, fast, accurate and you won't confuse the next person who has to work on your code.


问:
overflow in exp, python
can't really figure out why this error RuntimeWarning: overflow encountered in exp is showing up. The function I'm trying to implement is:

```<code>Id = lambda t_u, yp: Is * (np.exp((Vin(t_u) - L*yp)/(n_Ut*Ut)) - 1.0)
</code>```

with values:

```<code>Vin = lambda t: Vo * np.sin(2*np.pi*w*t)

L = 50e-3  # 50 mH
Vo = 5  # 5 V
w = 50  # 50 Hz
Is = 1e-9   # 1 nA
Ut = 25e-3  # 25 mV
n_Ut = 1.0
</code>```

The function Id is part of a ODE that I'm trying to solve with Runge-Kutta-Method.

EDIT:

```<code>Methoden\Serien\6>python circuit.py
Traceback (most recent call last):
  File "circuit.py", line 148, in <module>
    perform_experiment(exrk_o5(), "exrk_o5")
  File "circuit.py", line 48, in perform_experiment
    t, y = run_method(method, label)
  File "circuit.py", line 69, in run_method
    t, y = method.__call__(circuit_rhs, y[0, :], t_end, n_steps)
  File "C:\Users\-\numerical_methodes\6\rk.py", line 49, in __call__
    t[k+1],y[k+1, :] = self.step(f, y[k, :], t[k], dt)
  File "C:\Users\-\numerical_methodes\6\rk.py", line 102, in step
    dydt[:, i] = rhs(t + c[i] * dt, y0 + dt * np.dot(A[i, :].T, dydt.T))
  File "circuit.py", line 38, in circuit_rhs
    dydt=  np.array ([y[1] , Id(t, y [1]) /(C*L) - y [1]/( R*C) - y [0]/( C*L)])
  File "circuit.py", line 27, in <lambda>
    Id = lambda t_u, yp: Is * (np.exp((Vin(t_u) - L*yp)/(n_Ut*Ut)) - 1.0)
FloatingPointError: overflow encountered in exp
</code>```

where y[1]=0 and y[0]=0, as first values. The rk.py file is just the implementation of the Runge-Kutta-Method.

答:
Vin can return a value as high as 1.0; then you subtract something, resulting in an utterly unknown quantity.  You multiply that by 40, subtract 1, and raise epsilon to that power.  Depending on the value of yp, there's plenty of room for overflow in this process.  A sufficiently negative value for yp will cause the problem without reference to intermediate computations.

numpy.exp(n) overflows for some value in the range 700-800.  This suggests that yp is perhaps <= -350.

Title: overflow in exp, python

Body:
can't really figure out why this error RuntimeWarning: overflow encountered in exp is showing up. The function I'm trying to implement is:

Id = lambda t_u, yp: Is * (np.exp((Vin(t_u) - L*yp)/(n_Ut*Ut)) - 1.0)

with values:

Vin = lambda t: Vo * np.sin(2*np.pi*w*t)

L = 50e-3  # 50 mH
Vo = 5  # 5 V
w = 50  # 50 Hz
Is = 1e-9   # 1 nA
Ut = 25e-3  # 25 mV
n_Ut = 1.0

The function Id is part of a ODE that I'm trying to solve with Runge-Kutta-Method.

EDIT:

Methoden\Serien\6>python circuit.py
Traceback (most recent call last):
  File "circuit.py", line 148, in <module>
    perform_experiment(exrk_o5(), "exrk_o5")
  File "circuit.py", line 48, in perform_experiment
    t, y = run_method(method, label)
  File "circuit.py", line 69, in run_method
    t, y = method.__call__(circuit_rhs, y[0, :], t_end, n_steps)
  File "C:\Users\-\numerical_methodes\6\rk.py", line 49, in __call__
    t[k+1],y[k+1, :] = self.step(f, y[k, :], t[k], dt)
  File "C:\Users\-\numerical_methodes\6\rk.py", line 102, in step
    dydt[:, i] = rhs(t + c[i] * dt, y0 + dt * np.dot(A[i, :].T, dydt.T))
  File "circuit.py", line 38, in circuit_rhs
    dydt=  np.array ([y[1] , Id(t, y [1]) /(C*L) - y [1]/( R*C) - y [0]/( C*L)])
  File "circuit.py", line 27, in <lambda>
    Id = lambda t_u, yp: Is * (np.exp((Vin(t_u) - L*yp)/(n_Ut*Ut)) - 1.0)
FloatingPointError: overflow encountered in exp

where y[1]=0 and y[0]=0, as first values. The rk.py file is just the implementation of the Runge-Kutta-Method.


问:
Database installer failing to create DacStore object since updating to SQL Server 2016
I've got an installer that creates or updates a database associated with a website.  We originally wrote the installer for Microsoft Server 2008 R2, and SQL Server 2008.  We've updated to Microsoft Server 2016, and SQL Server 2016 and I now get failures with the dacpac.

I've updated the DAC framework on the system, but the issue persists.  The specific part that fails is in a PowerShell script.

New-Object : Cannot find type [Microsoft.SqlServer.Management.Dac.DacStore]: verify that the assembly containing this
      type is loaded.
      At C:\Users\Administrator\AppData\Local\Temp\2_ir_sf_temp_3\UnregisterDataTier.ps1:29 char:13
      + $dacstore = New-Object Microsoft.SqlServer.Management.Dac.DacStore($s ...
      +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
          + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

The section of code (line 29 is the final line):

```<code>If (!$uid -And !$pwd)
{
   $serverconnection.ConnectionString  = "Server = $server; Database = $database ; Integrated Security = True;"
}
Else
{
   $serverconnection.ConnectionString  = "Server = $server; Database = $database ; Integrated Security = False; User ID = $uid; Password = $pwd;"
}

$dacstore = New-Object Microsoft.SqlServer.Management.Dac.DacStore($serverconnection)
</code>```

I've been trying everything I can find with others getting the same error message, but it doesn't seem to apply to my situation.  I am assuming there was some minor change somewhere that I just cannot locate.

This works perfectly on 2008, but I cannot find what might have changed that causes this error.

答: Hmm are you running this directly on the SQL 2016 server (or on a computer with only the 2016 tool set?).  According to this MS Social page, the DAC library was deprecated starting with SQL 2012.   You should be able to use the SMO libraries or other .NET libraries to create the connection string at least.  However you may need to refactor your script a little for some of your commands.</p>

Body:
I've got an installer that creates or updates a database associated with a website.  We originally wrote the installer for Microsoft Server 2008 R2, and SQL Server 2008.  We've updated to Microsoft Server 2016, and SQL Server 2016 and I now get failures with the dacpac.

I've updated the DAC framework on the system, but the issue persists.  The specific part that fails is in a PowerShell script.

New-Object : Cannot find type [Microsoft.SqlServer.Management.Dac.DacStore]: verify that the assembly containing this
      type is loaded.
      At C:\Users\Administrator\AppData\Local\Temp\2_ir_sf_temp_3\UnregisterDataTier.ps1:29 char:13
      + $dacstore = New-Object Microsoft.SqlServer.Management.Dac.DacStore($s ...
      +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
          + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

The section of code (line 29 is the final line):

If (!$uid -And !$pwd)
{
   $serverconnection.ConnectionString  = "Server = $server; Database = $database ; Integrated Security = True;"
}
Else
{
   $serverconnection.ConnectionString  = "Server = $server; Database = $database ; Integrated Security = False; User ID = $uid; Password = $pwd;"
}

$dacstore = New-Object Microsoft.SqlServer.Management.Dac.DacStore($serverconnection)

I've been trying everything I can find with others getting the same error message, but it doesn't seem to apply to my situation.  I am assuming there was some minor change somewhere that I just cannot locate.

This works perfectly on 2008, but I cannot find what might have changed that causes this error.


问:
React prettifying URLs
My company has urls like this:

```<code>    http://helloworld.com/product?filter[category][0]=persian
</code>```

We want the url to look like this:

```<code>    http://helloworld.com/product-persian
</code>```

When a filter is applied to the current url it looks like this:

```<code>    http://helloworld.com/product?filter[category][0]=persian&filter[style][0]=border
</code>```

We'd like the filters to stay the same. so the new url we'd like would show this:

```<code>    http://helloworld.com/product-persian&filter[style][0]=border
</code>```

Using react, is there an optimal native way of doing this? I've briefly looked into https://github.com/reactjs/react-router-tutorial/tree/master/lessons/10-clean-urls and briefly into redis. What are the methods you use to do this when working with the react framework?

答:
To use URLs like these:
http://helloworld.com/product-persian, react-router is good option.

Write it like this:

```<code>import React from 'react'
import ReactDOM from 'react-dom'
import { Router, Route, browserHistory } from 'react-router'

const Routes = React.createClass({
  render(){
    return(
       <Router history={ browserHistory }>
         <Route path='/home' component={ homeComponent }/>
         <Route path='/product' component={ productsComponent }>
           <Route path='/product-:productName' component={ sigleProductComponent }/> {/* Single product route */}
           <Route path='/product-:productName&filter=:filterData' component={ productsComponent }/> {/* Product with filter params route */}
         </Route>
       </Router>
    )
  }
})

ReactDOM.render(<Routes />, document.getElementById('app'))
</code>```

In this code example you use params on urls, to call this prop use this.props.params.productName or this.props.params.filterData in your component sigleProductComponent.

This props are pass from react-router component.

I hope this code helps you.

Title: React prettifying URLs

Body:
My company has urls like this:

    http://helloworld.com/product?filter[category][0]=persian

We want the url to look like this:

    http://helloworld.com/product-persian

When a filter is applied to the current url it looks like this:

    http://helloworld.com/product?filter[category][0]=persian&filter[style][0]=border

We'd like the filters to stay the same. so the new url we'd like would show this:

    http://helloworld.com/product-persian&filter[style][0]=border

Using react, is there an optimal native way of doing this? I've briefly looked into https://github.com/reactjs/react-router-tutorial/tree/master/lessons/10-clean-urls and briefly into redis. What are the methods you use to do this when working with the react framework?

Body:
To use URLs like these:
http://helloworld.com/product-persian, react-router is good option.

Write it like this:

import React from 'react'
import ReactDOM from 'react-dom'
import { Router, Route, browserHistory } from 'react-router'

const Routes = React.createClass({
  render(){
    return(
       <Router history={ browserHistory }>
         <Route path='/home' component={ homeComponent }/>
         <Route path='/product' component={ productsComponent }>
           <Route path='/product-:productName' component={ sigleProductComponent }/> {/* Single product route */}
           <Route path='/product-:productName&filter=:filterData' component={ productsComponent }/> {/* Product with filter params route */}
         </Route>
       </Router>
    )
  }
})

ReactDOM.render(<Routes />, document.getElementById('app'))

In this code example you use params on urls, to call this prop use this.props.params.productName or this.props.params.filterData in your component sigleProductComponent.

This props are pass from react-router component.

I hope this code helps you.


问:
In rails, whether to use form helpers or not to?
In rails, is it recommended to use form helpers? Internally, everything boils down to plain html then why not write the html directly? Performance will obviously be better in writing direct html than using helpers. Is using form helpers like a convention or something that rails developers must follow?</p>

答: I think that form helpers is a reflection of the DRY (don't repeat yourself) principle.  Rather than writing the same code over to do similar tasks, creating a form helper that allows you to reuse that code is the way to go.  That way if you need to make a change or fix, you only need to do it in one place.  It also helps to make your code more compact and readable to abstract a complex action into a form helper.  The same is true of partial views, though partial views tend to encapsulate more complex mark up than a form helper.</p>

OwnerDisplayName: Chirantan

ContentLicense: CC BY-SA 2.5

LastEditorDisplayName: tvanfosson

OwnerDisplayName: tvanfosson


问:
How to add an icon to materialize tooltip?
We have this font awesome icon fa fa-twitter which at the current time sides next to the actual tooltip button of materialize what i need to do is to put it inside the tooltip

```<code><div class="center">
  <i class="fa fa-twitter"></i>
  <a class="btn tooltipped" data-position="bottom" data-delay="50" data-customcss="csszzz"  data-tooltip="I am tooltip" >Hover me!</a>
    </div>
</code>```

i know there is a data-customcss implementation aswell so theoritically we could implement it via .css also with url link of a photo icon but i can't figure out how that could be with angular

At the moment the data-customcss="csszzz" doesn't do anything as a pointed css class eg. .csszzz

答:
just add data-html="true" to your anchor element and move icon inside of tooltip:

```<code><div class="center">
  <a class="btn tooltipped"
      data-position="bottom"
      data-delay="50"
      data-html="true"
      data-tooltip="<i class='fa fa-twitter'></i> I am tooltip" >Hover me!</a>
</div>
</code>```

please look at http://materializecss.com/dialogs.html#tooltip for reference

plunker: https://plnkr.co/edit/3cZtphZ984R1ZqnkuLBX?p=preview

Body:
We have this font awesome icon fa fa-twitter which at the current time sides next to the actual tooltip button of materialize what i need to do is to put it inside the tooltip

  Hover me!

i know there is a data-customcss implementation aswell so theoritically we could implement it via .css also with url link of a photo icon but i can't figure out how that could be with angular

At the moment the data-customcss="csszzz" doesn't do anything as a pointed css class eg. .csszzz

Body:
just add data-html="true" to your anchor element and move icon inside of tooltip:

<div class="center">
  <a class="btn tooltipped"
      data-position="bottom"
      data-delay="50"
      data-html="true"
      data-tooltip="<i class='fa fa-twitter'></i> I am tooltip" >Hover me!</a>
</div>

please look at http://materializecss.com/dialogs.html#tooltip for reference

plunker: https://plnkr.co/edit/3cZtphZ984R1ZqnkuLBX?p=preview


问:
Ajax call is returning [object Object]
Html

```<code><div>
    <select id="method">
        <option value="get">GET</option>
        <option value="post">POST</option>
        <option value="put">PUT</option>
    </select>
    <input type="button" value="Try it" onclick="sendRequest()" />
    <span id='value1'>(Result)</span>
</div>
</code>```

Jquery

```<code>@section scripts {
    <script>
        // TODO: Replace with the URL of your WebService app
        var serviceUrl = 'http://localhost:99999/api/books';

        function sendRequest() {
            var method = $('#method').val();

            $.ajax({
                type: method,
                url: serviceUrl,
                dataType: 'json',
                success: function (data) {
                    $('#value1').text(data);
                    console.log(data);
                },
                error: function (jqXHR, textStatus, errorThrown) {
                    $('#value1').text(jqXHR.responseText || textStatus);
                }
            });
        }
    </script>
</code>```

The success function is showing up with arrays of [object, Object]. The console shows the data for each object, but it is not showing on the page.

For each book, I have to list the Id, Title, and the Author.

```<code>public IHttpActionResult Get()
        {
            IQueryable<BookDTO> books;
            books = from b in db.Books
                    select new BookDTO()
                    {
                        Id = b.Id,
                        Title = b.Title,
                        AuthorName = b.Author.Name
                    };
            return Ok(books);
</code>```

console.log Output

答:
```<code>$('#value1').text(data);
</code>```

That is a whole object you are setting as a text.

Probably you should want to look in to what data variable have and then append those values from that as text.

Title: Ajax call is returning [object Object]

Body:
Html

    <select id="method">
        <option value="get">GET</option>
        <option value="post">POST</option>
        <option value="put">PUT</option>
    </select>
    <input type="button" value="Try it" onclick="sendRequest()" />
    (Result)

Jquery

@section scripts {

The success function is showing up with arrays of [object, Object]. The console shows the data for each object, but it is not showing on the page.

For each book, I have to list the Id, Title, and the Author.

public IHttpActionResult Get()
        {
            IQueryable<BookDTO> books;
            books = from b in db.Books
                    select new BookDTO()
                    {
                        Id = b.Id,
                        Title = b.Title,
                        AuthorName = b.Author.Name
                    };
            return Ok(books);

console.log Output

Body:
$('#value1').text(data);

That is a whole object you are setting as a text.

Probably you should want to look in to what data variable have and then append those values from that as text.


问:
Simulating finding a randomly chosen number by asking binary questions
As a question in an assignment, I have been asked to write an Octave function that simulates 1000 experiments of finding a Random Variable X with alphabet {0, 1, 2, 3} and pmf:

Px(0) = 1/8

Px(1) = 1/4

Px(2) = 1/2

Px(3) = 1/8

by asking a sequence of binary, "yes" or "no" questions.

I have determined that the optimal sequence of binary questions asked to find the value of X is to simply ask "Is X = p?" where p are the possible values, in order of decreasing probabilty.

So the optimal sequence would be:

Is X = 2?

If not:

Is X = 1?

If not:

Is X = 0?

If not then X = 3

This is the function I have written:

```<code>function x = guessing_experiment(probabilities, n)
  % generates n simulations of finding a random number in an alphabet by asking binary questions,
  % where 'probabilities' is a list of the probabilities per number in the order the questions will be asked

  num_Qs = zeros(1,n);                            % allocate array of size n for number of questions asked per experiment
  [num_col, alphabet_size] = size(probabilities); % get size of alphabet

  for i = 1:n                                     % generate n experiments
    Qs = 0;                                       % number of questions asked in this experiment
    for j = 1:alphabet_size - 1                   % iterate through questions
      question = rand;                            % generate random number in range [0, 1]
      Qs++;                                       % incremenet number of questions asked
      if (question <= probabilities(j))           % if question produces a "yes" answer
        break;
      endif
    endfor
    num_Qs(i) = Qs;                               % store number of questions asked for this experiment
  endfor

  x = mean(num_Qs);                               % calculate mean number of questions asked over the n experiments

 end
</code>```

Which is called as guessing_experiment([1/2, 1/4, 1/8, 1/8], 1000)
where the array is the probability of each question producing a "yes" answer, in order of how they are to be asked, and n is the number of experiments.

Asking these questions should produce an average number of questions of 1.75, but my program is always producing an average of ~1.87. Where is my script in error?

I am assuming it has something to do with generating a new random number to simulate each of the 3 questions being asked.

答:
I deleted my previous wrong answer which was stated the your script is right and your calculation is wrong. I think about it once more and your calculation is right. I tried it myself using the following MATLAB script:

```<code>% probabilities for each number
p = [1/8,1/4,1/2,1/8];
% sort them from higher to lower
p = sort(p,'descend');
% number of questions per probability
nq = 1:length(p)-1;
% the last question can distinguish between two variables
nq(end+1) = nq(end);
% number of trials
n = 100000;
% random sample number of questions
q = randsample(nq,n,true,p);
% mean number of questions
avgQ = mean(q)
</code>```

and the obtained avg. was 1.75 - as you calculated.
I will try to take another look on your code to see what's wrong

EDIT

the problem with your script is that you ignored the conditional probability, i.e., when asking questions about the variable you ignored the information you already know about it. for example, when you're asking your third question, the probability that the value will be 0 is not p=1/8 but p=1/2 because you already know that it is not 1 or 2.
The fix you need to do is as dividing the probability by the possible events probabilities probabilities(j)/sum(probabilities(j:end)):

```<code>n = 10000;
p = [1/8,1/4,1/2,1/8];
% sort them from higher to lower
probabilities = sort(p,'descend');
probabilities(end-1) = probabilities(end-1) + probabilities(end);
probabilities(end) = [];
alphabet_size = numel(probabilities);
num_Qs = zeros(1,n);                            % allocate array of size n for number of questions asked per experiment

for i = 1:n                                     % generate n experiments
    Qs = 0;                                       % number of questions asked in this experiment
    for j = 1:alphabet_size                   % iterate through questions
        question = rand;                            % generate random number in range [0, 1]
        Qs = Qs + 1;                                       % incremenet number of questions asked
        if question < probabilities(j)/sum(probabilities(j:end))           % if question produces a "yes" answer
            break;
        end
    end
    num_Qs(i) = Qs;                               % store number of questions asked for this experiment
end

x = mean(num_Qs)
</code>```

x ~ 1.75

the vector of conditional probabilities in this scenario is:

```<code>p = [1/8,1/4,1/2,1/8];
p = sort(p,'descend');
cond_p = p./cumsum(p,'reverse')

cond_p =

    0.5000    0.5000    0.5000    1.0000
</code>```

Body:
As a question in an assignment, I have been asked to write an Octave function that simulates 1000 experiments of finding a Random Variable X with alphabet {0, 1, 2, 3} and pmf:

Px(0) = 1/8

Px(1) = 1/4

Px(2) = 1/2

Px(3) = 1/8

by asking a sequence of binary, "yes" or "no" questions.

I have determined that the optimal sequence of binary questions asked to find the value of X is to simply ask "Is X = p?" where p are the possible values, in order of decreasing probabilty.

So the optimal sequence would be:

Is X = 2?

If not:

Is X = 1?

If not:

Is X = 0?

If not then X = 3

This is the function I have written:

function x = guessing_experiment(probabilities, n)
  % generates n simulations of finding a random number in an alphabet by asking binary questions,
  % where 'probabilities' is a list of the probabilities per number in the order the questions will be asked

  num_Qs = zeros(1,n);                            % allocate array of size n for number of questions asked per experiment
  [num_col, alphabet_size] = size(probabilities); % get size of alphabet

  for i = 1:n                                     % generate n experiments
    Qs = 0;                                       % number of questions asked in this experiment
    for j = 1:alphabet_size - 1                   % iterate through questions
      question = rand;                            % generate random number in range [0, 1]
      Qs++;                                       % incremenet number of questions asked
      if (question <= probabilities(j))           % if question produces a "yes" answer
        break;
      endif
    endfor
    num_Qs(i) = Qs;                               % store number of questions asked for this experiment
  endfor

  x = mean(num_Qs);                               % calculate mean number of questions asked over the n experiments

 end

Which is called as guessing_experiment([1/2, 1/4, 1/8, 1/8], 1000)
where the array is the probability of each question producing a "yes" answer, in order of how they are to be asked, and n is the number of experiments.

Asking these questions should produce an average number of questions of 1.75, but my program is always producing an average of ~1.87. Where is my script in error?

I am assuming it has something to do with generating a new random number to simulate each of the 3 questions being asked.

Body:
I deleted my previous wrong answer which was stated the your script is right and your calculation is wrong. I think about it once more and your calculation is right. I tried it myself using the following MATLAB script:

% probabilities for each number
p = [1/8,1/4,1/2,1/8];
% sort them from higher to lower
p = sort(p,'descend');
% number of questions per probability
nq = 1:length(p)-1;
% the last question can distinguish between two variables
nq(end+1) = nq(end);
% number of trials
n = 100000;
% random sample number of questions
q = randsample(nq,n,true,p);
% mean number of questions
avgQ = mean(q)

and the obtained avg. was 1.75 - as you calculated.
I will try to take another look on your code to see what's wrong

EDIT

the problem with your script is that you ignored the conditional probability, i.e., when asking questions about the variable you ignored the information you already know about it. for example, when you're asking your third question, the probability that the value will be 0 is not p=1/8 but p=1/2 because you already know that it is not 1 or 2.
The fix you need to do is as dividing the probability by the possible events probabilities probabilities(j)/sum(probabilities(j:end)):

n = 10000;
p = [1/8,1/4,1/2,1/8];
% sort them from higher to lower
probabilities = sort(p,'descend');
probabilities(end-1) = probabilities(end-1) + probabilities(end);
probabilities(end) = [];
alphabet_size = numel(probabilities);
num_Qs = zeros(1,n);                            % allocate array of size n for number of questions asked per experiment

for i = 1:n                                     % generate n experiments
    Qs = 0;                                       % number of questions asked in this experiment
    for j = 1:alphabet_size                   % iterate through questions
        question = rand;                            % generate random number in range [0, 1]
        Qs = Qs + 1;                                       % incremenet number of questions asked
        if question < probabilities(j)/sum(probabilities(j:end))           % if question produces a "yes" answer
            break;
        end
    end
    num_Qs(i) = Qs;                               % store number of questions asked for this experiment
end

x = mean(num_Qs)

x ~ 1.75

the vector of conditional probabilities in this scenario is:

p = [1/8,1/4,1/2,1/8];
p = sort(p,'descend');
cond_p = p./cumsum(p,'reverse')

cond_p =

    0.5000    0.5000    0.5000    1.0000


问:
Java. How do I make code more effectly, HashMap?
I have started learn Java sometime ago, and now I have interesting quest from my teacher. I will be very grateful for everyone who can help me. And I don`t want to get directly answer for my question, I want to see way which help for me understand core.

This is fragment from my code.

Higher I have some text (String) which was convert to List.

```<code>List<String> list = Arrays.asList( text.split( " "));

Map<String, Integer> hashMap = new HashMap<>();
int uniqueWords = 0;
for (String word: list){
    Integer count = hashMap.get(word);
    hashMap.put(word, count == null ? 1 : count + 1);
    if (count == null){
        uniqueWords++;
    }else if (count == 1){
        uniqueWords --;
    }
}
</code>```

I need to do code more effective. I thinking several days and can`t prupose more effectly code. I need help )

答:
Use a Set:</p>

```<code>String text = "one two three one";
List<String> list = Arrays.asList( text.split( " "));

Set<String> uniqs = new HashSet<>();
uniqs.addAll(list);
System.out.println(uniqs.size()); // prints 3
</code>```

Body:
I have started learn Java sometime ago, and now I have interesting quest from my teacher. I will be very grateful for everyone who can help me. And I don`t want to get directly answer for my question, I want to see way which help for me understand core.

This is fragment from my code.

Higher I have some text (String) which was convert to List.

List<String> list = Arrays.asList( text.split( " "));

Map<String, Integer> hashMap = new HashMap<>();
int uniqueWords = 0;
for (String word: list){
    Integer count = hashMap.get(word);
    hashMap.put(word, count == null ? 1 : count + 1);
    if (count == null){
        uniqueWords++;
    }else if (count == 1){
        uniqueWords --;
    }
}

I need to do code more effective. I thinking several days and can`t prupose more effectly code. I need help )

Body:
Use a Set:

String text = "one two three one";
List<String> list = Arrays.asList( text.split( " "));

Set<String> uniqs = new HashSet<>();
uniqs.addAll(list);
System.out.println(uniqs.size()); // prints 3


问:
Azure-Functions: How to serve content from the root of domain
Is there any way to serve content out of a root of the domain with an azure function without using a proxy?

I defined a function with a route of / and it appears to be ignored.

I've found a way to use the Proxies (beta), to serve content from the root, but this doesn't work for local development (Proxies appear to only be on Azure, not in the azure-functions-cli).

In various issues I've found a few cryptic references indicating it may be possible, or was possible at one time, but haven't figured out how to.

答:
Not exactly what you're asking for, but my approach would be to use fiddler to emulate Proxies locally.

You can add the following to FiddlerScript:

```<code>OnBeforeRequest(oSession: Session) {
...
    if (oSession.host == "127.0.0.1:5050" && oSession.PathAndQuery == "/") {
        oSession.PathAndQuery = "/content";
    }
...
}
</code>```

This will catch any requests to localhost 127.0.0.1 at the specified port 5050 with root path / and redirect to /content path. You'll need to modify this for your specific port and path.

I'd also encourage you to file an issue on the azure functions cli github

Body:
Not exactly what you're asking for, but my approach would be to use fiddler to emulate Proxies locally.

You can add the following to FiddlerScript:

OnBeforeRequest(oSession: Session) {
...
    if (oSession.host == "127.0.0.1:5050" && oSession.PathAndQuery == "/") {
        oSession.PathAndQuery = "/content";
    }
...
}

This will catch any requests to localhost 127.0.0.1 at the specified port 5050 with root path / and redirect to /content path. You'll need to modify this for your specific port and path.

I'd also encourage you to file an issue on the azure functions cli github


问:
How to set default date of format 'MM/dd/yyyy'
Here is my code for creating calendar by using Bootstrap datetimepicker:

```<code>$('#datetimepicker12').datetimepicker({
    inline: true,
    format: 'MM/dd/yyyy',
    minDate: new Date(moment(new Date()).add(0, 'days').toDate()),
    maxDate: new Date(moment(new Date()).add(7, 'days').toDate()),
});
</code>```

Now I want to set default date of calendar to be my custom date, its format is like this '2017-03-30'

How can I convert that to be the same format as format of datetimepicker that I have set before?

答:
```<code>$.datepicker.setDefaults({
    dateFormat: 'yy-mm-dd'
});
</code>```

Body:
Here is my code for creating calendar by using Bootstrap datetimepicker:

$('#datetimepicker12').datetimepicker({
    inline: true,
    format: 'MM/dd/yyyy',
    minDate: new Date(moment(new Date()).add(0, 'days').toDate()),
    maxDate: new Date(moment(new Date()).add(7, 'days').toDate()),
});

Now I want to set default date of calendar to be my custom date, its format is like this '2017-03-30'

How can I convert that to be the same format as format of datetimepicker that I have set before?


问:
chartjs : how to set custom scale in bar chart
Chartjs is a pretty excellent open source tool, but had a quick question about a bar chart I'm trying to create. Given this chart data:

```<code>    var chartData = {
        labels : labels,
        datasets :[
            {
                fillColor : "rgba(220,220,220,0.5)",
                strokeColor : "rgba(220,220,220,0.8)",
                highlightFill: "rgba(220,220,220,0.75)",
                highlightStroke: "rgba(220,220,220,1)",
                scaleOverride: true,
                scaleSteps: 9,
                data : values
            }
        ]
    }
</code>```

I had hoped that the chart would draw with top value of 10, whether or not there were any values of 10. I thought the scaleOverride and scaleSteps would accomplish that.

Is it possible to get that output? I went thru the docs and did not see any other obvious options to set.

Update

got it to work. My orig post did not include the javascript function at the bottom.

```<code><div class="barChartTest" id="rating_12229" onload="drawChart();">
<input type="hidden" class="rating-component" comp-name="test1" comp-value="6"/>
<input type="hidden" class="rating-component" comp-name="test2" comp-value="7"/>
<input type="hidden" class="rating-component" comp-name="test3" comp-value="6"/>
<input type="hidden" class="rating-component" comp-name="test4" comp-value="5"/>
<input type="hidden" class="rating-component" comp-name="test5" comp-value="1"/>
</div>
<canvas id="rating_12229" width="50" height="20"></canvas>
</code>```

and here is my javascript:

```<code>function buildRatingChartData(ratingId){
    var comps = document.getElementById(ratingId).getElementsByClassName("rating-component");
    var labels = [];
    var values = [];

    for(var i=0; i<comps.length; i++){

            labels.push(comps[i].getAttribute("comp-name"));
            values.push(comps[i].getAttribute("comp-value"));
    }

    var chartData = {
        labels : labels,
        datasets :[
            {
                scale: 10,
                fillColor : "rgba(220,220,220,0.5)",
                strokeColor : "rgba(220,220,220,0.8)",
                highlightFill: "rgba(220,220,220,0.75)",
                highlightStroke: "rgba(220,220,220,1)",
                scaleOverride:true,
                scaleSteps:9,
                scaleStartValue:0,
                scaleStepWidth:1,
                data : values
            }
        ]
    }

    return chartData;

}
</code>```

here was where I added those options and got the output I wanted:

```<code>function drawCharts(){
    var ratings = document.getElementsByClassName("brewRatingData");
    for(var i=0; i<ratings.length; i++){

        var ratingId = ratings[i].getAttribute("id");
        var canvasId = ratingId.replace("brewRating", "coffeeBarChart");

      var brewChartData = buildRatingChartData(ratingId);
      var ctx = document.getElementById(canvasId).getContext("2d");
      window.myBar = new Chart(ctx).Bar(brewChartData, {
        responsive : true,
            scaleOverride:true,
            scaleSteps:10,
            scaleStartValue:0,
            scaleStepWidth:1,
      });

    }
}
</code>```

答:
```<code>var myChart = new Chart(ctx, {
                type: 'bar',
                data:data,
                options: {
                    responsive: true,
                    scales: {
                        yAxes: [{
                            ticks: {
                                beginAtZero: true,
                                steps: 10,
                                stepValue: 6,
                                max: 60 //max value for the chart is 60
                                }
                            }
                        }]
                    }
                }
            });
</code>```

Body:
Chartjs is a pretty excellent open source tool, but had a quick question about a bar chart I'm trying to create. Given this chart data:

    var chartData = {
        labels : labels,
        datasets :[
            {
                fillColor : "rgba(220,220,220,0.5)",
                strokeColor : "rgba(220,220,220,0.8)",
                highlightFill: "rgba(220,220,220,0.75)",
                highlightStroke: "rgba(220,220,220,1)",
                scaleOverride: true,
                scaleSteps: 9,
                data : values
            }
        ]
    }

I had hoped that the chart would draw with top value of 10, whether or not there were any values of 10. I thought the scaleOverride and scaleSteps would accomplish that.

Is it possible to get that output? I went thru the docs and did not see any other obvious options to set.

Update

got it to work. My orig post did not include the javascript function at the bottom.

<input type="hidden" class="rating-component" comp-name="test1" comp-value="6"/>
<input type="hidden" class="rating-component" comp-name="test2" comp-value="7"/>
<input type="hidden" class="rating-component" comp-name="test3" comp-value="6"/>
<input type="hidden" class="rating-component" comp-name="test4" comp-value="5"/>
<input type="hidden" class="rating-component" comp-name="test5" comp-value="1"/>

<canvas id="rating_12229" width="50" height="20"></canvas>

and here is my javascript:

function buildRatingChartData(ratingId){
    var comps = document.getElementById(ratingId).getElementsByClassName("rating-component");
    var labels = [];
    var values = [];

    for(var i=0; i<comps.length; i++){

            labels.push(comps[i].getAttribute("comp-name"));
            values.push(comps[i].getAttribute("comp-value"));
    }

    var chartData = {
        labels : labels,
        datasets :[
            {
                scale: 10,
                fillColor : "rgba(220,220,220,0.5)",
                strokeColor : "rgba(220,220,220,0.8)",
                highlightFill: "rgba(220,220,220,0.75)",
                highlightStroke: "rgba(220,220,220,1)",
                scaleOverride:true,
                scaleSteps:9,
                scaleStartValue:0,
                scaleStepWidth:1,
                data : values
            }
        ]
    }

    return chartData;

}

here was where I added those options and got the output I wanted:

function drawCharts(){
    var ratings = document.getElementsByClassName("brewRatingData");
    for(var i=0; i<ratings.length; i++){

        var ratingId = ratings[i].getAttribute("id");
        var canvasId = ratingId.replace("brewRating", "coffeeBarChart");

      var brewChartData = buildRatingChartData(ratingId);
      var ctx = document.getElementById(canvasId).getContext("2d");
      window.myBar = new Chart(ctx).Bar(brewChartData, {
        responsive : true,
            scaleOverride:true,
            scaleSteps:10,
            scaleStartValue:0,
            scaleStepWidth:1,
      });

    }
}


问:
AWK syntax error while using the IF statement
I am very new to AWK although I have previously used the command prompt/terminal.

I have this script below where I am creating subsets of data based on Country Code and State Code. But I get a syntax error.

```<code>BEGIN{
   FS = "\t"
   OFS = "\t"
   }

 # Subset data from the states you need for all years
 if ($5 == "IN-GA" || $5 == "IN-DD" || $5 == "IN-DN" || $5 == "IN-KA" || $5 == "IN-KL" || $5 == "IN-MH" || $5 == "IN-TN" || $5 == "IN-GJ"){
        if (substr($17, 1, 4) == "2000"){
            print $5, $12, $13, $14, $15, $16, $17, $22, $23, $24, $25, $26, $28 > "Y2000_India_sampling_output.txt"
        }
    }
</code>```

On Cygwin, I refer to the script and I run the below lines of code and you see the syntax error immediately:

```<code>$ gawk -f sampling_India.awk sampling_relFeb-2017.txt
gawk: sampling_India.awk:20:  gawk if ($5 == "IN-GA" || $5 == "IN-DD" || $5 == "IN-DN" || $5 == "IN-KA" || $5 == "IN-KL" || $5 == "IN-MH" || $5 == "IN-TN" || $5 == "IN-GJ"){
gawk: sampling_India.awk:20:       ^ syntax error
</code>```

Any thoughts?

答:
Your if condition is not enclosed in {...} block.

Have it like this:

```<code>BEGIN {
   FS = OFS = "\t"
}
# Subset data from the states you need for all years
$5 ~ /^IN-(GA|DD|DN|KA|KL|MH|TN|GJ)$/ && substr($17, 1, 4) == "2000" {
    print $5, $12, $13, $14, $15, $16, $17, $22, $23, $24, $25, $26, $28 > "Y2000_India_sampling_output.txt"
}
</code>```

Note how using regex you can combine multiple == conditions into a single condition.

Body:
I am very new to AWK although I have previously used the command prompt/terminal.

I have this script below where I am creating subsets of data based on Country Code and State Code. But I get a syntax error.

BEGIN{
   FS = "\t"
   OFS = "\t"
   }

 # Subset data from the states you need for all years
 if ($5 == "IN-GA" || $5 == "IN-DD" || $5 == "IN-DN" || $5 == "IN-KA" || $5 == "IN-KL" || $5 == "IN-MH" || $5 == "IN-TN" || $5 == "IN-GJ"){
        if (substr($17, 1, 4) == "2000"){
            print $5, $12, $13, $14, $15, $16, $17, $22, $23, $24, $25, $26, $28 > "Y2000_India_sampling_output.txt"
        }
    }

On Cygwin, I refer to the script and I run the below lines of code and you see the syntax error immediately:

$ gawk -f sampling_India.awk sampling_relFeb-2017.txt
gawk: sampling_India.awk:20:  gawk if ($5 == "IN-GA" || $5 == "IN-DD" || $5 == "IN-DN" || $5 == "IN-KA" || $5 == "IN-KL" || $5 == "IN-MH" || $5 == "IN-TN" || $5 == "IN-GJ"){
gawk: sampling_India.awk:20:       ^ syntax error

Any thoughts?

Body:
Your if condition is not enclosed in {...} block.

Have it like this:

BEGIN {
   FS = OFS = "\t"
}
# Subset data from the states you need for all years
$5 ~ /^IN-(GA|DD|DN|KA|KL|MH|TN|GJ)$/ && substr($17, 1, 4) == "2000" {
    print $5, $12, $13, $14, $15, $16, $17, $22, $23, $24, $25, $26, $28 > "Y2000_India_sampling_output.txt"
}

Note how using regex you can combine multiple == conditions into a single condition.


问:
Python - Beautiful Soup to grab emails from website
I've been trying to figure out a simple way to run through a set of URLs that lead to pages that all have the same layout. We figured out that one issue is that in the original list the URLs are http but then they redirect to https. I am not sure if that then causes a problem in trying to pull the information from the page. I can see the structure of the page when I use Inspector in Chrome, but when I try to set up the code to grab relevant links I come up empty (literally). The most general code I have been using is:

```<code>soup = BeautifulSoup(urllib2.urlopen('https://ngcproject.org/program/algirls').read())
links = SoupStrainer('a')
print links
</code>```

which yields:

```<code>a|{}
</code>```

Given that I'm new to this I've been trying to work with anything that I think might work. I also tried:

```<code>mail = soup.find(attrs={'class':'tc-connect-details_send-email'}).a['href']
</code>```

and

```<code>spans = soup.find_all('span', {'class' : 'tc-connect-details_send-email'})
lines = [span.get_text() for span in spans]
print lines
</code>```

but these don't yield anything either.

I am assuming that it's an issue with my code and not one that the data are hidden from being scraped. Ideally I want to have the data passed to a CSV file for each URL I scrape but right now I need to be able to confirm that the code is actually grabbing the right information. Any suggestions welcome!

答:
If you press CTRL+U on Google Chrome or Right click > view source.
You'll see that the page is rendered by using javascript or other.
urllib is not going to be able to display/download what you're looking for.
You'll have to use automated browser (Selenium - most popular) and you can use it with Google Chrome / Firefox or a headless browser (PhantomJS).

You can then get the information from Selenium and store it then manipulate it in anyway you see fit.

Body:
I've been trying to figure out a simple way to run through a set of URLs that lead to pages that all have the same layout. We figured out that one issue is that in the original list the URLs are http but then they redirect to https. I am not sure if that then causes a problem in trying to pull the information from the page. I can see the structure of the page when I use Inspector in Chrome, but when I try to set up the code to grab relevant links I come up empty (literally). The most general code I have been using is:

soup = BeautifulSoup(urllib2.urlopen('https://ngcproject.org/program/algirls').read())
links = SoupStrainer('a')
print links

which yields:

a|{}

Given that I'm new to this I've been trying to work with anything that I think might work. I also tried:

mail = soup.find(attrs={'class':'tc-connect-details_send-email'}).a['href']

and

spans = soup.find_all('span', {'class' : 'tc-connect-details_send-email'})
lines = [span.get_text() for span in spans]
print lines

but these don't yield anything either.

I am assuming that it's an issue with my code and not one that the data are hidden from being scraped. Ideally I want to have the data passed to a CSV file for each URL I scrape but right now I need to be able to confirm that the code is actually grabbing the right information. Any suggestions welcome!


问:
DM-theory definition: To what side does a carinality of an entity refer?
Of cource there is a cardinality of a relation, for example (0-1 : 1-n).
But can you express something like: "The cardinality of A in this relation is 0-1" or "The cardinality of B in this relation is 1-n."

I am building a meta data model for data modeling and I want to name the properties for the two entities` cardinalities (something like cardinality_a_min, ..._max, ...b_min, ...b_max). Of cource I can write comments to ensure that everybody knows to which side these properties refer to. But I am wondering if this question is well defined in theory.

I know that in one ERD-Notation A (0-1) : (1-n) B means that there are many As and in another notation it means that there are many Bs. So is this true also for the data modeling theory? Or is it clearly defined?
The problem of saying "The cardinality of A is 1-n" could be the following: Do I have many As now or does an A point to many other entities?

答:
You can think of the cardinality of an attribute in a relation as the number of values from that attribute's domain that can be associated with each valid combination of values from the other attributes in the relation. For example:

In a 1:M relation (e.g. a relation (A,B) with candidate key B), each value in A can be associated with many values in B, and each value in B can be associated with a single value in A.

1:1 and M:N relations are straightforward, I won't enumerate them. This approach holds for higher-degree relations as well:

In a M:N:1 relation (e.g. a relation (A,B,C) with candidate key A,B), each combination of A,B values can be associated with a single value from C. Every other combination of values from one or more domains can be associated with many values from the remaining domains.

In a M:1:1 relation (e.g. a relation (A,B,C) with candidate key A), each A value can be associated with a single combination of B and C. Each B value can be associated with many combinations of A and C, and each C with many A,B.

In the last example, take care to avoid the notion that "each B is associated with a single C". Associations occur along functional or multivalued dependency lines, in this case via A. In other words, each B is associated with many A, each of which is associated with one C. Transitively, each B is associated with many C, and vice versa.

For more detail on this topic, I suggest you study the logical foundation of data modeling closely. Particular topics include functional and multivalued dependencies in relations and Armstrong's axioms.


问:
When to Denormalize a Database Design
I know that normalization has been extensively discussed on Stack Overflow. I've read many of the previous discussions. I've got some additional questions though.

I'm working on a legacy system with at least 100 tables. The database is has some un-normalized structure, tables that contain a variety of disparate data, and other problems. I've been given the task of trying to improve it. I can't just start again but need to modify the existing schema.

In the past I have always tried to design normalized databases. Now the questions. A senior developer has suggested that in some cases we can't normalize:

With temporal data. For example an invoice is created that links to a product. If a customer asks for a copy of this invoice a year later we must be able to produce an exact copy of the original. What if the product price, name or description have been updated? The senior guy suggested that the price and other product information should be copied to the invoice table. I'm thinking maybe we should have another table such as productPrice that has a date field so we can track changes in price over time. We would need the same thing for the product description and name I guess? Seems complicated. What do you think?

The database is an accounting system. I'm not very familiar with accounting. At the moment some summary data is derived and stored in the database. For example total sales for the year. My senior associate has said that accountants like to check things are correct by comparing this value with data that is actually calculated from invoices etc to give them confidence that the application is working correctly.

He said that at the moment for example we can tell if someone deleted an invoice from last year mistakenly because the totals will not be the same. He also pointed out that it could be quite slow to calculate these totals on the fly. Of course I said that data should not be duplicated and should always be calculated when needed. I suggested that we could use SQL Reporting Services or some other solution that will generate these reports overnight and cache them. Anyway he's not convinced. Any comments on this?

答:
Your senior colleague is a developer, not a data modeller. You are better off starting from scratch, without them. Normalisation is complicated only to those who will not read books. It is fair enough that he makes you think, but some of the issues are absurd.

Your numbers:

You need to appreciate the differences between actual online data, and historic data; then the difference between merely historic and archival needs.  All of them are right for the specific business requirement, and wrong for all others, there is no universal right and wrong.

why is there no paper-based copy of the invoice ?  In most countries that would be a legal and tax requirement, what exactly is the difficulty of fishing out the old invoice ?

where the database has the requirement of storing the closed invoices, then sure, as soon as the invoice is closed, you need a method of capturing that information.

ProductPrice (actually, I would call it ProductDate) is a good idea, but may not be necessary. But you are right, you need to evaluate the currency of data, in the full context of the whole database.

I cannot see how copying the product price to the invoice table would help (are not there many line items ?)

in modern databases, where the copy of the invoice is required to be regurgitated, the closed Invoice is additionally stored in a different form, eg XML. One customer saves the PDFs as BLOBs. So there is no messing around with what the product price was five years ago.  But the basic invoice data is online and current, even for closed invoices; you just cannot recalculate ancient invoice using current prices.

some people use an archive_invoice table, but that has problems because now every code segment or user report tool has to look in two places (note that these days some users understand databases better than most developers)

Anyway, that is all discussion, for your understanding.  Now for the technical intent.

The database serves current and archival purposes from the one set of tables (no "archive" tables

Once an Invoice is created, it is a legal document, and cannot be changed or deleted (it can be reversed or partially credited by a new Invoice, with negative values).  They are marked IsIssued/IsPaid/Etc

Products cannot be deleted, they can be marked IsObsolete

There are separate tables for InvoiceHeader and InvoiceItem

InvoiceItem has FKs to both InvoiceHeader and Product

for many reasons (not only those you mention), the InvoiceItem row contains the NumUnits; ProductPrice; TaxAmount; ExtendedPrice.  Sure, this looks like a "denormalisation" but it is not, because prices, taxation rates, etc, are subject to change.  But more important, the legal requirement is that we can reproduce the old invoice on demand.

(where it can be reproduced from paper files, this is not required)

the InvoiceTotalAmount is a derived column, just SUM() of the InvoiceItems

That is rubbish.  Accounting systems, and accountants do not "work" like that.

If it is a true accounting system, then it will have JournalEntries, or "double entry"; that is what a qualified account is required to use (by law).

Double Entry Accounting does not mean duplicate entries; it means every financial transaction (one amount) shall have a source account and target account that it is applied to; so there is no "denormalisation" or duplication.  In a banking database, because the financial transactions are against single accounts, that is commonly rendered as two separate financial transactions (rows) within one Db Transaction.  Ordinary commercial database constraints are used to ensure that there are two "sides" to every financial transaction.

Ensuring that Invoices are not deleteable is a separate issue, to do with security, etc.  if anyone is paranoid about things being deleted from their database, and their database was not secured by a qualified person, then they have more and different problems that have nothing to do with this question.  Obtain a security audit, and do whatever they tell you.

Wikipedia is not a reliable source of information about any technical subject, let alone Normalisation.

A Normalised database is always much faster than Un-Normalised database

So it is very important to understand what Normalisation and Denormalisaion is, and what it isn't.  The process is greatly hindered when people have fluid and amateur "definitions", it just leads to confusion and time-wasting "discussions".  When you have fixed definitions, you can avoid all that, and just get on with the job.

Summary tables are quite normal, to save the time and processing power, of recalculating info that does not change, eg: YTD totals for every year but this year; MTD totals for every month in this year but not this month. "Always recalculating" data is a bit silly when (a) the info is very large and (b) does not change.  Calculate for the current month only

In banking systems (millions of Trades per day), at EndOfDay, we calculate and store Daily Total as well.  These are overwritten for the last five days, because Auditors are making changes, and JournalEntries against financial transactions for the last 5 days are allowed.

non-banking systems generally do not need daily totals

Summary tables are not a "denormalisation" (except in the eyes of those who have just learned about "normalisation" from their magical, ever-changing fluid "source"; or as non-practitioners, who apply simple black-or-white rules to everything).  Again, the definition is not being argued here; it simply does not apply to Summary tables.

Summary tables do not affect data integrity (assuming of course that the data that they were sourced from was integral).

Summary tables are an addition to the current data, which are not required to have the same constraints as the current data.  There are essentially reporting tables or data warehouse tables, as opposed to current data tables.

There are no Update Anomalies (which is a strict definition) related to Summary tables.  You cannot change or delete an invoice from last year.  Update Anomalies apply to true Denormalised or Un-Normalised current data.

Response to Comments

so its ok to do denormalisation for the sakes of archiving right?

My explanation above appears not to be clear enough.  Let's look at an example, and compare the options.

So, no.  Archiving is a hideous option (I have corrected thousands of archived tables back to their home; corrected the indices, and restored normal performance, as well as returning the SELECT to query just one table instead of two).  But if you do archive, it is not denormalised, it is worse, a copy.

The best option by far.  Again, this is the simple version, the full version requires an understanding of the temporal definition in Codd's Relational Model (not the ever-changing nonsense marketed by the detractors), and an SQL-compliant platform.

I disagree strongly with the statement that "A Normalised database is always much faster than Unnormalised database". That is patently false even if I don't harp on the use of "always". There are many scenarios in which selective, coherent denormalization of a database can result in extreme performance improvement.

Example [of justified De-Normalisation]: a complex join for a date-limited data set, like a month-end summary for a complex business. If the data is collected over the course of a month; then forever stable; and queried often — it can make sense to pre-compute via a materialized view, trigger, or more sophisticated method.

Evidently we need clear definitions.

Normalisation

Purpose

to eliminate (not merely reduce) duplicate data

Note that there are no complex joins, just ordinary simple joins.

Method

Formal Relational data modelling

(not stepping through the NFs by number).

Un-Normalised

Failure to Normalise properly, leaving Update Anomalies in the database, and makes a mess of the Transactions

De-Normalised

After formal Normalisation, one or more columns additionally placed in chosen tables, for performance reasons.

Eg. the pretend "SQL" does not perform the normal query with acceptable speed.

Note that, over a period of 40 years of replacing poor databases,  all databases that were declared to be De-normalised have in fact been Un-Normalised.

Summary Table/Materialised View

As detailed above in the Answer (please read again), and as illustrated in the graphic Current Only, it is an additional table to serve the purpose of providing summary values for history, that does not change.  This is common.

There is no column that is duplicated, it cannot be categorised as "De-Normalisation".

You were not in disagreement with my statement [A Normalised database is always much faster than Unnormalised database], which relates to Normalised vs Un-Normalised, you were categorising the summary table incorrectly, as "de-Normalised".

If you can know in advance what complex, time-consuming queries a database will receive, you can precompute the results of those queries -- for example, replacing a 14-table join with a table that already contains the needed data.

That is different again.  The two common reasons for that are:

the database isn't Normalised in the first place, so queries on large tables are slow

the proper cure is to Normalise properly

your pretend "SQL" is so slow

the proper cure for that is to obtain a genuine SQL Platform.

Such that you need to build an additional file (or Materialised View) to service the slow queries.  Yes, that is de-Normalising, and on a grand scale, but it is worse, it is a 100% copy of those fields.

ContentLicense: CC BY-SA 4.0

Title: When to Denormalize a Database Design