问:
Multiple java processes in Tomcat
I am working on a web-based application which is deployed in the Tomcat server. In our local dev enviroemt, when we start the Tomcat server it spawns only one java process which keeps running. However, an issue has been reported in production where the CPU usage for java process has gone up and there are multiple java processes which have  been spawned.

There is no other java application running, so this must have been spawned from Tomcat itself. What is the reason that in our development enviroment there is only one java process while in production multiple java processes have been spawned by Tomcat and how to correct it.

Regards,
    Dev

答:
What you see are most likely multiple threads that the version of top or ps shows on the production box, while you don't see them on the local one.

In production you most likely face a much higher workload, so that requests are served in parallel, while on the local box Tomcat gets away with less threads.

Title: Multiple java processes in Tomcat


来源: stackexchange

问:
On-Demand Resources download but not accessible. Swif 4
In my iOS app I have a bunch of mp4 videos that I download at a certain time on the app using On Demand Resources. Using this tutorial:
https://www.raywenderlich.com/520-on-demand-resources-in-ios-tutorial

I download the resources like this at the start of the app, in a previous view controller:

```<code>func requestSceneWith(tag: String,
                  onSuccess: @escaping () -> Void,
                  onFailure: @escaping (NSError) -> Void) {

// 2
currentRequest = NSBundleResourceRequest(tags: [tag])

// 3
guard let request = currentRequest else { return }

request.beginAccessingResources { (error: Error?) in

    // 4
    if let error = error {
        onFailure(error as NSError)
        return
    }

    // 5
    onSuccess()

}
</code>```

The resource seem to download fine, and I know that they have been downloaded, by looking in the disk report in xcode.

However, when the videos are supposed to be played in the app, the app just shows a black screen. Here is my code to play the videos:

```<code>  let videoURL = Bundle.main.url(forResource: "cow2", withExtension: "mp4", subdirectory: "Videos/Animals")

    self.player = AVPlayer(url: videoURL!)

    self.myPlayerController.player = self.player

 self.myPlayerController.player?.play()
</code>```

Now, when the resources are not tagged, and they come with the app and not downloaded later, they work fine. And the console prints the file name, like I did (print(videoURL.absoluteString). But after they are tagged and downloaded later, they dont work, and nothing prints in the console. Just a black screen appears in the app.

I've been stuck on this for ages, and help with really help.

Thanks

答: I think you didn't request the on-demand resources before accessing the video file. Maybe you can try to declare the NSBundleResourceRequest instance as a global variable in AppDelegate.</p>

ContentLicense: CC BY-SA 4.0

Body:
In my iOS app I have a bunch of mp4 videos that I download at a certain time on the app using On Demand Resources. Using this tutorial:
https://www.raywenderlich.com/520-on-demand-resources-in-ios-tutorial

I download the resources like this at the start of the app, in a previous view controller:

func requestSceneWith(tag: String,
                  onSuccess: @escaping () -> Void,
                  onFailure: @escaping (NSError) -> Void) {

// 2
currentRequest = NSBundleResourceRequest(tags: [tag])

// 3
guard let request = currentRequest else { return }

request.beginAccessingResources { (error: Error?) in

    // 4
    if let error = error {
        onFailure(error as NSError)
        return
    }

    // 5
    onSuccess()

}

The resource seem to download fine, and I know that they have been downloaded, by looking in the disk report in xcode.

However, when the videos are supposed to be played in the app, the app just shows a black screen. Here is my code to play the videos:

  let videoURL = Bundle.main.url(forResource: "cow2", withExtension: "mp4", subdirectory: "Videos/Animals")

    self.player = AVPlayer(url: videoURL!)

    self.myPlayerController.player = self.player

 self.myPlayerController.player?.play()

Now, when the resources are not tagged, and they come with the app and not downloaded later, they work fine. And the console prints the file name, like I did (print(videoURL.absoluteString). But after they are tagged and downloaded later, they dont work, and nothing prints in the console. Just a black screen appears in the app.

I've been stuck on this for ages, and help with really help.

Thanks


问:
What is the difference between component and directive?
I have just started working with Angular 2.

I was wondering what are the differences between components and directives in Angular 2?

答:
Actually components are also directives, but have differences between them.

Attribute Directives :

Attribute directives are classes that are able to modify the behavior or appearance of a single element. For creating an attribute directive apply the @Directive to a class.

```<code>import { Directive, ElementRef } from "@angular/core";

@Directive({
     selector: "[custom-attr]", })

export class CustomAttrDirective {

   constructor(element: ElementRef) {
      element.nativeElement.classList.add("bg-success", "text-white");
   }
}
</code>```

Adding a directive attribute template.html File

```<code><tr *ngFor="let item of getProducts(); let i = index" custom-attr>
   <td>{{i + 1}}</td>
   <td>{{item.name}}</td>
</tr>
</code>```

Structural Directives :

Structural directives change the layout of the HTML document by adding and removing elements, as a micro-templates. Structural directives allow content to be added conditionally based on the result of an expression such as*ngIf or for the same content to be repeated for each object in a data source such as *ngFor.

You can use the built-in directives for common tasks, but writing custom structural directives provides ability to tailor behavior to your application.

```<code><p *ngIf="true">
  Expression is true and ngIf is true.
  This paragraph is in the DOM.
</p>
<p *ngIf="false">
  Expression is false and ngIf is false.
  This paragraph is not in the DOM.
</p>
</code>```

Components :

Components are directives that their own templates, rather than relying on content provided from elsewhere. Components have access to all directive features, still have a host element, can still define input and output properties, and so on.But they also define their own content.

It can be easy to underestimate the importance of the template, but attribute and structural directives have limitations. Directives can do useful and powerful work, but they don’t have much insight into the elements they are applied to. Directives are most useful when they are general-purpose tools, such the ngModel directive, which can be applied to any data model property and any form element, without regard to what the data or the element is being used for.

Components, by contrast, are closely tied to the contents of their templates. Components provide the data and logic that will be used by the data bindings that are applied to the HTML elements in the template, which provide the context used to evaluate data binding expressions and act as the glue between the directives and the rest of the application. Components are also a useful tool in allowing large Angular projects to be broken up into manageable chunks.

```<code>import { Component, Input } from '@angular/core';

import { Hero } from './hero';

@Component({
  selector: 'app-hero-child',
  template: `
    <h3>{{hero.name}} says:</h3>
    <p>I, {{hero.name}}, am at your service, {{masterName}}.</p>
  `
})
export class HeroChildComponent {
  @Input() hero: Hero;
  @Input('master') masterName: string;
}
</code>```

from official angular

 from Pro-Angular book

Body:
Actually components are also directives, but have differences between them.

Attribute Directives :

Attribute directives are classes that are able to modify the behavior or appearance of a single element. For creating an attribute directive apply the @Directive to a class.

import { Directive, ElementRef } from "@angular/core";

@Directive({
     selector: "[custom-attr]", })

export class CustomAttrDirective {

   constructor(element: ElementRef) {
      element.nativeElement.classList.add("bg-success", "text-white");
   }
}

Adding a directive attribute template.html File

<tr *ngFor="let item of getProducts(); let i = index" custom-attr>
   <td>{{i + 1}}</td>
   <td>{{item.name}}</td>
</tr>

Structural Directives :

Structural directives change the layout of the HTML document by adding and removing elements, as a micro-templates. Structural directives allow content to be added conditionally based on the result of an expression such as*ngIf or for the same content to be repeated for each object in a data source such as *ngFor.

You can use the built-in directives for common tasks, but writing custom structural directives provides ability to tailor behavior to your application.

<p *ngIf="true">
  Expression is true and ngIf is true.
  This paragraph is in the DOM.
</p>
<p *ngIf="false">
  Expression is false and ngIf is false.
  This paragraph is not in the DOM.
</p>

Components :

Components are directives that their own templates, rather than relying on content provided from elsewhere. Components have access to all directive features, still have a host element, can still define input and output properties, and so on.But they also define their own content.

It can be easy to underestimate the importance of the template, but attribute and structural directives have limitations. Directives can do useful and powerful work, but they don’t have much insight into the elements they are applied to. Directives are most useful when they are general-purpose tools, such the ngModel directive, which can be applied to any data model property and any form element, without regard to what the data or the element is being used for.

Components, by contrast, are closely tied to the contents of their templates. Components provide the data and logic that will be used by the data bindings that are applied to the HTML elements in the template, which provide the context used to evaluate data binding expressions and act as the glue between the directives and the rest of the application. Components are also a useful tool in allowing large Angular projects to be broken up into manageable chunks.

import { Component, Input } from '@angular/core';

import { Hero } from './hero';

@Component({
  selector: 'app-hero-child',
  template: `
    <h3>{{hero.name}} says:</h3>
    <p>I, {{hero.name}}, am at your service, {{masterName}}.</p>
  `
})
export class HeroChildComponent {
  @Input() hero: Hero;
  @Input('master') masterName: string;
}

from official angular

 from Pro-Angular book


问:
Python array loop not looping/validator issues
I'm not exactly sure what I did, but when testing my code it either crashes immediately or gets stuck in a loop. If the first input is a value error (string) and the next is a number it loops as long as the pattern is kept. But if first user entry is int then program crashes. Please any help would be appreciated. </p>

```<code>def main():
    courseArray = []
    keepGoing = "y"
    while keepGoing == "y":
        courseArray = getValidateCourseScore()
        total = getTotal(courseArray)
        average = total/len(courseArray)
        print('The lowest score is: ', min(courseArray))
        print('The highest score is: ', max(courseArray))
        print('the average is: ', average)
        keepGoing = validateRunAgain(input(input("Do you want to run this program again? (Y/n)")))

def getValidateCourseScore():
    courseArray = []
    counter = 1
    while counter < 6:
        try:
            courseArray.append(int(input("Enter the number of points received for course: ")))
            valScore(courseArray)
        except ValueError:
            print("Please enter a valid score between 0-100")
            courseArray.append(int(input("Enter a number between 1 and 100: ")))
    counter += 1
    return courseArray

def valScore(courseArray):
    score = int(courseArray)
    if score < 0:
        print("Please enter a valid score between 0-100")
        courseArray.append(int(input("Enter a number between 1 and 100: ")))
    elif score > 100:
        print("Please enter a valid score between 0-100")
        courseArray.append(int(input("Enter a number between 1 and 100: ")))
    else:
        return courseArray

def validateRunAgain(userInput):
    if userInput == "n" or userInput == "N":
        print("Thanks for using my program")
        return "n"
    elif userInput == "y" or userInput == "Y":
        return "y"
    else:
        print("Please enter y or n")
        validateRunAgain(input("Do you want to run this program again? (Y/n)"))
        return getValidateCourseScore()

def getTotal(valueList):
    total = 0
    for num in valueList:
        total += num
    return total

main()
</code>```

答:
There are too many inputs from the user, so I have cut down on them and changed it.

Here are the sections of your code which I have changed :

Here valScore() I presume validates the input score so, I also gave the index of element to be validated. If it is not valid we remove it from the array and raise ValueError, since it raises error our counter is not updated.

```<code>keepGoing = validateRunAgain()

def getValidateCourseScore():
    courseArray = []
    counter = 1
    while counter < 6:
        try:
            courseArray.append(int(input("Enter the number of points received for course: ")))
            valScore(courseArray, counter - 1)
            counter += 1
        except ValueError:
            print("Please enter a valid score between 0-100")
            continue
    return courseArray

def valScore(courseArray, counter):
    score = courseArray[counter]
    if score < 0 or score > 100:
        courseArray.pop()
        raise ValueError

def validateRunAgain():
    while True:
        userInput = input("Do you want to run this program again? (Y/n)")
        if userInput == 'y' or userInput == 'Y':
            return 'y'
        elif userInput == 'n' or userInput == 'N':
            print('thank you for using my program')
            return 'n'
        else:
             print('Enter Y or N')
</code>```

Body:
I'm not exactly sure what I did, but when testing my code it either crashes immediately or gets stuck in a loop. If the first input is a value error (string) and the next is a number it loops as long as the pattern is kept. But if first user entry is int then program crashes. Please any help would be appreciated.

def main():
    courseArray = []
    keepGoing = "y"
    while keepGoing == "y":
        courseArray = getValidateCourseScore()
        total = getTotal(courseArray)
        average = total/len(courseArray)
        print('The lowest score is: ', min(courseArray))
        print('The highest score is: ', max(courseArray))
        print('the average is: ', average)
        keepGoing = validateRunAgain(input(input("Do you want to run this program again? (Y/n)")))

def getValidateCourseScore():
    courseArray = []
    counter = 1
    while counter < 6:
        try:
            courseArray.append(int(input("Enter the number of points received for course: ")))
            valScore(courseArray)
        except ValueError:
            print("Please enter a valid score between 0-100")
            courseArray.append(int(input("Enter a number between 1 and 100: ")))
    counter += 1
    return courseArray

def valScore(courseArray):
    score = int(courseArray)
    if score < 0:
        print("Please enter a valid score between 0-100")
        courseArray.append(int(input("Enter a number between 1 and 100: ")))
    elif score > 100:
        print("Please enter a valid score between 0-100")
        courseArray.append(int(input("Enter a number between 1 and 100: ")))
    else:
        return courseArray

def validateRunAgain(userInput):
    if userInput == "n" or userInput == "N":
        print("Thanks for using my program")
        return "n"
    elif userInput == "y" or userInput == "Y":
        return "y"
    else:
        print("Please enter y or n")
        validateRunAgain(input("Do you want to run this program again? (Y/n)"))
        return getValidateCourseScore()

def getTotal(valueList):
    total = 0
    for num in valueList:
        total += num
    return total

main()

Body:
There are too many inputs from the user, so I have cut down on them and changed it.

Here are the sections of your code which I have changed :

Here valScore() I presume validates the input score so, I also gave the index of element to be validated. If it is not valid we remove it from the array and raise ValueError, since it raises error our counter is not updated.

keepGoing = validateRunAgain()

def getValidateCourseScore():
    courseArray = []
    counter = 1
    while counter < 6:
        try:
            courseArray.append(int(input("Enter the number of points received for course: ")))
            valScore(courseArray, counter - 1)
            counter += 1
        except ValueError:
            print("Please enter a valid score between 0-100")
            continue
    return courseArray

def valScore(courseArray, counter):
    score = courseArray[counter]
    if score < 0 or score > 100:
        courseArray.pop()
        raise ValueError

def validateRunAgain():
    while True:
        userInput = input("Do you want to run this program again? (Y/n)")
        if userInput == 'y' or userInput == 'Y':
            return 'y'
        elif userInput == 'n' or userInput == 'N':
            print('thank you for using my program')
            return 'n'
        else:
             print('Enter Y or N')


问:
Cypress: Test if element does not exist
I want to be able to click on a check box and test that an element is no longer in the DOM in Cypress. Can someone suggest how you do it?

// This is the Test when the checkbox is clicked and the element is there

cy.get('[type="checkbox"]').click();
cy.get('.check-box-sub-text').contains('Some text in this div.')
```

I want to do the opposite of the test above.

So when I click it again the div with the class check-box-sub-text should not be in the DOM.

答:
Here's what worked for me:

```<code>cy.get('[data-cy=parent]').should('not.have.descendants', 'img')
</code>```

I check that some
 has no images inside.
Regarding original question, you can set data-cy="something, i.e. child" attribute on inner nodes and use this assertion:

```<code>cy.get('[data-cy=parent]').should('not.have.descendants', '[data-cy=child]')
</code>```

Title: Cypress: Test if element does not exist

Body:
I want to be able to click on a check box and test that an element is no longer in the DOM in Cypress. Can someone suggest how you do it?

// This is the Test when the checkbox is clicked and the element is there

cy.get('[type="checkbox"]').click();
cy.get('.check-box-sub-text').contains('Some text in this div.')

I want to do the opposite of the test above.

So when I click it again the div with the class check-box-sub-text should not be in the DOM.

Body:
Here's what worked for me:

cy.get('[data-cy=parent]').should('not.have.descendants', 'img')

I check that some <div data-cy="parent"> has no images inside.
Regarding original question, you can set data-cy="something, i.e. child" attribute on inner nodes and use this assertion:

cy.get('[data-cy=parent]').should('not.have.descendants', '[data-cy=child]')


问:
Saving the output of spider in a variable rather than in a file
I am looking for a way to save the spider output in a python variable instead of saving it in a json file and reading it back in the program.

```<code>import scrapy
from scrapy.crawler import CrawlerProcess

class TestSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://www.wikipedia.org']

    def parse(self, response):
        yield {
                'text' : response.css(".jsl10n.localized-slogan::text").extract_first()
             }

if __name__ == "__main__":
    os.remove('result.json')
    process = CrawlerProcess({
        'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
        'FEED_FORMAT': 'json',
        'FEED_URI': 'result.json'
    })

    process.crawl(TestSpider)
    process.start()
</code>```

I want to avoid the below step and directly read the value instead of saving it on disk first

```<code>with io.open('result.json', encoding='utf-8') as json_data:
        d = json.load(json_data)
        text = d[0]['text']
</code>```

答:
You can also pass an object into spider and change it, like this:

class TestSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://www.wikipedia.org']

    def parse(self, response):
        self.outputResponse['text'] = response.css(".jsl10n.localized-slogan::text").extract_first()

if __name__ == "__main__":
    os.remove('result.json')

    outputResponse = {}

    process = CrawlerProcess({
        'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
    })
    process.crawl(TestSpider, outputResponse=outputResponse)
    process.start()
```

This works, because every named argument passed to spider constructor is assigned to an instance as an attribute, that's why you can use self.outputResponse inside parse method and have access to an external object.

ContentLicense: CC BY-SA 3.0

Body:
I am looking for a way to save the spider output in a python variable instead of saving it in a json file and reading it back in the program.

import scrapy
from scrapy.crawler import CrawlerProcess

class TestSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://www.wikipedia.org']

    def parse(self, response):
        yield {
                'text' : response.css(".jsl10n.localized-slogan::text").extract_first()
             }

if __name__ == "__main__":
    os.remove('result.json')
    process = CrawlerProcess({
        'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
        'FEED_FORMAT': 'json',
        'FEED_URI': 'result.json'
    })

    process.crawl(TestSpider)
    process.start()

I want to avoid the below step and directly read the value instead of saving it on disk first

with io.open('result.json', encoding='utf-8') as json_data:
        d = json.load(json_data)
        text = d[0]['text']

Body:
You can also pass an object into spider and change it, like this:

class TestSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://www.wikipedia.org']

    def parse(self, response):
        self.outputResponse['text'] = response.css(".jsl10n.localized-slogan::text").extract_first()

if __name__ == "__main__":
    os.remove('result.json')

    outputResponse = {}

    process = CrawlerProcess({
        'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
    })
    process.crawl(TestSpider, outputResponse=outputResponse)
    process.start()

This works, because every named argument passed to spider constructor is assigned to an instance as an attribute, that's why you can use self.outputResponse inside parse method and have access to an external object.


问:
Recursive List - Computing the sum of all elements up to and including the last occurrence
It´s not only a Java question. It is a question of algorithms in general:

The task is to compute the sum of all the elements up to and including the last occurred value "val", which is a parameter in the method.

Method looks like:

int sumLastOcc (Node n, int val)

// n is the node of the list. Starts with the head when the method will be called.

// val is the value up to which the total sum will be formed

Example: 3 5 2 7 5 1 4

Result: 22 for val = 5 (call: sumLastOcc (head, 5))

Result: 0 for val = 9 (call: sumLastOcc (head, 9))

```<code>public class RecursiveList {
    static Node head;
    static boolean found = false;

    public static void main(String[] args) {
        RecursiveList list = new RecursiveList();

        list.addNumber(3);
        list.addNumber(5);
        list.addNumber(2);
        list.addNumber(7);
        list.addNumber(5);
        list.addNumber(1);
        list.addNumber(4);
        list.printList();

        int result = list.sumLastOcc(head, 5);

        System.out.println();
        System.out.println(result);
    }

    private int sumLastOcc(Node n, int val) {
        // CODE TO IMPLEMENT
        }

        return sum;
    }

    private void addNumber(int number) {
        Node curr = head;
        Node prev = null;

        if (head == null) {
            head = new Node(number, null);
        } else {
            while (curr != null) {
                prev = curr;
                curr = curr.next;
            }

            Node newNode = new Node(number, null);
            prev.next = newNode;
        }
    }

    // inner node class
    private class Node {
        Node next;
        int value;

        Node(int value, Node next) {
            this.value = value;
            this.next = next;
        }
    }

    private void printList() {
        Node curr = head;
        Node prev = null;

        while (curr != null) {
            System.out.print(curr.value + " ");

            prev = curr;
            curr = curr.next;
        }
    }
}
</code>```

It´s okay to use/add any global static variables like:

static boolean found = false;

as already defined.
We should use no loops (for, while, ...) and we should also use no additional method. Just the method "private int sumLastOcc(Node n, int val)" should do that charm.

答:
I can explain  -
   Traverse the list until the end and keep on adding the values. Also from the match set the flag (found to true) to keep track of deductions until you reach the end or find another match. Reset the counter on finding the match.

In pseudo-code like

```<code>private int sumLastOcc(Node n, int val) {
   if n.next != null
     //traverse the list until the end
     //add the value to sum
     sum = sum + n.value

     //keep track of values from the match
     if(found)
     deduct = deduct + n.value;

     //reset the flag when match found
     deduct = 0
     found = true

     //call until reach the end
     sumLastOcc(n.next, val)
   else
     //add the last node value
     sum = sum + n.value

    //even for the deduct
    if (found)
     deduct = deduct + n.value;

  //lastly return sum - deduct
  return found?sum - deduct:0
}
</code>```

Body:
It´s not only a Java question. It is a question of algorithms in general:

The task is to compute the sum of all the elements up to and including the last occurred value "val", which is a parameter in the method.

Method looks like:

int sumLastOcc (Node n, int val)

// n is the node of the list. Starts with the head when the method will be called.

// val is the value up to which the total sum will be formed

Example: 3 5 2 7 5 1 4

Result: 22 for val = 5 (call: sumLastOcc (head, 5))

Result: 0 for val = 9 (call: sumLastOcc (head, 9))

public class RecursiveList {
    static Node head;
    static boolean found = false;

    public static void main(String[] args) {
        RecursiveList list = new RecursiveList();

        list.addNumber(3);
        list.addNumber(5);
        list.addNumber(2);
        list.addNumber(7);
        list.addNumber(5);
        list.addNumber(1);
        list.addNumber(4);
        list.printList();

        int result = list.sumLastOcc(head, 5);

        System.out.println();
        System.out.println(result);
    }

    private int sumLastOcc(Node n, int val) {
        // CODE TO IMPLEMENT
        }

        return sum;
    }

    private void addNumber(int number) {
        Node curr = head;
        Node prev = null;

        if (head == null) {
            head = new Node(number, null);
        } else {
            while (curr != null) {
                prev = curr;
                curr = curr.next;
            }

            Node newNode = new Node(number, null);
            prev.next = newNode;
        }
    }

    // inner node class
    private class Node {
        Node next;
        int value;

        Node(int value, Node next) {
            this.value = value;
            this.next = next;
        }
    }

    private void printList() {
        Node curr = head;
        Node prev = null;

        while (curr != null) {
            System.out.print(curr.value + " ");

            prev = curr;
            curr = curr.next;
        }
    }
}

It´s okay to use/add any global static variables like:

static boolean found = false;

as already defined.
We should use no loops (for, while, ...) and we should also use no additional method. Just the method "private int sumLastOcc(Node n, int val)" should do that charm.

Body:
I can explain  -
   Traverse the list until the end and keep on adding the values. Also from the match set the flag (found to true) to keep track of deductions until you reach the end or find another match. Reset the counter on finding the match.

In pseudo-code like

private int sumLastOcc(Node n, int val) {
   if n.next != null
     //traverse the list until the end
     //add the value to sum
     sum = sum + n.value

     //keep track of values from the match
     if(found)
     deduct = deduct + n.value;

     //reset the flag when match found
     deduct = 0
     found = true

     //call until reach the end
     sumLastOcc(n.next, val)
   else
     //add the last node value
     sum = sum + n.value

    //even for the deduct
    if (found)
     deduct = deduct + n.value;

  //lastly return sum - deduct
  return found?sum - deduct:0
}


问:
Speed up datetime formatting when reading csv
I have a csv file with 4+m records.

I import it using pd.read_csv('big_file.csv', dtype=object)

This file has 2 columns with dates in the following format:
'yyyy-mm-ddThh:mm:ss.nsTZ' e.g.'2018-05-05T04:39:09.447Z'

I need to transform them to
'yyyy-mm-dd H:M:S' e.g. '2018-09-23 06:03:12'

I use the following code to do so:

df['created'] = pd.to_datetime(arg=df.created).dt.strftime('%Y-%m-%d %H:%M:%S')
df['lastLogin'] = pd.to_datetime(arg=df.lastLogin).dt.strftime('%Y-%m-%d %H:%M:%S')
df['lastUpdated'] = pd.to_datetime(arg=df.lastUpdated).dt.strftime('%Y-%m-%d %H:%M:%S')
df['created'] = pd.to_datetime(arg=df.created)
df['lastLogin'] = pd.to_datetime(arg=df.lastLogin)
df['lastUpdated'] = pd.to_datetime(arg=df.lastUpdated)
```

<p>This process is extremely slow:</p>

```CPU times: user 1min 48s, sys: 1.19 s, total: 1min 49s
Wall time: 1min 49s
```

Is there a way to speed it up?

答:
From the comment, I understand that you don't need the meaning of the dates but you just want to change the aesthetic appearance of the strings representing the dates. Then, you can just treat the data simply as strings. So, I did it like this.

```<code>#!/usr/bin/python3

import numpy as np
import sys

def gen_sample(numdata, outfname):
    yy=np.random.randint(1905, 2018, 2*numdata)
    mm=np.random.randint(   1,   13, 2*numdata)
    dd=np.random.randint(   1,   29, 2*numdata)
    hhh=np.random.randint(   0,   25, 2*numdata)
    mmm=np.random.randint(   0,   61, 2*numdata)
    sss=np.random.randint(   0,   61, 2*numdata)
    baboon=np.random.randint(   0, 1000, 2*numdata)
    with open(outfname, 'w') as outf:
        for jj in range(numdata):
            outf.write('%4.4i-%2.2i-%2.2iT%2.2i:%2.2i:%2.2i.%3.3iZ,%4.4i-%2.2i-%2.2iT%2.2i:%2.2i:%2.2i.%3.3iZ\n'
                       %(yy[2*jj],   mm[2*jj],   dd[2*jj],
                         hhh[2*jj],  mmm[2*jj],  sss[2*jj],  baboon[2*jj],
                         yy[2*jj+1], mm[2*jj+1], dd[2*jj+1],
                         hhh[2*jj+1], mmm[2*jj+1], sss[2*jj+1], baboon[2*jj+1]))

def convert(infname,outfname):

    data=np.loadtxt(infname, dtype=np.str, delimiter=',', ndmin=2)
    with open(outfname,'w') as outf:
        for jr in range(data.shape[0]):
            outf.write('%s %s,%s %s\n'%(
                data[jr,0][0:10],
                data[jr,0][11:19],
                data[jr,1][0:10],
                data[jr,1][11:19] ))

if __name__=='__main__':
    sample_fname= 'daa.csv'
    out_fname= 'daadaa.csv'
    if len(sys.argv)>1:
        numdata=int(sys.argv[1])
        gen_sample(numdata, sample_fname)
    else:
        convert(sample_fname, out_fname)
</code>```

and it took me about 15 seconds for 4M*2 data on my computer. Please see this

```<code>#!/bin/bash

for jj in 0 1 2
do
  echo "generating sample.."
  ./main.py 4000000
  echo "loading, converting, and writing.."
  echo "----"
  /usr/bin/time ./main.py
  echo "----"
done
</code>```

and this

```<code>$ ./run.sh
generating sample..
loading, converting, and writing..
----
14.96user 0.94system 0:15.05elapsed 105%CPU (0avgtext+0avgdata 818724maxresident)k
8inputs+312504outputs (0major+315787minor)pagefaults 0swaps
----
generating sample..
loading, converting, and writing..
----
14.91user 0.93system 0:14.99elapsed 105%CPU (0avgtext+0avgdata 818848maxresident)k
16inputs+312504outputs (0major+315864minor)pagefaults 0swaps
----
generating sample..
loading, converting, and writing..
----
15.39user 0.95system 0:15.52elapsed 105%CPU (0avgtext+0avgdata 818736maxresident)k
8inputs+312504outputs (0major+315857minor)pagefaults 0swaps
----
</code>```

Input file is like

```<code>$ head daa.csv
2016-10-05T08:07:03.214Z,1973-10-01T12:36:21.367Z
1961-08-24T02:08:57.436Z,1953-03-06T00:56:12.486Z
1986-09-07T17:15:60.322Z,1952-11-19T19:02:56.159Z
1939-08-17T05:13:19.659Z,1920-12-15T16:46:52.628Z
2004-11-09T02:29:25.905Z,1925-02-07T10:37:49.142Z
2011-12-12T10:46:38.583Z,1992-02-10T08:58:60.284Z
1968-01-23T05:05:05.151Z,1935-09-17T07:12:49.392Z
1916-04-05T18:55:35.281Z,1919-10-12T10:05:10.249Z
1970-10-04T21:45:16.751Z,1951-01-08T16:58:51.190Z
1910-01-19T22:12:04.088Z,2006-03-08T09:26:45.690Z
</code>```

Output file was like

```<code>$ head daadaa.csv
2016-10-05 08:07:03,1973-10-01 12:36:21
1961-08-24 02:08:57,1953-03-06 00:56:12
1986-09-07 17:15:60,1952-11-19 19:02:56
1939-08-17 05:13:19,1920-12-15 16:46:52
2004-11-09 02:29:25,1925-02-07 10:37:49
2011-12-12 10:46:38,1992-02-10 08:58:60
1968-01-23 05:05:05,1935-09-17 07:12:49
1916-04-05 18:55:35,1919-10-12 10:05:10
1970-10-04 21:45:16,1951-01-08 16:58:51
1910-01-19 22:12:04,2006-03-08 09:26:45
</code>```

If you don't need to write the converted data back to a file,
maybe the run time will be faster.
You will need to make the function more robust for the variation of your own data, but I hope the idea is delivered.

Body:
I have a csv file with 4+m records.

I import it using pd.read_csv('big_file.csv', dtype=object)

This file has 2 columns with dates in the following format:
'yyyy-mm-ddThh:mm:ss.nsTZ' e.g.'2018-05-05T04:39:09.447Z'

I need to transform them to
'yyyy-mm-dd H:M:S' e.g. '2018-09-23 06:03:12'

I use the following code to do so:

df['created'] = pd.to_datetime(arg=df.created).dt.strftime('%Y-%m-%d %H:%M:%S')
df['lastLogin'] = pd.to_datetime(arg=df.lastLogin).dt.strftime('%Y-%m-%d %H:%M:%S')
df['lastUpdated'] = pd.to_datetime(arg=df.lastUpdated).dt.strftime('%Y-%m-%d %H:%M:%S')
df['created'] = pd.to_datetime(arg=df.created)
df['lastLogin'] = pd.to_datetime(arg=df.lastLogin)
df['lastUpdated'] = pd.to_datetime(arg=df.lastUpdated)

This process is extremely slow:

CPU times: user 1min 48s, sys: 1.19 s, total: 1min 49s
Wall time: 1min 49s

Is there a way to speed it up?

Body:
From the comment, I understand that you don't need the meaning of the dates but you just want to change the aesthetic appearance of the strings representing the dates. Then, you can just treat the data simply as strings. So, I did it like this.

#!/usr/bin/python3

import numpy as np
import sys

def gen_sample(numdata, outfname):
    yy=np.random.randint(1905, 2018, 2*numdata)
    mm=np.random.randint(   1,   13, 2*numdata)
    dd=np.random.randint(   1,   29, 2*numdata)
    hhh=np.random.randint(   0,   25, 2*numdata)
    mmm=np.random.randint(   0,   61, 2*numdata)
    sss=np.random.randint(   0,   61, 2*numdata)
    baboon=np.random.randint(   0, 1000, 2*numdata)
    with open(outfname, 'w') as outf:
        for jj in range(numdata):
            outf.write('%4.4i-%2.2i-%2.2iT%2.2i:%2.2i:%2.2i.%3.3iZ,%4.4i-%2.2i-%2.2iT%2.2i:%2.2i:%2.2i.%3.3iZ\n'
                       %(yy[2*jj],   mm[2*jj],   dd[2*jj],
                         hhh[2*jj],  mmm[2*jj],  sss[2*jj],  baboon[2*jj],
                         yy[2*jj+1], mm[2*jj+1], dd[2*jj+1],
                         hhh[2*jj+1], mmm[2*jj+1], sss[2*jj+1], baboon[2*jj+1]))

def convert(infname,outfname):

    data=np.loadtxt(infname, dtype=np.str, delimiter=',', ndmin=2)
    with open(outfname,'w') as outf:
        for jr in range(data.shape[0]):
            outf.write('%s %s,%s %s\n'%(
                data[jr,0][0:10],
                data[jr,0][11:19],
                data[jr,1][0:10],
                data[jr,1][11:19] ))

if __name__=='__main__':
    sample_fname= 'daa.csv'
    out_fname= 'daadaa.csv'
    if len(sys.argv)>1:
        numdata=int(sys.argv[1])
        gen_sample(numdata, sample_fname)
    else:
        convert(sample_fname, out_fname)

and it took me about 15 seconds for 4M*2 data on my computer. Please see this

#!/bin/bash

for jj in 0 1 2
do
  echo "generating sample.."
  ./main.py 4000000
  echo "loading, converting, and writing.."
  echo "----"
  /usr/bin/time ./main.py
  echo "----"
done

and this

$ ./run.sh
generating sample..
loading, converting, and writing..
----
14.96user 0.94system 0:15.05elapsed 105%CPU (0avgtext+0avgdata 818724maxresident)k
8inputs+312504outputs (0major+315787minor)pagefaults 0swaps
----
generating sample..
loading, converting, and writing..
----
14.91user 0.93system 0:14.99elapsed 105%CPU (0avgtext+0avgdata 818848maxresident)k
16inputs+312504outputs (0major+315864minor)pagefaults 0swaps
----
generating sample..
loading, converting, and writing..
----
15.39user 0.95system 0:15.52elapsed 105%CPU (0avgtext+0avgdata 818736maxresident)k
8inputs+312504outputs (0major+315857minor)pagefaults 0swaps
----

Input file is like

$ head daa.csv
2016-10-05T08:07:03.214Z,1973-10-01T12:36:21.367Z
1961-08-24T02:08:57.436Z,1953-03-06T00:56:12.486Z
1986-09-07T17:15:60.322Z,1952-11-19T19:02:56.159Z
1939-08-17T05:13:19.659Z,1920-12-15T16:46:52.628Z
2004-11-09T02:29:25.905Z,1925-02-07T10:37:49.142Z
2011-12-12T10:46:38.583Z,1992-02-10T08:58:60.284Z
1968-01-23T05:05:05.151Z,1935-09-17T07:12:49.392Z
1916-04-05T18:55:35.281Z,1919-10-12T10:05:10.249Z
1970-10-04T21:45:16.751Z,1951-01-08T16:58:51.190Z
1910-01-19T22:12:04.088Z,2006-03-08T09:26:45.690Z

Output file was like

$ head daadaa.csv
2016-10-05 08:07:03,1973-10-01 12:36:21
1961-08-24 02:08:57,1953-03-06 00:56:12
1986-09-07 17:15:60,1952-11-19 19:02:56
1939-08-17 05:13:19,1920-12-15 16:46:52
2004-11-09 02:29:25,1925-02-07 10:37:49
2011-12-12 10:46:38,1992-02-10 08:58:60
1968-01-23 05:05:05,1935-09-17 07:12:49
1916-04-05 18:55:35,1919-10-12 10:05:10
1970-10-04 21:45:16,1951-01-08 16:58:51
1910-01-19 22:12:04,2006-03-08 09:26:45

If you don't need to write the converted data back to a file,
maybe the run time will be faster.
You will need to make the function more robust for the variation of your own data, but I hope the idea is delivered.


问:
Display text message after clicking on menu button in telegram

Add a fourth category "About"

Clicking on "About" will send a text message e.g. "Operating since 1999"

Here is the code snippet:

def main_menu_keyboard():
    keyboard = [[InlineKeyboardButton('Bubble Tea', callback_data='m1')],
                  [InlineKeyboardButton('Waffle', callback_data='m2')],
                  [InlineKeyboardButton('Otah', callback_data='m3')]]
    return InlineKeyboardMarkup(keyboard)
```

<p><img src="https://i.stack.imgur.com/KRtZ2.png" alt="enter image description here"></p>

<pre class="lang-py prettyprint-override"><code># Add command handler to dispatcher
updater.dispatcher.add_handler(CommandHandler('start', start))
updater.dispatcher.add_handler(CallbackQueryHandler(main_menu, pattern='main'))
updater.dispatcher.add_handler(CallbackQueryHandler(first_menu, pattern='m1'))
updater.dispatcher.add_handler(CallbackQueryHandler(second_menu, pattern='m2'))
updater.dispatcher.add_handler(CallbackQueryHandler(third_menu,pattern='m3'))
</code>```

I have been trying with different ways of displaying text as shown below but not too sure how to piece them together with the dispatcher

bot.send_message(chat_id=chat_id, text="About xxxxx details ")
update.message.reply_text("About xxxx details ")
```

答:
Add a fourth category "About"

def main_menu_keyboard():
    keyboard = [[InlineKeyboardButton('Bubble Tea', callback_data='m1')],
                  [InlineKeyboardButton('Waffle', callback_data='m2')],
                  [InlineKeyboardButton('Otah', callback_data='m3')],
                  [InlineKeyboardButton('About', callback_data='m4')]]  # new button
    return InlineKeyboardMarkup(keyboard)
```

<blockquote>
  <p>Clicking on "About" will send a text message e.g. "Operating since 1999"</p>
</blockquote>

<pre class="lang-py prettyprint-override"><code>def about_message(bot, update):  # handler for "About" button
    bot.send_message(chat_id=update.callback_query.from_user.id, text="Operating since 1999")

# ... other existing code

# Add command handler to dispatcher
updater.dispatcher.add_handler(CommandHandler('start', start))
updater.dispatcher.add_handler(CallbackQueryHandler(main_menu, pattern='main'))
updater.dispatcher.add_handler(CallbackQueryHandler(first_menu, pattern='m1'))
updater.dispatcher.add_handler(CallbackQueryHandler(second_menu, pattern='m2'))
updater.dispatcher.add_handler(CallbackQueryHandler(third_menu, pattern='m3'))
updater.dispatcher.add_handler(CallbackQueryHandler(about_message, pattern='m4'))  # add handler for "About" button

# ... other existing code
</code>```

Body:
Add a fourth category "About"

Clicking on "About" will send a text message e.g. "Operating since 1999"

Here is the code snippet:

def main_menu_keyboard():
    keyboard = [[InlineKeyboardButton('Bubble Tea', callback_data='m1')],
                  [InlineKeyboardButton('Waffle', callback_data='m2')],
                  [InlineKeyboardButton('Otah', callback_data='m3')]]
    return InlineKeyboardMarkup(keyboard)

# Add command handler to dispatcher
updater.dispatcher.add_handler(CommandHandler('start', start))
updater.dispatcher.add_handler(CallbackQueryHandler(main_menu, pattern='main'))
updater.dispatcher.add_handler(CallbackQueryHandler(first_menu, pattern='m1'))
updater.dispatcher.add_handler(CallbackQueryHandler(second_menu, pattern='m2'))
updater.dispatcher.add_handler(CallbackQueryHandler(third_menu,pattern='m3'))

I have been trying with different ways of displaying text as shown below but not too sure how to piece them together with the dispatcher

bot.send_message(chat_id=chat_id, text="About xxxxx details ")
update.message.reply_text("About xxxx details ")

Body:
Add a fourth category "About"

def main_menu_keyboard():
    keyboard = [[InlineKeyboardButton('Bubble Tea', callback_data='m1')],
                  [InlineKeyboardButton('Waffle', callback_data='m2')],
                  [InlineKeyboardButton('Otah', callback_data='m3')],
                  [InlineKeyboardButton('About', callback_data='m4')]]  # new button
    return InlineKeyboardMarkup(keyboard)

Clicking on "About" will send a text message e.g. "Operating since 1999"

def about_message(bot, update):  # handler for "About" button
    bot.send_message(chat_id=update.callback_query.from_user.id, text="Operating since 1999")

# ... other existing code

# Add command handler to dispatcher
updater.dispatcher.add_handler(CommandHandler('start', start))
updater.dispatcher.add_handler(CallbackQueryHandler(main_menu, pattern='main'))
updater.dispatcher.add_handler(CallbackQueryHandler(first_menu, pattern='m1'))
updater.dispatcher.add_handler(CallbackQueryHandler(second_menu, pattern='m2'))
updater.dispatcher.add_handler(CallbackQueryHandler(third_menu, pattern='m3'))
updater.dispatcher.add_handler(CallbackQueryHandler(about_message, pattern='m4'))  # add handler for "About" button

# ... other existing code


问:
Is it possible to use Invoke Webrequest to login to a site that is using Azure AD B2C authentication?
I am trying to configure my build-server to warmup my website by using a PowerShell script, that visits the website and login. The entire website is behind login.

My website is using Azure AD B2C for authentication and has been configured using the starter guide from Microsoft: https://github.com/AzureADQuickStarts/B2C-WebApp-OpenIdConnect-DotNet

When I login through the browser everything works as expected.

My current code for login looks like this:

```<code>$session = Get-AuthenticatedSession $DomainName $config.authenticationDetails $UserName $Password

Function Get-AuthenticatedSession {
  param(
    [Parameter(Mandatory=$true,Position=0)]
    [string]$domainName,
    [Parameter(Mandatory=$true,Position=1)]
    [object]$authenticationDetails,
    [Parameter(Mandatory=$true,Position=2)]
    [string]$username,
    [Parameter(Mandatory=$true,Position=3)]
    [string]$password
  )

  # Login - to create web session with authorisation cookies
  $loginPage = "$domainName$($authenticationDetails.url)"

  Write-Host "Getting Login Page $loginPage"

  try{
    $login = Invoke-WebRequest $loginPage -SessionVariable webSession -TimeoutSec 600 -UseBasicParsing
  }catch{
    Write-Host "First attempt failed with $($_.Exception.Response.StatusCode.value__) , retrying"
    $login = Invoke-WebRequest $loginPage -SessionVariable webSession -TimeoutSec 600 -UseBasicParsing
  }

  Write-Host "Got Login Page, filling out form"

  $fields = @{}
  $fields["$($authenticationDetails.userNameField)"] = $username
  $fields["$($authenticationDetails.passwordField)"] = $password

  Write-Host "logging in"

  $request = Invoke-WebRequest -Uri $loginPage -WebSession $webSession -Method POST -Body $form  -TimeoutSec 600| Out-Null

  $webSession

  Write-Host "login done"
}
</code>```

This is my code for requesting a page behind login, which is run afterwards:

```<code>foreach ($page in $config.urls) {
    RequestPage "$DomainName$($page.url)" $session
}

Function RequestPage {
    param(
        [Parameter(Mandatory=$true,Position=0)]
        [string]$url,
        [Parameter(Mandatory=$true,Position=1)]
        [object]$webSession
    )
    Get-Date
    Write-Host "requesting $url ..."
    try { $request = Invoke-WebRequest $url -WebSession $webSession -TimeoutSec 60000 -UseBasicParsing} catch {
      $status = $_.Exception.Response.StatusCode.Value__
      if ($status -ne 200){
        Write-Host "ERROR Something went wrong while requesting $url" -foregroundcolor red
      }
    }

    Write-Host $request

    Get-Date
    Write-Host "Done"
    Write-Host ""
}
</code>```

The problem is that the script does not seem to login correctly. Whenever I request a page behind login, I always get the html of the login page back. Any help is much appreciated!

EDIT - It seems like there is no FORM element on the page - doh! The actual signin is a button-element with a javascript click event attached. The login page is hosted by Microsoft, so I have little control over the DOM elements:

```<code><div class="buttons">
            <button id="next" tabindex="1">Sign in</button>
          </div>
</code>```

So I guess my question is whether it is possible to trigger a button click using PowerShell? When I also want the script to run on my build server where IE might not be available

答:
I've not had a reason to mess with Azure AD B2C, so, no experience as to what that site is or what you can do on it. However, as for ...

So I guess my question is whether it is possible to trigger a button
  click using PowerShell? When I also want the script to run on my build
  server where IE might not be available

...  the simple answer is, yes, as discussed and demoed in the article... Teaching PowerShell to Click. Red the article for full explanation

In a PowerShell Editor - In this case I'm using the ISE - of course in VSCode as well.

```<code># View the page live to se what it does - just do stuff
Start-Process 'http://www.minasi.com/addit.htm'

# Now, inspect the site page/form.
# View the page code to see what is there - If in the ISE

$minasi = Invoke-WebRequest -Uri 'http://www.minasi.com/addit.htm'
$minasi.Content | Out-ISETab
$minasi.AllElements | Out-ISETab
$minasi.Forms | Out-ISETab
$minasi.Forms.Fields | Out-ISETab
$minasi.InputFields | Out-ISETab

# View the page code to see what is there - If in VSCode

$minasi = Invoke-WebRequest -Uri 'http://www.minasi.com/addit.htm'
$minasi.Content | New-EditorFile
$minasi.AllElements | New-EditorFile
$minasi.Forms | New-EditorFile
$minasi.Forms.Fields | New-EditorFile
$minasi.InputFields | New-EditorFile

# Interact with the page to achieve the same results as GUI would by going direct to the 'results page' not the 'form page'.

$MethodPostURI = 'http://www.minasi.com/addit-worker.asp'
$Body ='addend1=3&addend2=9&B1=SUBMIT%21'
$page = Invoke-WebRequest $MethodPostURI -body $Body -method POST
$page.content

# You can also find elements by looping to find something specific, like the below...
$minasi.AllElements.FindById('SomeIdNameString')

# Or

$SubmitElement = $minasi.AllElements |
Where{
        $PSItem.tagName -eq 'INPUT' -and
        $PSItem.value -eq 'SUBMIT!'
     }
$SubmitElement # The use dot lookup to see if there is a click option.
</code>```

Lastly, here is something you may want to take a look at.

pQuery

This module is meant to create a variable that we can use jQuery-like
  syntax to do jQuery-like manipulations on a website.

Body:
I am trying to configure my build-server to warmup my website by using a PowerShell script, that visits the website and login. The entire website is behind login.

My website is using Azure AD B2C for authentication and has been configured using the starter guide from Microsoft: https://github.com/AzureADQuickStarts/B2C-WebApp-OpenIdConnect-DotNet

When I login through the browser everything works as expected.

My current code for login looks like this:

$session = Get-AuthenticatedSession $DomainName $config.authenticationDetails $UserName $Password

Function Get-AuthenticatedSession {
  param(
    [Parameter(Mandatory=$true,Position=0)]
    [string]$domainName,
    [Parameter(Mandatory=$true,Position=1)]
    [object]$authenticationDetails,
    [Parameter(Mandatory=$true,Position=2)]
    [string]$username,
    [Parameter(Mandatory=$true,Position=3)]
    [string]$password
  )

  # Login - to create web session with authorisation cookies
  $loginPage = "$domainName$($authenticationDetails.url)"

  Write-Host "Getting Login Page $loginPage"

  try{
    $login = Invoke-WebRequest $loginPage -SessionVariable webSession -TimeoutSec 600 -UseBasicParsing
  }catch{
    Write-Host "First attempt failed with $($_.Exception.Response.StatusCode.value__) , retrying"
    $login = Invoke-WebRequest $loginPage -SessionVariable webSession -TimeoutSec 600 -UseBasicParsing
  }

  Write-Host "Got Login Page, filling out form"

  $fields = @{}
  $fields["$($authenticationDetails.userNameField)"] = $username
  $fields["$($authenticationDetails.passwordField)"] = $password

  Write-Host "logging in"

  $request = Invoke-WebRequest -Uri $loginPage -WebSession $webSession -Method POST -Body $form  -TimeoutSec 600| Out-Null

  $webSession

  Write-Host "login done"
}

This is my code for requesting a page behind login, which is run afterwards:

foreach ($page in $config.urls) {
    RequestPage "$DomainName$($page.url)" $session
}

Function RequestPage {
    param(
        [Parameter(Mandatory=$true,Position=0)]
        [string]$url,
        [Parameter(Mandatory=$true,Position=1)]
        [object]$webSession
    )
    Get-Date
    Write-Host "requesting $url ..."
    try { $request = Invoke-WebRequest $url -WebSession $webSession -TimeoutSec 60000 -UseBasicParsing} catch {
      $status = $_.Exception.Response.StatusCode.Value__
      if ($status -ne 200){
        Write-Host "ERROR Something went wrong while requesting $url" -foregroundcolor red
      }
    }

    Write-Host $request

    Get-Date
    Write-Host "Done"
    Write-Host ""
}

The problem is that the script does not seem to login correctly. Whenever I request a page behind login, I always get the html of the login page back. Any help is much appreciated!

EDIT - It seems like there is no FORM element on the page - doh! The actual signin is a button-element with a javascript click event attached. The login page is hosted by Microsoft, so I have little control over the DOM elements:

            <button id="next" tabindex="1">Sign in</button>

So I guess my question is whether it is possible to trigger a button click using PowerShell? When I also want the script to run on my build server where IE might not be available

Body:
I've not had a reason to mess with Azure AD B2C, so, no experience as to what that site is or what you can do on it. However, as for ...

So I guess my question is whether it is possible to trigger a button
  click using PowerShell? When I also want the script to run on my build
  server where IE might not be available

...  the simple answer is, yes, as discussed and demoed in the article... Teaching PowerShell to Click. Red the article for full explanation

In a PowerShell Editor - In this case I'm using the ISE - of course in VSCode as well.

# View the page live to se what it does - just do stuff
Start-Process 'http://www.minasi.com/addit.htm'

# Now, inspect the site page/form.
# View the page code to see what is there - If in the ISE

$minasi = Invoke-WebRequest -Uri 'http://www.minasi.com/addit.htm'
$minasi.Content | Out-ISETab
$minasi.AllElements | Out-ISETab
$minasi.Forms | Out-ISETab
$minasi.Forms.Fields | Out-ISETab
$minasi.InputFields | Out-ISETab

# View the page code to see what is there - If in VSCode

$minasi = Invoke-WebRequest -Uri 'http://www.minasi.com/addit.htm'
$minasi.Content | New-EditorFile
$minasi.AllElements | New-EditorFile
$minasi.Forms | New-EditorFile
$minasi.Forms.Fields | New-EditorFile
$minasi.InputFields | New-EditorFile

# Interact with the page to achieve the same results as GUI would by going direct to the 'results page' not the 'form page'.

$MethodPostURI = 'http://www.minasi.com/addit-worker.asp'
$Body ='addend1=3&addend2=9&B1=SUBMIT%21'
$page = Invoke-WebRequest $MethodPostURI -body $Body -method POST
$page.content

# You can also find elements by looping to find something specific, like the below...
$minasi.AllElements.FindById('SomeIdNameString')

# Or

$SubmitElement = $minasi.AllElements |
Where{
        $PSItem.tagName -eq 'INPUT' -and
        $PSItem.value -eq 'SUBMIT!'
     }
$SubmitElement # The use dot lookup to see if there is a click option.

Lastly, here is something you may want to take a look at.

pQuery

This module is meant to create a variable that we can use jQuery-like
  syntax to do jQuery-like manipulations on a website.


问:
Need Help in understanding AWK script logic
I need help in understanding the below code logic written in AWK by one of our collegues in the past.
Can anyone with good AWK knowledge help me to understand this code.

Thanks,
Sandeep

```<code>sed -i 's/\r//g' $1 $2
sed -i 's/,/;/g' $1 $2
awk -F"|" '{if(FILENAME=="Parameter.txt"){a[$1]=NR;aa[NR]=$1;d=NR;if(NR==1){e=$1}else{e=e","$1};}
                        else if(FILENAME=="Traffic.csv"){h[FNR","$2]=$3;x[FNR","$4]=$5;k[FNR","$2","$3","$4","$5]=$1;y=FNR;}
                        else if(FILENAME=="Filter.txt"){for(i=1;i<=NF;i=i+3){if($(i+1)!="ne"){FE[FNR","$i]=$(i+2)}else{FNE[FNR","$i]=$(i+2)}};FILTER=FNR;FC[FNR]=(NF/3);}
                        else{
                                if(g==""){print e",TRAFFIC_CASE";g=1};
                                for(i=1;i<=d;i++){c[i]=""};l="";m="";z="";FINAL="";for(j=1;j<=FILTER;j++){FI[j]=""};
                                for(i=1;i<=NF;i++){
                                                                        split($i,b,"=");
                                                                        if(a[b[1]]!=""){c[a[b[1]]]=b[2];};
                                                                        for(j=1;j<=y;j++){
                                                                                if(h[j","a[b[1]]]==b[2] && h[j","a[b[1]]]!=""){l=a[b[1]]","b[2]};
                                                                                if(x[j","a[b[1]]]==b[2] && x[j","a[b[1]]]!=""){z=a[b[1]]","b[2]};
                                                                                if(k[j","l","z]!=""){m=k[j","l","z]};};
                                                                        };
                                if(substr(FILENAME,20,3)=="SVC"){Q[c[a[MSISDN]]]=1}
                                else if(Q[c[a[MSISDN]]]!=1){
                                        for(i=1;i<=d;i++){
                                                                                if(i==1){f=c[1];}else{f=f","c[i]};
                                                                                if(c[i]==""){c[i]="B"};
                                                                                for(j=1;j<=FILTER;j++){
                                                                                if(FE[j","i]!=""){if(FE[j","i]==c[i] && (FI[j]=="" || FI[j]<=FC[j])){FI[j]=FI[j]+1;}else{FI[j]=FI[j]-1;}};
                                                                                if(FNE[j","i]!=""){if(FNE[j","i]!=c[i] && (FI[j]=="" || FI[j]<=FC[j])){FI[j]=FI[j]+1;}else{FI[j]=FI[j]-1;}};
                                                                                };
                                                                                };
                                                                        }
                                for(j=1;j<=FILTER;j++){if(FI[j]==FC[j]){FINAL=1};};
                                if(FINAL!=1){print f","m;};
                                };
                        }' Parameter.txt Traffic.csv Filter.txt $2 $1
</code>```

答:
We have the parameters, traffic cases and filters in files... this is a big enough script that it also wants to be in a stand-alone file.

My process:

I reformatted and dropped the result in vim

I eliminated one nest by pattern matching outside the first set of brackets.

I saw the concatenation of "," in arrays... can use SUBSEP for this (after some further refactoring, can probably eliminate the SUBSEP definition... it's not relevant

Those places I saw a check for if (foo[bar] == ""), I replaced that for the more intuitive if (bar in foo)

I started replacing with s/\<x\>/something_x/g -- vim is really nice for this because you can isolate variable names using \< \>...

I replaced blanking arrays with split("", arr)

I saw the b[1] was a key and b[2] was a value... best to create intermediate variables to make the traffic match section more readable.

Going through each variable started revealing unused variables like aa.

I changed things like foo = foo + 1 to foo++

I reversed logic conditions in if statements where it seemed appropriate and got rid of superfluous {}.

I did some refactoring around the filtering bit to reduce the number of array lookups.

Here's my guess:

```<code>#! /usr/bin/awk -f

BEGIN {
    FS = "|"
    fileno = 0
    SUBSEP = ","
}

FILENAME == "Parameter.txt" {
    param[$1] = NR
    param_sz = NR
    if (FNR == 1)
        param_list = $1
    else
        param_list = param_list "," $1
    next
}

FILENAME == "Traffic.csv" {
    traffic_h[FNR, $2] = $3
    traffic_x[FNR, $4] = $5
    traffic_key[FNR, $2, $3, $4, $5] = $1
    traffic_sz = FNR
    next
}

FILENAME == "Filter.txt" {
    for (i = 1; i <= NF; i += 3) {
        if ($(i + 1) == "ne")
            filter_ne[FNR, $i] = $(i + 2)
        else
            filter_eq[FNR, $i] = $(i + 2)
    }
    filter_sz = FNR
    filters[FNR] = (NF / 3)
    next
}

fileno == 0  {
    print param_list ",TRAFFIC_CASE"
    fileno = FNR
}

{
    split("", case)
    l = ""
    traffic_match = ""
    z = ""
    final_filter = 0
    split("", filtered)

    for (i = 1; i <= NF; i++) {
        split($i, b, "=")
        key = b[1]
        val = b[2]
        if (key in param) {
            this_param = param[key]
            case[this_param] = val
            for (t = 1; t <= traffic_sz; t++) {
                if ((t, this_param == val && traffic_h[t, this_param]) in traffic_h)
                    l = val
                if ((t, this_param == val && traffic_x[j, this_param]) in traffic_x)
                    z = val
                if ((t, this_param, l, this_param, z) in traffic_key)
                    traffic_match = traffic_key[t, this_param, l, this_param, z]
            }
        }
    }
    if (substr(FILENAME, 20, 3) == "SVC") {
        svc_case[case[param["MSISDN"]]] = "" # << MSISDN was = "" so... arbitrary string should be OK or... should fix your bug?
    }
    if (!(case[param["MSISDN"]] in svc_case)) {
        for (p = 1; p <= param_sz; p++) {
            if (p == 1)
                case_list = case[1]
            else
                case_list = case_list "," case[i]
            if (!(p in case))
                case[p] = "B"
            for (f = 1; f <= filter_sz; f++) {
                if ((f, p) in filter_eq) {
                    if (filter_eq[f, p] == case[p] && (!f in filtered) || filtered[f] <= filters[f]))
                        ++filtered[f]
                    else
                        --filtered[f]
                } else if ((j, i) in filter_ne) {
                    if (filter_ne[f, p] != case[p] && (!(f in filtered) || filtered[f] <= filters[f]))
                        ++filtered[f]
                    else
                        --filtered[f]
                }
            }
        }
    }
    for (f = 1; f <= filter_sz; f++)
        if (filtered[f] == filters[f])
            final_filter = 1
    if (!final_filter)
        print case_list "," traffic_match
}
</code>```

So, it's very, very likely I got something wrong because I have no I/O to work with for a test-case -- but you can repeat my steps to figure out reality.  I'd work through that filtering piece a bit more... I'd check "MSISDN" (like Ed suggested) and see if that looks interesting to you...

Another thing that looks suspect is that you are ostensibly trying to parse a .csv file with "|" as a delimiter... that's probably not going to work.

Body:
I need help in understanding the below code logic written in AWK by one of our collegues in the past.
Can anyone with good AWK knowledge help me to understand this code.

Thanks,
Sandeep

sed -i 's/\r//g' $1 $2
sed -i 's/,/;/g' $1 $2
awk -F"|" '{if(FILENAME=="Parameter.txt"){a[$1]=NR;aa[NR]=$1;d=NR;if(NR==1){e=$1}else{e=e","$1};}
                        else if(FILENAME=="Traffic.csv"){h[FNR","$2]=$3;x[FNR","$4]=$5;k[FNR","$2","$3","$4","$5]=$1;y=FNR;}
                        else if(FILENAME=="Filter.txt"){for(i=1;i<=NF;i=i+3){if($(i+1)!="ne"){FE[FNR","$i]=$(i+2)}else{FNE[FNR","$i]=$(i+2)}};FILTER=FNR;FC[FNR]=(NF/3);}
                        else{
                                if(g==""){print e",TRAFFIC_CASE";g=1};
                                for(i=1;i<=d;i++){c[i]=""};l="";m="";z="";FINAL="";for(j=1;j<=FILTER;j++){FI[j]=""};
                                for(i=1;i<=NF;i++){
                                                                        split($i,b,"=");
                                                                        if(a[b[1]]!=""){c[a[b[1]]]=b[2];};
                                                                        for(j=1;j<=y;j++){
                                                                                if(h[j","a[b[1]]]==b[2] && h[j","a[b[1]]]!=""){l=a[b[1]]","b[2]};
                                                                                if(x[j","a[b[1]]]==b[2] && x[j","a[b[1]]]!=""){z=a[b[1]]","b[2]};
                                                                                if(k[j","l","z]!=""){m=k[j","l","z]};};
                                                                        };
                                if(substr(FILENAME,20,3)=="SVC"){Q[c[a[MSISDN]]]=1}
                                else if(Q[c[a[MSISDN]]]!=1){
                                        for(i=1;i<=d;i++){
                                                                                if(i==1){f=c[1];}else{f=f","c[i]};
                                                                                if(c[i]==""){c[i]="B"};
                                                                                for(j=1;j<=FILTER;j++){
                                                                                if(FE[j","i]!=""){if(FE[j","i]==c[i] && (FI[j]=="" || FI[j]<=FC[j])){FI[j]=FI[j]+1;}else{FI[j]=FI[j]-1;}};
                                                                                if(FNE[j","i]!=""){if(FNE[j","i]!=c[i] && (FI[j]=="" || FI[j]<=FC[j])){FI[j]=FI[j]+1;}else{FI[j]=FI[j]-1;}};
                                                                                };
                                                                                };
                                                                        }
                                for(j=1;j<=FILTER;j++){if(FI[j]==FC[j]){FINAL=1};};
                                if(FINAL!=1){print f","m;};
                                };
                        }' Parameter.txt Traffic.csv Filter.txt $2 $1

Body:
We have the parameters, traffic cases and filters in files... this is a big enough script that it also wants to be in a stand-alone file.

My process:

I reformatted and dropped the result in vim

I eliminated one nest by pattern matching outside the first set of brackets.

I saw the concatenation of "," in arrays... can use SUBSEP for this (after some further refactoring, can probably eliminate the SUBSEP definition... it's not relevant

Those places I saw a check for if (foo[bar] == ""), I replaced that for the more intuitive if (bar in foo)

I started replacing with s/\<x\>/something_x/g -- vim is really nice for this because you can isolate variable names using \< \>...

I replaced blanking arrays with split("", arr)

I saw the b[1] was a key and b[2] was a value... best to create intermediate variables to make the traffic match section more readable.

Going through each variable started revealing unused variables like aa.

I changed things like foo = foo + 1 to foo++

I reversed logic conditions in if statements where it seemed appropriate and got rid of superfluous {}.

I did some refactoring around the filtering bit to reduce the number of array lookups.

Here's my guess:

#! /usr/bin/awk -f

BEGIN {
    FS = "|"
    fileno = 0
    SUBSEP = ","
}

FILENAME == "Parameter.txt" {
    param[$1] = NR
    param_sz = NR
    if (FNR == 1)
        param_list = $1
    else
        param_list = param_list "," $1
    next
}

FILENAME == "Traffic.csv" {
    traffic_h[FNR, $2] = $3
    traffic_x[FNR, $4] = $5
    traffic_key[FNR, $2, $3, $4, $5] = $1
    traffic_sz = FNR
    next
}

FILENAME == "Filter.txt" {
    for (i = 1; i <= NF; i += 3) {
        if ($(i + 1) == "ne")
            filter_ne[FNR, $i] = $(i + 2)
        else
            filter_eq[FNR, $i] = $(i + 2)
    }
    filter_sz = FNR
    filters[FNR] = (NF / 3)
    next
}

fileno == 0  {
    print param_list ",TRAFFIC_CASE"
    fileno = FNR
}

{
    split("", case)
    l = ""
    traffic_match = ""
    z = ""
    final_filter = 0
    split("", filtered)

    for (i = 1; i <= NF; i++) {
        split($i, b, "=")
        key = b[1]
        val = b[2]
        if (key in param) {
            this_param = param[key]
            case[this_param] = val
            for (t = 1; t <= traffic_sz; t++) {
                if ((t, this_param == val && traffic_h[t, this_param]) in traffic_h)
                    l = val
                if ((t, this_param == val && traffic_x[j, this_param]) in traffic_x)
                    z = val
                if ((t, this_param, l, this_param, z) in traffic_key)
                    traffic_match = traffic_key[t, this_param, l, this_param, z]
            }
        }
    }
    if (substr(FILENAME, 20, 3) == "SVC") {
        svc_case[case[param["MSISDN"]]] = "" # << MSISDN was = "" so... arbitrary string should be OK or... should fix your bug?
    }
    if (!(case[param["MSISDN"]] in svc_case)) {
        for (p = 1; p <= param_sz; p++) {
            if (p == 1)
                case_list = case[1]
            else
                case_list = case_list "," case[i]
            if (!(p in case))
                case[p] = "B"
            for (f = 1; f <= filter_sz; f++) {
                if ((f, p) in filter_eq) {
                    if (filter_eq[f, p] == case[p] && (!f in filtered) || filtered[f] <= filters[f]))
                        ++filtered[f]
                    else
                        --filtered[f]
                } else if ((j, i) in filter_ne) {
                    if (filter_ne[f, p] != case[p] && (!(f in filtered) || filtered[f] <= filters[f]))
                        ++filtered[f]
                    else
                        --filtered[f]
                }
            }
        }
    }
    for (f = 1; f <= filter_sz; f++)
        if (filtered[f] == filters[f])
            final_filter = 1
    if (!final_filter)
        print case_list "," traffic_match
}

So, it's very, very likely I got something wrong because I have no I/O to work with for a test-case -- but you can repeat my steps to figure out reality.  I'd work through that filtering piece a bit more... I'd check "MSISDN" (like Ed suggested) and see if that looks interesting to you...

Another thing that looks suspect is that you are ostensibly trying to parse a .csv file with "|" as a delimiter... that's probably not going to work.


问:
How i can send orderId received after answering from api?Swift
Hoe i can save orderId and retrieve from Disk and send second Request
answer from base:

```<code>{"orderId" : 241,"fullName" : "Поесть"}
</code>```

SearchResponse.swift

```<code>struct SearchResponse: Codable {

    var orderId: Int
    var fullName: String
    var retailId: Int

    private enum CodingKeys: String, CodingKey {
        case orderId
        case fullName
        case retailId
    }
}
</code>```

after search request i  try to save orderId in disk like this in status == 200(service.swift):

```<code>if let value = result.value {
    var code = 0;
    do {
        if let responseDictionary = try JSONSerialization.jsonObject(with: value.data, options: .allowFragments) as? [String:Any] {
            code = responseDictionary["status"] as? Int ?? 0;
        }

    } catch {
        if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
            observer.onError(e);
        }
    }
    if (value.response?.statusCode == 200) {
        do{

            let response = try JSONDecoder().decode(SearchResponse.self, from: value.data);
            try Disk.save(response.retailId, to: .caches, as: FileName.retailName);
            try Disk.save(response.orderId, to: .caches, as: FileName.orderId);

        } catch {
            if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
                observer.onError(e);
            }
        }
        observer.onNext(true)
    } else {
        if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
            observer.onError(e);
        }
    }
}
</code>```

viewmodel.swift

```<code>private var orderId: String {
    let orderId = try? Disk.retrieve(FileName.orderId, from: .caches, as: SearchResponse.self).orderId
    return String(orderId!) <-- there error "Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value"
}
</code>```

答:
This is not a complete answer but I think it is clearer to explain as an answer. You should not use try? but instead catch any error and print them and furthermore avoid forced unwrapping.

```<code>private var orderId: String? {
    do {
        let orderId = try Disk.retrieve(FileName.orderId, from: .caches, as: SearchResponse.self).orderId
        return String(orderId)
    } catch {
        print(error)
    }
    return nil
}
</code>```

Now you will get an error message printed to the console if anything goes wrong or you will get nil back if nothing was found which separate two possible issues.

When saving to disk your error handling is only concerned with json decoding errors so you need to print/handle any possible errors generated from try Disk.save(... as well

Body:
Hoe i can save orderId and retrieve from Disk and send second Request
answer from base:

{"orderId" : 241,"fullName" : "Поесть"}

SearchResponse.swift

struct SearchResponse: Codable {

    var orderId: Int
    var fullName: String
    var retailId: Int

    private enum CodingKeys: String, CodingKey {
        case orderId
        case fullName
        case retailId
    }
}

after search request i  try to save orderId in disk like this in status == 200(service.swift):

if let value = result.value {
    var code = 0;
    do {
        if let responseDictionary = try JSONSerialization.jsonObject(with: value.data, options: .allowFragments) as? [String:Any] {
            code = responseDictionary["status"] as? Int ?? 0;
        }

    } catch {
        if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
            observer.onError(e);
        }
    }
    if (value.response?.statusCode == 200) {
        do{

            let response = try JSONDecoder().decode(SearchResponse.self, from: value.data);
            try Disk.save(response.retailId, to: .caches, as: FileName.retailName);
            try Disk.save(response.orderId, to: .caches, as: FileName.orderId);

        } catch {
            if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
                observer.onError(e);
            }
        }
        observer.onNext(true)
    } else {
        if let e = try? JSONDecoder().decode(ErrorResponse.self, from: value.data) {
            observer.onError(e);
        }
    }
}

viewmodel.swift

private var orderId: String {
    let orderId = try? Disk.retrieve(FileName.orderId, from: .caches, as: SearchResponse.self).orderId
    return String(orderId!) <-- there error "Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value"
}

Body:
This is not a complete answer but I think it is clearer to explain as an answer. You should not use try? but instead catch any error and print them and furthermore avoid forced unwrapping.

private var orderId: String? {
    do {
        let orderId = try Disk.retrieve(FileName.orderId, from: .caches, as: SearchResponse.self).orderId
        return String(orderId)
    } catch {
        print(error)
    }
    return nil
}

Now you will get an error message printed to the console if anything goes wrong or you will get nil back if nothing was found which separate two possible issues.

When saving to disk your error handling is only concerned with json decoding errors so you need to print/handle any possible errors generated from try Disk.save(... as well


问:
Using the keys of an object literal as a Typescript type?
I have an object which contains some predefined data for my application, which is stored in a const variable like this:

const data:{[key:string]:any} =Object.freeze({
    some: 123,
    long: {"a":"b"},
    list: ["c"],
    of: "",
    arbitrary: null,
    things: 1.2,
});

```

<p>The keys of this object are known to the rest of the application. Consider this function which accesses the data object:</p>

<pre class="lang-js prettyprint-override"><code>function doWork(k) {
    if(!data.hasOwnProperty(k)) throw Error();
    let value = data[k];
    //...
}
</code>```

This is called with strings like

doWork("things");
```

<p>I would like to replace that runtime error for invalid keys with a Typescript compile-time check. I would like to be able to write</p>

<pre class="lang-js prettyprint-override"><code>function doWork(k: keyof data) {
    let value = data[k];
    //...
}
</code>```

But apparently the keyof operator does not work that way. I get an error TS2304: Cannot find name 'data'.

My workaround:
I can extract the keys of the object with something like this:

console.log("\""+Object.keys(data).join("\"|\"")+"\"");
```

<p>Which I can then copy/paste and define as a type.</p>

<pre class="lang-js prettyprint-override"><code>type data_key = "some"|"long"|"list"|"of"|"arbitrary"|"things"
export function doWork(k:data_key) {
    let value = data[k];
    //...
}
</code>```

This feels like a silly hack and is quite inconvenient whenever I have to make a change, because I have to remember to put in this statement at the right place, run the program, and manually copy in the values back into the source code (or realistically, just type in the changes myself).

I am open to a better solution. Is there a language feature that provides the functionality I am looking for?

答:
Let TypeScript infer the type of data, then extract the keys from the type it infers by using type data_key = keyof typeof data;:

```<code>const data = Object.freeze({
    some: 123,
    long: {"a":"b"},
    list: ["c"],
    of: "",
    arbitrary: null,
    things: 1.2,
});

type data_key = keyof typeof data;

function doWork(k: data_key) {
    let value = data[k];
    //...
}
</code>```

On the playground.

How that works:

TypeScript has advanced type inference and so is able to infer the type of the object initializer passed into Object.freeze with the keys some, long, list, etc. Object.freeze is defined as freeze<T>(o: T): Readonly<T>¹ so it returns a Readonly version of that same inferred type.

keyof gets the keys of a type.

typeof in this context is TypeScript's typeof, not JavaScript's. In TypeScript, there are places it expects runtime value, and other places it expects compile-time types. keyof's operand is a place where a compile-time type is expected, so typeof data returns the compile-time type of data, which is the Readonly version of the inferred type from freeze (with the keys some, long, list, etc.). (If you had x = typeof data, that would be typeof in a context where a runtime value is expected, so it would be JavaScript's typeof, and x would get the runtime value "object".)

¹ There are actually three definitions of freeze in lib.es5.d.ts (one for arrays, one for functions, and one for all other kinds of objects; this is that last one).

Body:
I have an object which contains some predefined data for my application, which is stored in a const variable like this:

const data:{[key:string]:any} =Object.freeze({
    some: 123,
    long: {"a":"b"},
    list: ["c"],
    of: "",
    arbitrary: null,
    things: 1.2,
});

The keys of this object are known to the rest of the application. Consider this function which accesses the data object:

function doWork(k) {
    if(!data.hasOwnProperty(k)) throw Error();
    let value = data[k];
    //...
}

This is called with strings like

doWork("things");

I would like to replace that runtime error for invalid keys with a Typescript compile-time check. I would like to be able to write

function doWork(k: keyof data) {
    let value = data[k];
    //...
}

But apparently the keyof operator does not work that way. I get an error TS2304: Cannot find name 'data'.

My workaround:
I can extract the keys of the object with something like this:

console.log("\""+Object.keys(data).join("\"|\"")+"\"");

Which I can then copy/paste and define as a type.

type data_key = "some"|"long"|"list"|"of"|"arbitrary"|"things"
export function doWork(k:data_key) {
    let value = data[k];
    //...
}

This feels like a silly hack and is quite inconvenient whenever I have to make a change, because I have to remember to put in this statement at the right place, run the program, and manually copy in the values back into the source code (or realistically, just type in the changes myself).

I am open to a better solution. Is there a language feature that provides the functionality I am looking for?

Body:
Let TypeScript infer the type of data, then extract the keys from the type it infers by using type data_key = keyof typeof data;:

const data = Object.freeze({
    some: 123,
    long: {"a":"b"},
    list: ["c"],
    of: "",
    arbitrary: null,
    things: 1.2,
});

type data_key = keyof typeof data;

function doWork(k: data_key) {
    let value = data[k];
    //...
}

<a href="http://www.typescriptlang.org/play/index.html#src=const%20data%20%3D%20Object.freeze(%7B%0D%0A%20%20%20%20some%3A%20123%2C%0D%0A%20%20%20%20long%3A%20%7B%22a%22%3A%22b%22%7D%2C%0D%0A%20%20%20%20list%3A%20%5B%22c%22%5D%2C%0D%0A%20%20%20%20of%3A%20%22%22%2C%0D%0A%20%20%20%20arbitrary%3A%20null%2C%0D%0A%20%20%20%20things%3A%201.2%2C%0D%0A%7D)%3B%0D%0A%0D%0Atype%20data_key%20%3D%20keyof%20typeof%20data%3B%0D%0A%0D%0Afunction%20doWork(k%3A%20data_key)%20%7B%0D%0A%20%20%20%20let%20value%20%3D%20data%5Bk%5D%3B%0D%0A%20%20%20%20%2F%2F...%0D%0A%7D" rel="noreferrer">On the playground.

How that works:

TypeScript has advanced type inference and so is able to infer the type of the object initializer passed into Object.freeze with the keys some, long, list, etc. Object.freeze is defined as freeze<T>(o: T): Readonly<T>¹ so it returns a Readonly version of that same inferred type.

keyof gets the keys of a type.

typeof in this context is TypeScript's typeof, not JavaScript's. In TypeScript, there are places it expects runtime value, and other places it expects compile-time types. keyof's operand is a place where a compile-time type is expected, so typeof data returns the compile-time type of data, which is the Readonly version of the inferred type from freeze (with the keys some, long, list, etc.). (If you had x = typeof data, that would be typeof in a context where a runtime value is expected, so it would be JavaScript's typeof, and x would get the runtime value "object".)

¹ There are actually three definitions of freeze in lib.es5.d.ts (one for arrays, one for functions, and one for all other kinds of objects; this is that last one).


问:
Optimize query selecting a period
Given the following table:

```<code>Table events
id
start_time
end_time
</code>```

Is there a way to quickly search for a constant?

E.g.

```<code>SELECT *
FROM events
WHERE start_time<='2009-02-18 16:27:12'
AND     end_time>='2009-02-18 16:27:12'
</code>```

I am using MySQL. Having an index on either field still has to check a range. Moreover an index on both fields will not make a difference (only the first will be used).

I can add fields / indexes to the table (so adding an indexed constructed field containing the info of both fields would be acceptable).

P.S. The need for this came from this question: Optimize SQL that uses between clause

答:
You've basically got a query with 2 distinctly separate range conditions. You are using >=, to MySQL this is always a range scan. There is documentation here to optimize range scans.

The bottom line is that MySQL performs an additional check to filter out rows that satisfy the range condition, and then satisfies the rest of the  WHERE clause, which in your case is another range condition.

OwnerDisplayName: daremon

ContentLicense: CC BY-SA 2.5

Title: Optimize query selecting a period

Body:
Given the following table:

Table events
id
start_time
end_time

Is there a way to quickly search for a constant?

E.g.

SELECT *
FROM events
WHERE start_time<='2009-02-18 16:27:12'
AND     end_time>='2009-02-18 16:27:12'

I am using MySQL. Having an index on either field still has to check a range. Moreover an index on both fields will not make a difference (only the first will be used).

I can add fields / indexes to the table (so adding an indexed constructed field containing the info of both fields would be acceptable).

P.S. The need for this came from this question: Optimize SQL that uses between clause

OwnerDisplayName: mluebke


问:
for and foreach statements in D
Besides the syntactic differences, are the two inherently the same?  Are both of them implemented in the core language? or is foreach part of the standard library?  And as far as performance, does it make a difference if I choose one over the other?

答:
The main difference between foreach and for is the higher level of
abstraction of a foreach-loop. A foreach-loop is typically lowered to some
for-loop by the compiler. This has (at least) four advantages:

Readability: foreach (a; someArray) doSomething(a); is inherently more
readable than for (size_t i = 0; i < someArray.length; i++) doSomething(someArray[i]);.
This gets even clearer if the type of someArray is not a simple array.

Flexibility: if, at one point in time, you decide that the type of
someArray has to be changed from some array to, say, a range or an object
(e.g. to implement a parallel loop), foreach stays unchanged whereas the
for-loop has to be changed to use either empty, front and popFront
(in the case of a range) or opApply or some other mechanism in the case of a
class or struct.

Special features, e.g. iterating over type tuples, decoding of UTF-8 and
UTF-16 strings.

Performance: the foreach-loop lets the compiler decide how to optimally
implement the loop based on the type (iterating over an array, a range,
a string, an object ...) and possibly other information (e.g. the size of the
type). This allows for efficient implementation for all types and other compiler
optimizations without you having to worry too much about implementation
details. In reality, the performance of foreach relative to hand-coded for
is mixed. foreach(dchar c; someString) {...} (i.e. the decoding of an UTF-8
string while looping) is very fast. But foreach(a; someObject) {...}, where
someObject implements opApply, is a tad slower (because the loop body is
wrapped into a delegate and opApply typically calls this delegate inside a loop, which
generates some overhead). As usual, this won't matter to your code in 99.99% of
the cases, as foreach will always yield an (at least) decent implementation.

The main disadvantage (besides, occasionally, speed) is that some things cannot
be done with foreach, namely many mutations of the thing being looped over
(e.g. resizing of an array within the loop body).

Title: for and foreach statements in D


问:
Handle WebApi Core methods with Id as string and int
I have two methods:

```<code>    [HttpGet("{id}")]
    public IActionResult GetTask([FromRoute] int id)
    {
    }

    [HttpGet("{userId}")]
    public IActionResult GetUserTask([FromRoute] string userId)
    {
    }
</code>```

As you can see, i want to pass to my API routes like:

https://localhost:44365/Task/1

and

https://localhost:44365/Task/string

But my WebApi project cant handle it. When i pass route like this:

```<code>https://localhost:44365/Task/7dd2514618c4-4575b3b6f2e9731edd61
</code>```

i get an 400 http and this response:

```<code>{
"id": [
    "The value '7dd2514618c4-4575b3b6f2e9731edd61' is not valid."
]
}
</code>```

While debugging, im not hitting any methods (when i pass string instead of int)

My question is, how to verload methods with one parameters with string or int? These methods do diffrent things

EDIT

When i pass something like:

```<code>https://localhost:44365/Task/dddd
</code>```

I still get response with invalid id

答:
You can define parameter type like [HttpGet("{id:int}")]. For more information refer below link.

https://learn.microsoft.com/en-us/aspnet/web-api/overview/web-api-routing-and-actions/attribute-routing-in-web-api-2#route-constraints

Your actions should be like below.

```<code>    [HttpGet("{id:int}")]
    public IActionResult GetTask([FromRoute] int id)
    {
    }

    [HttpGet("{userId}")]
    public IActionResult GetUserTask([FromRoute] string userId)
    {
    }
</code>```

Body:
I have two methods:

    [HttpGet("{id}")]
    public IActionResult GetTask([FromRoute] int id)
    {
    }

    [HttpGet("{userId}")]
    public IActionResult GetUserTask([FromRoute] string userId)
    {
    }

As you can see, i want to pass to my API routes like:

https://localhost:44365/Task/1

and

https://localhost:44365/Task/string

But my WebApi project cant handle it. When i pass route like this:

https://localhost:44365/Task/7dd2514618c4-4575b3b6f2e9731edd61

i get an 400 http and this response:

{
"id": [
    "The value '7dd2514618c4-4575b3b6f2e9731edd61' is not valid."
]
}

While debugging, im not hitting any methods (when i pass string instead of int)

My question is, how to verload methods with one parameters with string or int? These methods do diffrent things

EDIT

When i pass something like:

https://localhost:44365/Task/dddd

I still get response with invalid id

Body:
You can define parameter type like [HttpGet("{id:int}")]. For more information refer below link.

https://learn.microsoft.com/en-us/aspnet/web-api/overview/web-api-routing-and-actions/attribute-routing-in-web-api-2#route-constraints

Your actions should be like below.

    [HttpGet("{id:int}")]
    public IActionResult GetTask([FromRoute] int id)
    {
    }

    [HttpGet("{userId}")]
    public IActionResult GetUserTask([FromRoute] string userId)
    {
    }


问:
Trying to parse logfile based on start and end time
I am trying to parse large zipped logfile and would like to collect all matching parameters within a certain time range:

Wed Nov  3 09:27:20 2010 : remote IP address 209.151.64.18
Wed Nov  3 11:57:22 2010 : secondary DNS address 204.117.214.10

I am able to grep other parameter using the line below:
gzcat  jfk-gw10-asr1.20100408.log.gz | egrep gabriel|98.126.209.144\|13.244.137.58\|16.151.65.121

I have been unable to parse for the start time and/or end time.
Any assistance is greatly appreciated.

答:
you can access space separated field using awk:

```<code>cat you_file_name | awk '/ / { print $1;}'
</code>```

Use $[n] to print the desired field.

Body:
you can access space separated field using awk:

cat you_file_name | awk '/ / { print $1;}'

Use $[n] to print the desired field.


问:
error: cannot find symbol method crossFade() in 4.7.1
Before anyone duplicate it. I was using the below code before in v3.7.0 of glide. Now when I have updated it to 4.7.1 it is showing the error: cannot find symbol method crossFade().

I have searched in different places but could not get the code work.

```<code>Glide.with(this)
                    .load(uriProfileImage)
                    .crossFade()
                    .bitmapTransform(new CircleTransform(EditProfile.this))
                    .diskCacheStrategy(DiskCacheStrategy.ALL)
                    .into(changeImage);
</code>```

答:
I had the same issue, use old version solve the problem.
 implementation 'com.github.bumptech.glide:glide:3.7.0' than implementation 'com.github.bumptech.glide:glide:4.9.0' which is the last version

Body:
Before anyone duplicate it. I was using the below code before in v3.7.0 of glide. Now when I have updated it to 4.7.1 it is showing the error: cannot find symbol method crossFade().

I have searched in different places but could not get the code work.

Glide.with(this)
                    .load(uriProfileImage)
                    .crossFade()
                    .bitmapTransform(new CircleTransform(EditProfile.this))
                    .diskCacheStrategy(DiskCacheStrategy.ALL)
                    .into(changeImage);


问:
Wordpress AJAX 400 when registered in Shortcode Plugin
I'm writing a custom shortcode that will need to call some PHP file via AJAX, after some user event. But I am getting Bad Request 400 as if my wp_ajax_* actions aren't being registered.

I have tried rearranging the order in which I add actions and enqueue my scripts but nothing seems to work.

Here is my simple plugin code

function aj_ajax_demo_shortcode() {
    return '
Shortcode
';
}
add_shortcode( 'ajax_demo', 'aj_ajax_demo_shortcode' );

add_action( 'wp_ajax_nopriv_aj_ajax_demo', 'aj_ajax_demo_process' );
add_action( 'wp_ajax_aj_ajax_demo', 'aj_ajax_demo_process' );
function aj_ajax_demo_process() {
    wp_send_json((object) array('msg' => 'hello world'));
}

add_action( 'wp_enqueue_scripts', 'aj_enqueue_scripts' );
function aj_enqueue_scripts() {
    wp_enqueue_script(
        'aj-demo',
        plugin_dir_url( __FILE__ ) . 'aj-demo-ajax-code.js'
    );

    wp_localize_script(
        'aj-demo',
        'aj_ajax_demo',
        array(
            'ajax_url' => admin_url( 'admin-ajax.php' ),
            'aj_demo_nonce' => wp_create_nonce('aj-demo-nonce')
        )
    );
}
```

<p>And the JS:</p>

```fetch(aj_ajax_demo.ajax_url, {
    method: 'POST',
    data: {
        action : 'aj_ajax_demo',
        nonce : aj_ajax_demo.aj_demo_nonce,
    }
}).then(response => {
    if (response.ok) {
        response.json().then(response => {
            console.log(response);
        });
    }
});
```

I'm struggling to get my AJAX function to even be called, and I can't see where I have done something wrong, especially given the small amount of code I have.

答:
You don't send any post variable to server. Use FormData()

```<code>  var formData  = new FormData();
  formData.append('action','aj_ajax_demo');
  formData.append('nonce',aj_ajax_demo.aj_demo_nonce);
</code>```

and then:

```<code>fetch(aj_ajax_demo.ajax_url, {
    method: 'POST',
    body: formData
}).then(response => {
    if (response.ok) {
        response.json().then(response => {
            console.log(response);
        });
    }
});
</code>```

Body:
I'm writing a custom shortcode that will need to call some PHP file via AJAX, after some user event. But I am getting Bad Request 400 as if my wp_ajax_* actions aren't being registered.

I have tried rearranging the order in which I add actions and enqueue my scripts but nothing seems to work.

Here is my simple plugin code

function aj_ajax_demo_shortcode() {
    return '
Shortcode
';
}
add_shortcode( 'ajax_demo', 'aj_ajax_demo_shortcode' );

add_action( 'wp_ajax_nopriv_aj_ajax_demo', 'aj_ajax_demo_process' );
add_action( 'wp_ajax_aj_ajax_demo', 'aj_ajax_demo_process' );
function aj_ajax_demo_process() {
    wp_send_json((object) array('msg' => 'hello world'));
}

add_action( 'wp_enqueue_scripts', 'aj_enqueue_scripts' );
function aj_enqueue_scripts() {
    wp_enqueue_script(
        'aj-demo',
        plugin_dir_url( __FILE__ ) . 'aj-demo-ajax-code.js'
    );

    wp_localize_script(
        'aj-demo',
        'aj_ajax_demo',
        array(
            'ajax_url' => admin_url( 'admin-ajax.php' ),
            'aj_demo_nonce' => wp_create_nonce('aj-demo-nonce')
        )
    );
}

And the JS:

fetch(aj_ajax_demo.ajax_url, {
    method: 'POST',
    data: {
        action : 'aj_ajax_demo',
        nonce : aj_ajax_demo.aj_demo_nonce,
    }
}).then(response => {
    if (response.ok) {
        response.json().then(response => {
            console.log(response);
        });
    }
});

I'm struggling to get my AJAX function to even be called, and I can't see where I have done something wrong, especially given the small amount of code I have.

Body:
You don't send any post variable to server. Use FormData()

  var formData  = new FormData();
  formData.append('action','aj_ajax_demo');
  formData.append('nonce',aj_ajax_demo.aj_demo_nonce);

and then:

fetch(aj_ajax_demo.ajax_url, {
    method: 'POST',
    body: formData
}).then(response => {
    if (response.ok) {
        response.json().then(response => {
            console.log(response);
        });
    }
});


问:
How do I generate random string array urls without repeating?
I am trying to generate random string array urls in picasso, everything is working fine but it repeats, like i had 28 string array items when i start app some items are repeating but i want only 1 item at one time when random start

This is my code

```<code>     ImageView imageView = itemView.findViewById(R.id.imageview);
        random = new Random();
        int p=  random.nextInt(icons.length);
        Picasso.get().load(icons[p]).into(imageView);
</code>```

答:
You can keep track of previously generated integers in an array/list and check the array each time you generate a new random number. This way, if the new integer generated already exists in the array, you generate a new one, until you have generated 28 numbers, after which you will have to clear the array and start over.</p>

```<code>ImageView imageView = itemView.findViewById(R.id.imageview);
Random random = new Random();
List<Integer> prevInts = new ArrayList<>();
Picasso.get().load(icons[randomUniqueInteger()]).into(imageView);

public int randomUniqueInteger(){
    int p = 0;
    do {
        p = random.nextInt(icons.length);
    } while(prevInts.contains(p));

    if ((prevInts.size + 1) == icons.length){
       prevInts.clear();
    }

    prevInts.add(p);

    return p;
}
</code>```

Body:
I am trying to generate random string array urls in picasso, everything is working fine but it repeats, like i had 28 string array items when i start app some items are repeating but i want only 1 item at one time when random start

This is my code

     ImageView imageView = itemView.findViewById(R.id.imageview);
        random = new Random();
        int p=  random.nextInt(icons.length);
        Picasso.get().load(icons[p]).into(imageView);

Body:
You can keep track of previously generated integers in an array/list and check the array each time you generate a new random number. This way, if the new integer generated already exists in the array, you generate a new one, until you have generated 28 numbers, after which you will have to clear the array and start over.

ImageView imageView = itemView.findViewById(R.id.imageview);
Random random = new Random();
List<Integer> prevInts = new ArrayList<>();
Picasso.get().load(icons[randomUniqueInteger()]).into(imageView);

public int randomUniqueInteger(){
    int p = 0;
    do {
        p = random.nextInt(icons.length);
    } while(prevInts.contains(p));

    if ((prevInts.size + 1) == icons.length){
       prevInts.clear();
    }

    prevInts.add(p);

    return p;
}


问:
how to enable xpack.security.enabled?
I have installed Elasticsearch 7, on Ubuntu. I believe X-Pack is installed by default, but I need to enable it.

In which file should I set this setting?

My cluster settings are in: /etc/elasticsearch/elasticsearch.yml and /etc/default/elasticsearch, but I cannot see xpack.security.enabled in either of files...

答:
write the following line in /etc/elasticsearch/elasticsearch.yml :

```<code>xpack.security.enabled: true
</code>```

when you enable xpack.security, you need to create user and password unless you cannot use elasticsearch or kibana. follow this guide:
https://www.elastic.co/guide/en/elastic-stack-overview/current/get-started-enable-security.html

Title: how to enable xpack.security.enabled?

Body:
write the following line in /etc/elasticsearch/elasticsearch.yml :

xpack.security.enabled: true

when you enable xpack.security, you need to create user and password unless you cannot use elasticsearch or kibana. follow this guide:
https://www.elastic.co/guide/en/elastic-stack-overview/current/get-started-enable-security.html