问:
How to edit creation date of MP4 files?
Currently, I'm working on a python3 script that helps me sort the Google Photos takeout files. The Takeout service for Google Photos actually strips all the metadata of an image/video into a separate JSON file.

This script that I'm working on helps me to merge the timestamp present in the JSON file into its subsequent photo or video. In order to achieve this, I'm currently using - ExifTool by Phil Harvey, which is a Perl executable. I call this tool in a subprocess to edit the Date tags in EXIF/Metadata.

This process is quite hefty and is taking a large amount of time. Then I realised that most of my photos are JPG and videos are MP4, it is very easy to edit Exif data of JPG files in python using some of the libraries present & for the lesser proportion of photos like PNG I can use exiftool.

This has drastically improved the runtime of my script. Now I want to know that is there any way to edit the creation dates of MP4 files natively in python which can theoretically execute faster than the subprocess method.

Please help! Thanks in advance.

答:
Im not too familiar with it, but ffmpeg seems like an option for just the mp4's.

```<code>cmd = 'ffmpeg -i "file.mp4" -codec copy -metadata timestamp="new_time_here" "output.mp4"'
subprocess.call(shlex.split(cmd))
</code>```

modified from:
https://www.reddit.com/r/learnpython/comments/3yotj2/comment/cyfiyb7/?utm_source=share&utm_medium=web2x&context=3

Title: How to edit creation date of MP4 files?

Body:
Im not too familiar with it, but ffmpeg seems like an option for just the mp4's.

cmd = 'ffmpeg -i "file.mp4" -codec copy -metadata timestamp="new_time_here" "output.mp4"'
subprocess.call(shlex.split(cmd))

modified from:
https://www.reddit.com/r/learnpython/comments/3yotj2/comment/cyfiyb7/?utm_source=share&utm_medium=web2x&context=3


来源: stackexchange

问:
Browser Extension Background Module
I am working on a browser extension that runs a background script. So far I have all of that working. My issue is I need to use a module in my background script. When I write the import statement I get the follwing exception:

```<code>Uncaught SyntaxError: import declarations may only appear at top level of a module
</code>```

How does one turn a background script into a module?

Manifest.json

```<code>{
  "name": "Extension Demo",
  "version": "1.0",
  "description": "Build an Basic Extension!",
  "permissions": [
    "activeTab",
    "webRequest",
    "webRequestBlocking",
    "<all_urls>"
  ],
  "browser_action": {
    "default_popup": "index.html",
    "default_icon": {
      "16": "img/icon16.png",
      "32": "img/icon32.png",
      "48": "img/icon48.png"
    }
  },
  "manifest_version": 2,
  "background": {
    "scripts": [ "background.js" ]
  }
}
</code>```

bakground.js

```<code>import * as myMod from './myMod.js'

function redirect(requestDetails) {
    //do some awesome stuff
}

browser.webRequest.onBeforeRequest.addListener(
    redirect,
    { urls: ["<all_urls>"], types: ["page"] },
    ["blocking"]
);
</code>```

答:
You are using manifest V2, and I highly recommend updating to Manifest v3 to latest features and your error fix.

In your manifest.json file, pass "type": "module" inside the object to allow ESM imports.

```<code>"background": {
    "service_worker": "background.js",
    "type": "module"
}
</code>```

Alternatively, in case you still want to use V2, use a dynamic import instead. Keep in mind that those returns promises and you will have to await it or use a .then() statement.

```<code>(async () => {

  const myMod = await import("./myMod.js");
  // if myMod.js uses an `export default` statement, use `(await import("./myMod.js")).default` instead.

  function redirect(requestDetails) {
    //do some awesome stuff
  }

  browser.webRequest.onBeforeRequest.addListener(
    redirect,
    { urls: ["<all_urls>"], types: ["page"] },
    ["blocking"]
  );
})();

</code>```

ContentLicense: CC BY-SA 4.0

Title: Browser Extension Background Module

Body:
I am working on a browser extension that runs a background script. So far I have all of that working. My issue is I need to use a module in my background script. When I write the import statement I get the follwing exception:

Uncaught SyntaxError: import declarations may only appear at top level of a module

How does one turn a background script into a module?

Manifest.json

{
  "name": "Extension Demo",
  "version": "1.0",
  "description": "Build an Basic Extension!",
  "permissions": [
    "activeTab",
    "webRequest",
    "webRequestBlocking",
    "<all_urls>"
  ],
  "browser_action": {
    "default_popup": "index.html",
    "default_icon": {
      "16": "img/icon16.png",
      "32": "img/icon32.png",
      "48": "img/icon48.png"
    }
  },
  "manifest_version": 2,
  "background": {
    "scripts": [ "background.js" ]
  }
}

bakground.js

import * as myMod from './myMod.js'

function redirect(requestDetails) {
    //do some awesome stuff
}

browser.webRequest.onBeforeRequest.addListener(
    redirect,
    { urls: ["<all_urls>"], types: ["page"] },
    ["blocking"]
);

Body:
You are using manifest V2, and I highly recommend updating to Manifest v3 to latest features and your error fix.

In your manifest.json file, pass "type": "module" inside the object to allow ESM imports.

"background": {
    "service_worker": "background.js",
    "type": "module"
}

Alternatively, in case you still want to use V2, use a dynamic import instead. Keep in mind that those returns promises and you will have to await it or use a .then() statement.

(async () => {

  const myMod = await import("./myMod.js");
  // if myMod.js uses an `export default` statement, use `(await import("./myMod.js")).default` instead.

  function redirect(requestDetails) {
    //do some awesome stuff
  }

  browser.webRequest.onBeforeRequest.addListener(
    redirect,
    { urls: ["<all_urls>"], types: ["page"] },
    ["blocking"]
  );
})();


问:
Maintaining message integrity
I have a message that I am passing to myself which will be subject to man-in-the-middle attacks.  Because of that, I am concerned about the integrity of the message being maintained between the time I send it, and the time that I receive it back.

It should be assumed that once I send the message to myself, no information about the message sent will be available to me in the future.  The message is completely self-contained.

To that end, I know that should hash the message contents and compare the hashes before I send the message, and after I send the message, if they differ, then the message has been tampered with.

Of course, if the man-in-the-middle knows that the hash is really just the hash of the message contents, as-is, then because the message is self-contained, he can just create new contents and apply the same hash algorithm to the contents.

The question is, to what lengths should I go to randomize the message contents when generating the hash?  When does it reach the point of diminishing returns?

In this scenario, I have a set of key/value pairs.  To that end, the steps that I know I HAVE to take are:

Add a salt to the message.  The salt is a secret to the rest of the world.  It gets attached to the contents of the message before hashing.

Order the key/value pairs in a consistent manner before generating the hash.

While not directly relevant, a timestamp is going to be added to the contents of each message before hashing, to prevent replay attacks.

These are the optional steps that I am considering:

Transforming the keys before I order them.  I've considered reversing them, then ordering by count/key.

Playing with the separators that separate key/value pairs (both for the separator for the key/value and the separator for the pair).

NOTE

Message privacy is not a requirement here, so I am not looking for encryption.  The values must be transmitted in plain-text.

Finally, what hashing algorithms should I avoid?

Specifics

I have an ASP.NET MVC site which I have a controller which handles input validation and persistence.

If (based on a heuristic, it's not important which) the input is determined to be an automated spam attempt, a model of IDictionary<string, string> is created with the input values and a ViewResult is sent to a general CAPTCHA page.

In that view, in the form that contains the CAPTCHA control, the contents of the IDictionary<string, string> will be written out in hidden input fields, and the action of the form will be the same action that the contents were originally posted to.  This way, MVC can pick up the values when the form is resubmitted.

It's because of this I can't encrypt the key/value pairs (or maybe I can and should, tell me why and how!).

Of course, I need to add one more value, which contains the hashed message contents.  If that value is there, then the controller will check to see that the message integrity is maintained, and allow the input to be persisted if it has.

Solution

I've opted to go with the SignedCms class in the System.Security.Cyrptography.Pkcs namespace, which represents the signging and verifying of CMS/PKCS #7 messages.

To elaborate, I've created a self-issued certificate with MAKECERT.EXE and then in my code, I use the example here to digitally sign the data:

http://blogs.msdn.com/shawnfa/archive/2006/02/27/539990.aspx

Now, it should be a matter of keeping the password on the exported private key secure, as well as security on the server, which makes it less about programming.

I'll have to add an extra key for the timestamp for replay attacks, but that won't be too hard.

The answer goes to Kalium, not for his initial post, but for his follow up comments which pointed the way to digital signatures, and eventually my discovery of how to utilize them in .NET.

Thanks to everyone who contributed.

答: I think PGP/GPG is what you want here.</p>

OwnerDisplayName: casperOne

ContentLicense: CC BY-SA 2.5

Title: Maintaining message integrity

LastEditorDisplayName: casperOne

Body: I think PGP/GPG is what you want here.

OwnerDisplayName: Kalium


问:
JavaScript/DOM - Is there a difference between a "CSS Selector" and an attribute?
Excluding declarative event handlers:

href='#' onclick=<handler> ... />

Is there any material difference between an Attribute and a CSS Selector?  If I provide my own attribute:

href='#' my-data-node-type='1'/>

Is "my-data-node-type" an attribute, a CSS Selector or both?  I think what I'm really asking here is, are the terms "attribute" and "css selector" synonymous?  Or is a "css selector" only an attribute recognized by CSS as eligible for styling?

答:
Actually selectors use attributes (and tag names) and classes and pseudo-classes.

Selectors are of different types from universal to more specific ones.

See: http://www.w3.org/TR/CSS2/selector.html

So you can use your attributes to build a pattern (a selector) to apply a rule to it. (Or to find the element in the DOM if using some parsing library like JQuery).

What selectors you can use depend a little on the CSS engine implementation, some browsers don't recognize some pseudo-classes for example.

ContentLicense: CC BY-SA 3.0

Body:
Excluding declarative event handlers:

<a href='#' onclick=<handler> ... />

Is there any material difference between an Attribute and a CSS Selector?  If I provide my own attribute:

Is "my-data-node-type" an attribute, a CSS Selector or both?  I think what I'm really asking here is, are the terms "attribute" and "css selector" synonymous?  Or is a "css selector" only an attribute recognized by CSS as eligible for styling?


问:
Why does -INT_MIN == INT_MIN, and how the negative sign operate in bit level?
Can't understand how -INT_MIN transform to INT_MIN in bit level</p>

答:
In common architecture negative values use 2's complement.

This has an immediate outcome: -INT_MIN will be greater (by 1) that INT_MAX. That means the the result value will be undefined(*). And still for common architectures it just happen to be -INT_MIN again.

Just look at what happens for signed bytes (values are lower so easier to handle) MIN is -128 and MAX is 127. In hexa, they are respectively 0x80 and 0x7F. But the representation of 128 as an int value is again 0x80. And when you assign that back to a signed byte type you end with (- (-128)) gives -128...

As char has a lower rank than int, the char is promoted to int, the operation gives a result that can be represented as an int and the conversion to a char gives an implementation defined result. But - INT_MIN is an expression that would give a result that cannot be represented as a int value. This is an exceptional condition and the behaviour is explicitely undefined per standard. That being said, common implementation handle everything the same...


问:
How to generate combinations of elements of different lists?
I have a list of lists that goes like

```<code>[[1,2,3],[3,4,5],[5,6,7,8]]
</code>```

I want to create a program/function that creates a combinations like

```<code>1,3,5
1,3,6
1,3,7
1,3,8
1,4,5
1,4,6
.
.
.
</code>```

If there's a way you can do it without using the itertools module, that would be even more appreciated.

答:
As you asked for a solution without itertools, this one is a recursive function that takes a list of any length and does the combination you need:

def combine(elems):
    if len(elems) == 0:
        return [[]]
    result = []
    subcombinations =  combine(elems[1:])
    for x in elems[0]:
        for y in subcombinations:
            result.append([x, *y])
    return result
```
<p>Or a much shorter version</p>
<pre class="lang-py prettyprint-override"><code>def combine(elems):
    if len(elems) == 0:
        return [[]]
    return [[x, *y] for x in elems[0] for y in combine(elems[1:])]

</code>```

Body:
I have a list of lists that goes like

[[1,2,3],[3,4,5],[5,6,7,8]]

I want to create a program/function that creates a combinations like

1,3,5
1,3,6
1,3,7
1,3,8
1,4,5
1,4,6
.
.
.

If there's a way you can do it without using the itertools module, that would be even more appreciated.

Body:
As you asked for a solution without itertools, this one is a recursive function that takes a list of any length and does the combination you need:

def combine(elems):
    if len(elems) == 0:
        return [[]]
    result = []
    subcombinations =  combine(elems[1:])
    for x in elems[0]:
        for y in subcombinations:
            result.append([x, *y])
    return result

Or a much shorter version

def combine(elems):
    if len(elems) == 0:
        return [[]]
    return [[x, *y] for x in elems[0] for y in combine(elems[1:])]


问:
Turn list of Result into Result of list inside a computation expression?
I have a Result<'T, 'E> list that I would like to turn into a single Result<'T list, 'E> following these rules:

If any Result is an Error then the result should be an Error

If the result is an Error it should be the first Error in the list

If every result is an OK then the result should be an Ok and the list order should be maintained

So I had a go and implemented this as follows:

```<code>let all xs =
  let folder = fun state next ->
    match (state, next) with
    | (Result.Ok ys, Result.Ok y) -> ys |> List.append [ y ] |> Result.Ok
    | (Result.Error e, _) -> Result.Error e
    | (_, Result.Error e) -> Result.Error e
  Seq.fold folder (Result.Ok []) xs
</code>```

However, this seems like something that might already have been implemented in the standard library. Has it?

Second, I have a computation expression for Result like this:

```<code>type ResultBuilder () =
  member this.Bind(x, f) =
    match x with
    | Result.Ok o -> f o
    | Result.Error e -> Result.Error e
  member this.Return(value) = Result.Ok value
  member this.ReturnFrom(value) = value

let result = new ResultBuilder()
</code>```

I can use all inside of a result { ... } but is further integration possible? For example by implementing ResultBuilder.For?

答:
(Noting that other answers here more than suffice!)

If working specifically with Lists, this can be done using the sequence function (Result<'c, 'd> list -> Result<'c list, 'd>) exposed by the Result module of the Fsharpx.Extras library (source code).

However, for more general sequences, this can be done with Seq.sequenceResultM function provided by the FsToolkit.ErrorHandling library.

Body:
I have a Result<'T, 'E> list that I would like to turn into a single Result<'T list, 'E> following these rules:

If any Result is an Error then the result should be an Error

If the result is an Error it should be the first Error in the list

If every result is an OK then the result should be an Ok and the list order should be maintained

So I had a go and implemented this as follows:

let all xs =
  let folder = fun state next ->
    match (state, next) with
    | (Result.Ok ys, Result.Ok y) -> ys |> List.append [ y ] |> Result.Ok
    | (Result.Error e, _) -> Result.Error e
    | (_, Result.Error e) -> Result.Error e
  Seq.fold folder (Result.Ok []) xs

However, this seems like something that might already have been implemented in the standard library. Has it?

Second, I have a computation expression for Result like this:

type ResultBuilder () =
  member this.Bind(x, f) =
    match x with
    | Result.Ok o -> f o
    | Result.Error e -> Result.Error e
  member this.Return(value) = Result.Ok value
  member this.ReturnFrom(value) = value

let result = new ResultBuilder()

I can use all inside of a result { ... } but is further integration possible? For example by implementing ResultBuilder.For?


问:
Get text from pressed button
How can I get the text from a pressed button? (Android)

I can get the text from a button:

```<code>String buttonText = button.getText();
</code>```

I can get the id from a pressed button:

```<code>int buttinID = view.getId();
</code>```

What I can't find out at this moment is how to get the text on the pressed button.

```<code>public void onClick(View view) {
  // Get the text on the pressed button
}
</code>```

答:
Typecast the view into a button and access the text content using getText().

```<code>   public String onClick(View view) {
        // Get the text on the pressed button
        return ((Button)view).getText().toString()
    }
</code>```

Title: Get text from pressed button

Body:
How can I get the text from a pressed button? (Android)

I can get the text from a button:

String buttonText = button.getText();

I can get the id from a pressed button:

int buttinID = view.getId();

What I can't find out at this moment is how to get the text on the pressed button.

public void onClick(View view) {
  // Get the text on the pressed button
}

Body:
Typecast the view into a button and access the text content using getText().

   public String onClick(View view) {
        // Get the text on the pressed button
        return ((Button)view).getText().toString()
    }


问:
Oracle - Where clause parameter change query speed
Using a query with a table and a View, however if i change the where column the speed drastically changes.
What can be the problem?
its quite hard to google "performance changes on where clause" since all exemples are not even close to this.

```<code>SELECT ADRE.*
FROM INVOICE N
INNER JOIN ENTRY_INVOICE NAF
     ON NAF.COMPANY = N.COMPANY
     AND NAF.ID_INVOICE = N.ID_INVOICE
INNER JOIN UV_ENTRY_ALL ADRE
    ON ADRE.COMPANY = NAF.COMPANY
    AND ADRE.ID_ENTRY = NAF.ID_ENTRY
WHERE
    NAF.COMPANY = 1
    /*
    AND NAF.ID_INVOICE = 113806
        => 40 SECONDS

    AND  NAF.ID_ENTRY = 387473
        => 0,6 SECONDS

    AND EXISTS (SELECT 1 FROM ENTRY_INVOICE WHERE COMPANY = 1 AND ID_INVOICE=113806)
        => 1,6 SECONDS
    */
</code>```

ENTRY_INVOICE
IT HAS AT MOST 2 RECORDS FOR EACH INVOICE.

COMPANY
ID_INVOICE
ID_ENTRY

1
113706
387224

1
113706
387225

1
113707
387226

1
113806
387473

PLAN EXPLANATION
https://exists-stack.tiiny.site/

As questioned, even if remove the table INVOICE, and select especific columns, it wont change the speed.

答:
I would change the filtering criteria to WHERE N.COMPANY = 1 to nudge the optimizer to use that table as the driving table. The query could look like:

```<code>SELECT ADRE.*, N.*
FROM INVOICE N
INNER JOIN ENTRY_INVOICE NAF
     ON NAF.COMPANY = N.COMPANY
     AND NAF.ID_INVOICE = N.ID_INVOICE
INNER JOIN UV_ENTRY_ALL ADRE
    ON ADRE.COMPANY = NAF.COMPANY
    AND ADRE.ID_ENTRY = NAF.ID_ENTRY
WHERE
    N.COMPANY = 1 -- changed here
</code>```

Once that is changed, the query could benefit from the following indexes:

```<code>create index ix1 on invoice (company);

create index ix2 on entry_invoice (company, id_invoice);

create index ix3 on uv_entry_all (company, id_entry);
</code>```

Keep in mind that any extra search criteria may impact performance. In particular, I ignored the predicates in comments at the end of the query.

Body:
Using a query with a table and a View, however if i change the where column the speed drastically changes.
What can be the problem?
its quite hard to google "performance changes on where clause" since all exemples are not even close to this.

SELECT ADRE.*
FROM INVOICE N
INNER JOIN ENTRY_INVOICE NAF
     ON NAF.COMPANY = N.COMPANY
     AND NAF.ID_INVOICE = N.ID_INVOICE
INNER JOIN UV_ENTRY_ALL ADRE
    ON ADRE.COMPANY = NAF.COMPANY
    AND ADRE.ID_ENTRY = NAF.ID_ENTRY
WHERE
    NAF.COMPANY = 1
    /*
    AND NAF.ID_INVOICE = 113806
        => 40 SECONDS

    AND  NAF.ID_ENTRY = 387473
        => 0,6 SECONDS

    AND EXISTS (SELECT 1 FROM ENTRY_INVOICE WHERE COMPANY = 1 AND ID_INVOICE=113806)
        => 1,6 SECONDS
    */

ENTRY_INVOICE
IT HAS AT MOST 2 RECORDS FOR EACH INVOICE.

COMPANY
ID_INVOICE
ID_ENTRY

1
113706
387224

1
113706
387225

1
113707
387226

1
113806
387473

PLAN EXPLANATION
https://exists-stack.tiiny.site/

As questioned, even if remove the table INVOICE, and select especific columns, it wont change the speed.

Body:
I would change the filtering criteria to WHERE N.COMPANY = 1 to nudge the optimizer to use that table as the driving table. The query could look like:

SELECT ADRE.*, N.*
FROM INVOICE N
INNER JOIN ENTRY_INVOICE NAF
     ON NAF.COMPANY = N.COMPANY
     AND NAF.ID_INVOICE = N.ID_INVOICE
INNER JOIN UV_ENTRY_ALL ADRE
    ON ADRE.COMPANY = NAF.COMPANY
    AND ADRE.ID_ENTRY = NAF.ID_ENTRY
WHERE
    N.COMPANY = 1 -- changed here

Once that is changed, the query could benefit from the following indexes:

create index ix1 on invoice (company);

create index ix2 on entry_invoice (company, id_invoice);

create index ix3 on uv_entry_all (company, id_entry);

Keep in mind that any extra search criteria may impact performance. In particular, I ignored the predicates in comments at the end of the query.


问:
Enhancement SQL query ORACLE
I have complex SQL query below:

SELECT 'SA',
       V.CFTCN,
       V.COMIC,
       V.MARCHE,
       REGEXP_SUBSTR(V.TRDAC,'[^,]+',1,column_value) RELAC
FROM   (SELECT V.CFTCN,
               V.COMIC,
               V.MARCHE,
               V.TRDAC
        FROM   DEC2A V
        WHERE  V.ISREP='O'
        UNION
        SELECT V.CFTCN,
               V.COMIC,
               V.MARCHE,
               V.TRDAC
        FROM   DCFT2B V
        WHERE V.ISREP='O'
        AND   V.MARCHE NOT IN (SELECT DISTINCT MARCHE
                               FROM LTFMAR)
       ) V,
       TABLE(
         CAST(
           MULTISET(
             SELECT LEVEL
             FROM   DUAL
             CONNECT BY LEVEL <= LENGTH(REGEXP_REPLACE(V.TRDAC,'[^,]')) +1
           ) as sys.OdciNumberList
         )
       )
UNION
SELECT 'VT',
       TRDAC,
       COMIC,
       MARCHE,
       NULL RELAC
FROM   DCFT3B V
WHERE V.ISREP='O'
UNION
SELECT DISTINCT 'DA',
       CFTCN,
       NULL COMIC,
       MARCHE,
       NULL RELAC
FROM   CFTCSW
WHERE  DAHIS=TO_DATE('23122021','DDMMYYYY');
```

how can I rewrite my query to get enhance code?
thanks for all

答:
UNION is much slower than UNION ALL; if you do not need to eliminate duplicates than replace all the UNIONs with UNION ALL.

V.MARCHE NOT IN (SELECT DISTINCT MARCHE FROM LTFMAR)
```
<p>Does not need the <code>DISTINCT</code> clause as you do not need to filter out duplicates.</p>
<hr />
<p>Regular expressions are much slower than simple string functions. Your cross join <code>CAST</code> & <code>MULTISET</code> expression can be rewritten to something like:</p>
<pre class="lang-sql prettyprint-override"><code>WITH rsqfc (cftcn, /*...,*/ trdac, spos, epos) AS (
  SELECT cftcn,
         /* ..., */
         trdac,
         1,
         INSTR(trdac, ',', 1)
  FROM   dec2a
UNION ALL
  SELECT cftcn,
         /* ..., */
         trdac,
         epos + 1,
         INSTR(trdac, ',', epos + 1)
  FROM   rsqfc
  WHERE  epos > 0
)
SELECT cftcn,
       /* ..., */
       CASE epos
       WHEN 0
       THEN SUBSTR(trdac, spos)
       ELSE SUBSTR(trdac, spos, epos - spos)
       END AS relac
FROM   rsqfc;
</code>```

And, although the code is longer, it should be more performant.

db<>fiddle here

Title: Enhancement SQL query ORACLE

Body:
I have complex SQL query below:

SELECT 'SA',
       V.CFTCN,
       V.COMIC,
       V.MARCHE,
       REGEXP_SUBSTR(V.TRDAC,'[^,]+',1,column_value) RELAC
FROM   (SELECT V.CFTCN,
               V.COMIC,
               V.MARCHE,
               V.TRDAC
        FROM   DEC2A V
        WHERE  V.ISREP='O'
        UNION
        SELECT V.CFTCN,
               V.COMIC,
               V.MARCHE,
               V.TRDAC
        FROM   DCFT2B V
        WHERE V.ISREP='O'
        AND   V.MARCHE NOT IN (SELECT DISTINCT MARCHE
                               FROM LTFMAR)
       ) V,
       TABLE(
         CAST(
           MULTISET(
             SELECT LEVEL
             FROM   DUAL
             CONNECT BY LEVEL <= LENGTH(REGEXP_REPLACE(V.TRDAC,'[^,]')) +1
           ) as sys.OdciNumberList
         )
       )
UNION
SELECT 'VT',
       TRDAC,
       COMIC,
       MARCHE,
       NULL RELAC
FROM   DCFT3B V
WHERE V.ISREP='O'
UNION
SELECT DISTINCT 'DA',
       CFTCN,
       NULL COMIC,
       MARCHE,
       NULL RELAC
FROM   CFTCSW
WHERE  DAHIS=TO_DATE('23122021','DDMMYYYY');

how can I rewrite my query to get enhance code?
thanks for all

Body:
UNION is much slower than UNION ALL; if you do not need to eliminate duplicates than replace all the UNIONs with UNION ALL.

V.MARCHE NOT IN (SELECT DISTINCT MARCHE FROM LTFMAR)

Does not need the DISTINCT clause as you do not need to filter out duplicates.

Regular expressions are much slower than simple string functions. Your cross join CAST & MULTISET expression can be rewritten to something like:

WITH rsqfc (cftcn, /*...,*/ trdac, spos, epos) AS (
  SELECT cftcn,
         /* ..., */
         trdac,
         1,
         INSTR(trdac, ',', 1)
  FROM   dec2a
UNION ALL
  SELECT cftcn,
         /* ..., */
         trdac,
         epos + 1,
         INSTR(trdac, ',', epos + 1)
  FROM   rsqfc
  WHERE  epos > 0
)
SELECT cftcn,
       /* ..., */
       CASE epos
       WHEN 0
       THEN SUBSTR(trdac, spos)
       ELSE SUBSTR(trdac, spos, epos - spos)
       END AS relac
FROM   rsqfc;

And, although the code is longer, it should be more performant.

db<>fiddle here


问:
how do I only instantiate every constructor variable ones with multiple constructors?
I need to avoid duplicate code with these constructors, so I wondered if there is a way to instantiate these variables only ones? I saw a lot of posts about chaining but I haven't gotten it  to work properly.</p>
```<code>class Trainingsschema
{
    // create an arrayList for every oefening
    private String klant;
    private String trainer;
    private Integer nummer;
    private ArrayList<Oefening> alleOefeningen;

    // two different constructors to correctly initialise the class variables
    public Trainingsschema(String klant, String trainer)
    {
        this.klant = klant;
        this.trainer = trainer;
        this.nummer = this.volgendUniekNummer();
        this.alleOefeningen = this.initialiseerOefeningen();
    }

    public Trainingsschema(String klant)
    {
        this.klant = klant;
        this.trainer = null;
        this.nummer = this.volgendUniekNummer();
        this.alleOefeningen = this.initialiseerOefeningen();
    }
</code>```

答:
You should have one constructor to initialize all the variables.  Call it with defaults from other constructors, like this:</p>
```<code>class Trainingsschema
{
    // create an arrayList for every oefening
    private String klant;
    private String trainer;
    private Integer nummer;
    private List<Oefening> alleOefeningen;

    // two different constructors to correctly initialise the class variables
    public Trainingsschema(String klant, String trainer)
    {
        this.klant = klant;
        this.trainer = trainer;
        this.nummer = this.volgendUniekNummer();
        // Show the method where you initialize the array.  Why do you need method?
        this.alleOefeningen = new ArrayList<>();
    }

    public Trainingsschema(String klant)
    {
        this(klant, null);
    }
}
</code>```

Body:
I need to avoid duplicate code with these constructors, so I wondered if there is a way to instantiate these variables only ones? I saw a lot of posts about chaining but I haven't gotten it  to work properly.

class Trainingsschema
{
    // create an arrayList for every oefening
    private String klant;
    private String trainer;
    private Integer nummer;
    private ArrayList<Oefening> alleOefeningen;

    // two different constructors to correctly initialise the class variables
    public Trainingsschema(String klant, String trainer)
    {
        this.klant = klant;
        this.trainer = trainer;
        this.nummer = this.volgendUniekNummer();
        this.alleOefeningen = this.initialiseerOefeningen();
    }

    public Trainingsschema(String klant)
    {
        this.klant = klant;
        this.trainer = null;
        this.nummer = this.volgendUniekNummer();
        this.alleOefeningen = this.initialiseerOefeningen();
    }

Body:
You should have one constructor to initialize all the variables.  Call it with defaults from other constructors, like this:

class Trainingsschema
{
    // create an arrayList for every oefening
    private String klant;
    private String trainer;
    private Integer nummer;
    private List<Oefening> alleOefeningen;

    // two different constructors to correctly initialise the class variables
    public Trainingsschema(String klant, String trainer)
    {
        this.klant = klant;
        this.trainer = trainer;
        this.nummer = this.volgendUniekNummer();
        // Show the method where you initialize the array.  Why do you need method?
        this.alleOefeningen = new ArrayList<>();
    }

    public Trainingsschema(String klant)
    {
        this(klant, null);
    }
}


问:
Auto renew the Kerberos ticket
I had to use Kerberos authentication for the first time, it kinda works, but I feel like missing a lot of understanding what is going on and how to cook it properly.

Basically, what I need is my Python script to run every couple of hours and send a request to a remote web server using domain account in MS AD environment.

The following code provides me with the ready to go session instance:

```<code>from requests import Session
import gssapi
from requests_gssapi import HTTPSPNEGOAuth

session = Session()
session.auth = HTTPSPNEGOAuth(mech=gssapi.mechs.Mechanism.from_sasl_name("SPNEGO"))
</code>```

The script was added to the crontab of a user in a linux box and kinit was used to obtain a ticket-granting ticket:

```<code>kinit -kt ~/ad_user.keytab ad_user@DOMAIN.COM
</code>```

But after a while it all stopped because of the expired ticket. The solution was simple: adding the kinit to the crontab to run every 8 hours solved the issue.

I'm wondering if there is a better and more proper way to achieve the same? If I don't want/need to create a principal for the server in the AD, but simply want some code to always have a valid ticket - can I avoid having a dedicated task in users' crontab?

答:
Why don't you initiate the ticket cache directly in your code?  This might help be more transparent that your job relies on a kerberos login and where it's located.  (In 5 years when you come back to this code it will be hard to remember and this might save you some grief later.)

It will also help to ensure that someone can't accidentally disable your job by removing a cron job.

kinit from python script using keytab

I would do this in every script you require authentication to reduce external dependencies.

Title: Auto renew the Kerberos ticket

Body:
I had to use Kerberos authentication for the first time, it kinda works, but I feel like missing a lot of understanding what is going on and how to cook it properly.

Basically, what I need is my Python script to run every couple of hours and send a request to a remote web server using domain account in MS AD environment.

The following code provides me with the ready to go session instance:

from requests import Session
import gssapi
from requests_gssapi import HTTPSPNEGOAuth

session = Session()
session.auth = HTTPSPNEGOAuth(mech=gssapi.mechs.Mechanism.from_sasl_name("SPNEGO"))

The script was added to the crontab of a user in a linux box and kinit was used to obtain a ticket-granting ticket:

kinit -kt ~/ad_user.keytab ad_user@DOMAIN.COM

But after a while it all stopped because of the expired ticket. The solution was simple: adding the kinit to the crontab to run every 8 hours solved the issue.

I'm wondering if there is a better and more proper way to achieve the same? If I don't want/need to create a principal for the server in the AD, but simply want some code to always have a valid ticket - can I avoid having a dedicated task in users' crontab?


问:
Using an @if statement in an attribute doesn't work
I'm fairly new to C# and Blazor. I have the following attribute in a div tag.

```<code> class="menu-header @if(expanded[j_ind]){expanded}"
</code>```

I get the following error from Visual Studio 2019

RZ9979: Code blocks delimited by '@{...}' like '@{ if(expanded[j_ind]){expanded} }' for attributes are no longer supported These features have been changed to use attribute suntax. Use 'attr="@(x => {...}"'.

Edit: The comment below has the solution:

```<code>class="menu-header @(expanded[j_ind] ? "expanded" : null)"
</code>```

答:
You are doing it in a wrong way, The correct way is</p>
```<code>class="menu-header @(expanded[j_ind] ? "expanded" : "")"
</code>```

Body:
I'm fairly new to C# and Blazor. I have the following attribute in a div tag.

 class="menu-header @if(expanded[j_ind]){expanded}"

I get the following error from Visual Studio 2019

RZ9979: Code blocks delimited by '@{...}' like '@{ if(expanded[j_ind]){expanded} }' for attributes are no longer supported These features have been changed to use attribute suntax. Use 'attr="@(x => {...}"'.

Edit: The comment below has the solution:

class="menu-header @(expanded[j_ind] ? "expanded" : null)"

Body:
You are doing it in a wrong way, The correct way is

class="menu-header @(expanded[j_ind] ? "expanded" : "")"


问:
how to make sure one process start before other process?
How can I make shure that one process starts before the other?

Background: I am doing computations in two separate processes (each of them running in a loop). But for the computations in one process I need the results from the computations of the other process. The first thought might be: why not do it synchronous? But this one calculation runs faster than the other one. And if there are no new results available, it should just continue with the last available value.

Example Code:

```<code>from multiprocessing import Process, Queue
import time
import random

queue = Queue()

def some_calculation():
    # this is slower
    time.sleep(2)
    return random.randint(0, 10)

def some_other_calculation(required):
    # this is faster
    time.sleep(0.5)
    return required + random.randint(0, 10)

def first_target(queue):
    while True:
        # doing some computations
        res = some_calculation()
        queue.put(res)

def second_target(queue):
    res_list= []
    # req = 5 as an initial guess might also work, but lets
    # assume this is not an option
    while True:
        try:
            req = queue.get(block=False)
        except:
            print('no new value available, reuse old value')
        res = some_other_calculation(req)
        res_list.append(res)
        print('results so far:', res_list)

if __name__ == '__main__':
    proc1 = Process(target=first_target, args=(queue,))
    proc2 = Process(target=second_target, args=(queue,))
    proc1.start()
    # time.sleep(3) here everything works fine, but is there
    # maybe a more elegant solution?
    proc2.start()

</code>```

Is there a way -- aside from just sleeping or predefining a value for req -- to make sure, that this one process really starts before the other one does?

Many thanks for any help in advance!

答:
The idea is for second_target to start a thread that will issue blocking get calls on the queue and thus will constantly update the req variable with the latest value available. second_target also needs to make an initial synchronous call to some_calculation to get its initial value of req in order to greatly simplify the logic. I have also replace the infinite while True loop in second_target with a more limited loop for demo purposes in order that the program terminates:

from multiprocessing import Process, Queue
import time
import random
from threading import Thread

def some_calculation():
    # this is slower
    time.sleep(2)
    return random.randint(0, 10)

def some_other_calculation(required):
    # this is faster
    time.sleep(0.5)
    return required + random.randint(0, 10)

def first_target(queue):
    while True:
        # doing some computations
        res = some_calculation()
        queue.put(res)

def second_target(queue):
    def monitor_queue():
        nonlocal queue, req
        while True:
            req = queue.get()

    res_list= []
    req = some_calculation() # Call it synchronously for initial value
    last_req = req
    Thread(target=monitor_queue, daemon=True).start()
    for _ in range(10): # for testing purposes so we terminate:
    #while True:
        if req == last_req:
            print('no new value available, reuse old value')
        else:
            print('got a new value')
            last_req = req
        res = some_other_calculation(req)
        res_list.append(res)
        print('results so far:', res_list)

if __name__ == '__main__':
    queue = Queue()
    p = Process(target=first_target, args=(queue,), daemon=True)
    p.start()
    second_target(queue)
```
<p>Prints:</p>
<pre class="lang-none prettyprint-override"><code>no new value available, reuse old value
results so far: [5]
got a new value
results so far: [5, 2]
no new value available, reuse old value
results so far: [5, 2, 2]
no new value available, reuse old value
results so far: [5, 2, 2, 7]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11]
got a new value
results so far: [5, 2, 2, 7, 11, 11]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13, 13]
got a new value
results so far: [5, 2, 2, 7, 11, 11, 13, 13, 7]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13, 13, 7, 11]
</code>```

Body:
How can I make shure that one process starts before the other?

Background: I am doing computations in two separate processes (each of them running in a loop). But for the computations in one process I need the results from the computations of the other process. The first thought might be: why not do it synchronous? But this one calculation runs faster than the other one. And if there are no new results available, it should just continue with the last available value.

Example Code:

from multiprocessing import Process, Queue
import time
import random

queue = Queue()

def some_calculation():
    # this is slower
    time.sleep(2)
    return random.randint(0, 10)

def some_other_calculation(required):
    # this is faster
    time.sleep(0.5)
    return required + random.randint(0, 10)

def first_target(queue):
    while True:
        # doing some computations
        res = some_calculation()
        queue.put(res)

def second_target(queue):
    res_list= []
    # req = 5 as an initial guess might also work, but lets
    # assume this is not an option
    while True:
        try:
            req = queue.get(block=False)
        except:
            print('no new value available, reuse old value')
        res = some_other_calculation(req)
        res_list.append(res)
        print('results so far:', res_list)

if __name__ == '__main__':
    proc1 = Process(target=first_target, args=(queue,))
    proc2 = Process(target=second_target, args=(queue,))
    proc1.start()
    # time.sleep(3) here everything works fine, but is there
    # maybe a more elegant solution?
    proc2.start()

Is there a way -- aside from just sleeping or predefining a value for req -- to make sure, that this one process really starts before the other one does?

Many thanks for any help in advance!

Body:
The idea is for second_target to start a thread that will issue blocking get calls on the queue and thus will constantly update the req variable with the latest value available. second_target also needs to make an initial synchronous call to some_calculation to get its initial value of req in order to greatly simplify the logic. I have also replace the infinite while True loop in second_target with a more limited loop for demo purposes in order that the program terminates:

from multiprocessing import Process, Queue
import time
import random
from threading import Thread

def some_calculation():
    # this is slower
    time.sleep(2)
    return random.randint(0, 10)

def some_other_calculation(required):
    # this is faster
    time.sleep(0.5)
    return required + random.randint(0, 10)

def first_target(queue):
    while True:
        # doing some computations
        res = some_calculation()
        queue.put(res)

def second_target(queue):
    def monitor_queue():
        nonlocal queue, req
        while True:
            req = queue.get()

    res_list= []
    req = some_calculation() # Call it synchronously for initial value
    last_req = req
    Thread(target=monitor_queue, daemon=True).start()
    for _ in range(10): # for testing purposes so we terminate:
    #while True:
        if req == last_req:
            print('no new value available, reuse old value')
        else:
            print('got a new value')
            last_req = req
        res = some_other_calculation(req)
        res_list.append(res)
        print('results so far:', res_list)

if __name__ == '__main__':
    queue = Queue()
    p = Process(target=first_target, args=(queue,), daemon=True)
    p.start()
    second_target(queue)

Prints:

no new value available, reuse old value
results so far: [5]
got a new value
results so far: [5, 2]
no new value available, reuse old value
results so far: [5, 2, 2]
no new value available, reuse old value
results so far: [5, 2, 2, 7]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11]
got a new value
results so far: [5, 2, 2, 7, 11, 11]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13, 13]
got a new value
results so far: [5, 2, 2, 7, 11, 11, 13, 13, 7]
no new value available, reuse old value
results so far: [5, 2, 2, 7, 11, 11, 13, 13, 7, 11]


问:
jQuery date/time picker
I've been looking around for a decent jQuery plugin that can handle both dates and times.  The core UI DatePicker is great, but unfortunately I need to be able to take time in as well.

I've found a few hacks for the DatePicker to work with times, but they all seem pretty inelegant and Google isn't turning up anything nice.

Is there a good jQuery plugin for selecting dates and times in a single UI control with a usable interface?

答:
In my view, dates and times should be handled as two separate input boxes for it to be most usable and efficient for the user to input. Let the user input one thing at a time is a good principle, imho.

I use the core UI DatePicker, and the following time picker.

This one is inspired by the one Google Calendar uses:

jQuery timePicker:

examples: http://labs.perifer.se/timedatepicker/

project on github: https://github.com/perifer/timePicker

I found it to be the best among all of the alternatives. User can input fast, it looks clean, is simple, and allows user to input specific times down to the minute.

PS:
In my view: sliders (used by some alternative time pickers) take too many clicks and require mouse precision from the user (which makes input slower).

Title: jQuery date/time picker


问:
How to remove the border of last child in list items using after
In my application I have some font icons in i tag within the list items and after the each icons I have placed some border line .

But my requirement is to remove the border of last element in list items.

.component.html

```<code><div class="list-container">
<ul>
   <li *ngFor="let icon of iconsArr">
    <span class="icons" [style.font-weight]="icon.active ? 'bold' : 'normal'" ></span><i (click)="onIconClick(icon)" class="smiley-icon "   [ngClass]="icon.class"
    [style.color]="icon.active === true ? '#FF0000' : '#ffffff'" >

 </i>
   </li>
 </ul>
      </div>

</code>```

.component.css

```<code>.list-container{
  li{
    display: inline;
  }
.icons{
    font-size: 2.9em;
    display:inline-block;
    margin: 0px;
    text-shadow: -1.3px 0 #000, 0 1.3px #000, 1.3px 0 #000, 0 -1.3px #000;
    color: white;
    margin-right: -8.9px;
    //z-index:1000;
}
 .icons::after{
    content: "";
    display: inline-block;
    width: 0.8em;
    border-top: 5px solid #e8e8e8;
    top: 50%;
    z-index: -10;
    margin-bottom: 15px;
    margin-left: -8px;
}

</code>```

If I write

```<code>
.icons:last-of-type:after{
content:"";
border:none;
}
</code>```

It is not working, I want to remove the border for last icon from the list of icons

答:
```<code>ul li:last-of-type .icons {
    border: none;
}
</code>```

Please try something like that, or

```<code>ul li:last-of-child .icons {
    border: none;
}
</code>```

Body:
In my application I have some font icons in i tag within the list items and after the each icons I have placed some border line .

But my requirement is to remove the border of last element in list items.

.component.html

.component.css

.list-container{
  li{
    display: inline;
  }
.icons{
    font-size: 2.9em;
    display:inline-block;
    margin: 0px;
    text-shadow: -1.3px 0 #000, 0 1.3px #000, 1.3px 0 #000, 0 -1.3px #000;
    color: white;
    margin-right: -8.9px;
    //z-index:1000;
}
 .icons::after{
    content: "";
    display: inline-block;
    width: 0.8em;
    border-top: 5px solid #e8e8e8;
    top: 50%;
    z-index: -10;
    margin-bottom: 15px;
    margin-left: -8px;
}

If I write

.icons:last-of-type:after{
content:"";
border:none;
}

It is not working, I want to remove the border for last icon from the list of icons

Body:
ul li:last-of-type .icons {
    border: none;
}

Please try something like that, or

ul li:last-of-child .icons {
    border: none;
}


问:
EF Core 6, { character in an SQL VALUES() block is causing string not in a correct format exception
This one can't be correct, probably is somehow a mis-configuration of EF Core in our project.

Background: We have to export an Oracle DB from Aqua Data studio to a file.  I have been manually transposing the export from Oracle to SQL Server.  We came up with the idea of putting the manual process into a Visual Studio project.

From the project, certain strings in the VALUES() section of some INSERT INTO "SomeTable" blocks fail.

In all cases the same code copied over to SSMS works with no modification to it.

This unit test does prove that the { character in the string throws an exception.  Removing it from the string, the ExecuteRawSQL() method will now insert the data. I've put Bold / Italics around the {.

We are running the latest SQL Server Express, on local host.  EF Core 6, .Net Core 6.

The entire input string for ANSWER_TEXT is after the Test Method code block.

I've tried escaping the {, etc.  Also putting a space between { and Project.  Not that I thought this would fix it, just to rule it out.

The { causing the issue can't possibly be right.  The same block with the entire string will insert via SSMS.

It's got to be a Visual Studio project or EF Core config issue, something like that.

It's got me stumped -- didn't expect to trim this string down and be able to duplicate this malfunction.

```<code> [TestMethod]
    public void Test_CE_USER_ANSWERS()
    {
        File.Delete(@"C:\ProjectsTFS-DOIT\Online CE\Docs\New_TSQL_DB\Test_CE_USER_ANSWERS.txt");
        List<string> sqlDataStatements = new ();
        string ceUserAnswers = @"INSERT INTO ""CE_TRANSMITTAL_ANSWERS""(""FORM_ID"", ""TRANSMITTAL_ID"", ""QUESTION_ID"", ""ANSWER_ORDER"", ""ANSWER_TEXT"")
            VALUES(9804, 1, 808, 0, 'A history/architecture memo to file dated 3/25/ 2021 has been uploaded to Environet ***{*** Project File>Cultural Resources>Project ')
            ";
        sqlDataStatements.Add(ceUserAnswers);
        using (CETESTContext ctx = new())
        {
            foreach (string statement in sqlDataStatements)
            {
                try
                {
                    ctx.Database.ExecuteSqlRaw(statement);
                }
                catch (Exception ex)
                {
                    File.AppendAllText(@"C:\ProjectsTFS-DOIT\Online CE\Docs\New_TSQL_DB\Test_CE_USER_ANSWERS.txt", "Exception Details: \n ================ \n" + ex.ToString() + "\n ================  \n" + "SQL Statement: \n ================ \n" + statement + "\n ================ \nEnd of Exception and Statement Message \n ================ \n");
                }
            }
        }
    }
</code>```

Here is the entire input string:

```<code>'<div>There are no previously recorded archaeological sites within the project APE. The project area is located within a suburban corridor with commercial and dense residential development. Areas of new right-of-way beyond the existing operational right-of-way have been disturbed by modern development including commercial development, gas stations, driveways and streets, parking areas, underground utilities, sidewalks, and residential development to the northeast and east. No further archaeological investigations are recommended.</div><div><br></div>The project area is largely developed with commercial buildings less than 50 years old. A history/architecture memo to file dated 3/25/ 2021 has been uploaded to Environet {Project File>Cultural Resources>Project Information>Memo-to-File History Architecture]. No history/architecture properties within the APE were recommended as eligible for the NRHP. Based on the scope of the project and review of the project APE, no additional research is recommended for history/architecture properties, and no NRHP-listed or eligible properties will be affected by the project.'
</code>```

Edit 1

Regarding this from Lasse V. Karlsen from the comments:

If I'm not mistaken, whole ExecuteSqlRaw implementation is an anti-pattern because it uses the same syntax and features as string.Format, thus promoting SQL concatenation. Try doubling up the brackets, {{ instead of just {, see if this ends up inserting with just 1 {

This does work.  But we have some 4.9 million lines of code with INSERT INTO ... VALUES(), any of the values might have { or } which need to now be {{ or }}.

Would prefer not to alter data although technically this really does insert one { or }.

Is there some way to tell EF to not worry about {} characters?  That would be much easier.

Edit 2

I wasn't crazy about doing a blanket replace of { with {{, } with }} but this works.

Here is our code for our replacements -- takes out unneeded Oracle syntax and now replaces {}:

```<code>        private static string RemoveOracleSyntax(string statement)
    {
        string cleanStatement = statement;
        if (cleanStatement.Contains("TO_DATE"))
        {
            cleanStatement = cleanStatement.Replace("TO_DATE(", "");
            cleanStatement = cleanStatement.Replace(",'YYYY-MM-DD HH24:MI:SS')", "");
        }

        if (cleanStatement.Contains("TO_CLOB"))
        {
            cleanStatement = cleanStatement.Replace("') || TO_CLOB('", "");
            cleanStatement = cleanStatement.Replace("TO_CLOB(", "");
            cleanStatement = cleanStatement.Replace("'))", "') ");
            cleanStatement = cleanStatement.Replace("'),", "', ");
        }

        if (cleanStatement.Contains("{"))
        {
            cleanStatement = cleanStatement.Replace("{", "{{");
        }

        if (cleanStatement.Contains("}"))
        {
            cleanStatement = cleanStatement.Replace("}", "}}");
        }

        return cleanStatement;
    }
</code>```

答:
The input from Lasse V. Karlsen is the answer.

If I'm not mistaken, whole ExecuteSqlRaw implementation is an anti-pattern because it uses the same syntax and features as string.Format, thus promoting SQL concatenation. Try doubling up the brackets, {{ instead of just {, see if this ends up inserting with just 1 {.

Our Lead Developer is wants to use ExecuteSQL, not ExecuteSQLRaw.  It was brought up this is no longer in the API.  This supposedly would not fail on a { or } character.

Body:
This one can't be correct, probably is somehow a mis-configuration of EF Core in our project.

Background: We have to export an Oracle DB from Aqua Data studio to a file.  I have been manually transposing the export from Oracle to SQL Server.  We came up with the idea of putting the manual process into a Visual Studio project.

From the project, certain strings in the VALUES() section of some INSERT INTO "SomeTable" blocks fail.

In all cases the same code copied over to SSMS works with no modification to it.

This unit test does prove that the { character in the string throws an exception.  Removing it from the string, the ExecuteRawSQL() method will now insert the data. I've put Bold / Italics around the {.

We are running the latest SQL Server Express, on local host.  EF Core 6, .Net Core 6.

The entire input string for ANSWER_TEXT is after the Test Method code block.

I've tried escaping the {, etc.  Also putting a space between { and Project.  Not that I thought this would fix it, just to rule it out.

The { causing the issue can't possibly be right.  The same block with the entire string will insert via SSMS.

It's got to be a Visual Studio project or EF Core config issue, something like that.

It's got me stumped -- didn't expect to trim this string down and be able to duplicate this malfunction.

 [TestMethod]
    public void Test_CE_USER_ANSWERS()
    {
        File.Delete(@"C:\ProjectsTFS-DOIT\Online CE\Docs\New_TSQL_DB\Test_CE_USER_ANSWERS.txt");
        List<string> sqlDataStatements = new ();
        string ceUserAnswers = @"INSERT INTO ""CE_TRANSMITTAL_ANSWERS""(""FORM_ID"", ""TRANSMITTAL_ID"", ""QUESTION_ID"", ""ANSWER_ORDER"", ""ANSWER_TEXT"")
            VALUES(9804, 1, 808, 0, 'A history/architecture memo to file dated 3/25/ 2021 has been uploaded to Environet ***{*** Project File>Cultural Resources>Project ')
            ";
        sqlDataStatements.Add(ceUserAnswers);
        using (CETESTContext ctx = new())
        {
            foreach (string statement in sqlDataStatements)
            {
                try
                {
                    ctx.Database.ExecuteSqlRaw(statement);
                }
                catch (Exception ex)
                {
                    File.AppendAllText(@"C:\ProjectsTFS-DOIT\Online CE\Docs\New_TSQL_DB\Test_CE_USER_ANSWERS.txt", "Exception Details: \n ================ \n" + ex.ToString() + "\n ================  \n" + "SQL Statement: \n ================ \n" + statement + "\n ================ \nEnd of Exception and Statement Message \n ================ \n");
                }
            }
        }
    }

Here is the entire input string:

'
There are no previously recorded archaeological sites within the project APE. The project area is located within a suburban corridor with commercial and dense residential development. Areas of new right-of-way beyond the existing operational right-of-way have been disturbed by modern development including commercial development, gas stations, driveways and streets, parking areas, underground utilities, sidewalks, and residential development to the northeast and east. No further archaeological investigations are recommended.

The project area is largely developed with commercial buildings less than 50 years old. A history/architecture memo to file dated 3/25/ 2021 has been uploaded to Environet {Project File>Cultural Resources>Project Information>Memo-to-File History Architecture]. No history/architecture properties within the APE were recommended as eligible for the NRHP. Based on the scope of the project and review of the project APE, no additional research is recommended for history/architecture properties, and no NRHP-listed or eligible properties will be affected by the project.'

Edit 1

Regarding this from Lasse V. Karlsen from the comments:

If I'm not mistaken, whole ExecuteSqlRaw implementation is an anti-pattern because it uses the same syntax and features as string.Format, thus promoting SQL concatenation. Try doubling up the brackets, {{ instead of just {, see if this ends up inserting with just 1 {

This does work.  But we have some 4.9 million lines of code with INSERT INTO ... VALUES(), any of the values might have { or } which need to now be {{ or }}.

Would prefer not to alter data although technically this really does insert one { or }.

Is there some way to tell EF to not worry about {} characters?  That would be much easier.

Edit 2

I wasn't crazy about doing a blanket replace of { with {{, } with }} but this works.

Here is our code for our replacements -- takes out unneeded Oracle syntax and now replaces {}:

        private static string RemoveOracleSyntax(string statement)
    {
        string cleanStatement = statement;
        if (cleanStatement.Contains("TO_DATE"))
        {
            cleanStatement = cleanStatement.Replace("TO_DATE(", "");
            cleanStatement = cleanStatement.Replace(",'YYYY-MM-DD HH24:MI:SS')", "");
        }

        if (cleanStatement.Contains("TO_CLOB"))
        {
            cleanStatement = cleanStatement.Replace("') || TO_CLOB('", "");
            cleanStatement = cleanStatement.Replace("TO_CLOB(", "");
            cleanStatement = cleanStatement.Replace("'))", "') ");
            cleanStatement = cleanStatement.Replace("'),", "', ");
        }

        if (cleanStatement.Contains("{"))
        {
            cleanStatement = cleanStatement.Replace("{", "{{");
        }

        if (cleanStatement.Contains("}"))
        {
            cleanStatement = cleanStatement.Replace("}", "}}");
        }

        return cleanStatement;
    }


问:
How to convert date from character to datetime in R to insert into Oracle
I've got a problem with dates when I insert dates from R into Oracle using ROracle package.

I have a simple dataframe

```<code>dt = c('2022-01-01', '2005-04-01', '2011-10-02')
var = c('sgdsg', 'hjhgjg', 'rurtur')
num = c(165, 1658.5, 8978.12354)

data = data.frame(dt, var, num)%>%
  mutate(dt = as.Date(dt))
</code>```

I need to insert this dataframe inro Oracle database using the following code

```<code>data %>%
  dbWriteTable(
    oracle_con,
    value = .,
    'D_TEST',
    append = T,
    row.names=F,
    overwrite = F
  )
</code>```

where D_TEST is a test table in my database.

Hovewer I've got the following results

```<code>DT                  VAR     NUM
2022-01-01 03:00:00 sgdsg   165
2005-04-01 03:00:00 hjhgjg  1 658.5
2011-10-02 03:00:00 rurtur  8 978.12354
</code>```

As you can see there is an additional time component. I need to get the 00:00:00 insted.
How can I fix the problem?

P.S.
I tried to change the timezone

```<code>Sys.setenv(TZ = "Europe/Moscow")
Sys.setenv(ORA_SDTZ = "Europe/Moscow")
</code>```

But I didn't help.

答:
Basically what the function dbWriteTable is doing is to convert your Date variable back to a character

```<code>> as.character(data$dt)
[1] "2022-01-01" "2005-04-01" "2011-10-02"
</code>```

and taking this string in the insert statement (simplified) as follows

```<code>insert into  tab(col) values ( '2022-01-01');
</code>```

So you can see this will work allways fine if your column is VARCHAR2 (and you convert to DATE explicitely later).

If your column is of datatype DATE or some TIMESTAMP you performs an implicite conversion which is allways potentialy bad as you may fail when the session mask is other than the data string or you observe some default timezone shift (as you did).

So my recomendation is keep the column VARCHAR2and cast to DATE explicitely.

Body:
I've got a problem with dates when I insert dates from R into Oracle using ROracle package.

I have a simple dataframe

dt = c('2022-01-01', '2005-04-01', '2011-10-02')
var = c('sgdsg', 'hjhgjg', 'rurtur')
num = c(165, 1658.5, 8978.12354)

data = data.frame(dt, var, num)%>%
  mutate(dt = as.Date(dt))

I need to insert this dataframe inro Oracle database using the following code

data %>%
  dbWriteTable(
    oracle_con,
    value = .,
    'D_TEST',
    append = T,
    row.names=F,
    overwrite = F
  )

where D_TEST is a test table in my database.

Hovewer I've got the following results

DT                  VAR     NUM
2022-01-01 03:00:00 sgdsg   165
2005-04-01 03:00:00 hjhgjg  1 658.5
2011-10-02 03:00:00 rurtur  8 978.12354

As you can see there is an additional time component. I need to get the 00:00:00 insted.
How can I fix the problem?

P.S.
I tried to change the timezone

Sys.setenv(TZ = "Europe/Moscow")
Sys.setenv(ORA_SDTZ = "Europe/Moscow")

But I didn't help.

Body:
Basically what the function dbWriteTable is doing is to convert your Date variable back to a character

> as.character(data$dt)
[1] "2022-01-01" "2005-04-01" "2011-10-02"

and taking this string in the insert statement (simplified) as follows

insert into  tab(col) values ( '2022-01-01');

So you can see this will work allways fine if your column is VARCHAR2 (and you convert to DATE explicitely later).

If your column is of datatype DATE or some TIMESTAMP you performs an implicite conversion which is allways potentialy bad as you may fail when the session mask is other than the data string or you observe some default timezone shift (as you did).

So my recomendation is keep the column VARCHAR2and cast to DATE explicitely.


问:
start an application included in the same solution
I have a little C# project and I want to start an application with endings(bad english) like executable.exe -o 99. But I want the "executable.exe" being within my C# project. So I copied the target executable into the project workspace.
The compiled executable now should contain the whole other executable in its memory.
The main application now uses the other application as utility while runtime.
This may have installer program like characteristics, but the utility app is just used temporary.
To build the structure for achieving this functionality would take enough effort.

So my question is: How to provide already compiled and runnable utility application inside of another main application?

Somehow like an assert.

答:
If the executable also includes a project with compiled libraries you can add it as a reference in your project and use its public classes.

If not you may want to do something like:

```<code>using System;
using System.Diagnostics;

...

ProcessStartInfo startInfo = new ProcessStartInfo(@"C:\path\to\application\executable.exe");
startInfo.Arguments = "-o 99";
Proccess.Start(startInfo);
</code>```

Not sure if this is what you need but its the only thing that comes to mind.

Body:
If the executable also includes a project with compiled libraries you can add it as a reference in your project and use its public classes.

If not you may want to do something like:

using System;
using System.Diagnostics;

...

ProcessStartInfo startInfo = new ProcessStartInfo(@"C:\path\to\application\executable.exe");
startInfo.Arguments = "-o 99";
Proccess.Start(startInfo);

Not sure if this is what you need but its the only thing that comes to mind.


Title: How to generate combinations of elements of different lists?

Body:
I have a list of lists that goes like

[[1,2,3],[3,4,5],[5,6,7,8]]

I want to create a program/function that creates a combinations like

1,3,5
1,3,6
1,3,7
1,3,8
1,4,5
1,4,6
.
.
.

If there's a way you can do it without using the itertools module, that would be even more appreciated.

Body:
This is a roughly equivalent implementation of itertools.product() from documentation in case you want/need to build the function without using a library.

def product(*args, repeat=1):
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = [x+[y] for x in result for y in pool]
    for prod in result:
        yield tuple(prod)

a = [[1,2,3],[3,4,5],[5,6,7,8]]

print(list(product(*a)))

Output:

[(1, 3, 5), (1, 3, 6), (1, 3, 7), (1, 3, 8), (1, 4, 5), (1, 4, 6), (1, 4, 7), (1, 4, 8), (1, 5, 5), (1, 5, 6), (1, 5, 7), (1, 5, 8), (2, 3, 5), (2, 3, 6), (2, 3, 7), (2, 3, 8), (2, 4, 5), (2, 4, 6), (2, 4, 7), (2, 4, 8), (2, 5, 5), (2, 5, 6), (2, 5, 7), (2, 5, 8), (3, 3, 5), (3, 3, 6), (3, 3, 7), (3, 3, 8), (3, 4, 5), (3, 4, 6), (3, 4, 7), (3, 4, 8), (3, 5, 5), (3, 5, 6), (3, 5, 7), (3, 5, 8)]

To print as you wish:

results = list(product(*a))

print('\n'.join([','.join(list(map(str, res))) for res in results]))

Output:

1,3,5
1,3,6
1,3,7
1,3,8
1,4,5
1,4,6
1,4,7
1,4,8
1,5,5
1,5,6
1,5,7
1,5,8
2,3,5
2,3,6
2,3,7
2,3,8
2,4,5
2,4,6
2,4,7
2,4,8
2,5,5
2,5,6
2,5,7
2,5,8
3,3,5
3,3,6
3,3,7
3,3,8
3,4,5
3,4,6
3,4,7
3,4,8
3,5,5
3,5,6
3,5,7
3,5,8


问:
Fast way to do word co_occurrence count in python on big corpus
I am trying to make a co_occurrence count work fast in python.

I have a list of context windows. Which is a list of words in some way connected by a dependency label after I parsed the corpus with a transition parser. Each context window has in average length 3. So a context window is not simply the 10 words before and after the focus word. That is why I need a list of lists. And why I can't just use a list of words.

I also have a dictionary where every distinct word of the corpus has a distinct index as value. But I don't think that using that speeds it up a lot. Right now I am only working with a small test corpus with 50,000 words and about 300,000 context windows my code is fast enough for that but I need it to be a lot faster because I need it to work on a much larger corpus in the end.

```<code>class CoOc:
    def __init__(self):
        self.co_occurrence = Counter()

    def add_count(self, token_index, context, token_to_index):
        try:
            # some words are not in the token dictionary if they dont appear frequent enough
            context_index = token_to_index[context]
        except KeyError:
            context_index = -1
        if context_index != -1:
            self.co_occurrence[(token_index, context_index)] += 1

    def count_co_occurrence(self, context_chains, token_index_dic):
        token_index_list = []
        for key in token_index_dic:
            token_index_list.append([key, token_index_dic[key]])
        time1 = current_milli_time()
        for token_key in token_index_list[0:100]:
            token = token_key[0]
            token_index = token_key[1]
            for context_chain in context_chains:
                if token in context_chain:
                    for context_word in context_chain:
                        if token != context_word:
                            self.add_count(token_index, context_word, token_index_dic)
        time2 = current_milli_time()
        print(time2-time1)
</code>```

So in the count function from all the tokens, which are stored in the token_index dictionary, I make a list containing only a small section of it to check the time it takes.

I tried to loop through the list of all the context windows to create a list of context windows only consisting of indices before I loop through the tokens so I would not even need the add count function and just replace the line where I call add_count with the last line of add_count but that did not speed up the process.

Like this it takes 3 seconds to run, for only 100 focus words. I need it to work a lot faster to be working on the corpus I want to use it on. Is that even possible in python or do I need to implement it in another language?

So the first 10 context windows look like this:

```<code>['Gerücht', 'drücken', 'Kurs']
['Konkursgerüchte', 'drücken', 'Kurs']
['Kurs', 'Aktie']
['Kurs', 'Amazon-Aktie']
['der', 'Amazon-Aktie']
['der', 'Aktie']
['Begleitet', 'Gerücht', 'Konkurs']
['Begleitet', 'Marktgerüchten', 'Konkurs']
['begleiten', 'Gerücht', 'Konkurs']
['begleiten', 'setzt fort', 'Aktie', 'Talfahrt']
</code>```

Each context word appears in the token_index_dic with some index unless it doesn't appear frequent enough in the corpus.

The first 10 elements of the token_index_list look like this:

```<code>['Browser-gestützte', 0]
['Akzeptanzstellen', 1]
['Nachschubplanung', 2]
['persönlichen', 3]
['Unionsfraktion', 4]
['Wired', 21122]
['Hauptfigur', 6]
['Strafgesetz', 7]
['Computer-Hersteller', 8]
['Rückschläge', 9]
</code>```

and then when I print self.co_occurrence it looks like this:

```<code># (focus_word_index, context_word_index): count
Counter({(1, 9479): 11, (1, 20316): 7, (2, 1722): 7, (2, 20217): 7, (2, 19842): 7, (2, 2934): 7, (3, 11959): 7, (3, 2404): 7, (3, 1105): 7, (4, 12047): 7, (4, 19262): 7, (0, 13585): 4, (1, 18525): 4, (1, 1538): 4, (1, 9230): 4, (1, 20606): 4, (1, 1486): 4, (2, 13166): 4, (2, 6948): 4, (2, 23028): 4, (2, 14051): 4, (3, 3854): 4, (3, 7908): 4, (3, 4902): 4, (3, 13222): 4, (4, 23737): 4, (4, 6785): 4, (4, 18718): 4, (5, 15424): 4, (5, 4394): 4, (5, 21534): 4, (5, 5829): 4, (5, 6513): 4, (6, 23331): 4, (6, 7234): 4, (6, 20606): 4, (6, 22951): 4, (6, 7318): 4, (6, 15332): 4, (9, 21183): 4, (9, 23028): 4, (9, 1572): 4, (1, 25031): 3, (1, 5829): 3, (1, 14458): 3, (3, 14387): 3, (3, 9574): 3, (3, 21061): 3, (4, 18143): 3, (4, 3306): 3, (4, 17798): 3, (4, 2250): 3, (5, 9982): 3, (5, 5999): 3, (6, 15727): 3, (0, 16008): 2, (0, 1304): 2, (0, 5210): 2, (0, 17798): 2, (1, 20000): 2, (1, 19326): 2, (1, 3820): 2, (1, 25173): 2, (1, 21843): 2, (2, 20662): 2, (3, 7521): 2, (3, 14479): 2, (3, 8109): 2, (3, 18311): 2, (4, 2556): 2, (5, 23940): 2, (5, 1823): 2, (5, 18733): 2, (6, 3131): 2, (6, 947): 2, (6, 18540): 2, (6, 4756): 2, (6, 2743): 2, (6, 7692): 2, (6, 20263): 2, (6, 8670): 2, (6, 2674): 2, (6, 20050): 2, (6, 13274): 2, (6, 17876): 2, (6, 7538): 2, (6, 11098): 2, (6, 4296): 2, (6, 2417): 2, (6, 21421): 2, (6, 19256): 2, (6, 16739): 2, (7, 10908): 2, (7, 4439): 2, (7, 9492): 2, (8, 7027): 2, (8, 599): 2, (8, 4439): 2, (9, 16030): 2, (9, 6856): 2, (9, 24320): 2, (9, 15978): 2, (1, 6454): 1, (1, 14482): 1, (1, 2643): 1, (1, 7338): 1, (2, 21061): 1, (2, 1486): 1, (4, 4296): 1, (4, 23940): 1, (4, 5775): 1, (5, 24133): 1, (5, 2743): 1, (5, 11472): 1, (5, 19336): 1, (5, 20606): 1, (5, 2740): 1, (5, 9479): 1, (5, 14200): 1, (6, 22175): 1, (6, 13104): 1, (6, 10435): 1, (6, 1891): 1, (6, 22353): 1, (6, 4852): 1, (6, 20943): 1, (6, 23965): 1, (6, 13494): 1, (7, 1300): 1, (7, 12497): 1, (7, 2788): 1, (8, 13879): 1, (8, 2404): 1})
</code>```

答:
First let's define a decorator so that we can time the whole function :

```<code>import time

def measured_duration(func):  # decorator to return the call duration and the result
    def wrapper(*args, **kwargs):
        time_before = time.time()
        result = func(*args, **kwargs)
        time_after = time.time()
        return (time_after - time_before) * 1000, result
    return wrapper
</code>```

which can be used like that :

```<code>import os
duration, result = measured_duration(os.listdir)(".")
print(duration, result)
# 1.5974044799804688e-05 ['venv', 'test.csv', '__pycache__', ...]
</code>```

Then let's define some types (it does not speed things up, but it helped me understand the problem) :

```<code>from collections import Counter
from typing import Tuple, Dict, Sequence

ContextWindow = Tuple[str, ...]
TokensIndexes = Dict[str, int]
Cooccurrences = Counter[Tuple[int, int], int]
#                 token index ^      ^ index in context window

def count_cooccurrences(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    ...
</code>```

(yes I could have used NewType, but did not wanted to)

So we can use it like that :

```<code>def main():
    # the example data
    windows = [
        ['Gerücht', 'drücken', 'Kurs'],
        ['Konkursgerüchte', 'drücken', 'Kurs'],
        ['Kurs', 'Aktie'],
        ['Kurs', 'Amazon-Aktie'],
        ['der', 'Amazon-Aktie'],
        ['der', 'Aktie', 'Begleitet'],  # <------ changed
        ['Begleitet', 'Gerücht', 'Konkurs'],
        ['Begleitet', 'Marktgerüchten', 'Konkurs'],
        ['begleiten', 'Gerücht', 'Konkurs'],
        ['begleiten', 'setzt fort', 'Aktie', 'Talfahrt'],
    ]
    token_index_list = {
        'Begleitet': 0,  # <------- modified
        # 'Browser-gestützte': 0,  # -----
        'Akzeptanzstellen': 1,
        'Nachschubplanung': 2,
        'persönlichen': 3,
        'Unionsfraktion': 4,
        'Wired': 21122,
        'Hauptfigur': 6,
        'Strafgesetz': 7,
        'Computer-Hersteller': 8,
        'Rückschläge': 9,
    }

    duration, cooccurrences = measured_duration(count_cooccurrences)(windows, token_index_list)
    print(duration, cooccurrences)

main()
</code>```

I changed a bit the data your provided so that there is some matches.

Now for the function :

```<code>def count_cooccurrences(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    return Counter(
        (token_index, context_index)
        for (token, token_index), window_tokens in itertools.product(
            index_by_token.items(), windows
        )
        for context_index in (
            index_in_window
            for index_in_window, window_token in enumerate(window_tokens)
            if token == window_token
        )
        if context_index != -1
    )
</code>```
```<code>5.269050598144531e-05 Counter({(0, 0): 2, (0, 2): 1})
# on my computer, I always get ~0.055 ms
</code>```

I made it directly return a Counter object, which gets fed an iterable for initialisation, which contains pairs of (token_index_in_corpus, token_index_in_context_window).
Sadly, we want to count only the matches and there is no find method on tuple/list (it exists for str only) so that we have a whole for context_index loop that is meant to iterate usually 0 times, sometime 1 time. But it supports having several times the token in the context window, if this were ever the case.

Please let me know if this run fast enough for you. Next step would be to make it parallel using several processes (because it is a CPU-bound task).

EDIT

For multiprocessing your data, first let's define a helper function :

from itertools import zip_longest
################################################################################
# copied from https://docs.python.org/3/library/itertools.html#itertools-recipes
def grouper(iterable, n, *, incomplete='fill', fillvalue=None):
    "Collect data into non-overlapping fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, fillvalue='x') --> ABC DEF Gxx
    # grouper('ABCDEFG', 3, incomplete='strict') --> ABC DEF ValueError
    # grouper('ABCDEFG', 3, incomplete='ignore') --> ABC DEF
    args = [iter(iterable)] * n
    if incomplete == 'ignore':
        return zip(*args)
    if incomplete == 'fill':
        return zip_longest(*args, fillvalue=fillvalue)
    if incomplete == 'strict':
        return zip(*args, strict=True)
    else:
        raise ValueError('Expected fill, strict, or ignore')
################################################################################
```
<p>Then the parallel implementation :</p>
<pre class="lang-py prettyprint-override"><code>import concurrent.futures
from functools import reduce

WINDOW_CHUNK_SIZE = 5
CORPUS_TOKENS_CHUNK_SIZE = 5
# means that each task will consist of searching for CORPUS_TOKENS_CHUNK_SIZE tokens in WINDOW_CHUNK_SIZE windows
# you can fiddle with these to find an optimum, here I used 5 and 5 for demo purposes

def unpack_count_cooccurrences(args):
    # cf https://stackoverflow.com/q/60002957/11384184
    # the ProcessPoolExecutor.map make the target function be called with a tuple of its args, so we unpack it
    return count_cooccurrences(args[0], args[1])

def count_cooccurrences_parallel(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    with concurrent.futures.ProcessPoolExecutor(max_workers=None) as executor:
        all_params = (  # generator
            (windows_chunk, dict(index_by_token__chunk))
            for windows_chunk, index_by_token__chunk in itertools.product(
                grouper(windows, WINDOW_CHUNK_SIZE, incomplete="ignore"),
                grouper(index_by_token.items(), CORPUS_TOKENS_CHUNK_SIZE, incomplete="ignore"),
            )
        )
        tasks = (executor.submit(unpack_count_cooccurrences, params) for params in all_params)  # generator
        return reduce(lambda counter, future_counter: counter + future_counter.result(),  # could not use operator.add
                      concurrent.futures.as_completed(tasks),  # to reduce progressively
                      Counter())  # starting from an initial value
</code>```

which can be called similarly to the first one :

    duration, cooccurrences = measured_duration(count_cooccurrences_parallel)(windows, token_index_list)
    print(duration, cooccurrences)
```
```13.42153549194336 Counter({(0, 0): 2, (0, 2): 1})
```

The result is much slower (13ms vs 0.05) because of the cost overhead. For a dataset this small, multiprocessing is really overkill. But for your larger dataset, it should provide good performances.

Again, let me know of the time it took to run.

Body:
I am trying to make a co_occurrence count work fast in python.

I have a list of context windows. Which is a list of words in some way connected by a dependency label after I parsed the corpus with a transition parser. Each context window has in average length 3. So a context window is not simply the 10 words before and after the focus word. That is why I need a list of lists. And why I can't just use a list of words.

I also have a dictionary where every distinct word of the corpus has a distinct index as value. But I don't think that using that speeds it up a lot. Right now I am only working with a small test corpus with 50,000 words and about 300,000 context windows my code is fast enough for that but I need it to be a lot faster because I need it to work on a much larger corpus in the end.

class CoOc:
    def __init__(self):
        self.co_occurrence = Counter()

    def add_count(self, token_index, context, token_to_index):
        try:
            # some words are not in the token dictionary if they dont appear frequent enough
            context_index = token_to_index[context]
        except KeyError:
            context_index = -1
        if context_index != -1:
            self.co_occurrence[(token_index, context_index)] += 1

    def count_co_occurrence(self, context_chains, token_index_dic):
        token_index_list = []
        for key in token_index_dic:
            token_index_list.append([key, token_index_dic[key]])
        time1 = current_milli_time()
        for token_key in token_index_list[0:100]:
            token = token_key[0]
            token_index = token_key[1]
            for context_chain in context_chains:
                if token in context_chain:
                    for context_word in context_chain:
                        if token != context_word:
                            self.add_count(token_index, context_word, token_index_dic)
        time2 = current_milli_time()
        print(time2-time1)

So in the count function from all the tokens, which are stored in the token_index dictionary, I make a list containing only a small section of it to check the time it takes.

I tried to loop through the list of all the context windows to create a list of context windows only consisting of indices before I loop through the tokens so I would not even need the add count function and just replace the line where I call add_count with the last line of add_count but that did not speed up the process.

Like this it takes 3 seconds to run, for only 100 focus words. I need it to work a lot faster to be working on the corpus I want to use it on. Is that even possible in python or do I need to implement it in another language?

So the first 10 context windows look like this:

['Gerücht', 'drücken', 'Kurs']
['Konkursgerüchte', 'drücken', 'Kurs']
['Kurs', 'Aktie']
['Kurs', 'Amazon-Aktie']
['der', 'Amazon-Aktie']
['der', 'Aktie']
['Begleitet', 'Gerücht', 'Konkurs']
['Begleitet', 'Marktgerüchten', 'Konkurs']
['begleiten', 'Gerücht', 'Konkurs']
['begleiten', 'setzt fort', 'Aktie', 'Talfahrt']

Each context word appears in the token_index_dic with some index unless it doesn't appear frequent enough in the corpus.

The first 10 elements of the token_index_list look like this:

['Browser-gestützte', 0]
['Akzeptanzstellen', 1]
['Nachschubplanung', 2]
['persönlichen', 3]
['Unionsfraktion', 4]
['Wired', 21122]
['Hauptfigur', 6]
['Strafgesetz', 7]
['Computer-Hersteller', 8]
['Rückschläge', 9]

and then when I print self.co_occurrence it looks like this:

# (focus_word_index, context_word_index): count
Counter({(1, 9479): 11, (1, 20316): 7, (2, 1722): 7, (2, 20217): 7, (2, 19842): 7, (2, 2934): 7, (3, 11959): 7, (3, 2404): 7, (3, 1105): 7, (4, 12047): 7, (4, 19262): 7, (0, 13585): 4, (1, 18525): 4, (1, 1538): 4, (1, 9230): 4, (1, 20606): 4, (1, 1486): 4, (2, 13166): 4, (2, 6948): 4, (2, 23028): 4, (2, 14051): 4, (3, 3854): 4, (3, 7908): 4, (3, 4902): 4, (3, 13222): 4, (4, 23737): 4, (4, 6785): 4, (4, 18718): 4, (5, 15424): 4, (5, 4394): 4, (5, 21534): 4, (5, 5829): 4, (5, 6513): 4, (6, 23331): 4, (6, 7234): 4, (6, 20606): 4, (6, 22951): 4, (6, 7318): 4, (6, 15332): 4, (9, 21183): 4, (9, 23028): 4, (9, 1572): 4, (1, 25031): 3, (1, 5829): 3, (1, 14458): 3, (3, 14387): 3, (3, 9574): 3, (3, 21061): 3, (4, 18143): 3, (4, 3306): 3, (4, 17798): 3, (4, 2250): 3, (5, 9982): 3, (5, 5999): 3, (6, 15727): 3, (0, 16008): 2, (0, 1304): 2, (0, 5210): 2, (0, 17798): 2, (1, 20000): 2, (1, 19326): 2, (1, 3820): 2, (1, 25173): 2, (1, 21843): 2, (2, 20662): 2, (3, 7521): 2, (3, 14479): 2, (3, 8109): 2, (3, 18311): 2, (4, 2556): 2, (5, 23940): 2, (5, 1823): 2, (5, 18733): 2, (6, 3131): 2, (6, 947): 2, (6, 18540): 2, (6, 4756): 2, (6, 2743): 2, (6, 7692): 2, (6, 20263): 2, (6, 8670): 2, (6, 2674): 2, (6, 20050): 2, (6, 13274): 2, (6, 17876): 2, (6, 7538): 2, (6, 11098): 2, (6, 4296): 2, (6, 2417): 2, (6, 21421): 2, (6, 19256): 2, (6, 16739): 2, (7, 10908): 2, (7, 4439): 2, (7, 9492): 2, (8, 7027): 2, (8, 599): 2, (8, 4439): 2, (9, 16030): 2, (9, 6856): 2, (9, 24320): 2, (9, 15978): 2, (1, 6454): 1, (1, 14482): 1, (1, 2643): 1, (1, 7338): 1, (2, 21061): 1, (2, 1486): 1, (4, 4296): 1, (4, 23940): 1, (4, 5775): 1, (5, 24133): 1, (5, 2743): 1, (5, 11472): 1, (5, 19336): 1, (5, 20606): 1, (5, 2740): 1, (5, 9479): 1, (5, 14200): 1, (6, 22175): 1, (6, 13104): 1, (6, 10435): 1, (6, 1891): 1, (6, 22353): 1, (6, 4852): 1, (6, 20943): 1, (6, 23965): 1, (6, 13494): 1, (7, 1300): 1, (7, 12497): 1, (7, 2788): 1, (8, 13879): 1, (8, 2404): 1})

Body:
First let's define a decorator so that we can time the whole function :

import time

def measured_duration(func):  # decorator to return the call duration and the result
    def wrapper(*args, **kwargs):
        time_before = time.time()
        result = func(*args, **kwargs)
        time_after = time.time()
        return (time_after - time_before) * 1000, result
    return wrapper

which can be used like that :

import os
duration, result = measured_duration(os.listdir)(".")
print(duration, result)
# 1.5974044799804688e-05 ['venv', 'test.csv', '__pycache__', ...]

Then let's define some types (it does not speed things up, but it helped me understand the problem) :

from collections import Counter
from typing import Tuple, Dict, Sequence

ContextWindow = Tuple[str, ...]
TokensIndexes = Dict[str, int]
Cooccurrences = Counter[Tuple[int, int], int]
#                 token index ^      ^ index in context window

def count_cooccurrences(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    ...

(yes I could have used NewType, but did not wanted to)

So we can use it like that :

def main():
    # the example data
    windows = [
        ['Gerücht', 'drücken', 'Kurs'],
        ['Konkursgerüchte', 'drücken', 'Kurs'],
        ['Kurs', 'Aktie'],
        ['Kurs', 'Amazon-Aktie'],
        ['der', 'Amazon-Aktie'],
        ['der', 'Aktie', 'Begleitet'],  # <------ changed
        ['Begleitet', 'Gerücht', 'Konkurs'],
        ['Begleitet', 'Marktgerüchten', 'Konkurs'],
        ['begleiten', 'Gerücht', 'Konkurs'],
        ['begleiten', 'setzt fort', 'Aktie', 'Talfahrt'],
    ]
    token_index_list = {
        'Begleitet': 0,  # <------- modified
        # 'Browser-gestützte': 0,  # -----
        'Akzeptanzstellen': 1,
        'Nachschubplanung': 2,
        'persönlichen': 3,
        'Unionsfraktion': 4,
        'Wired': 21122,
        'Hauptfigur': 6,
        'Strafgesetz': 7,
        'Computer-Hersteller': 8,
        'Rückschläge': 9,
    }

    duration, cooccurrences = measured_duration(count_cooccurrences)(windows, token_index_list)
    print(duration, cooccurrences)

main()

I changed a bit the data your provided so that there is some matches.

Now for the function :

def count_cooccurrences(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    return Counter(
        (token_index, context_index)
        for (token, token_index), window_tokens in itertools.product(
            index_by_token.items(), windows
        )
        for context_index in (
            index_in_window
            for index_in_window, window_token in enumerate(window_tokens)
            if token == window_token
        )
        if context_index != -1
    )

5.269050598144531e-05 Counter({(0, 0): 2, (0, 2): 1})
# on my computer, I always get ~0.055 ms

I made it directly return a Counter object, which gets fed an iterable for initialisation, which contains pairs of (token_index_in_corpus, token_index_in_context_window).
Sadly, we want to count only the matches and there is no find method on tuple/list (it exists for str only) so that we have a whole for context_index loop that is meant to iterate usually 0 times, sometime 1 time. But it supports having several times the token in the context window, if this were ever the case.

Please let me know if this run fast enough for you. Next step would be to make it parallel using several processes (because it is a CPU-bound task).

EDIT

For multiprocessing your data, first let's define a helper function :

from itertools import zip_longest
################################################################################
# copied from https://docs.python.org/3/library/itertools.html#itertools-recipes
def grouper(iterable, n, *, incomplete='fill', fillvalue=None):
    "Collect data into non-overlapping fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, fillvalue='x') --> ABC DEF Gxx
    # grouper('ABCDEFG', 3, incomplete='strict') --> ABC DEF ValueError
    # grouper('ABCDEFG', 3, incomplete='ignore') --> ABC DEF
    args = [iter(iterable)] * n
    if incomplete == 'ignore':
        return zip(*args)
    if incomplete == 'fill':
        return zip_longest(*args, fillvalue=fillvalue)
    if incomplete == 'strict':
        return zip(*args, strict=True)
    else:
        raise ValueError('Expected fill, strict, or ignore')
################################################################################

Then the parallel implementation :

import concurrent.futures
from functools import reduce

WINDOW_CHUNK_SIZE = 5
CORPUS_TOKENS_CHUNK_SIZE = 5
# means that each task will consist of searching for CORPUS_TOKENS_CHUNK_SIZE tokens in WINDOW_CHUNK_SIZE windows
# you can fiddle with these to find an optimum, here I used 5 and 5 for demo purposes

def unpack_count_cooccurrences(args):
    # cf https://stackoverflow.com/q/60002957/11384184
    # the ProcessPoolExecutor.map make the target function be called with a tuple of its args, so we unpack it
    return count_cooccurrences(args[0], args[1])

def count_cooccurrences_parallel(windows: Sequence[ContextWindow], index_by_token: TokensIndexes) -> Cooccurrences:
    with concurrent.futures.ProcessPoolExecutor(max_workers=None) as executor:
        all_params = (  # generator
            (windows_chunk, dict(index_by_token__chunk))
            for windows_chunk, index_by_token__chunk in itertools.product(
                grouper(windows, WINDOW_CHUNK_SIZE, incomplete="ignore"),
                grouper(index_by_token.items(), CORPUS_TOKENS_CHUNK_SIZE, incomplete="ignore"),
            )
        )
        tasks = (executor.submit(unpack_count_cooccurrences, params) for params in all_params)  # generator
        return reduce(lambda counter, future_counter: counter + future_counter.result(),  # could not use operator.add
                      concurrent.futures.as_completed(tasks),  # to reduce progressively
                      Counter())  # starting from an initial value

which can be called similarly to the first one :

    duration, cooccurrences = measured_duration(count_cooccurrences_parallel)(windows, token_index_list)
    print(duration, cooccurrences)

13.42153549194336 Counter({(0, 0): 2, (0, 2): 1})

The result is much slower (13ms vs 0.05) because of the cost overhead. For a dataset this small, multiprocessing is really overkill. But for your larger dataset, it should provide good performances.

Again, let me know of the time it took to run.


问:
What is the use of PreferredSize widget in flutter?
I'm new in flutter, and I see there are few widgets for the layout design such as SizedBox and Container.
There is one widget which is PreferredSize Widget that I don't know and cannot find any example about it.
What makes it different from other widgets such as container and SizedBox which can set height and width?.

Can someone give an example?

答:
If you want to create your own custom prefered size widget all you need to do is to implement PreferredSizeWidget interface in your widget.

Example:

```<code>import 'package:flutter/material.dart';

class MyWidget extends StatelessWidget implements PreferredSizeWidget {

  @override
  Size get preferredSize => Size.fromHeight(kToolbarHeight);

  @override
  Widget build(BuildContext context) {
    return Container(); // Your custom widget implementation.
  }

}
</code>```

Body:
If you want to create your own custom prefered size widget all you need to do is to implement PreferredSizeWidget interface in your widget.

Example:

import 'package:flutter/material.dart';

class MyWidget extends StatelessWidget implements PreferredSizeWidget {

  @override
  Size get preferredSize => Size.fromHeight(kToolbarHeight);

  @override
  Widget build(BuildContext context) {
    return Container(); // Your custom widget implementation.
  }

}


问:
What Role or Scopes Does An Azure Service Principal Need to Create Applications
I currently create a service principal using the Azure CLI:

az ad sp create-for-rbac --name foo --role Contributor

I need the service principal to have enough permissions to create/modify/delete various Azure AD resources including Applications, other Service Principals and Service Principal Passwords. When I use the above service principal to create other service principals, I currently get 403 Forbidden errors.

I have also tried using the 'Owner' and 'User Access Administrator' roles but these still give me a 403 error. What do I need to add to the above Azure CLI command or what additional role assignments do I need to add?

I'd like to use the service principal in a Pulumi program with their Azure AD provider (based on Terraform's Azure AD provider). See:

https://github.com/pulumi/pulumi-azuread/issues/246

答:
You need to add the scope of this service principal and also change the Azure role of this Service Principal to 'User Access Administrator' to enable you to modify resources in Azure AD. Also, 'User Access Administrator' role will give the service principal the required permissions for that Azure role to assign RBAC permissions. Please refer the below command for more details: -

```<code>  az ad sp create-for-rbac --name foo --role User Access Administrator --scopes /subscriptions/{subscriptionId}/resourceGroups/{resourceGroup1}
</code>```

Also, ensure that the user ID through which you are creating this service principal and assigning the role to it has permissions to register and create applications in Azure AD. If not, then please assign that ID 'Application Administrator' Azure AD role or you should be allowed to create and register applications by an administrator even though being a 'User'.

Body:
You need to add the scope of this service principal and also change the Azure role of this Service Principal to 'User Access Administrator' to enable you to modify resources in Azure AD. Also, 'User Access Administrator' role will give the service principal the required permissions for that Azure role to assign RBAC permissions. Please refer the below command for more details: -

  az ad sp create-for-rbac --name foo --role User Access Administrator --scopes /subscriptions/{subscriptionId}/resourceGroups/{resourceGroup1}

Also, ensure that the user ID through which you are creating this service principal and assigning the role to it has permissions to register and create applications in Azure AD. If not, then please assign that ID 'Application Administrator' Azure AD role or you should be allowed to create and register applications by an administrator even though being a 'User'.


问:
Why can't Integer.MAX_VALUE also work in this AP CSA array problem?
The AP CSA problem is: Which replacement(s) for /* some value */ will always result in correct execution of findMin?

```<code>// arr is the array to be processed
public static int findMin (int[] arr)
{
    int min = /* some value */;
    int index = 0;
    while (index < arr.length)
    {
        if (arr[index] < min)
        {
            min = arr[index];
        }
        index++;
    }
    return min;
}
</code>```

The answer choices are:
I. Integer.MAX_VALUE
II. Integer.MIN_VALUE
III. arr[0]

The correct answer is II and III, but I want to know why Integer.MAX_VALUE would not also work?
The if statement would just run right away & min would just be changed a lot more.

答:
choice II is clearly wrong because the function will always return Integer.MIN_VALUE, whatever the content of the input array.

I and III are both possible, but they will differ in the way they deal with an empty input array:

I will return Integer.MAX_VALUE

III will throw an ArrayIndexOutOfBoundsException

I prefer the latter.

Body:
The AP CSA problem is: Which replacement(s) for /* some value */ will always result in correct execution of findMin?

// arr is the array to be processed
public static int findMin (int[] arr)
{
    int min = /* some value */;
    int index = 0;
    while (index < arr.length)
    {
        if (arr[index] < min)
        {
            min = arr[index];
        }
        index++;
    }
    return min;
}

The answer choices are:
I. Integer.MAX_VALUE
II. Integer.MIN_VALUE
III. arr[0]

The correct answer is II and III, but I want to know why Integer.MAX_VALUE would not also work?
The if statement would just run right away & min would just be changed a lot more.