问:
How to find all nodes and their links to other nodes by traversing them in both directions (fathers and children)?
I have a client application consuming data from an external API based on a configuration during startup.

Since calculating the necessary data is not easy for me, I tried to fake the implementation and calculate it locally for now.

There are many WorkItems linked to each other and the configuration only knows about the relations to consider.

Some explanations:

Workitem

A workitem has a unique id and a type

e.g.

```<code>{
    id: 1,
    type: "person"
}
</code>```

so you know that Workitem 1 is of type person

Linkrole

A linkrole is a composite key of three keys

The parrent workitem type

The child workitem type

The name of the role

e.g.

```<code>{
    parentWorkItemType: "person",
    childWorkItemType: "house",
    name: "owns",
}
</code>```

which represents Person owns House

Links

The backend knows about the relations, a link between workitems might look like

```<code>{
    /* each link is unique */
    linkRole: {
        parentWorkItemType: "person",
        childWorkItemType: "house",
        name: "owns",
    },
    parentId: 1,
    childId: 2
}
</code>```

so you know that Person 1 owns House 2

Fake prerequisites:

Fake backend

I created some fake data for my calculations. The file backend.js represents the backend / database

```<code>// backend.js

export const workItems = [{
    id: 1,
    type: "car"
}, {
    id: 2,
    type: "car"
}, {
    id: 3,
    type: "bird"
}, {
    id: 4,
    type: "bird"
}, {
    id: 5,
    type: "bird"
}, {
    id: 6,
    type: "invalid"
}, {
    id: 7,
    type: "house"
}, {
    id: 8,
    type: "house"
}, {
    id: 9,
    type: "house",
}, {
    id: 10,
    type: "person",
}];

export const workItemLinks = [{
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "bird",
        name: "isChildOf",
    },
    parentId: 1,
    childId: 3
}, {
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "bird",
        name: "isChildOf",
    },
    parentId: 1,
    childId: 4
}, {
    linkRole: {
        parentWorkItemType: "bird",
        childWorkItemType: "house",
        name: "with",
    },
    parentId: 3,
    childId: 7
}, {
    linkRole: {
        parentWorkItemType: "bird",
        childWorkItemType: "house",
        name: "with",
    },
    parentId: 3,
    childId: 8
}, {
    linkRole: {
        parentWorkItemType: "person",
        childWorkItemType: "car",
        name: "owns",
    },
    parentId: 10,
    childId: 1
}, {
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "car",
        name: "references",
    },
    parentId: 1,
    childId: 2
}];
</code>```

Fake API

I created some methods inside the file api.js to fake an API asking the backend for data.

Important sidenote: These methods just help me to find the data I need. I can add more "helpers" if needed (In the real world I can ask to add more API endpoints) so everything can be queried. Please feel free to modify the "Fake API".

```<code>// api.js

import { workItems, workItemLinks } from "./backend.js";

export function getWorkItemsByType(workItemType) {
    return workItems.filter(workItem => workItem.type === workItemType);
}

export function getWorkItemsByIds(workItemIds) {
    return workItems.filter(workItem => workItemIds.some(workItemId => workItemId === workItem.id));
}

export function getWorkItemLinksByLinkRoleAndLeftSideIds(linkRole, leftSideIds, leftSideIsParentSide) {
    return workItemLinks.filter(workItemLink => {

        // Pseudo equality check

        if (workItemLink.linkRole.parentWorkItemType === linkRole.parentWorkItemType &&
            workItemLink.linkRole.childWorkItemType === linkRole.childWorkItemType &&
            workItemLink.linkRole.name === linkRole.name) {
            const leftSideIdInLink = leftSideIsParentSide ? workItemLink.parentId : workItemLink.childId;

            // Return this link if it matches with the left side id ( you're looking for the right side id )

            return leftSideIds.some(leftSideId => leftSideId === leftSideIdInLink);
        }

        return false;
    });
}
</code>```

Fake configuration

I created definitions inside configuration.js to start the calculations based on those

```<code>// configuration.js

export const definitions = [
    {                                           // fetch linked birds from every root car
        linkRole: {
            parentWorkItemType: "car",
            childWorkItemType: "bird",
            name: "isChildOf",
        }
    },
    {                                           // !! Root Element !! Fetch work items based on type
        workItemType: "car",
        isRootWorkItem: true
    },
    {                                            // fetch linked houses from every bird ( car - bird - house )
        linkRole: {
            parentWorkItemType: "bird",
            childWorkItemType: "house",
            name: "with",
        }
    },
    {                                             // fetch every person from every car ( person - car )
        linkRole: {
            parentWorkItemType: "person",
            childWorkItemType: "car",
            name: "owns",
        }
    },
    {                                          // Self reference, fetch linked cars from every root car
        linkRole: {
            parentWorkItemType: "car",
            childWorkItemType: "car",
            name: "references",
        }
    },
];
</code>```

Problem to solve:

I am looking for a performant way to fetch every related link and workitem from the backend based on the chain of relations starting with each root item.

Expected output based on the configuration:

```<code>export const workItems = [{
    id: 1,
    type: "car"
}, {
    id: 2,
    type: "car"
}, {
    id: 3,
    type: "bird"
}, {
    id: 4,
    type: "bird"
}, /* no car is linked to bird 5, no links for invalid 6 */ {
    id: 7,
    type: "house"
}, {
    id: 8,
    type: "house"
}, /* no bird is linked to house 9 */ {
    id: 10,
    type: "person",
}];

/* based on the configuration every workItemLink from the backend got fetched */
</code>```

My first implementation approach:

```<code>import { inspect } from "util";

import { getWorkItemsByType, getWorkItemsByIds, getWorkItemLinksByLinkRoleAndLeftSideIds } from "./api.js";
import { definitions } from "./configuration.js";

// Find the root definition

const { workItemType: rootWorkItemType } = definitions.find(definition => definition.isRootWorkItem);

// Fetch all the root workitems and initialize the store holding unique workitems

const workItems = getWorkItemsByType(rootWorkItemType); // will hold car 1 and car 2

const initialWorkItemIds = workItems.map(workItem => workItem.id);

const workItemIdsToLoad = []; // try to read all missing ids from fetched links and fetch all the workitems at once later on

const workItemLinks = []; // global store for links

const definitionsToTraverse = definitions.filter(definition => definition.linkRole !== undefined); // do not consider the root definition

run(initialWorkItemIds, rootWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks);

const missingWorkItemIdsToLoad = workItemIdsToLoad.filter(workItemIdToLoad => !rootWorkItemIds.some(rootWorkItemId => rootWorkItemId === workItemIdToLoad)); // filter out all the existing root item ids

const workItemsToAdd = getWorkItemsByIds(missingWorkItemIdsToLoad); // fetch all the missing workitems at once

workItems.push(...workItemsToAdd); // add them to the global store

console.log(inspect({
    workItems,
    workItemLinks
}, false, null, true))

function run(leftSideWorkItemIds, leftSideWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks) {
    /*

        consider all the defitions from "definitionsToTraverse" directly related to "leftSideWorkItemType"
        remove them from "definitionsToTraverse" to prevent endless loops
        loop backwards so splice won't struggle with the indices

    */

    for (let definitionIndex = definitionsToTraverse.length - 1; definitionIndex >= 0; definitionIndex--) {
        const currentDefinition = definitionsToTraverse[definitionIndex];
        const { linkRole } = currentDefinition;
        const { parentWorkItemType, childWorkItemType } = linkRole;

        const leftSideWorkItemTypeIsParent = parentWorkItemType === leftSideWorkItemType;
        const leftSideWorkItemTypeIsChild = childWorkItemType === leftSideWorkItemType;

        // Check the direct relation

        if (leftSideWorkItemTypeIsParent || leftSideWorkItemTypeIsChild) {
            // Remove the inspected definition

            definitionsToTraverse.splice(definitionIndex, 1);

            const relatedWorkItemLinks = getWorkItemLinksByLinkRoleAndLeftSideIds(linkRole, leftSideWorkItemIds, leftSideWorkItemTypeIsParent);

            // store all the right side workitem IDs for the next run

            const rightSideWorkItemIds = [];

            for (let relatedWorkItemLinkIndex = 0; relatedWorkItemLinkIndex < relatedWorkItemLinks.length; relatedWorkItemLinkIndex++) {
                const relatedWorkItemLink = relatedWorkItemLinks[relatedWorkItemLinkIndex];

                // Process the link if not processed yet

                if (!workItemLinks.some(workItemLink => /* !!! pseudo equality check !!! */
                        workItemLink.linkRole.parentWorkItemType === relatedWorkItemLink.linkRole.parentWorkItemType &&
                        workItemLink.linkRole.childWorkItemType === relatedWorkItemLink.linkRole.childWorkItemType &&
                        workItemLink.linkRole.name === relatedWorkItemLink.linkRole.name &&
                        workItemLink.parentId === relatedWorkItemLink.parentId &&
                        workItemLink.childId === relatedWorkItemLink.childId)) {

                    // Push the link to the store

                    workItemLinks.push(relatedWorkItemLink);

                    // Get the right side id from the link

                    const rightSideWorkItemId = leftSideWorkItemTypeIsParent ? relatedWorkItemLink.childId : relatedWorkItemLink.parentId;

                    rightSideWorkItemIds.push(rightSideWorkItemId);

                    // Push the id to the store if it doesn't exist

                    if (!workItemIdsToLoad.some(workItemIdToLoad => workItemIdToLoad === rightSideWorkItemId)) {
                        workItemIdsToLoad.push(rightSideWorkItemId);
                    }
                }
            }

            // Find the opposite workitem type of the current definition

            const rightSideWorkItemType = leftSideWorkItemTypeIsParent ? childWorkItemType : parentWorkItemType;

            // Run again but use this definition as the previous one

            run(rightSideWorkItemIds, rightSideWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks);
        }
    }
}
</code>```

The problem is that my approach crashes because the backwards loop even runs if definitionsToTraverse is empty. And I'm mutating the arrays from the parameters directly, I think I shouldn't do that.

So any help would be appreciated a lot!

答:
Update: Added a substantially cleaner version of the code.  I was bothered by the previous implementation.

I may be oversimplifying what you need to do here.  I'm treating this as described in my comment on the question:

Do I have this right? You have (here) a single root type of car. and so you want to include both cars, and then you want to include all nodes which have either of those cars as parent or child values in a link, and then include all nodes which have any of those results a a parent or child in a link, and so on recursively, collecting all the results not in a tree but in a simple list.

If that's correct, then this seems like a reasonable approach:

const getLinkedIds = (links, ids, toAdd = links
  .filter (({childId, parentId}) => ids .includes (childId) || ids .includes (parentId))
  .flatMap (({childId, parentId}) => [childId, parentId])
  .filter (id => !ids .includes (id))
) => toAdd .length == 0 ? ids : getLinkedIds (links, ids .concat (toAdd))

const linkedItems = (defs, items, links,
  rootTypes = defs.filter (({isRootWorkItem = false}) => isRootWorkItem)
    .map (({workItemType}) => workItemType),
  roots = items .filter (({type}) => rootTypes .includes (type)),
  ids = getLinkedIds (links, roots.flatMap (({id}) => id))
) => items .filter (({id}) => ids .includes (id))

const definitions = [ /* linkTypes deleted */  {workItemType: "car", isRootWorkItem: true}]
const workItems = [{id: 1, type: "car"}, {id: 2, type: "car"}, {id: 3, type: "bird"}, {id: 4, type: "bird"}, {id: 5, type: "bird"}, {id: 6, type: "invalid"}, {id: 7, type: "house"}, {id: 8, type: "house"}, {id: 9, type: "house"}, {id: 10, type: "person"}]
const workItemLinks = [ /* linkTypes deleted */ {parentId: 1, childId: 3}, {parentId: 1, childId: 4}, {parentId: 3, childId: 7}, {parentId: 3, childId: 8}, {parentId: 10, childId: 1}, {parentId: 1, childId: 2}]

console .log (
  linkedItems (definitions, workItems, workItemLinks)
)```
<pre class="snippet-code-css lang-css prettyprint-override"><code>.as-console-wrapper {max-height: 100% !important; top: 0}</code>```

In linkedItems, we start with the definitions to collect the root types, then use those to collect all the root elements.  We then call our main function, getLinkedIds using links, and the ids of these elements.  Then we filter the items to find those whose ids we chose.

getLinkedIds finds all the links that include any of our ids as either parent or child, and adds both parent and child to a list of ids that will be included, and when its done, filters out any that are already in our list.  If this list of ids to add is empty, we're done and return our current list.  Otherwise we recur on the combination of the two lists.  While there may be some inefficiencies here, I think the lists would have to be quite large for them to matter.

I haven't tried to read your code and understand how it differs from this, but if my guess is correct about your requirements, this is relatively simple.

Old Version

This is not my usual coding style, and it bothers me.  I tend to work with direct recursion, and I usually use immutable data structures.  I would love to see an approach that handles this using that style, but I didn't come up with one right away.

Update: The comment above applied to this older version of the code.  The newer version seems fine to me.

Here is a description of the original implementation of getLinkedIds (the other function has only cosmetic changes):

getLinkedIds keeps two lists of ids: those still to process and those we've already processed, repeatedly removing an id from the first list, adding it to the second and then queueing up all the unprocessed parentIds for which we have a childId link to the current value and then all the unprocessed childIds for which we have a parentId link to the current value.  When the toProcess queue is empty, we return the processed one.

And you can expand this snippet to see that version (which I don't recommend:

const getLinkedIds = (links, ids) => {
  const toProcess = [...ids], processed = []
  while (toProcess .length > 0) {
     const id = toProcess .pop ()
     processed .push (id)
     links .filter (({parentId}) => id == parentId)
       .forEach (({childId}) => {
         if (! processed .includes (childId) && ! toProcess .includes (childId)) {
           toProcess .push (childId)
         }
       })
     links .filter (({childId}) => id == childId)
       .forEach (({parentId}) => {
         if (! processed .includes (parentId) && ! toProcess .includes (parentId)) {
           toProcess .push (parentId)
         }
       })
  }
  return [...processed]
}

const linkedItems = (defs, items, links) => {
  const rootTypes = defs .filter (({isRootWorkItem = false}) => isRootWorkItem)
    .map (({workItemType}) => workItemType)
  const roots = items .filter (({type}) => rootTypes .includes (type))
  const ids = getLinkedIds (links, roots.flatMap (({id}) => id))
  return items .filter (({id}) => ids .includes (id))
}

const definitions = [ /* linkTypes deleted */  {workItemType: "car", isRootWorkItem: true}]
const workItems = [{id: 1, type: "car"}, {id: 2, type: "car"}, {id: 3, type: "bird"}, {id: 4, type: "bird"}, {id: 5, type: "bird"}, {id: 6, type: "invalid"}, {id: 7, type: "house"}, {id: 8, type: "house"}, {id: 9, type: "house"}, {id: 10, type: "person"}]
const workItemLinks = [ /* linkTypes deleted */ {parentId: 1, childId: 3}, {parentId: 1, childId: 4}, {parentId: 3, childId: 7}, {parentId: 3, childId: 8}, {parentId: 10, childId: 1}, {parentId: 1, childId: 2}]

console .log (
  linkedItems (definitions, workItems, workItemLinks)
)```
<pre class="snippet-code-css lang-css prettyprint-override"><code>.as-console-wrapper {max-height: 100% !important; top: 0}</code>```

Body:
I have a client application consuming data from an external API based on a configuration during startup.

Since calculating the necessary data is not easy for me, I tried to fake the implementation and calculate it locally for now.

There are many WorkItems linked to each other and the configuration only knows about the relations to consider.

Some explanations:

Workitem

A workitem has a unique id and a type

e.g.

{
    id: 1,
    type: "person"
}

so you know that Workitem 1 is of type person

Linkrole

A linkrole is a composite key of three keys

The parrent workitem type

The child workitem type

The name of the role

e.g.

{
    parentWorkItemType: "person",
    childWorkItemType: "house",
    name: "owns",
}

which represents Person owns House

Links

The backend knows about the relations, a link between workitems might look like

{
    /* each link is unique */
    linkRole: {
        parentWorkItemType: "person",
        childWorkItemType: "house",
        name: "owns",
    },
    parentId: 1,
    childId: 2
}

so you know that Person 1 owns House 2

Fake prerequisites:

Fake backend

I created some fake data for my calculations. The file backend.js represents the backend / database

// backend.js

export const workItems = [{
    id: 1,
    type: "car"
}, {
    id: 2,
    type: "car"
}, {
    id: 3,
    type: "bird"
}, {
    id: 4,
    type: "bird"
}, {
    id: 5,
    type: "bird"
}, {
    id: 6,
    type: "invalid"
}, {
    id: 7,
    type: "house"
}, {
    id: 8,
    type: "house"
}, {
    id: 9,
    type: "house",
}, {
    id: 10,
    type: "person",
}];

export const workItemLinks = [{
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "bird",
        name: "isChildOf",
    },
    parentId: 1,
    childId: 3
}, {
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "bird",
        name: "isChildOf",
    },
    parentId: 1,
    childId: 4
}, {
    linkRole: {
        parentWorkItemType: "bird",
        childWorkItemType: "house",
        name: "with",
    },
    parentId: 3,
    childId: 7
}, {
    linkRole: {
        parentWorkItemType: "bird",
        childWorkItemType: "house",
        name: "with",
    },
    parentId: 3,
    childId: 8
}, {
    linkRole: {
        parentWorkItemType: "person",
        childWorkItemType: "car",
        name: "owns",
    },
    parentId: 10,
    childId: 1
}, {
    linkRole: {
        parentWorkItemType: "car",
        childWorkItemType: "car",
        name: "references",
    },
    parentId: 1,
    childId: 2
}];

Fake API

I created some methods inside the file api.js to fake an API asking the backend for data.

Important sidenote: These methods just help me to find the data I need. I can add more "helpers" if needed (In the real world I can ask to add more API endpoints) so everything can be queried. Please feel free to modify the "Fake API".

// api.js

import { workItems, workItemLinks } from "./backend.js";

export function getWorkItemsByType(workItemType) {
    return workItems.filter(workItem => workItem.type === workItemType);
}

export function getWorkItemsByIds(workItemIds) {
    return workItems.filter(workItem => workItemIds.some(workItemId => workItemId === workItem.id));
}

export function getWorkItemLinksByLinkRoleAndLeftSideIds(linkRole, leftSideIds, leftSideIsParentSide) {
    return workItemLinks.filter(workItemLink => {

        // Pseudo equality check

        if (workItemLink.linkRole.parentWorkItemType === linkRole.parentWorkItemType &&
            workItemLink.linkRole.childWorkItemType === linkRole.childWorkItemType &&
            workItemLink.linkRole.name === linkRole.name) {
            const leftSideIdInLink = leftSideIsParentSide ? workItemLink.parentId : workItemLink.childId;

            // Return this link if it matches with the left side id ( you're looking for the right side id )

            return leftSideIds.some(leftSideId => leftSideId === leftSideIdInLink);
        }

        return false;
    });
}

Fake configuration

I created definitions inside configuration.js to start the calculations based on those

// configuration.js

export const definitions = [
    {                                           // fetch linked birds from every root car
        linkRole: {
            parentWorkItemType: "car",
            childWorkItemType: "bird",
            name: "isChildOf",
        }
    },
    {                                           // !! Root Element !! Fetch work items based on type
        workItemType: "car",
        isRootWorkItem: true
    },
    {                                            // fetch linked houses from every bird ( car - bird - house )
        linkRole: {
            parentWorkItemType: "bird",
            childWorkItemType: "house",
            name: "with",
        }
    },
    {                                             // fetch every person from every car ( person - car )
        linkRole: {
            parentWorkItemType: "person",
            childWorkItemType: "car",
            name: "owns",
        }
    },
    {                                          // Self reference, fetch linked cars from every root car
        linkRole: {
            parentWorkItemType: "car",
            childWorkItemType: "car",
            name: "references",
        }
    },
];

Problem to solve:

I am looking for a performant way to fetch every related link and workitem from the backend based on the chain of relations starting with each root item.

Expected output based on the configuration:

export const workItems = [{
    id: 1,
    type: "car"
}, {
    id: 2,
    type: "car"
}, {
    id: 3,
    type: "bird"
}, {
    id: 4,
    type: "bird"
}, /* no car is linked to bird 5, no links for invalid 6 */ {
    id: 7,
    type: "house"
}, {
    id: 8,
    type: "house"
}, /* no bird is linked to house 9 */ {
    id: 10,
    type: "person",
}];

/* based on the configuration every workItemLink from the backend got fetched */

My first implementation approach:

import { inspect } from "util";

import { getWorkItemsByType, getWorkItemsByIds, getWorkItemLinksByLinkRoleAndLeftSideIds } from "./api.js";
import { definitions } from "./configuration.js";

// Find the root definition

const { workItemType: rootWorkItemType } = definitions.find(definition => definition.isRootWorkItem);

// Fetch all the root workitems and initialize the store holding unique workitems

const workItems = getWorkItemsByType(rootWorkItemType); // will hold car 1 and car 2

const initialWorkItemIds = workItems.map(workItem => workItem.id);

const workItemIdsToLoad = []; // try to read all missing ids from fetched links and fetch all the workitems at once later on

const workItemLinks = []; // global store for links

const definitionsToTraverse = definitions.filter(definition => definition.linkRole !== undefined); // do not consider the root definition

run(initialWorkItemIds, rootWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks);

const missingWorkItemIdsToLoad = workItemIdsToLoad.filter(workItemIdToLoad => !rootWorkItemIds.some(rootWorkItemId => rootWorkItemId === workItemIdToLoad)); // filter out all the existing root item ids

const workItemsToAdd = getWorkItemsByIds(missingWorkItemIdsToLoad); // fetch all the missing workitems at once

workItems.push(...workItemsToAdd); // add them to the global store

console.log(inspect({
    workItems,
    workItemLinks
}, false, null, true))

function run(leftSideWorkItemIds, leftSideWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks) {
    /*

        consider all the defitions from "definitionsToTraverse" directly related to "leftSideWorkItemType"
        remove them from "definitionsToTraverse" to prevent endless loops
        loop backwards so splice won't struggle with the indices

    */

    for (let definitionIndex = definitionsToTraverse.length - 1; definitionIndex >= 0; definitionIndex--) {
        const currentDefinition = definitionsToTraverse[definitionIndex];
        const { linkRole } = currentDefinition;
        const { parentWorkItemType, childWorkItemType } = linkRole;

        const leftSideWorkItemTypeIsParent = parentWorkItemType === leftSideWorkItemType;
        const leftSideWorkItemTypeIsChild = childWorkItemType === leftSideWorkItemType;

        // Check the direct relation

        if (leftSideWorkItemTypeIsParent || leftSideWorkItemTypeIsChild) {
            // Remove the inspected definition

            definitionsToTraverse.splice(definitionIndex, 1);

            const relatedWorkItemLinks = getWorkItemLinksByLinkRoleAndLeftSideIds(linkRole, leftSideWorkItemIds, leftSideWorkItemTypeIsParent);

            // store all the right side workitem IDs for the next run

            const rightSideWorkItemIds = [];

            for (let relatedWorkItemLinkIndex = 0; relatedWorkItemLinkIndex < relatedWorkItemLinks.length; relatedWorkItemLinkIndex++) {
                const relatedWorkItemLink = relatedWorkItemLinks[relatedWorkItemLinkIndex];

                // Process the link if not processed yet

                if (!workItemLinks.some(workItemLink => /* !!! pseudo equality check !!! */
                        workItemLink.linkRole.parentWorkItemType === relatedWorkItemLink.linkRole.parentWorkItemType &&
                        workItemLink.linkRole.childWorkItemType === relatedWorkItemLink.linkRole.childWorkItemType &&
                        workItemLink.linkRole.name === relatedWorkItemLink.linkRole.name &&
                        workItemLink.parentId === relatedWorkItemLink.parentId &&
                        workItemLink.childId === relatedWorkItemLink.childId)) {

                    // Push the link to the store

                    workItemLinks.push(relatedWorkItemLink);

                    // Get the right side id from the link

                    const rightSideWorkItemId = leftSideWorkItemTypeIsParent ? relatedWorkItemLink.childId : relatedWorkItemLink.parentId;

                    rightSideWorkItemIds.push(rightSideWorkItemId);

                    // Push the id to the store if it doesn't exist

                    if (!workItemIdsToLoad.some(workItemIdToLoad => workItemIdToLoad === rightSideWorkItemId)) {
                        workItemIdsToLoad.push(rightSideWorkItemId);
                    }
                }
            }

            // Find the opposite workitem type of the current definition

            const rightSideWorkItemType = leftSideWorkItemTypeIsParent ? childWorkItemType : parentWorkItemType;

            // Run again but use this definition as the previous one

            run(rightSideWorkItemIds, rightSideWorkItemType, definitionsToTraverse, workItemIdsToLoad, workItemLinks);
        }
    }
}

The problem is that my approach crashes because the backwards loop even runs if definitionsToTraverse is empty. And I'm mutating the arrays from the parameters directly, I think I shouldn't do that.

So any help would be appreciated a lot!

Body:
Update: Added a substantially cleaner version of the code.  I was bothered by the previous implementation.

I may be oversimplifying what you need to do here.  I'm treating this as described in my comment on the question:

Do I have this right? You have (here) a single root type of car. and so you want to include both cars, and then you want to include all nodes which have either of those cars as parent or child values in a link, and then include all nodes which have any of those results a a parent or child in a link, and so on recursively, collecting all the results not in a tree but in a simple list.

If that's correct, then this seems like a reasonable approach:

const getLinkedIds = (links, ids, toAdd = links
  .filter (({childId, parentId}) => ids .includes (childId) || ids .includes (parentId))
  .flatMap (({childId, parentId}) => [childId, parentId])
  .filter (id => !ids .includes (id))
) => toAdd .length == 0 ? ids : getLinkedIds (links, ids .concat (toAdd))

const linkedItems = (defs, items, links,
  rootTypes = defs.filter (({isRootWorkItem = false}) => isRootWorkItem)
    .map (({workItemType}) => workItemType),
  roots = items .filter (({type}) => rootTypes .includes (type)),
  ids = getLinkedIds (links, roots.flatMap (({id}) => id))
) => items .filter (({id}) => ids .includes (id))

const definitions = [ /* linkTypes deleted */  {workItemType: "car", isRootWorkItem: true}]
const workItems = [{id: 1, type: "car"}, {id: 2, type: "car"}, {id: 3, type: "bird"}, {id: 4, type: "bird"}, {id: 5, type: "bird"}, {id: 6, type: "invalid"}, {id: 7, type: "house"}, {id: 8, type: "house"}, {id: 9, type: "house"}, {id: 10, type: "person"}]
const workItemLinks = [ /* linkTypes deleted */ {parentId: 1, childId: 3}, {parentId: 1, childId: 4}, {parentId: 3, childId: 7}, {parentId: 3, childId: 8}, {parentId: 10, childId: 1}, {parentId: 1, childId: 2}]

console .log (
  linkedItems (definitions, workItems, workItemLinks)
)

.as-console-wrapper {max-height: 100% !important; top: 0}

In linkedItems, we start with the definitions to collect the root types, then use those to collect all the root elements.  We then call our main function, getLinkedIds using links, and the ids of these elements.  Then we filter the items to find those whose ids we chose.

getLinkedIds finds all the links that include any of our ids as either parent or child, and adds both parent and child to a list of ids that will be included, and when its done, filters out any that are already in our list.  If this list of ids to add is empty, we're done and return our current list.  Otherwise we recur on the combination of the two lists.  While there may be some inefficiencies here, I think the lists would have to be quite large for them to matter.

I haven't tried to read your code and understand how it differs from this, but if my guess is correct about your requirements, this is relatively simple.

Old Version

This is not my usual coding style, and it bothers me.  I tend to work with direct recursion, and I usually use immutable data structures.  I would love to see an approach that handles this using that style, but I didn't come up with one right away.

Update: The comment above applied to this older version of the code.  The newer version seems fine to me.

Here is a description of the original implementation of getLinkedIds (the other function has only cosmetic changes):

getLinkedIds keeps two lists of ids: those still to process and those we've already processed, repeatedly removing an id from the first list, adding it to the second and then queueing up all the unprocessed parentIds for which we have a childId link to the current value and then all the unprocessed childIds for which we have a parentId link to the current value.  When the toProcess queue is empty, we return the processed one.

And you can expand this snippet to see that version (which I don't recommend:

const getLinkedIds = (links, ids) => {
  const toProcess = [...ids], processed = []
  while (toProcess .length > 0) {
     const id = toProcess .pop ()
     processed .push (id)
     links .filter (({parentId}) => id == parentId)
       .forEach (({childId}) => {
         if (! processed .includes (childId) && ! toProcess .includes (childId)) {
           toProcess .push (childId)
         }
       })
     links .filter (({childId}) => id == childId)
       .forEach (({parentId}) => {
         if (! processed .includes (parentId) && ! toProcess .includes (parentId)) {
           toProcess .push (parentId)
         }
       })
  }
  return [...processed]
}

const linkedItems = (defs, items, links) => {
  const rootTypes = defs .filter (({isRootWorkItem = false}) => isRootWorkItem)
    .map (({workItemType}) => workItemType)
  const roots = items .filter (({type}) => rootTypes .includes (type))
  const ids = getLinkedIds (links, roots.flatMap (({id}) => id))
  return items .filter (({id}) => ids .includes (id))
}

const definitions = [ /* linkTypes deleted */  {workItemType: "car", isRootWorkItem: true}]
const workItems = [{id: 1, type: "car"}, {id: 2, type: "car"}, {id: 3, type: "bird"}, {id: 4, type: "bird"}, {id: 5, type: "bird"}, {id: 6, type: "invalid"}, {id: 7, type: "house"}, {id: 8, type: "house"}, {id: 9, type: "house"}, {id: 10, type: "person"}]
const workItemLinks = [ /* linkTypes deleted */ {parentId: 1, childId: 3}, {parentId: 1, childId: 4}, {parentId: 3, childId: 7}, {parentId: 3, childId: 8}, {parentId: 10, childId: 1}, {parentId: 1, childId: 2}]

console .log (
  linkedItems (definitions, workItems, workItemLinks)
)

.as-console-wrapper {max-height: 100% !important; top: 0}


来源: stackexchange

问:
Mockito verify state halfway through test
I have some code that put simply, sets an object to a state of PROCESSING, does some stuff, then sets it to SUCCESS. I want to verify that the PROCESSING save is done with the correct values.

The problem is when the verify() tests are performed, .equals() is called on the object as it is at the end of the test, rather than halfway through.

For example the code:

```<code>public void process(Thing thing) {
    thing.setValue(17);
    thing.setStatus(Status.PROCESSING);
    dao.save(thing);

    doSomeMajorProcessing(thing);

    thing.setStatus(Status.SUCCESS);
    dao.save(thing);
}
</code>```

I want to test:

```<code>public void test() {
    Thing actual = new Thing();
    processor.process(actual);

    Thing expected = new Thing();
    expected.setValue(17);
    expected.setStatus(Status.PROCESSING);
    verify(dao).save(expected);

    // ....

    expected.setStatus(Status.SUCCESS);
    verify(dao).save(expected);
}
</code>```

On the first verify, actual.getStatus() is Status.SUCCESS, as Mockito just keeps a reference to the object and can only test it's value at the end.

I have considered that if a when(...) where involved then .equals() would be called at the correct time and the result would only happen if Thing was what I wanted it to be. However, in this case .save() returns nothing.

How can I verify that the object is put into the correct states?

答:
After reviewing https://code.google.com/archive/p/mockito/issues/126
I was able to get my version of this working (Java 15, Mockito 3.6.28):

// ========= CODE ==========
public void process(Thing thing) {
    thing.setValue(17);
    thing.setStatus(Status.PROCESSING);
    dao.save(thing);

    doSomeMajorProcessing(thing);

    thing.setStatus(Status.SUCCESS);
    dao.save(thing);
}

// ========= TEST ==========

// ++++++ NOTE - put this at class level

private final Dao dao = mock(Dao.class, withSettings().defaultAnswer(new ClonesArguments()));

public void test() {
    Thing actual = new Thing();
    processor.process(actual);

    ArgumentCaptor<Thing> captor = ArgumentCaptor.for(Thing.class);
    verify(dao, times(2)).save(captor.capture());
    List<Things> savedCalls = captor.getAllValues();
    assertEquals(Status.PROCESSING, savedCalls.get(0).getStatus());
    assertEquals(Status.SUCCESS, savedCalls.get(1).getStatus());
}
```

ContentLicense: CC BY-SA 3.0

Body:
I have some code that put simply, sets an object to a state of PROCESSING, does some stuff, then sets it to SUCCESS. I want to verify that the PROCESSING save is done with the correct values.

The problem is when the verify() tests are performed, .equals() is called on the object as it is at the end of the test, rather than halfway through.

For example the code:

public void process(Thing thing) {
    thing.setValue(17);
    thing.setStatus(Status.PROCESSING);
    dao.save(thing);

    doSomeMajorProcessing(thing);

    thing.setStatus(Status.SUCCESS);
    dao.save(thing);
}

I want to test:

public void test() {
    Thing actual = new Thing();
    processor.process(actual);

    Thing expected = new Thing();
    expected.setValue(17);
    expected.setStatus(Status.PROCESSING);
    verify(dao).save(expected);

    // ....

    expected.setStatus(Status.SUCCESS);
    verify(dao).save(expected);
}

On the first verify, actual.getStatus() is Status.SUCCESS, as Mockito just keeps a reference to the object and can only test it's value at the end.

I have considered that if a when(...) where involved then .equals() would be called at the correct time and the result would only happen if Thing was what I wanted it to be. However, in this case .save() returns nothing.

How can I verify that the object is put into the correct states?

ContentLicense: CC BY-SA 4.0


问:
Check if string exists in JSON array in SQL
Trying to return rows that the JSON value contains a certain value.

```<code>Table Structure
ID     Mapping
1      {"DepartmentID":"Finishing","ShipToIDs":[],"ItemIDs":[]}
2      {"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":["300-VLR1022A"]}
3      {"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":[]}
</code>```

For example, passing in ShipToIDs = 100 would return rows 2-3 and passing in ItemID = 300-VLR1022A would return row 2.

答:
You will need SQL Server 2016 or later to make use of this, but it seems that you are looking for JSON_QUERY (Transact-SQL), e.g.:

/*
 * Data setup...
 */
create table dbo.Example (
  ID int,
  Mapping nvarchar(max)
);

insert dbo.Example (ID, Mapping) values
  (1, N'{"DepartmentID":"Finishing","ShipToIDs":[],"ItemIDs":[]}'),
  (2, N'{"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":["300-VLR1022A"]}'),
  (3, N'{"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":[]}');

/*
 * JSON_QUERY examples...
 */
select ID
from dbo.Example
cross apply openjson(Mapping, '$.ShipToIDs') ShipToID
where ShipToID.Value = '100';

select ID
from dbo.Example
cross apply openjson(Mapping, '$.ItemIDs') ItemID
where ItemID.Value = '300-VLR1022A';
```

Which yields the results:

ID

2

3

and:

ID

2

Body:
Trying to return rows that the JSON value contains a certain value.

Table Structure
ID     Mapping
1      {"DepartmentID":"Finishing","ShipToIDs":[],"ItemIDs":[]}
2      {"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":["300-VLR1022A"]}
3      {"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":[]}

For example, passing in ShipToIDs = 100 would return rows 2-3 and passing in ItemID = 300-VLR1022A would return row 2.

Body:
You will need SQL Server 2016 or later to make use of this, but it seems that you are looking for JSON_QUERY (Transact-SQL), e.g.:

/*
 * Data setup...
 */
create table dbo.Example (
  ID int,
  Mapping nvarchar(max)
);

insert dbo.Example (ID, Mapping) values
  (1, N'{"DepartmentID":"Finishing","ShipToIDs":[],"ItemIDs":[]}'),
  (2, N'{"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":["300-VLR1022A"]}'),
  (3, N'{"DepartmentID":"Finishing","ShipToIDs":["100"],"ItemIDs":[]}');

/*
 * JSON_QUERY examples...
 */
select ID
from dbo.Example
cross apply openjson(Mapping, '$.ShipToIDs') ShipToID
where ShipToID.Value = '100';

select ID
from dbo.Example
cross apply openjson(Mapping, '$.ItemIDs') ItemID
where ItemID.Value = '300-VLR1022A';

Which yields the results:

ID

2

3

and:

ID

2


问:
PHP print message when a div is present on specific page
I am trying to print a message (link) through PHP if a div is present on a page. Basically the div is present on some product pages on an ecommerce website, while it is not present on other product pages. The ecommerce is a Woocommerce and that's why I am using Woo hooks in the code.

Basically I get nothing in return (even when visiting pages where the div is present). What am I missing?

```<code>add_filter('woocommerce_before_add_to_cart_button', 'size_guide_display', 15);
function size_guide_display() {
    $sizeguide = $html->find("div#tab-SizeChart_tab");
    if ($sizeguide) :
        echo '<a href="#tab-SizeChart_tab" title="Storleksguide" class="size_guide_link">Storleksguide</a>';
    endif;
}
</code>```

I'll be very happy for any reply :-) Pretty novice with PHP.

Thanks!

答:
Due to the information you've provided I guess using JavaScript on the client-side would be your way to go.

```<code>document.addEventListener("DOMContentLoaded", () => {
    const sizeChartTabEle = document.querySelector("div#tab-SizeChart_tab")
    const storleksGuideLinkEle = document.createElement("a")
    storleksGuideLinkEle.title = "Storleksguide"
    storleksGuideLinkEle.href = "#tab-SizeChart_tab"
    storleksGuideLinkEle.classList.add("size_guide_link")
    storleksGuideLinkEle.innerText = "Storleksguide"
    sizeChartTabEle?.append(storleksGuideLinkEle)
})
</code>```

This code creates an event listener DOMContentLoaded (see this StackOverflow answer to learn more about it, speaking very basic - it waits for your contents being loaded) and executes the part in the curly brackets.

Your div#tab-SizeChart_tab is queried and afterwards appended by a newly created a element storleksGuideLinkEle.

If you make sure this code is only executed on pages that you need it, this should do the job.

Body:
I am trying to print a message (link) through PHP if a div is present on a page. Basically the div is present on some product pages on an ecommerce website, while it is not present on other product pages. The ecommerce is a Woocommerce and that's why I am using Woo hooks in the code.

Basically I get nothing in return (even when visiting pages where the div is present). What am I missing?

add_filter('woocommerce_before_add_to_cart_button', 'size_guide_display', 15);
function size_guide_display() {
    $sizeguide = $html->find("div#tab-SizeChart_tab");
    if ($sizeguide) :
        echo 'Storleksguide';
    endif;
}

I'll be very happy for any reply :-) Pretty novice with PHP.

Thanks!

Body:
Due to the information you've provided I guess using JavaScript on the client-side would be your way to go.

document.addEventListener("DOMContentLoaded", () => {
    const sizeChartTabEle = document.querySelector("div#tab-SizeChart_tab")
    const storleksGuideLinkEle = document.createElement("a")
    storleksGuideLinkEle.title = "Storleksguide"
    storleksGuideLinkEle.href = "#tab-SizeChart_tab"
    storleksGuideLinkEle.classList.add("size_guide_link")
    storleksGuideLinkEle.innerText = "Storleksguide"
    sizeChartTabEle?.append(storleksGuideLinkEle)
})

This code creates an event listener DOMContentLoaded (see this StackOverflow answer to learn more about it, speaking very basic - it waits for your contents being loaded) and executes the part in the curly brackets.

Your div#tab-SizeChart_tab is queried and afterwards appended by a newly created a element storleksGuideLinkEle.

If you make sure this code is only executed on pages that you need it, this should do the job.


问:
Layout align checkbox and date textview right in table layout?
```<code><?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:paddingTop="4dip" android:paddingBottom="6dip"
    android:layout_width="fill_parent" android:layout_height="wrap_content"
    android:orientation="horizontal">
    <TableLayout android:layout_width="fill_parent"
        android:layout_height="wrap_content">
        <TableRow android:layout_width="fill_parent"
            android:layout_height="20dp" >
            <TextView android:id="@+id/text_sender" android:textStyle="bold" android:textColor="#ffffff" android:focusable="false"
                android:layout_width="0dp" android:layout_height="20dp" android:layout_weight=".75" />
            <TextView android:id="@+id/text_date" android:text="date"
                android:layout_width="0dp" android:layout_gravity="right"  android:layout_weight=".25" android:focusable="false"
                android:layout_height="20dp" />
        </TableRow>
        <TableRow android:layout_width="fill_parent"
            android:layout_height="35dp" android:gravity="top">
            <TextView android:id="@+id/text_msg" android:layout_width="0dp" android:layout_weight=".75"
                android:layout_height="35dp" android:focusable="false" />

            <CheckBox android:layout_width="0dp" android:id="@+id/ck_msg" android:layout_weight=".25"
                android:checked="false" android:layout_height="35dp" android:focusable="false"
                android:layout_gravity="right"  />
        </TableRow>
    </TableLayout>
</LinearLayout>
</code>```

i want to aligh both of them to extream right

答:
It's better to use RelativeLayout for this.</p>

```<code><?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:paddingTop="4dip" android:paddingBottom="6dip"
    android:layout_width="fill_parent" android:layout_height="fill_parent">

    <TextView android:text="TextSender"
        android:id="@+id/text_sender"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content" />

    <TextView android:text="TextDate"
        android:id="@+id/text_date"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentRight="true" />

    <CheckBox android:id="@+id/checkBox1"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentRight="true"
        android:layout_below="@id/text_sender"
        android:text=""
         />
    <TextView android:text="TextView"
        android:id="@+id/textView1"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignBaseline="@id/checkBox1"
        android:layout_alignParentLeft="true"
        android:layout_below="@id/text_sender" />

</RelativeLayout>
</code>```

Body:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:paddingTop="4dip" android:paddingBottom="6dip"
    android:layout_width="fill_parent" android:layout_height="wrap_content"
    android:orientation="horizontal">
    <TableLayout android:layout_width="fill_parent"
        android:layout_height="wrap_content">
        <TableRow android:layout_width="fill_parent"
            android:layout_height="20dp" >
            <TextView android:id="@+id/text_sender" android:textStyle="bold" android:textColor="#ffffff" android:focusable="false"
                android:layout_width="0dp" android:layout_height="20dp" android:layout_weight=".75" />
            <TextView android:id="@+id/text_date" android:text="date"
                android:layout_width="0dp" android:layout_gravity="right"  android:layout_weight=".25" android:focusable="false"
                android:layout_height="20dp" />
        </TableRow>
        <TableRow android:layout_width="fill_parent"
            android:layout_height="35dp" android:gravity="top">
            <TextView android:id="@+id/text_msg" android:layout_width="0dp" android:layout_weight=".75"
                android:layout_height="35dp" android:focusable="false" />

            <CheckBox android:layout_width="0dp" android:id="@+id/ck_msg" android:layout_weight=".25"
                android:checked="false" android:layout_height="35dp" android:focusable="false"
                android:layout_gravity="right"  />
        </TableRow>
    </TableLayout>
</LinearLayout>

i want to aligh both of them to extream right

Body:
It's better to use RelativeLayout for this.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:paddingTop="4dip" android:paddingBottom="6dip"
    android:layout_width="fill_parent" android:layout_height="fill_parent">

    <TextView android:text="TextSender"
        android:id="@+id/text_sender"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content" />

    <TextView android:text="TextDate"
        android:id="@+id/text_date"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentRight="true" />

    <CheckBox android:id="@+id/checkBox1"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentRight="true"
        android:layout_below="@id/text_sender"
        android:text=""
         />
    <TextView android:text="TextView"
        android:id="@+id/textView1"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignBaseline="@id/checkBox1"
        android:layout_alignParentLeft="true"
        android:layout_below="@id/text_sender" />

</RelativeLayout>


问:
Outside context error when working from blueprint flask python
I have this simple webapp written in python (Flask)

models.py

```<code>from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()

class Coin(db.Model):
    __tablename__ = "coins"
    id = db.Column(db.Integer, primary_key=True)
    pair = db.Column(db.String)
    sell_amt = db.Column(db.Float)
    buy_amt = db.Column(db.Float)
</code>```

app.py

```<code>from flask import Flask
from ui import ui
from models import db , Coin

app = Flask(__name__)
app.register_blueprint(ui)
db.init_app(app)

if __name__ == "__main__":
    app.run(port=8080)
</code>```

__init__.py in ui folder

```<code>from flask import Blueprint ,current_app
from models import db, Coin
from threading import Thread
ui = Blueprint('ui', __name__)

def intro():
    global bot_state
    with current_app.app_context():
        all_coins = Coin.query.filter_by().all()
    while bot_state:
        sleep(3)
        print (f" Current time : {time()}")

@ui.route('/startbot')
def start_bot():
    global bot_thread, bot_state
    bot_state = True

    bot_thread = Thread(target=intro ,daemon=True)
    bot_thread.start()
    return "bot started "

@ui.route('/stopbot')
def stop_bot():
    global bot_state
    bot_state = False
    bot_thread.join()
    return  " bot stopped"
</code>```

When create a request to /startbot the app throws the error the it is working outside the app context

```<code>RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context().  See the
documentation for more information.
</code>```

but when trying to create a database object for example new = Coin() it works fine, how do you give a function the context of the application without making a function that returns the app, because doing so creates another error that is (circular import)

Note this is the bare minimum example and there are other files that require access to the models.py folder (to add orders to the data base created by the bot )

答:
The trick is to pass the application object to the thread. This also works with the proxy current_app. In this case, however, you need access to the underlying application object. You can find a short note on this within the documentation here.

```<code>from flask import current_app

# ...

def intro(app):
    with app.app_context():
        all_coins = Coin.query.all()

@ui.route('/startbot')
def start_bot():
    bot_thread = Thread(
        target=intro,
        args=(current_app._get_current_object(),), # <- !!!
        daemon=True
    )
    bot_thread.start()
    return "bot started"
</code>```

Since you don't seem to have fully understood my explanations, the following is how the complete contents of the __init__.py file would look like.

```<code>from flask import Blueprint, current_app, render_template
from models import Coin, db
from threading import Event, Lock, Thread
from time import sleep, time

ui = Blueprint('ui', __name__)
thread = None
thread_event = Event()
thread_lock = Lock()

def intro(app, event):
    app.logger.info('bot started')
    try:
        while event.is_set():
            tm = time()
            app.logger.info('current time %s', tm)

            with app.app_context():
                all_coins = Coin.query.all()
                # ...

            dur = 3 - (time() - tm)
            if dur > 0: sleep(dur)
    finally:
        event.clear()
        app.logger.info('bot stopped')

@ui.route('/startbot')
def start_bot():
    global thread
    thread_event.set()
    with thread_lock:
        if thread is None:
            thread = Thread(
                target=intro,
                args=(current_app._get_current_object(), thread_event),
                daemon=True
            )
            thread.start()
    return '', 200

@ui.route('/stopbot')
def stop_bot():
    global thread
    thread_event.clear()
    with thread_lock:
        if thread is not None:
            thread.join()
            thread = None
    return '', 200
</code>```

Have fun and success with the further implementation of your project.

Body:
I have this simple webapp written in python (Flask)

models.py

from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()

class Coin(db.Model):
    __tablename__ = "coins"
    id = db.Column(db.Integer, primary_key=True)
    pair = db.Column(db.String)
    sell_amt = db.Column(db.Float)
    buy_amt = db.Column(db.Float)

app.py

from flask import Flask
from ui import ui
from models import db , Coin

app = Flask(__name__)
app.register_blueprint(ui)
db.init_app(app)

if __name__ == "__main__":
    app.run(port=8080)

__init__.py in ui folder

from flask import Blueprint ,current_app
from models import db, Coin
from threading import Thread
ui = Blueprint('ui', __name__)

def intro():
    global bot_state
    with current_app.app_context():
        all_coins = Coin.query.filter_by().all()
    while bot_state:
        sleep(3)
        print (f" Current time : {time()}")

@ui.route('/startbot')
def start_bot():
    global bot_thread, bot_state
    bot_state = True

    bot_thread = Thread(target=intro ,daemon=True)
    bot_thread.start()
    return "bot started "

@ui.route('/stopbot')
def stop_bot():
    global bot_state
    bot_state = False
    bot_thread.join()
    return  " bot stopped"

When create a request to /startbot the app throws the error the it is working outside the app context

RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context().  See the
documentation for more information.

but when trying to create a database object for example new = Coin() it works fine, how do you give a function the context of the application without making a function that returns the app, because doing so creates another error that is (circular import)

Note this is the bare minimum example and there are other files that require access to the models.py folder (to add orders to the data base created by the bot )

Body:
The trick is to pass the application object to the thread. This also works with the proxy current_app. In this case, however, you need access to the underlying application object. You can find a short note on this within the documentation here.

from flask import current_app

# ...

def intro(app):
    with app.app_context():
        all_coins = Coin.query.all()

@ui.route('/startbot')
def start_bot():
    bot_thread = Thread(
        target=intro,
        args=(current_app._get_current_object(),), # <- !!!
        daemon=True
    )
    bot_thread.start()
    return "bot started"

Since you don't seem to have fully understood my explanations, the following is how the complete contents of the __init__.py file would look like.

from flask import Blueprint, current_app, render_template
from models import Coin, db
from threading import Event, Lock, Thread
from time import sleep, time

ui = Blueprint('ui', __name__)
thread = None
thread_event = Event()
thread_lock = Lock()

def intro(app, event):
    app.logger.info('bot started')
    try:
        while event.is_set():
            tm = time()
            app.logger.info('current time %s', tm)

            with app.app_context():
                all_coins = Coin.query.all()
                # ...

            dur = 3 - (time() - tm)
            if dur > 0: sleep(dur)
    finally:
        event.clear()
        app.logger.info('bot stopped')

@ui.route('/startbot')
def start_bot():
    global thread
    thread_event.set()
    with thread_lock:
        if thread is None:
            thread = Thread(
                target=intro,
                args=(current_app._get_current_object(), thread_event),
                daemon=True
            )
            thread.start()
    return '', 200

@ui.route('/stopbot')
def stop_bot():
    global thread
    thread_event.clear()
    with thread_lock:
        if thread is not None:
            thread.join()
            thread = None
    return '', 200

Have fun and success with the further implementation of your project.


问:
Python: what does Thread.is_alive *exactly* mean?
In Python 3.9.10, I am stumbling on the following very unsettling behaviour:

```<code>class MyThread(threading.Thread):
    def run(self):
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # prints FALSE!!!
        ...  # do some work involving asyncio and networking
        ...  # (specifically, I'm using aiohttp) and I know this work is
        ...  # actually being done because I can see its side-effects
        ...  # from across the network.
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # prints False AGAIN!!!
        ...  # go on with that work (still detectable and detected)
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # still False...
</code>```

Under some conditions, that call to is_alive() is returning False. Now, I'm not doing anything weird like redefining methods in MyThread that I'm not supposed to, or messing around with internals of anything.

My question is, under normal circumstances, is there any situation in which Thread.is_alive would return False after the thread has started, but while it is still doing work? (By the way, doing mostly Python work, not some C code running in the background.)

More Details

There is a main thread, and two worker threads. It goes on something like this. The following code runs in the main thread:

```<code>exit_signal = threading.Event()
workers = {}
# pass them the exit_signal so they know when to stop:
workers['connect_to_server_1'] = MyThread("server1.com", exit_signal)
workers['connect_to_server_1'].start()
workers['connect_to_server_2'] = MyThread("server2.com", exit_signal)
workers['connect_to_server_2'].start()

# wait until the process gets a SIGINT (user hits ^C)
try:
    for w in workers.values():
        w.join()  # will never return
except KeyboardInterrupt:
    logging.info("ok, user wants to quit, let's quit")
else:
    logging.critical("threads have quit on their own")  # never happens

# list thread statuses
w = workers['connect_to_server_1']
logging.debug(f"Is {w} alive? {w.is_alive()}")  # prints FALSE
w = workers['connect_to_server_2']
logging.debug(f"Is {w} alive? {w.is_alive()}")  # prints TRUE

# For debugging purposes, give the workers some more time to keep doing
# their jobs. This here is an interesting time window: the main thread
# has already received ^C, but the workers are supposedly not aware of
# that.
time.sleep(10)
# Finally, tell workers to stop, and wait for them to go:
exit_signal.set()
workers['connect_to_server_1'].join()
workers['connect_to_server_2'].join()
logging.info("all good, bye!")
</code>```

What happens is this

Before I hit ^C, I see in my log output messages from both workers telling me that they are heatlhy and successfully doing their jobs; more importantly, those "Am I alive?" messages (from the first code snippet at the top) always say True.

After I hit ^C, I see the log messages from the main thread checking the is_alive() status of both workers. I expected it to tell me that both workers are alive, since the interrupt signal always interrupts just the main thread. However, it tells me that the second worker is alive, but the first is not.

After that, while the main thread is blocked on that time.sleep(10) call, I still see messages from both workers in the log output. Both workers tell me that they are successfully doing their jobs (which can be verified by log messages from the other server they're talking to). However, everytime the first worker logs the "Am I alive?" message, it says False. WTF?

Finally, I set the exit_signal.

I see a message from the second worker (the one that was saying "Am I alive? True"), telling me that it received the signal, and then it goes on doing its shutdown routine, closing files and sockets etc.

I don't see any message from the other worker, and I can't see anywhere anything that indicates that he has received that signal, except for the fact that the join method on its thread returned successfully!

Closing thoughts

This code has been running in Python 3.6 for months, usually with around 15 workers instead of 2, and this issue never happened. It only happens when I try to run it in Python 3.9. It's somewhat easily reproducible: when I run that service with Python 3.9, around half of the time everything works perfectly, but the in the other half I get scared by this zombie thread telling me that it's dead, yet it's talking to me.

Also, the zombie thread is always the one talking to one specific server, which makes me think that this might be a problem with that one server's SSL certificate, or its implementation of the websocket protocol, but whatever, I don't control that one server. What I do control is the this instance of threading.Thread which should be either dead or walking upright, but not both.

What am I missing here?

答:
Turns out this is a recently-introduced bug in the threading implementation. Thread.join calls the internal method Thread._wait_for_tstate_lock, and that method was recently changed to look like this:

```<code>try:
    if lock.acquire(block, timeout):
        lock.release()
        self._stop()
except:
    if lock.locked():
        # bpo-45274: lock.acquire() acquired the lock, but the function
        # was interrupted with an exception before reaching the
        # lock.release(). It can happen if a signal handler raises an
        # exception, like CTRL+C which raises KeyboardInterrupt.
        lock.release()
        self._stop()
    raise
</code>```

The if lock.locked() check was an attempt to fix a hanging issue that occurred if this method was interrupted by Ctrl-C between the lock.acquire and the lock.release immediately afterward, but the check is wrong. It doesn't check if the previous lock.acquire call acquired the lock. It just checks if the lock is locked at all! The lock is almost always locked, and particularly, it's supposed to be locked the entire time a thread is alive.

This means that if you interrupt the lock.acquire call in this method with Ctrl-C, the code releases the lock (that someone else is holding) and calls self._stop to perform end-of-thread cleanup, including marking the thread as not alive any more. That's why your is_alive calls are returning False.

Body:
In Python 3.9.10, I am stumbling on the following very unsettling behaviour:

class MyThread(threading.Thread):
    def run(self):
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # prints FALSE!!!
        ...  # do some work involving asyncio and networking
        ...  # (specifically, I'm using aiohttp) and I know this work is
        ...  # actually being done because I can see its side-effects
        ...  # from across the network.
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # prints False AGAIN!!!
        ...  # go on with that work (still detectable and detected)
        liveness = self.is_alive()
        logging.debug(f"Am I alive? {liveness}")  # still False...

Under some conditions, that call to is_alive() is returning False. Now, I'm not doing anything weird like redefining methods in MyThread that I'm not supposed to, or messing around with internals of anything.

My question is, under normal circumstances, is there any situation in which Thread.is_alive would return False after the thread has started, but while it is still doing work? (By the way, doing mostly Python work, not some C code running in the background.)

More Details

There is a main thread, and two worker threads. It goes on something like this. The following code runs in the main thread:

exit_signal = threading.Event()
workers = {}
# pass them the exit_signal so they know when to stop:
workers['connect_to_server_1'] = MyThread("server1.com", exit_signal)
workers['connect_to_server_1'].start()
workers['connect_to_server_2'] = MyThread("server2.com", exit_signal)
workers['connect_to_server_2'].start()

# wait until the process gets a SIGINT (user hits ^C)
try:
    for w in workers.values():
        w.join()  # will never return
except KeyboardInterrupt:
    logging.info("ok, user wants to quit, let's quit")
else:
    logging.critical("threads have quit on their own")  # never happens

# list thread statuses
w = workers['connect_to_server_1']
logging.debug(f"Is {w} alive? {w.is_alive()}")  # prints FALSE
w = workers['connect_to_server_2']
logging.debug(f"Is {w} alive? {w.is_alive()}")  # prints TRUE

# For debugging purposes, give the workers some more time to keep doing
# their jobs. This here is an interesting time window: the main thread
# has already received ^C, but the workers are supposedly not aware of
# that.
time.sleep(10)
# Finally, tell workers to stop, and wait for them to go:
exit_signal.set()
workers['connect_to_server_1'].join()
workers['connect_to_server_2'].join()
logging.info("all good, bye!")

What happens is this

Before I hit ^C, I see in my log output messages from both workers telling me that they are heatlhy and successfully doing their jobs; more importantly, those "Am I alive?" messages (from the first code snippet at the top) always say True.

After I hit ^C, I see the log messages from the main thread checking the is_alive() status of both workers. I expected it to tell me that both workers are alive, since the interrupt signal always interrupts just the main thread. However, it tells me that the second worker is alive, but the first is not.

After that, while the main thread is blocked on that time.sleep(10) call, I still see messages from both workers in the log output. Both workers tell me that they are successfully doing their jobs (which can be verified by log messages from the other server they're talking to). However, everytime the first worker logs the "Am I alive?" message, it says False. WTF?

Finally, I set the exit_signal.

I see a message from the second worker (the one that was saying "Am I alive? True"), telling me that it received the signal, and then it goes on doing its shutdown routine, closing files and sockets etc.

I don't see any message from the other worker, and I can't see anywhere anything that indicates that he has received that signal, except for the fact that the join method on its thread returned successfully!

Closing thoughts

This code has been running in Python 3.6 for months, usually with around 15 workers instead of 2, and this issue never happened. It only happens when I try to run it in Python 3.9. It's somewhat easily reproducible: when I run that service with Python 3.9, around half of the time everything works perfectly, but the in the other half I get scared by this zombie thread telling me that it's dead, yet it's talking to me.

Also, the zombie thread is always the one talking to one specific server, which makes me think that this might be a problem with that one server's SSL certificate, or its implementation of the websocket protocol, but whatever, I don't control that one server. What I do control is the this instance of threading.Thread which should be either dead or walking upright, but not both.

What am I missing here?

Body:
Turns out this is a recently-introduced bug in the threading implementation. Thread.join calls the internal method Thread._wait_for_tstate_lock, and that method was recently changed to look like this:

try:
    if lock.acquire(block, timeout):
        lock.release()
        self._stop()
except:
    if lock.locked():
        # bpo-45274: lock.acquire() acquired the lock, but the function
        # was interrupted with an exception before reaching the
        # lock.release(). It can happen if a signal handler raises an
        # exception, like CTRL+C which raises KeyboardInterrupt.
        lock.release()
        self._stop()
    raise

The if lock.locked() check was an attempt to fix a hanging issue that occurred if this method was interrupted by Ctrl-C between the lock.acquire and the lock.release immediately afterward, but the check is wrong. It doesn't check if the previous lock.acquire call acquired the lock. It just checks if the lock is locked at all! The lock is almost always locked, and particularly, it's supposed to be locked the entire time a thread is alive.

This means that if you interrupt the lock.acquire call in this method with Ctrl-C, the code releases the lock (that someone else is holding) and calls self._stop to perform end-of-thread cleanup, including marking the thread as not alive any more. That's why your is_alive calls are returning False.


问:
php preg replace include slash(/)
i use preg replace since my column database does not support "strange letters"
but after regex i need keep "/", in this code bellow "/" is always missing
in code bellow i need to get all letter complete

```<code><?php
$jurnalName = "TL 110/90-12 K93-N02 AHM+";
$name = htmlspecialchars(htmlentities($jurnalName));
$name = preg_replace('/[^A-Za-z0-9|\-  +]/', '', $name);

var_dump($name);
</code>```

the result is always "TL 11090-12 K93-N02 AHM+" what i expecting is complete "TL 110/90-12 K93-N02 AHM+"

答:
Add / to the list of chars you want to keep

```<code><?php
$jurnalName = "TL 110/90-12 K93-N02 AHM+";
$name = htmlspecialchars(htmlentities($jurnalName));
$name = preg_replace('/[^A-Za-z0-9|\-\s\+\/]/', '', $name);

var_dump($name);
</code>```

(I also changed the space for \s and scaped the plus +, its optional here but I think its a good practice for characters that have spetial meaning inside regex

Title: php preg replace include slash(/)

Body:
i use preg replace since my column database does not support "strange letters"
but after regex i need keep "/", in this code bellow "/" is always missing
in code bellow i need to get all letter complete

<?php
$jurnalName = "TL 110/90-12 K93-N02 AHM+";
$name = htmlspecialchars(htmlentities($jurnalName));
$name = preg_replace('/[^A-Za-z0-9|\-  +]/', '', $name);

var_dump($name);

the result is always "TL 11090-12 K93-N02 AHM+" what i expecting is complete "TL 110/90-12 K93-N02 AHM+"

Body:
Add / to the list of chars you want to keep

<?php
$jurnalName = "TL 110/90-12 K93-N02 AHM+";
$name = htmlspecialchars(htmlentities($jurnalName));
$name = preg_replace('/[^A-Za-z0-9|\-\s\+\/]/', '', $name);

var_dump($name);

(I also changed the space for \s and scaped the plus +, its optional here but I think its a good practice for characters that have spetial meaning inside regex


问:
Nginx switches back to the first node after it comes back online
I am using Nginx as a failover cluster, where a service is given as an upstream.
The service is running in both the nodes simultaneously and the Nginx is set up in third node which redirects to other nodes.
When we switch off the service in one node it successfully switches to the backup node, but when the service in the 1st node is turned back on it switches back to the 1st node.
Below is the relevant part of the configuration in Nginx:

```<code>http {
include       mime.types;
default_type  application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"'
                  '"upstream: $upstream_addr"'
                  '"status: $upstream_status"';

access_log  logs\access.log  main;

sendfile        on;
#tcp_nopush     on;

#keepalive_timeout  0;
keepalive_timeout  65;

#gzip  on;

upstream backend {
    server ip1:port1 fail_timeout=5s max_fails=1;
    server ip2:port2 backup;
}

server {
    listen       8000;
    server_name  ip3;

    ssl_certificate      ...\certificate.crt;
    ssl_certificate_key  ...\privateKey.key;

    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

location / {
        proxy_pass https://backend;

}
</code>```

How can I stop switching the node to the primary node once the service is up again?

Thanks.

答: Write a bash code to change primary node to backup node once the primary node is backed up. Do let me know if you need the bash code</p>

Body:
I am using Nginx as a failover cluster, where a service is given as an upstream.
The service is running in both the nodes simultaneously and the Nginx is set up in third node which redirects to other nodes.
When we switch off the service in one node it successfully switches to the backup node, but when the service in the 1st node is turned back on it switches back to the 1st node.
Below is the relevant part of the configuration in Nginx:

http {
include       mime.types;
default_type  application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"'
                  '"upstream: $upstream_addr"'
                  '"status: $upstream_status"';

access_log  logs\access.log  main;

sendfile        on;
#tcp_nopush     on;

#keepalive_timeout  0;
keepalive_timeout  65;

#gzip  on;

upstream backend {
    server ip1:port1 fail_timeout=5s max_fails=1;
    server ip2:port2 backup;
}

server {
    listen       8000;
    server_name  ip3;

    ssl_certificate      ...\certificate.crt;
    ssl_certificate_key  ...\privateKey.key;

    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

location / {
        proxy_pass https://backend;

}

How can I stop switching the node to the primary node once the service is up again?

Thanks.


问:
Can SUMIFS calculate data from current month with 2 or more criteria?
I have a spreadsheet that uses a few SUMIFS formulas to calculate data based on 2 criteria and a normal range. Currently I'm having to change this manually every month to capture metrics of tickets reported that meet some of those criteria. Ideally i'd like to implement some logic to only calculate data that has dates from the current month.

Here's my original sumifs:

```<code>=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary",B2:B100,"little")
=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary",B2:B100,"panic")
</code>```

etc..

I was told something to the extent of this would get me where I needed to be, but it doesn't seem to be actually calculating only data with dates within this month.

```<code>=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary*",$B$2:$B$101,"little")+SUMIFS($G$2:$G$100,$F$2:$F$100,">="&DATE(YEAR(TODAY()),MONTH(TODAY()),1))
</code>```

Any help would be greatly appreciated!!

Spreadsheet_data

答:
Yep, you can!

You'll need formulas to get the first date of this month and the next month:

```<code>Date(Year(Today()), Month(Today())    , 1))  -- first day of this month
Date(Year(Today()), Month(Today()) + 1, 1))  -- first day of next month
</code>```

(This wraps the months intelligently i.e. Date(2021, 13, 1) is 2022-01-01)

You can then use these with greater-than and less-than operators within SUMIFS:

```<code>= SUMIFS(
    value_column,
    date_column, ">=" & Date(Year(Today()), Month(Today())    , 1),
    date_column, "<"  & Date(Year(Today()), Month(Today()) + 1, 1)
)
</code>```

For example:

```<code>= SUMIFS(
    G:G,
    F:F, ">=" & Date(Year(Today()), Month(Today())    , 1),
    F:F, "<"  & Date(Year(Today()), Month(Today()) + 1, 1),
    A:A, "primary",
    B:B, "panic"
)
</code>```

So here it's picking up the 33 and 57 to calculate the sum of 90.

Why this instead of SumProduct?

You can refer to entire columns e.g. G:G rather than a fixed range e.g. G$2:G$9999. SumProduct gives a #VALUE! error if any of the rows contain text, i.e. the column labels in the first row. SumIfs happily ignores the text.

Edit:

There may be a fault in this formula:

```<code>=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary*",$B$2:$B$101,"little")+SUMIFS($G$2:$G$100,$F$2:$F$100,">="&DATE(YEAR(TODAY()),MONTH(TODAY()),1))
</code>```

Making the format a little friendlier:

```<code>= SUMIFS(
    $G$2:$G$100,
    $A$2:$A$100, "primary*",
    $B$2:$B$101, "little"
)
+ SUMIFS(
    $G$2:$G$100,
    $F$2:$F$100, ">=" & DATE(YEAR(TODAY()),MONTH(TODAY()),1)
)
</code>```

It's adding all the Primary Littles from any month, and then also adding all the entries from this month onwards - double-counting any Primary Littles it's already counted.

Body:
I have a spreadsheet that uses a few SUMIFS formulas to calculate data based on 2 criteria and a normal range. Currently I'm having to change this manually every month to capture metrics of tickets reported that meet some of those criteria. Ideally i'd like to implement some logic to only calculate data that has dates from the current month.

Here's my original sumifs:

=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary",B2:B100,"little")
=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary",B2:B100,"panic")

etc..

I was told something to the extent of this would get me where I needed to be, but it doesn't seem to be actually calculating only data with dates within this month.

=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary*",$B$2:$B$101,"little")+SUMIFS($G$2:$G$100,$F$2:$F$100,">="&DATE(YEAR(TODAY()),MONTH(TODAY()),1))

Any help would be greatly appreciated!!

Spreadsheet_data

Body:
Yep, you can!

You'll need formulas to get the first date of this month and the next month:

Date(Year(Today()), Month(Today())    , 1))  -- first day of this month
Date(Year(Today()), Month(Today()) + 1, 1))  -- first day of next month

(This wraps the months intelligently i.e. Date(2021, 13, 1) is 2022-01-01)

You can then use these with greater-than and less-than operators within SUMIFS:

= SUMIFS(
    value_column,
    date_column, ">=" & Date(Year(Today()), Month(Today())    , 1),
    date_column, "<"  & Date(Year(Today()), Month(Today()) + 1, 1)
)

For example:

= SUMIFS(
    G:G,
    F:F, ">=" & Date(Year(Today()), Month(Today())    , 1),
    F:F, "<"  & Date(Year(Today()), Month(Today()) + 1, 1),
    A:A, "primary",
    B:B, "panic"
)

So here it's picking up the 33 and 57 to calculate the sum of 90.

Why this instead of SumProduct?

You can refer to entire columns e.g. G:G rather than a fixed range e.g. G$2:G$9999. SumProduct gives a #VALUE! error if any of the rows contain text, i.e. the column labels in the first row. SumIfs happily ignores the text.

Edit:

There may be a fault in this formula:

=SUMIFS($G$2:$G$100,$A$2:$A$100,"primary*",$B$2:$B$101,"little")+SUMIFS($G$2:$G$100,$F$2:$F$100,">="&DATE(YEAR(TODAY()),MONTH(TODAY()),1))

Making the format a little friendlier:

= SUMIFS(
    $G$2:$G$100,
    $A$2:$A$100, "primary*",
    $B$2:$B$101, "little"
)
+ SUMIFS(
    $G$2:$G$100,
    $F$2:$F$100, ">=" & DATE(YEAR(TODAY()),MONTH(TODAY()),1)
)

It's adding all the Primary Littles from any month, and then also adding all the entries from this month onwards - double-counting any Primary Littles it's already counted.


问:
Remove duplicates with the same NAME and keep the one with a higher USER RATING COUNT. (python)
I have sorted a list of tuples by name, each tuple has the elements id, name, average_rating, rating_count, developers, and size.

I'm now trying to remove the duplicate name and only keep the one with the higher rating_count.

A snip of my sorted list of tuples

```<code>[(320469536, '4_IN_1_ROW_Lite', 2.5, 3593, 'AOBO Co.,Ltd', 7750656),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 16, 'Yoshikazu Kakinoki', 3940352),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 900, 'Yoshikazu Kakinoki', 3940352)
]
</code>```

I would like an output of

```<code>[(320469536, '4_IN_1_ROW_Lite', 2.5, 3593, 'AOBO Co.,Ltd', 7750656),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 900, 'Yoshikazu Kakinoki', 3940352)
]
</code>```

only keeping '5x5 Shogi (MiniShogi) K55' with the highest rating_count which is 900.

My take

```<code>    for i in range(len(list_of_games)):
        if list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] > list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i+1])
        elif list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] < list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i])
</code>```

But I got an out of range error

```<code>line 222, in main
    if list_of_games[i][1] == list_of_games[i+1][1] and \
IndexError: list index out of range
</code>```

May I know how do I solve this?

答:
You don't have to iterate over last the element. You got out of range, because by iterating over last element you are comparing it with last + 1th element, which doesn't exist.

This should solve it:

```<code>for i in range(len(list_of_games)-1):
        if list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] > list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i+1])
        elif list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] < list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i])
</code>```

Body:
I have sorted a list of tuples by name, each tuple has the elements id, name, average_rating, rating_count, developers, and size.

I'm now trying to remove the duplicate name and only keep the one with the higher rating_count.

A snip of my sorted list of tuples

[(320469536, '4_IN_1_ROW_Lite', 2.5, 3593, 'AOBO Co.,Ltd', 7750656),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 16, 'Yoshikazu Kakinoki', 3940352),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 900, 'Yoshikazu Kakinoki', 3940352)
]

I would like an output of

[(320469536, '4_IN_1_ROW_Lite', 2.5, 3593, 'AOBO Co.,Ltd', 7750656),
(299139129, '5x5 Shogi (MiniShogi) K55', 4.0, 900, 'Yoshikazu Kakinoki', 3940352)
]

only keeping '5x5 Shogi (MiniShogi) K55' with the highest rating_count which is 900.

My take

    for i in range(len(list_of_games)):
        if list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] > list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i+1])
        elif list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] < list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i])

But I got an out of range error

line 222, in main
    if list_of_games[i][1] == list_of_games[i+1][1] and \
IndexError: list index out of range

May I know how do I solve this?

Body:
You don't have to iterate over last the element. You got out of range, because by iterating over last element you are comparing it with last + 1th element, which doesn't exist.

This should solve it:

for i in range(len(list_of_games)-1):
        if list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] > list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i+1])
        elif list_of_games[i][1] == list_of_games[i+1][1] and \
                list_of_games[i][3] < list_of_games[i+1][3]:
            list_of_games.remove(list_of_games[i])


问:
Continue iteration in 3 dimensions i, j, k using itertools.combinations from certain value
I post this question since all existing answers solve this issue in one dimension using islice, which is not the same (at least I couldn't figure out how to transfer current solutions to three dimensions).

Suppose we have the following script that produces all discrete combinations of integers for three dimensions ranging from (1 to 127):

```<code>for j, k, l in itertools.combinations(range(1, 128), 3):
    result = calculations([128, i, j, k, 0])
    writeFile(result+"\n")
</code>```

and suppose that the script gets interrupted at a random (recorded) point, e.g. [128, 121, 98, 45]. How can we use islice (or any other library) to continue with next iteration, i.e. [128, 121, 98, 46] and onwards until [128,127,126,125]?

Thank you for your time,

答:
as a somewhat dirty hack, iter(f, sentinel) produces an iterator that repeatedly calls the given function until it returns the given sentinel value, so if you knew the last finished value you could pass combos.__next__ and the last given value to iter and exhaust that iterator to skip to the next viable item:

```<code>import itertools, collections
all_combos = iter(map(set, itertools.combinations(range(1, 128), 3)))
LAST_RECORDED_COMBO = {121, 98, 45}
ITER_WE_WANT_TO_SKIP = iter(all_combos.__next__, LAST_RECORDED_COMBO)
# taken right from https://stackoverflow.com/questions/36763084/how-to-efficiently-exhaust-an-iterator-in-a-oneliner
collections.deque(ITER_WE_WANT_TO_SKIP, maxlen=0)

for combo in all_combos: # all_combos has now skipped forward
    print(combo) # original loop etc.
</code>```

The above is little more than than just looping through the combos stopping when it sees the last one used like this:

```<code>for c in all_combos:
    if c == LAST_RECORDED_COMBO:
        break
for combo in all_combos: # all_combos has now skipped forward
    print(combo) # original loop etc.
</code>```

But optimized so it can run entirely on the C layer much like islice to help performance.

I used map(set, ..) here because the sentinel value must be equal which means the ordering of a list or tuple would matter for it to stop properly, otherwise this has really bad feedback since it will just exhaust the entire iterator without a clear indication of why.

Body:
I post this question since all existing answers solve this issue in one dimension using islice, which is not the same (at least I couldn't figure out how to transfer current solutions to three dimensions).

Suppose we have the following script that produces all discrete combinations of integers for three dimensions ranging from (1 to 127):

for j, k, l in itertools.combinations(range(1, 128), 3):
    result = calculations([128, i, j, k, 0])
    writeFile(result+"\n")

and suppose that the script gets interrupted at a random (recorded) point, e.g. [128, 121, 98, 45]. How can we use islice (or any other library) to continue with next iteration, i.e. [128, 121, 98, 46] and onwards until [128,127,126,125]?

Thank you for your time,

Body:
as a somewhat dirty hack, iter(f, sentinel) produces an iterator that repeatedly calls the given function until it returns the given sentinel value, so if you knew the last finished value you could pass combos.__next__ and the last given value to iter and exhaust that iterator to skip to the next viable item:

import itertools, collections
all_combos = iter(map(set, itertools.combinations(range(1, 128), 3)))
LAST_RECORDED_COMBO = {121, 98, 45}
ITER_WE_WANT_TO_SKIP = iter(all_combos.__next__, LAST_RECORDED_COMBO)
# taken right from https://stackoverflow.com/questions/36763084/how-to-efficiently-exhaust-an-iterator-in-a-oneliner
collections.deque(ITER_WE_WANT_TO_SKIP, maxlen=0)

for combo in all_combos: # all_combos has now skipped forward
    print(combo) # original loop etc.

The above is little more than than just looping through the combos stopping when it sees the last one used like this:

for c in all_combos:
    if c == LAST_RECORDED_COMBO:
        break
for combo in all_combos: # all_combos has now skipped forward
    print(combo) # original loop etc.

But optimized so it can run entirely on the C layer much like islice to help performance.

I used map(set, ..) here because the sentinel value must be equal which means the ordering of a list or tuple would matter for it to stop properly, otherwise this has really bad feedback since it will just exhaust the entire iterator without a clear indication of why.


问:
Could adding jquery to a page cause Selenium css selectors to not work when using the contains function?
We are using Selenium 1.0.1 to test our web application.  We use css and xpath selectors.  The css selectors are usually more reliable and they work in both FireFox and IE.  The problem we're having is the contains function stopped working for us on February 24th.  The day before in our code we added jquery to the page that is having the problem.

We're not sure if it is that change that caused the problem or if there was some kind of browser update that day.  The problem is in both IE and FireFox so I doubt it was that.

Here is the CSS selector we are using:
    css=table#dashLayout_gauge0 option:contains('New Range')

Before this would return the correct element.  Now Selenium cannot find the element using this selector.  If I re-write it using XPath I can get it working in FireFox, just not in IE.

答: Turns out it didn't have to do with jQuery per se.  It was more what we were doing with jQuery.  We were changing the text of the options in certain cases using jQuery and that was causing the css selectors not to work.</p>

ContentLicense: CC BY-SA 2.5


问:
Correlation Matrix Heatmap - use legend to toggle visible items
I am using Plotly to create a heatmap which reflects a correlation matrix.  I would like to know if it's possible to toggle which items are displayed in the plot using the legend.

For example, the below creates a 10x10 matrix and respective heatmap.  My aim is to be able to select only "A", "B" & "C" in the plot itself, and it correctly show a 3x3 heatmap only including those items.

```<code>import plotly.offline as pyo
import plotly.express as px
import plotly.graph_objs as go
import numpy as np
import pandas as pd

arr = np.random.rand(10,10)
np.fill_diagonal(arr,np.nan)

corr = pd.DataFrame(arr)
corr.columns = ['A','B','C','D','E','F','G','H','I','J']
corr.index = ['A','B','C','D','E','F','G','H','I','J']

trace = go.Heatmap(z=corr.values,x=corr.index.values,y=corr.columns.values,colorscale='RdBu_r',
                   zmax=1,zmin=-1,text=corr.values)
fig = go.Figure(data=[trace])
pyo.plot(fig)

</code>```

The plot can also be created using plotly express imshow as per the below

```<code>fig = px.imshow(corr.values, text_auto='.2f',
                zmin=-1, zmax=1,
                color_continuous_scale='RdBu_r',aspect="auto",
                labels=dict(color="Correlation"),
                x=corr.index.values,
                y=corr.columns.values)
pyo.plot(fig)
</code>```

For visualization, I'd expect to go from this;

to this;

by using the legend (or equivalent)

答:
An approach to controlling by a legend is create a trace for each combination of keys you want

limited to subset of powerset

for additional combinations, set to legend only

```<code>import plotly.offline as pyo
import plotly.express as px
import plotly.graph_objs as go
import numpy as np
import pandas as pd
import more_itertools

arr = np.random.rand(10, 10)
np.fill_diagonal(arr, np.nan)

corr = pd.DataFrame(arr)
corr.columns = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J"]
corr.index = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J"]

fig = px.imshow(
    corr.values,
    text_auto=".2f",
    zmin=-1,
    zmax=1,
    color_continuous_scale="RdBu_r",
    aspect="auto",
    labels=dict(color="Correlation"),
    x=corr.index.values,
    y=corr.columns.values,
)

fig.update_traces(name="All", showlegend=True)

ps = [c for c in more_itertools.powerset(corr.columns) if len(c) == 3]

for c in ps:
    fig.add_traces(
        px.imshow(
            corr.loc[list(c), list(c)].values,
            x=list(c),
            y=list(c),
            color_continuous_scale="RdBu_r",
            text_auto=".2f",
        )
        .update_traces(name=str(c), showlegend=True, visible="legendonly")
        .data
    )

fig.update_layout(coloraxis_colorbar_x=-0.2)
</code>```

Body:
I am using Plotly to create a heatmap which reflects a correlation matrix.  I would like to know if it's possible to toggle which items are displayed in the plot using the legend.

For example, the below creates a 10x10 matrix and respective heatmap.  My aim is to be able to select only "A", "B" & "C" in the plot itself, and it correctly show a 3x3 heatmap only including those items.

import plotly.offline as pyo
import plotly.express as px
import plotly.graph_objs as go
import numpy as np
import pandas as pd

arr = np.random.rand(10,10)
np.fill_diagonal(arr,np.nan)

corr = pd.DataFrame(arr)
corr.columns = ['A','B','C','D','E','F','G','H','I','J']
corr.index = ['A','B','C','D','E','F','G','H','I','J']

trace = go.Heatmap(z=corr.values,x=corr.index.values,y=corr.columns.values,colorscale='RdBu_r',
                   zmax=1,zmin=-1,text=corr.values)
fig = go.Figure(data=[trace])
pyo.plot(fig)

The plot can also be created using plotly express imshow as per the below

fig = px.imshow(corr.values, text_auto='.2f',
                zmin=-1, zmax=1,
                color_continuous_scale='RdBu_r',aspect="auto",
                labels=dict(color="Correlation"),
                x=corr.index.values,
                y=corr.columns.values)
pyo.plot(fig)

For visualization, I'd expect to go from this;

to this;

by using the legend (or equivalent)

Body:
An approach to controlling by a legend is create a trace for each combination of keys you want

limited to subset of powerset

for additional combinations, set to legend only

import plotly.offline as pyo
import plotly.express as px
import plotly.graph_objs as go
import numpy as np
import pandas as pd
import more_itertools

arr = np.random.rand(10, 10)
np.fill_diagonal(arr, np.nan)

corr = pd.DataFrame(arr)
corr.columns = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J"]
corr.index = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J"]

fig = px.imshow(
    corr.values,
    text_auto=".2f",
    zmin=-1,
    zmax=1,
    color_continuous_scale="RdBu_r",
    aspect="auto",
    labels=dict(color="Correlation"),
    x=corr.index.values,
    y=corr.columns.values,
)

fig.update_traces(name="All", showlegend=True)

ps = [c for c in more_itertools.powerset(corr.columns) if len(c) == 3]

for c in ps:
    fig.add_traces(
        px.imshow(
            corr.loc[list(c), list(c)].values,
            x=list(c),
            y=list(c),
            color_continuous_scale="RdBu_r",
            text_auto=".2f",
        )
        .update_traces(name=str(c), showlegend=True, visible="legendonly")
        .data
    )

fig.update_layout(coloraxis_colorbar_x=-0.2)