问:
Maximum number of subarrays having equal sum
I was asked this question in one of my interview. Given an array of integers (with both positive and negative values) we need to find the maximum number of disjoint subarrays having equal sum.

Example :

Input : [1, 2, 3]
Output : 2 {since we have at most 2 subarrays with sum = 3 i.e. [1, 2],[3]}

Input: [2 2 2 -2]
Output : 2 {two subarrays each with sum = 2 i.e. [2],[2, 2, -2]}

My Approach

First approach that came to my mind was to find the prefix sum array and then taking each element (prefix[i]) as target finding the number of subarrays with sum = prefix[i] .

But this failed in the case of negative numbers. How to handle the negative cases?

EDIT Complete array must be covered with these subarrays. That is why in example 2 we get 2 as output and not 3 ([2],[2],[2]).

答:
Update: OP has clarified that the entire array must be partitioned into complete subarrays. As such, the first half of this post constitutes a full answer, but the second half may still be useful as a soltuion to a generalization of this question.

If we're using the whole array, the problem can be solved in O(n) time, which is optimal. We get to use the fact that splitting A into k subarrays means all subarrays will have a sum of sum(A)/k. After computing the array sum, we can solve the problem in a single pass through the array, using just prefix sums and a hashmap.

If we're not using the whole array, the problem is harder. There's an O(n^2) solution where you compute all subarray sums, sort equal-sum subarrays by right endpoint, and use greedy interval scheduling. I have no idea if this is optimal, but I'm interested to see if there's a better solution.

Full array partition case

The precise problem definition here is to find the length of the longest sequence of points [x_0, x_1, ... x_m] such that 0 <= x_0 < x_1 < ... < x_m < length(A) such that sum(A[0:x_0]) == sum(A[x_0:x_1]) == ... == sum(A[x_m:length(A)]).

Suppose we've computed the sum of the array to be S. Then, we can split A into k parts of equal sum if and only if (k divides S and if we see S/k, 2S/k, ... , kS/k as a subsequence of the prefix sums of A). One easy way to do that is to keep a running sum: if our running sum r divides S, then save 2S/r in a hashmap as a 'sum we're searching for'. If our current sum is one we've been searching for, save the next number from that subsequence as a 'sum we're searching for', unless we've reached the end of that sequence.

For example, say that S is 32. Then, A can be partitioned into 8 equal sum subarrays if and only if 4, 8, 12, 16, 20, 24, 28 appears as a subsequence of the prefix sums of A (32 will always be present at the end). So as soon as we see 4 as a prefix sum, we check that it divides S, and then save 8 to our search set for prefix sums. We also keep a helper dictionary, mapping 8 to 4, so that after finding 8 as a prefix sum, we know that 8+4 is the next prefix sum to look for.

Python code:

def best_full_partition(nums: List[int]) -> int:
    """Given a list of integers (positive or negative) 'nums',
    return the maximum number of disjoint equal sum subarrays we can
    partition nums into (using all elements)"""

    total = sum(nums)

    # Special case where total sum is 0
    if total == 0:
        # Count the number of times 0 is a partial sum
        answer = 0
        current_sum = 0
        for x in nums:
            current_sum += x
            if current_sum == 0:
                answer += 1
        return answer

    best_found = 1

    # Prefix sums we're trying to find; all share common factor with total
    looking_for = set()

    """ Map from prefix sums to the common factor/original prefix sum.
        There may be several: e.g. total/6 and total/3 may be targets
        of 2*total/3. """
    looking_to_original_sums = collections.defaultdict(set)

    current_sum = 0

    for x in nums:
        current_sum += x
        if current_sum == 0:
            continue

        if current_sum in looking_for:
            for original_sum in looking_to_original_sums[current_sum]:
                new_target = current_sum + original_sum

                # If we've found all matches in this chain
                if new_target == total:
                    best_found = max(best_found, total // original_sum)
                    continue

                looking_for.add(new_target)
                looking_to_original_sums[new_target].add(original_sum)

            looking_to_original_sums.pop(current_sum)
            looking_for.discard(current_sum)

        # Check if current sum is a divisor of full array sum
        if total % current_sum == 0:
            # If this splits array in half by sum, we've reached its end
            if 2 * current_sum == total:
                best_found = max(best_found, 2)
            else:
                # Add the next multiple of this sum to our search set
                looking_for.add(2 * current_sum)
                looking_to_original_sums[2 * current_sum].add(current_sum)

    return best_found
```
<p>This takes <code>O(n)</code> time, which is optimal, and <code>O(n)</code> space.</p>
<hr />
<p><strong>Partial array partition</strong></p>
<p>This case is harder, because there's fewer conditions on what the valid subarray sums can be. The trick is to just compute the sum of all subarrays. We make a hashmap, mapping each sum to the index bounds of its subarray, so <code>sum(A[L, L+1, ... R])</code> maps to <code>[L, R]</code>. Since there are duplicates, we keep a list of all intervals which produced that sum, and we generate that list to be sorted by the right endpoint.</p>
<p>Now, we can use earliest deadline first scheduling, aka greedy scheduling, to find the maximum number of intervals we can take from that list without overlap. Both steps take quadratic time. It may be possible to improve this, but I have no ideas for how to do so.</p>
<p>Python code:</p>
<pre class="lang-py prettyprint-override"><code>def best_partial_partition(nums: List[int]) -> int:
    """Given a list of integers (positive or negative) 'nums',
    return the maximum number of disjoint equal sum subarrays we can
    create from nums (using all elements is not required)"""

    n = len(nums)

    best_found = 1

    # For each subarray sum, stores a list of all subarrays with that sum
    # Sorted by right endpoint, both ends inclusive
    sum_to_intervals = collections.defaultdict(list)

    for right_end in range(n):
        curr_sum = 0
        for left_end in reversed(range(right_end+1)):
            curr_sum += nums[left_end]
            sum_to_intervals[curr_sum].append([left_end, right_end])

    # Use greedy interval scheduling to get most intervals

    for interval_list in sum_to_intervals.values():
        # Can skip if we know no improvement is possible here
        if len(interval_list) <= best_found:
            continue

        curr_len = 0
        curr_right_end = -1
        for left, right in interval_list:
            if left > curr_right_end:
                curr_len += 1
                curr_right_end = right

        best_found = max(best_found, curr_len)

    return best_found
</code>```

This runs in O(n^2) time.

Edit: Fixed a bug in full partition solver when the array sum was 0; thanks to @josejuan for pointing this out. This case needs to be treated separately to avoid dividing by zero.

Body:
Update: OP has clarified that the entire array must be partitioned into complete subarrays. As such, the first half of this post constitutes a full answer, but the second half may still be useful as a soltuion to a generalization of this question.

If we're using the whole array, the problem can be solved in O(n) time, which is optimal. We get to use the fact that splitting A into k subarrays means all subarrays will have a sum of sum(A)/k. After computing the array sum, we can solve the problem in a single pass through the array, using just prefix sums and a hashmap.

If we're not using the whole array, the problem is harder. There's an O(n^2) solution where you compute all subarray sums, sort equal-sum subarrays by right endpoint, and use greedy interval scheduling. I have no idea if this is optimal, but I'm interested to see if there's a better solution.

Full array partition case

The precise problem definition here is to find the length of the longest sequence of points [x_0, x_1, ... x_m] such that 0 <= x_0 < x_1 < ... < x_m < length(A) such that sum(A[0:x_0]) == sum(A[x_0:x_1]) == ... == sum(A[x_m:length(A)]).

Suppose we've computed the sum of the array to be S. Then, we can split A into k parts of equal sum if and only if (k divides S and if we see S/k, 2S/k, ... , kS/k as a subsequence of the prefix sums of A). One easy way to do that is to keep a running sum: if our running sum r divides S, then save 2S/r in a hashmap as a 'sum we're searching for'. If our current sum is one we've been searching for, save the next number from that subsequence as a 'sum we're searching for', unless we've reached the end of that sequence.

For example, say that S is 32. Then, A can be partitioned into 8 equal sum subarrays if and only if 4, 8, 12, 16, 20, 24, 28 appears as a subsequence of the prefix sums of A (32 will always be present at the end). So as soon as we see 4 as a prefix sum, we check that it divides S, and then save 8 to our search set for prefix sums. We also keep a helper dictionary, mapping 8 to 4, so that after finding 8 as a prefix sum, we know that 8+4 is the next prefix sum to look for.

Python code:

def best_full_partition(nums: List[int]) -> int:
    """Given a list of integers (positive or negative) 'nums',
    return the maximum number of disjoint equal sum subarrays we can
    partition nums into (using all elements)"""

    total = sum(nums)

    # Special case where total sum is 0
    if total == 0:
        # Count the number of times 0 is a partial sum
        answer = 0
        current_sum = 0
        for x in nums:
            current_sum += x
            if current_sum == 0:
                answer += 1
        return answer

    best_found = 1

    # Prefix sums we're trying to find; all share common factor with total
    looking_for = set()

    """ Map from prefix sums to the common factor/original prefix sum.
        There may be several: e.g. total/6 and total/3 may be targets
        of 2*total/3. """
    looking_to_original_sums = collections.defaultdict(set)

    current_sum = 0

    for x in nums:
        current_sum += x
        if current_sum == 0:
            continue

        if current_sum in looking_for:
            for original_sum in looking_to_original_sums[current_sum]:
                new_target = current_sum + original_sum

                # If we've found all matches in this chain
                if new_target == total:
                    best_found = max(best_found, total // original_sum)
                    continue

                looking_for.add(new_target)
                looking_to_original_sums[new_target].add(original_sum)

            looking_to_original_sums.pop(current_sum)
            looking_for.discard(current_sum)

        # Check if current sum is a divisor of full array sum
        if total % current_sum == 0:
            # If this splits array in half by sum, we've reached its end
            if 2 * current_sum == total:
                best_found = max(best_found, 2)
            else:
                # Add the next multiple of this sum to our search set
                looking_for.add(2 * current_sum)
                looking_to_original_sums[2 * current_sum].add(current_sum)

    return best_found

This takes O(n) time, which is optimal, and O(n) space.

Partial array partition

This case is harder, because there's fewer conditions on what the valid subarray sums can be. The trick is to just compute the sum of all subarrays. We make a hashmap, mapping each sum to the index bounds of its subarray, so sum(A[L, L+1, ... R]) maps to [L, R]. Since there are duplicates, we keep a list of all intervals which produced that sum, and we generate that list to be sorted by the right endpoint.

Now, we can use earliest deadline first scheduling, aka greedy scheduling, to find the maximum number of intervals we can take from that list without overlap. Both steps take quadratic time. It may be possible to improve this, but I have no ideas for how to do so.

Python code:

def best_partial_partition(nums: List[int]) -> int:
    """Given a list of integers (positive or negative) 'nums',
    return the maximum number of disjoint equal sum subarrays we can
    create from nums (using all elements is not required)"""

    n = len(nums)

    best_found = 1

    # For each subarray sum, stores a list of all subarrays with that sum
    # Sorted by right endpoint, both ends inclusive
    sum_to_intervals = collections.defaultdict(list)

    for right_end in range(n):
        curr_sum = 0
        for left_end in reversed(range(right_end+1)):
            curr_sum += nums[left_end]
            sum_to_intervals[curr_sum].append([left_end, right_end])

    # Use greedy interval scheduling to get most intervals

    for interval_list in sum_to_intervals.values():
        # Can skip if we know no improvement is possible here
        if len(interval_list) <= best_found:
            continue

        curr_len = 0
        curr_right_end = -1
        for left, right in interval_list:
            if left > curr_right_end:
                curr_len += 1
                curr_right_end = right

        best_found = max(best_found, curr_len)

    return best_found

This runs in O(n^2) time.

Edit: Fixed a bug in full partition solver when the array sum was 0; thanks to @josejuan for pointing this out. This case needs to be treated separately to avoid dividing by zero.


来源: stackexchange

问:
tfs2010 - how to change build definition - build priority
I want to change the default priority of a build definition (the build is triggered each check-ins).
How do I do that?</p>

答:
The correct use of the MSBuild priority argument is:</p>

```<code>/p:Priority=High
</code>```

ContentLicense: CC BY-SA 3.0

Body:
The correct use of the MSBuild priority argument is:

/p:Priority=High


问:
Calling same function from two another functions works only for first in React Native
I have problem to call one function in another two functions.
Two functions, where I will call function is same. I am using mapping.
All three functions I added in constructor, but I have no idea why this happend.

Here is my code:

```<code>class medicalDataForAll extends React.Component{
    constructor(props) {
        super(props)
        this.getLastWeekHeartRate = this.getLastWeekHeartRate.bind(this)
        this.getLastWeekHeartRateX = this.getLastWeekHeartRateX.bind(this)
        this.getLastWeekHeartRateY = this.getLastWeekHeartRateY.bind(this)
        }

    getLastWeekHeartRate() {
        var daysValues = [];

        this.getLastWeekHeartRateDatesXY().forEach((a) => {

            if (!this[a[0].day]) {
                this[a[0].day] = { day: a[0].day, value: 0};
                daysValues.push(this[a[0].day]);
            }

            this[a[0].day].value += a[0].value;
            this[a[0].day].month = a[0].month;

        }, Object.create(null));

        var days = this.getLastWeekHeartRateDatesXY().map((a) => a[0].day)
        var daysCounts = [...new Set(days)].map(e =>({
                        day: e,
                        count: days.filter(n => n===e).length }))

        var newArray = daysValues.map(function(value, index) {
            return {
                xAxe:  "" + daysValues[index].day + '/' + daysValues[index].month + "",
                yAxe: parseInt(daysValues[index].value / daysCounts[index].count)
            }
          });

        //console.log(newArray.reverse())
        return newArray.reverse()
    }

    getLastWeekHeartRateX() {
        const value = this.getLastWeekHeartRate().map((a) => a.xAxe)

        console.log(value)
        return value
    }

    getLastWeekHeartRateY() {
        const value = this.getLastWeekHeartRate().map((a) => a.xAxe)

        console.log(value)
        return value
    }

</code>```

Problem is showing getLastWeekHeartRateY(), the output of function is empty.
Thank you!

答:
After searching problem I can load all functions in render().

Problem was timing. I had many functions and it was not enough to execute function in render(). I just created in state chartData [] and after this updated state und used in both functions.

Code below:

```<code>
class medicalDataForAll extends React.Component{
    constructor(props) {
        super(props)
            this.state = {
               chartData = {}
            }
        }

    componentDidMount(){
        this.setState({
            chartData: this.getLastWeekHeartRate()
        })
    }

    getLastWeekHeartRate() {...}

    getLastWeekHeartRateX() {
        let value = this.state.chartData.map((a) => a.xAxe)

        return value
    }

    getLastWeekHeartRateY() {
        let value = this.state.chartData.map((a) => a.xAxe)

        return value
    }

</code>```

Have a nice coding!

ContentLicense: CC BY-SA 4.0

Body:
I have problem to call one function in another two functions.
Two functions, where I will call function is same. I am using mapping.
All three functions I added in constructor, but I have no idea why this happend.

Here is my code:

class medicalDataForAll extends React.Component{
    constructor(props) {
        super(props)
        this.getLastWeekHeartRate = this.getLastWeekHeartRate.bind(this)
        this.getLastWeekHeartRateX = this.getLastWeekHeartRateX.bind(this)
        this.getLastWeekHeartRateY = this.getLastWeekHeartRateY.bind(this)
        }

    getLastWeekHeartRate() {
        var daysValues = [];

        this.getLastWeekHeartRateDatesXY().forEach((a) => {

            if (!this[a[0].day]) {
                this[a[0].day] = { day: a[0].day, value: 0};
                daysValues.push(this[a[0].day]);
            }

            this[a[0].day].value += a[0].value;
            this[a[0].day].month = a[0].month;

        }, Object.create(null));

        var days = this.getLastWeekHeartRateDatesXY().map((a) => a[0].day)
        var daysCounts = [...new Set(days)].map(e =>({
                        day: e,
                        count: days.filter(n => n===e).length }))

        var newArray = daysValues.map(function(value, index) {
            return {
                xAxe:  "" + daysValues[index].day + '/' + daysValues[index].month + "",
                yAxe: parseInt(daysValues[index].value / daysCounts[index].count)
            }
          });

        //console.log(newArray.reverse())
        return newArray.reverse()
    }

    getLastWeekHeartRateX() {
        const value = this.getLastWeekHeartRate().map((a) => a.xAxe)

        console.log(value)
        return value
    }

    getLastWeekHeartRateY() {
        const value = this.getLastWeekHeartRate().map((a) => a.xAxe)

        console.log(value)
        return value
    }

Problem is showing getLastWeekHeartRateY(), the output of function is empty.
Thank you!

Body:
After searching problem I can load all functions in render().

Problem was timing. I had many functions and it was not enough to execute function in render(). I just created in state chartData [] and after this updated state und used in both functions.

Code below:

class medicalDataForAll extends React.Component{
    constructor(props) {
        super(props)
            this.state = {
               chartData = {}
            }
        }

    componentDidMount(){
        this.setState({
            chartData: this.getLastWeekHeartRate()
        })
    }

    getLastWeekHeartRate() {...}

    getLastWeekHeartRateX() {
        let value = this.state.chartData.map((a) => a.xAxe)

        return value
    }

    getLastWeekHeartRateY() {
        let value = this.state.chartData.map((a) => a.xAxe)

        return value
    }

Have a nice coding!


问:
How to make a basic calendar with PHP?
I am a beginner learning PHP and I am trying to make a basic calendar with PHP. Simpele kalender is how it has to look. It has to be with if and/or loops. This is what I have so far:

```<code><?php

$month_start = 'Tue';

$number_days = 28;

// number of rows
$number_rows = $number_days / 7;
    if ($number_days % 7 != 0) {
// number of days
$number_rows = ($number_days / 7) + 1;
}
?>

<table>
<tr>
<th>Mon</th>
<th>Tue</th>
<th>Wed</th>
<th>Thu</th>
<th>Fri</th>
<th>Sat</th>
<th>Sun</th>
</tr>

<?php
for($y = 1; $y <= $number_rows;$y++){
echo "<tr>";
for($i = 1; $i <= $number_days; $i++){
    if($i % 7 != 0){
      echo "<td>". $i . "</td>";
    }
    else{
      echo "<td>". $i . "</td>";
      break;

    }

}
echo "</tr>";
}
?>
</table>
<style>
  table, th, td {
  border: 1px solid black;
  text-align: center;
}
</style>
</code>```

How do I start on Thursday (beginning of the month) and how can I get the days 1 to 28 in the columns? I made another calendar but that wasn't correct. According to the feedback I got on it, I don't need to use gmdate, cal_days_in_month and mktime. Anyone that can help me go into the right direction?

答:
This is how I created a calendar. This creates "non-days" i.e. greyed-out (you'll have to style yourself) squares so that the first day of the month will start on its proper weekday place.

    $year = 2022;
    $date = new DateTime("".$year."-01-01");
    $weekday = (int)$date->format('N');
    $weekn = (int)$date->format('W');
    $day = 1;
    $month = 1;
    $dayLabels = array("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun");

    // This creates a "table head" for each month
    $thead = '
M	W	';
    foreach($dayLabels as $lbl) $thead .= "".$lbl."	";
    $thead .= '
';

    // This variable will hold the html table of the calendar
    $calendar = '
';

    // Loop each month until 12
    for(;$month <= 12; $month++)
    {
        $calendar .= $thead;
        $monthday = 1;
        $monthmax = cal_days_in_month(CAL_GREGORIAN, $month, $year);
        $position = 0;

        while($monthday <= $monthmax)
        {
            // Create month and week columns
            if($position == 0)
            {
                $weekn = (int)$date->format('W');
                $calendar .= "
".$month."	".$weekn."	";
                $position++;
            } // Create "non-days"
            else if($position < $weekday)
            {
                $calendar .= "	";
                $position++;
            } // Create days
            else if($position == $weekday)
            {
                $calendar .= "".$monthday."	";
                $position++;
                $weekday++;
                $monthday++;
                $date->modify('+1 day');

                if($position == 8)
                {
                    $position = 0;
                    $weekday = 1;
                }
            }
        }

        // Finish "non-days"
        while($position != 0)
        {
            $calendar .= "	";
            $position++;
            if($position == 8) $position = 0;
        }
        $calendar .= "
";
        if($month != 12) $calendar .= '	';
    }

    return $calendar;
```

Title: How to make a basic calendar with PHP?

Body:
I am a beginner learning PHP and I am trying to make a basic calendar with PHP. Simpele kalender is how it has to look. It has to be with if and/or loops. This is what I have so far:

<?php

$month_start = 'Tue';

$number_days = 28;

// number of rows
$number_rows = $number_days / 7;
    if ($number_days % 7 != 0) {
// number of days
$number_rows = ($number_days / 7) + 1;
}
?>

Mon
Tue
Wed
Thu
Fri
Sat
Sun

<?php
for($y = 1; $y <= $number_rows;$y++){
echo "
";
for($i = 1; $i <= $number_days; $i++){
    if($i % 7 != 0){
      echo "". $i . "	";
    }
    else{
      echo "". $i . "	";
      break;

    }

}
echo "
";
}
?>

How do I start on Thursday (beginning of the month) and how can I get the days 1 to 28 in the columns? I made another calendar but that wasn't correct. According to the feedback I got on it, I don't need to use gmdate, cal_days_in_month and mktime. Anyone that can help me go into the right direction?

Body:
This is how I created a calendar. This creates "non-days" i.e. greyed-out (you'll have to style yourself) squares so that the first day of the month will start on its proper weekday place.

    $year = 2022;
    $date = new DateTime("".$year."-01-01");
    $weekday = (int)$date->format('N');
    $weekn = (int)$date->format('W');
    $day = 1;
    $month = 1;
    $dayLabels = array("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun");

    // This creates a "table head" for each month
    $thead = '<tr><th>M</th><th>W</th>';
    foreach($dayLabels as $lbl) $thead .= "<th>".$lbl."</th>";
    $thead .= '</tr>';

    // This variable will hold the html table of the calendar
    $calendar = '<table id="calendar" border="1" style="width: 550px; max-width: 100%;">';

    // Loop each month until 12
    for(;$month <= 12; $month++)
    {
        $calendar .= $thead;
        $monthday = 1;
        $monthmax = cal_days_in_month(CAL_GREGORIAN, $month, $year);
        $position = 0;

        while($monthday <= $monthmax)
        {
            // Create month and week columns
            if($position == 0)
            {
                $weekn = (int)$date->format('W');
                $calendar .= "<tr><th>".$month."</th><th>".$weekn."</th>";
                $position++;
            } // Create "non-days"
            else if($position < $weekday)
            {
                $calendar .= "<td class=\"nonday\"></td>";
                $position++;
            } // Create days
            else if($position == $weekday)
            {
                $calendar .= "<td class=\"day\">".$monthday."</td>";
                $position++;
                $weekday++;
                $monthday++;
                $date->modify('+1 day');

                if($position == 8)
                {
                    $position = 0;
                    $weekday = 1;
                }
            }
        }

        // Finish "non-days"
        while($position != 0)
        {
            $calendar .= "<td class=\"nonday\"></td>";
            $position++;
            if($position == 8) $position = 0;
        }
        $calendar .= "</tr>";
        if($month != 12) $calendar .= '<td colspan="9" class="monthlinebreak"></td>';
    }

    return $calendar;


问:
Scrapy : create folder structure out of downloaded images based on the url from which images are being downloaded
I have an array of links that define the structure of a website. While downloading images from these links, I want to simultaneously place the downloaded images in a folder structure similar to the website structure, and not just rename it (as answered in Scrapy image download how to use custom filename)

My code for the same is like this:

```<code>class MyImagesPipeline(ImagesPipeline):
    """Custom image pipeline to rename images as they are being downloaded"""
    page_url=None
    def image_key(self, url):
        page_url=self.page_url
        image_guid = url.split('/')[-1]
        return '%s/%s/%s' % (page_url,image_guid.split('_')[0],image_guid)

    def get_media_requests(self, item, info):
        #http://store.abc.com/b/n/s/m
        os.system('mkdir '+item['sku'][0].encode('ascii','ignore'))
        self.page_url = urlparse(item['start_url']).path #I store the parent page's url in start_url Field
        for image_url in item['image_urls']:
            yield Request(image_url)
</code>```

It creates the required folder structure but when I go into the folders in deapth, I see that the files have been misplaced in the folders.

I'm suspecting that it is happening because the "get_media_requests" and "image_key" functions might be executing asynchronously hence the value of "page_url" changes before it is used by the "image_key" function.

答:
This scrapy pipeline extension provides an easy way to store downloaded files into a folder tree.

You have to install it:

pip install scrapy_folder_tree
```
<p>and then, add the pipeline on your configuration:</p>
<pre class="lang-py prettyprint-override"><code>ITEM_PIPELINES = {
    'scrapy_folder_tree.ImagesHashTreePipeline': 300
}
</code>```

Disclaimer: I'm the author of scrapy-folder-tree

Body:
I have an array of links that define the structure of a website. While downloading images from these links, I want to simultaneously place the downloaded images in a folder structure similar to the website structure, and not just rename it (as answered in Scrapy image download how to use custom filename)

My code for the same is like this:

class MyImagesPipeline(ImagesPipeline):
    """Custom image pipeline to rename images as they are being downloaded"""
    page_url=None
    def image_key(self, url):
        page_url=self.page_url
        image_guid = url.split('/')[-1]
        return '%s/%s/%s' % (page_url,image_guid.split('_')[0],image_guid)

    def get_media_requests(self, item, info):
        #http://store.abc.com/b/n/s/m
        os.system('mkdir '+item['sku'][0].encode('ascii','ignore'))
        self.page_url = urlparse(item['start_url']).path #I store the parent page's url in start_url Field
        for image_url in item['image_urls']:
            yield Request(image_url)

It creates the required folder structure but when I go into the folders in deapth, I see that the files have been misplaced in the folders.

I'm suspecting that it is happening because the "get_media_requests" and "image_key" functions might be executing asynchronously hence the value of "page_url" changes before it is used by the "image_key" function.

Body:
This scrapy pipeline extension provides an easy way to store downloaded files into a folder tree.

You have to install it:

pip install scrapy_folder_tree

and then, add the pipeline on your configuration:

ITEM_PIPELINES = {
    'scrapy_folder_tree.ImagesHashTreePipeline': 300
}

Disclaimer: I'm the author of scrapy-folder-tree


问:
add unique constraint in room database to multiple column
I have one entity in room

```<code>@Entity(foreignKeys ={
        @ForeignKey(entity = Label.class, parentColumns = "_id", childColumns = "labelId", onDelete = CASCADE),
        @ForeignKey(entity = Task.class, parentColumns = "_id", childColumns = "taskId", onDelete = CASCADE)
})
public class LabelOfTask extends Data{
    @ColumnInfo(name = "labelId")
    private Integer labelId;
    @ColumnInfo(name = "taskId")
    private Integer taskId;
}
</code>```

sql syntax of this entity is as below

```<code>CREATE TABLE `LabelOfTask` (
    `_id` INTEGER PRIMARY KEY AUTOINCREMENT,
     `labelId` INTEGER,
     `taskId` INTEGER,
     FOREIGN KEY(`labelId`) REFERENCES `Label`(`_id`) ON UPDATE NO ACTION ON DELETE CASCADE ,
     FOREIGN KEY(`taskId`) REFERENCES `Task`(`_id`) ON UPDATE NO ACTION ON DELETE CASCADE
 );
</code>```

but what change or annotation I need to add in entity class if I want to append below constraint to the auto generated sql schema of the table

```<code>unique (labelId, taskId)
</code>```

Ultimately I want to make combination of labelId and taskId unique in a table(or entity of room) using room library.

答:
Can't comment, so I'll just add it here. If manually migrating your Room DB to add an index using SQL statements, your index name to be able to match the annotation on the table must be of the format:

```<code>...
database.execSQL("CREATE UNIQUE INDEX index_tableName_columnName ON tableName (columnName)")
...
</code>```

Not sure how long the index names can be, so not sure what it looks like with multi-column indexes.

Body:
I have one entity in room

@Entity(foreignKeys ={
        @ForeignKey(entity = Label.class, parentColumns = "_id", childColumns = "labelId", onDelete = CASCADE),
        @ForeignKey(entity = Task.class, parentColumns = "_id", childColumns = "taskId", onDelete = CASCADE)
})
public class LabelOfTask extends Data{
    @ColumnInfo(name = "labelId")
    private Integer labelId;
    @ColumnInfo(name = "taskId")
    private Integer taskId;
}

sql syntax of this entity is as below

CREATE TABLE `LabelOfTask` (
    `_id` INTEGER PRIMARY KEY AUTOINCREMENT,
     `labelId` INTEGER,
     `taskId` INTEGER,
     FOREIGN KEY(`labelId`) REFERENCES `Label`(`_id`) ON UPDATE NO ACTION ON DELETE CASCADE ,
     FOREIGN KEY(`taskId`) REFERENCES `Task`(`_id`) ON UPDATE NO ACTION ON DELETE CASCADE
 );

but what change or annotation I need to add in entity class if I want to append below constraint to the auto generated sql schema of the table

unique (labelId, taskId)

Ultimately I want to make combination of labelId and taskId unique in a table(or entity of room) using room library.

Body:
Can't comment, so I'll just add it here. If manually migrating your Room DB to add an index using SQL statements, your index name to be able to match the annotation on the table must be of the format:

...
database.execSQL("CREATE UNIQUE INDEX index_tableName_columnName ON tableName (columnName)")
...

Not sure how long the index names can be, so not sure what it looks like with multi-column indexes.


问:
Pass Query Result to a csv file and Send it as attachement using SQL Server 2005
I have two tasks and since I am new to it, I need some help/advice from the masters.

What I need to do is send the result of q select query to a csv file which can have delimeter as comma or tab and then send this file as an attachement to a particular receipient.

Hoping for some great advice

答:
SSIS is probably the way to go, but if you want T-SQL to do it, try this..

I'm not going to write the complete program for you, but here are the major points...

determine the split character:

```<code>SET @CSV=',' --use CHAR(9) for tab
</code>```

build your CSV string one line at a time, looping over the results (do this for each line):

```<code>SET @FileMessage=ISNULL(CONVERT(varchar(10),column1),'')+@CSV+ISNULL(CONVERT(varchar(10),column2),'')
SET @ExecuteString = RTRIM('echo ' + @FileMessage + ' >>' + @FileName)
--append it to you file, you'll need write permissions
EXEC @ReturnValue=master..xp_cmdshell @ExecuteString, no_output
</code>```

actually send the mail, after the file is completely built:

```<code>EXEC xp_sendmail { [ @recipients= ] 'recipients [ ;...n ]' }
     [ ,[ @message= ] 'message' ]
     [ ,[ @query= ] 'query' ]
     [ ,[ @attachments= ] 'attachments [ ;...n ]' ]
     [ ,[ @copy_recipients= ] 'copy_recipients [ ;...n ]'
     [ ,[ @blind_copy_recipients= ] 'blind_copy_recipients [ ;...n ]'
     [ ,[ @subject= ] 'subject' ]
     [ ,[ @type= ] 'type' ]
     [ ,[ @attach_results= ] 'attach_value' ]
     [ ,[ @no_output= ] 'output_value' ]
     [ ,[ @no_header= ] 'header_value' ]
     [ ,[ @width= ] width ]
     [ ,[ @separator= ] 'separator' ]
     [ ,[ @echo_error= ] 'echo_value' ]
     [ ,[ @set_user= ] 'user' ]
     [ ,[ @dbuse= ] 'database' ]
</code>```

OwnerDisplayName: Sandhurst

ContentLicense: CC BY-SA 2.5

Body:
SSIS is probably the way to go, but if you want T-SQL to do it, try this..

I'm not going to write the complete program for you, but here are the major points...

determine the split character:

SET @CSV=',' --use CHAR(9) for tab

build your CSV string one line at a time, looping over the results (do this for each line):

SET @FileMessage=ISNULL(CONVERT(varchar(10),column1),'')+@CSV+ISNULL(CONVERT(varchar(10),column2),'')
SET @ExecuteString = RTRIM('echo ' + @FileMessage + ' >>' + @FileName)
--append it to you file, you'll need write permissions
EXEC @ReturnValue=master..xp_cmdshell @ExecuteString, no_output

actually send the mail, after the file is completely built:

EXEC xp_sendmail { [ @recipients= ] 'recipients [ ;...n ]' }
     [ ,[ @message= ] 'message' ]
     [ ,[ @query= ] 'query' ]
     [ ,[ @attachments= ] 'attachments [ ;...n ]' ]
     [ ,[ @copy_recipients= ] 'copy_recipients [ ;...n ]'
     [ ,[ @blind_copy_recipients= ] 'blind_copy_recipients [ ;...n ]'
     [ ,[ @subject= ] 'subject' ]
     [ ,[ @type= ] 'type' ]
     [ ,[ @attach_results= ] 'attach_value' ]
     [ ,[ @no_output= ] 'output_value' ]
     [ ,[ @no_header= ] 'header_value' ]
     [ ,[ @width= ] width ]
     [ ,[ @separator= ] 'separator' ]
     [ ,[ @echo_error= ] 'echo_value' ]
     [ ,[ @set_user= ] 'user' ]
     [ ,[ @dbuse= ] 'database' ]

OwnerDisplayName: mike


问:
Can multiple javadoc commands output to the same directory hierarchy?
I have multiple targets, which I'll call Foo and Bar, in my build file.  Assume that Foo is built first.  Each target consists of tasks to run javac, jar, and javadoc.

I would like for both javadoc commands to use the same destination directory; however,Bar's javadoc command produces a package-list file that overwrite's Foo's package-list.  Additionally, I would like for Bar's javadoc to be able to link to javadoc generated for Foo, but it can only do so if there is a package-list it can read -- which would be at the same location I want it to write.

The reason I don't create a separate Javadoc target is that I would like to be able to build the javadoc for Foo, whether or not Bar compiles, and vice versa.

(How) can I have multiple javadoc commands write to the same base directory?

答:
I don't think there is a good way to do this - there is not only the package-list, but also a number of other files which will regenerated (differently) by both javadoc processes:

constant-values.html

allclasses-frame.html

allclasses-noframe.html

deprecated-list.html

index-all.html (or per-letter index files)

overview-frame.html

overview-summary.html

overview-tree.html

serialized-form.html

These are possible alternatives:

Generate separate outputs, but link them to each other (using the -link or -linkoffline option).

Generate a common output using the last version of the other project that was known to compile. Use your version control system for this.

Occassionally generate a big output with both (i.e. from stable versions).

Then regenerate only the part for your project (into a new directory), and then copy it (without the files in the top level directory, i.e. overview and the frameset files) over the big output. The index, tree, and class lists in the top-level frameset will not be up-to date, but how often do these change anyway?

If you want to invest lots of time, you could modify the third option to merge the new top-level files into the existing ones. Good luck.


问:
In MongoDB, what is the purpose of embedded documents if the same thing can be achieved without using them?
For example

Instead of a field named, COLUMN_1, that holds an embedded document with fields A, B, and C.

Why not 3 separate fields with names COLUMN_1_A, COLUMN_1_B, COLUMN_1_C.

答:
In mongoDB you prefer to work with documents the way  they exist naturaly
, afcourse you can create routines that can translate every time and assemble/disasemble the document:

```<code>  COLUMN_1:{A:X,B:Y,C:Z}
</code>```

to:

```<code> COLUMN_1_A:X,COLUMN_1_B:Y,COLUMN_1_C:Z
</code>```

But this is additonal work that you dont want to do every time , you are a lazy effective developer that prefer just store your json document the way it will be used unless there is a specific use case that make sence to do it this way ... :)

Also please, note embedding is possible up to 100 levels , but max document size is still limited to 16MB , so in your document model it is not expected that you embed all your database in single document...

When you need performance , you add indices and optimize queries , that way your most searched data stay in memory , there is no big difference if your fields are embeded or in the root if they are not indexed ...

Body:
In mongoDB you prefer to work with documents the way  they exist naturaly
, afcourse you can create routines that can translate every time and assemble/disasemble the document:

  COLUMN_1:{A:X,B:Y,C:Z}

to:

 COLUMN_1_A:X,COLUMN_1_B:Y,COLUMN_1_C:Z

But this is additonal work that you dont want to do every time , you are a lazy effective developer that prefer just store your json document the way it will be used unless there is a specific use case that make sence to do it this way ... :)

Also please, note embedding is possible up to 100 levels , but max document size is still limited to 16MB , so in your document model it is not expected that you embed all your database in single document...

When you need performance , you add indices and optimize queries , that way your most searched data stay in memory , there is no big difference if your fields are embeded or in the root if they are not indexed ...


问:
`refold :: Functor s => (a -> s a, a) -> (s b -> b) -> b` as a morphism between universal types
Various recursion scheme boil down to specific instantiation of refold

refold :: Functor s => (s b -> b) -> (a -> s a) -> a -> b
refold f g = go where go a = f (fmap go (g a))
```
<p>What is the meaningful interpretation of <code>refold</code> ?</p>
<p>The data type <code>data Nu f = forall a. Nu (a -> f a) a</code> and <code>newtype Mu f = Mu {unMu :: forall b. (f b -> b) -> b}</code> can be seen as the colimit and limit of the forget functor from coalgebras and algebras, and <code>refold</code> is a morphism between those, but does it shed light on <code>refold</code> ?</p>
<pre class="lang-hs prettyprint-override"><code>refold' :: forall s. Functor s => Nu s -> Mu s
refold' (Nu g (a :: a)) = Mu mu where

  mu :: forall b. (s b -> b) -> b
  mu f = go a where

    go :: a -> b
    go a = f (fmap go (g a))
</code>```

答:
I guess it depends what you mean by "meaningful interpretation".

If s is a base functor for a recursive data type and a corecursive codata type, like the following functor s ~ ListF e for the recursive list data type [e] (which, in Haskell, is also a corecursive stream codata type):

```<code>{-# LANGUAGE DeriveFunctor #-}
data ListF e b = Nil | Cons e b deriving (Show, Functor)
</code>```

then an s-coalgebra of type a -> s a together with a starting seed a can generate a value of codata type [e] by unfolding from that seed, while an s-algebra of type s b -> b can consume a value of data type [e] by folding into a value of type b.  The refold function just combines the operation of unfolding from a and folding into b, without actually creating an intermediate codata/data type.

For example, you can generate the (finite) codata stream [10,9..1] by unfolding from an Integer seed using the starting value / coalgebra pair (a,g) as follows:

```<code>a :: Integer
a = 10

g :: Integer -> (ListF Integer) Integer
g 0 = Nil
g n = Cons n (n-1)
</code>```

and fold a list to calculate its Int length using the algebra:

```<code>f :: (ListF Integer) Int -> Int
f Nil = 0
f (Cons _ b) = 1 + b
</code>```

The refold function just combines these operations:

```<code>main = print $ refold f g a
</code>```

In this particular case, it calculates the length 10 of the stream/list [1..10] without actually creating any intermediate stream/list.

I guess the intuition is that if an operation can be imagined as an F-recursion applied to an F-corecursion for the same functor F, then it's a refold.  Or, maybe more practically, if an algorithm has an internal recursive structure that matches the functor F, it can be expressed as a refold.  The documentation for refold in recursion-schemes gives the example of quicksort having a recursive structure that matches a binary tree, though you've presumably already seen that example.

Note: What follows is wrong or at best imprecise, but I'll try to think a little more about it.

<s>In practice, refold isn't only used as a morphism between universal data types, but if you have a final s-coalgebra for a codata type C associated with the functor s:

```<code>eatC :: C -> ListF Integer C
</code>```

and an initial s-algebra for a data type D also associated with the functor s:

```<code>makeD :: ListF Integer D -> D
</code>```

then refold makeD eatC should be a natural morphism from codata type C to data type D.  That is, it should be the unique morphism satsifying:

```<code>fmap h . refold makeD eatC = refold makeD eatC . fmap h
</code>```

I'm not sure that aspect is tremendously interesting...</s>

Body:
Various recursion scheme boil down to specific instantiation of refold

refold :: Functor s => (s b -> b) -> (a -> s a) -> a -> b
refold f g = go where go a = f (fmap go (g a))

What is the meaningful interpretation of refold ?

The data type data Nu f = forall a. Nu (a -> f a) a and newtype Mu f = Mu {unMu :: forall b. (f b -> b) -> b} can be seen as the colimit and limit of the forget functor from coalgebras and algebras, and refold is a morphism between those, but does it shed light on refold ?

refold' :: forall s. Functor s => Nu s -> Mu s
refold' (Nu g (a :: a)) = Mu mu where

  mu :: forall b. (s b -> b) -> b
  mu f = go a where

    go :: a -> b
    go a = f (fmap go (g a))

Body:
I guess it depends what you mean by "meaningful interpretation".

If s is a base functor for a recursive data type and a corecursive codata type, like the following functor s ~ ListF e for the recursive list data type [e] (which, in Haskell, is also a corecursive stream codata type):

{-# LANGUAGE DeriveFunctor #-}
data ListF e b = Nil | Cons e b deriving (Show, Functor)

then an s-coalgebra of type a -> s a together with a starting seed a can generate a value of codata type [e] by unfolding from that seed, while an s-algebra of type s b -> b can consume a value of data type [e] by folding into a value of type b.  The refold function just combines the operation of unfolding from a and folding into b, without actually creating an intermediate codata/data type.

For example, you can generate the (finite) codata stream [10,9..1] by unfolding from an Integer seed using the starting value / coalgebra pair (a,g) as follows:

a :: Integer
a = 10

g :: Integer -> (ListF Integer) Integer
g 0 = Nil
g n = Cons n (n-1)

and fold a list to calculate its Int length using the algebra:

f :: (ListF Integer) Int -> Int
f Nil = 0
f (Cons _ b) = 1 + b

The refold function just combines these operations:

main = print $ refold f g a

In this particular case, it calculates the length 10 of the stream/list [1..10] without actually creating any intermediate stream/list.

I guess the intuition is that if an operation can be imagined as an F-recursion applied to an F-corecursion for the same functor F, then it's a refold.  Or, maybe more practically, if an algorithm has an internal recursive structure that matches the functor F, it can be expressed as a refold.  The documentation for refold in recursion-schemes gives the example of quicksort having a recursive structure that matches a binary tree, though you've presumably already seen that example.

Note: What follows is wrong or at best imprecise, but I'll try to think a little more about it.

<s>In practice, refold isn't only used as a morphism between universal data types, but if you have a final s-coalgebra for a codata type C associated with the functor s:

eatC :: C -> ListF Integer C

and an initial s-algebra for a data type D also associated with the functor s:

makeD :: ListF Integer D -> D

then refold makeD eatC should be a natural morphism from codata type C to data type D.  That is, it should be the unique morphism satsifying:

fmap h . refold makeD eatC = refold makeD eatC . fmap h

I'm not sure that aspect is tremendously interesting...</s>


问:
Leaflet - Map "shadowed" and not clickable when size is set to a percentage
I am trying to display a map on a container. I successfully display it when setting its size to a specific amount of px.
However, I want the height to adapt to the container size.
In order to do that, I had to set the height of every parents of the map container. Once done, the map appear "shadowed" and is not clickable, just as it was behind something. I can only zoom in and out.

HTML :

```<code><div class="container-fluid">
  <div class="row">
    <div class="col">
      <div id="mapid"></div>
    </div>
    <div class="col-2" id="selection">
      //
    </div>
  </div>
</div>
</code>```

CSS:

```<code>body {
    padding-top: 65px;
    margin: 0;
}
html, body, div {
    height: 100%;
    width: 100%;
}
#mapid {
    height: 100%;
    width: 100%;
    align-self: center;
}
</code>```

The result:

答:
The culprit is most probably:

div {
    height: 100%;
    width: 100%;
}
```

...which looks to enlarge the Leaflet attribution control over your map container as well.

This explains the "shadow" effect and loss of interactivity.

Avoid such general selector for such layout styling.

Body:
I am trying to display a map on a container. I successfully display it when setting its size to a specific amount of px.
However, I want the height to adapt to the container size.
In order to do that, I had to set the height of every parents of the map container. Once done, the map appear "shadowed" and is not clickable, just as it was behind something. I can only zoom in and out.

HTML :

      //

CSS:

body {
    padding-top: 65px;
    margin: 0;
}
html, body, div {
    height: 100%;
    width: 100%;
}
#mapid {
    height: 100%;
    width: 100%;
    align-self: center;
}

The result:

Body:
The culprit is most probably:

div {
    height: 100%;
    width: 100%;
}

...which looks to enlarge the Leaflet attribution control over your map container as well.

This explains the "shadow" effect and loss of interactivity.

Avoid such general selector for such layout styling.


问:
iOS Splash Page while Loading Data
I am trying to write an app that displays a simple logo/splash page while the app retrieves some data. I cannot seem to find a tutorial anywhere.

I have a "MainWindow.xib" file that will have my splash page, and I'm setting that as my "Main Interface" in my Info.plist, however, I cannot seem to see how to replace that page with a .xib that contains a UINavigationController. I thought I would just create a new UIController, that had a UINavigationController and in my .xib I'll drag in a Navigation Controller and set all my information, but it's giving me real fits.

So, I figured I would have a UIController that I would "alloc" and "init" with my second .xib that contains all my navigation.

```<code>myMainController = [[UIController alloc] initWithNibName:@"MainNavController":nil];
</code>```

in "MainNavController.xib", I've dragged in a Navigation Controller, but I don't know what to connect it to????

I'm sure I'm going down the wrong, path; but I cannot find a decent tutorial for this.

Can someone give direction or link to a decent tutorial?

Thanks.

答:
Are you ever actually adding the new controller to your existing view?  If not, something like:

```<code>[self.view addSubview:myMainController.view];
</code>```

should do it for you.  A better alternative is likely to be:

```<code>self.window.rootViewController = myMainController;
</code>```

But I'm fairly new to this and maybe I've missed something ...

Title: iOS Splash Page while Loading Data

Body:
I am trying to write an app that displays a simple logo/splash page while the app retrieves some data. I cannot seem to find a tutorial anywhere.

I have a "MainWindow.xib" file that will have my splash page, and I'm setting that as my "Main Interface" in my Info.plist, however, I cannot seem to see how to replace that page with a .xib that contains a UINavigationController. I thought I would just create a new UIController, that had a UINavigationController and in my .xib I'll drag in a Navigation Controller and set all my information, but it's giving me real fits.

So, I figured I would have a UIController that I would "alloc" and "init" with my second .xib that contains all my navigation.

myMainController = [[UIController alloc] initWithNibName:@"MainNavController":nil];

in "MainNavController.xib", I've dragged in a Navigation Controller, but I don't know what to connect it to????

I'm sure I'm going down the wrong, path; but I cannot find a decent tutorial for this.

Can someone give direction or link to a decent tutorial?

Thanks.

Body:
Are you ever actually adding the new controller to your existing view?  If not, something like:

[self.view addSubview:myMainController.view];

should do it for you.  A better alternative is likely to be:

self.window.rootViewController = myMainController;

But I'm fairly new to this and maybe I've missed something ...


问:
pyspark .show() works but .collect() does not
I have a very simple pyspark program that uses dataframe to query data from a group of ORC files. I am using anaconda python in windows and installed pyspark on it.

The program goes like this:

```<code>from pyspark.sql import SparkSession

spark_session = SparkSession.builder.appName("test").getOrCreate()
df_orc = spark_session .read.orc("./raw_data/")
df_orc.createOrReplaceTempView("orc")
</code>```

This works fine:

```<code>spark.sql("select count(*) from orc").show()
</code>```

But this will generate errors:

```<code>spark.sql("select count(*) from orc").collect()
</code>```

The error messages are:

```<code>WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

Py4JJavaError: An error occurred while calling o81.collectToPython.
: java.lang.IllegalArgumentException
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.spark.util.ClosureCleaner$.getClassReader(ClosureCleaner.scala:46)
        at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.sca
        at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.sca
        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
        at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:103)
        at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:103)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
        at scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:103)
        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
        at org.apache.spark.util.FieldAccessFinder$$anon$3.visitMethodInsn(ClosureCleaner.scala:426)
        at org.apache.xbean.asm5.ClassReader.a(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.b(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.accept(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.accept(Unknown Source)
        at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(
        at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(
        at scala.collection.immutable.List.foreach(List.scala:381)
        at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.sc
        at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:156)
        at org.apache.spark.SparkContext.clean(SparkContext.scala:2294)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2068)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2094)
        at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:936)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
        at org.apache.spark.rdd.RDD.collect(RDD.scala:935)
        at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:278)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply$mcI$sp(Dataset.scala:2808)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply(Dataset.scala:2805)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply(Dataset.scala:2805)
        at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
        at org.apache.spark.sql.Dataset.withNewExecutionId(Dataset.scala:2828)
        at org.apache.spark.sql.Dataset.collectToPython(Dataset.scala:2805)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:280)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:214)
        at java.base/java.lang.Thread.run(Unknown Source)
</code>```

答:
The reason why this works fine:

spark.sql("select count(*) from orc").show()

is because .show() only works on the first 5 rows of your data

but when you run:

spark.sql("select count(*) from orc").collect()

.collect() will work on all your data

from your error message:

WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

According to the spark documentation:

The performance overhead of creating and logging strings for wide schemas can be large. To limit the impact, we bound the number of fields to include by default. This can be overriden by setting the 'spark.debug.maxToStringFields' conf in SparkEnv.

Howeevr, it might affect the performance of your job, so you need something like:

```<code>spark = SparkSession
.builder
.master('local[*]')
.appName('Notebook')
.config('spark.sql.debug.maxToStringFields', 200)
.getOrCreate()
</code>```

This is a new SQL config spark.sql.debug.maxToStringFields that controls maximum number fields up to which truncatedString cuts its input sequences.

Default value is: DEFAULT_MAX_TO_STRING_FIELDS = 25

You can also add spark.sql.debug.maxToStringFields=100 to spark-defaults.conf

Previous versions of Spark used spark.debug.maxToStringFields instead of spark.sql.debug.maxToStringFields

Body:
I have a very simple pyspark program that uses dataframe to query data from a group of ORC files. I am using anaconda python in windows and installed pyspark on it.

The program goes like this:

from pyspark.sql import SparkSession

spark_session = SparkSession.builder.appName("test").getOrCreate()
df_orc = spark_session .read.orc("./raw_data/")
df_orc.createOrReplaceTempView("orc")

This works fine:

spark.sql("select count(*) from orc").show()

But this will generate errors:

spark.sql("select count(*) from orc").collect()

The error messages are:

WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

Py4JJavaError: An error occurred while calling o81.collectToPython.
: java.lang.IllegalArgumentException
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.<init>(Unknown Source)
        at org.apache.spark.util.ClosureCleaner$.getClassReader(ClosureCleaner.scala:46)
        at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.sca
        at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.sca
        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
        at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:103)
        at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:103)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
        at scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:103)
        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
        at org.apache.spark.util.FieldAccessFinder$$anon$3.visitMethodInsn(ClosureCleaner.scala:426)
        at org.apache.xbean.asm5.ClassReader.a(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.b(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.accept(Unknown Source)
        at org.apache.xbean.asm5.ClassReader.accept(Unknown Source)
        at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(
        at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(
        at scala.collection.immutable.List.foreach(List.scala:381)
        at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.sc
        at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:156)
        at org.apache.spark.SparkContext.clean(SparkContext.scala:2294)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2068)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2094)
        at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:936)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
        at org.apache.spark.rdd.RDD.collect(RDD.scala:935)
        at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:278)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply$mcI$sp(Dataset.scala:2808)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply(Dataset.scala:2805)
        at org.apache.spark.sql.Dataset$$anonfun$collectToPython$1.apply(Dataset.scala:2805)
        at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
        at org.apache.spark.sql.Dataset.withNewExecutionId(Dataset.scala:2828)
        at org.apache.spark.sql.Dataset.collectToPython(Dataset.scala:2805)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:280)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:214)
        at java.base/java.lang.Thread.run(Unknown Source)

Body:
The reason why this works fine:

spark.sql("select count(*) from orc").show()

is because .show() only works on the first 5 rows of your data

but when you run:

spark.sql("select count(*) from orc").collect()

.collect() will work on all your data

from your error message:

WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

According to the spark documentation:

The performance overhead of creating and logging strings for wide schemas can be large. To limit the impact, we bound the number of fields to include by default. This can be overriden by setting the 'spark.debug.maxToStringFields' conf in SparkEnv.

Howeevr, it might affect the performance of your job, so you need something like:

spark = SparkSession
.builder
.master('local[*]')
.appName('Notebook')
.config('spark.sql.debug.maxToStringFields', 200)
.getOrCreate()

This is a new SQL config spark.sql.debug.maxToStringFields that controls maximum number fields up to which truncatedString cuts its input sequences.

Default value is: DEFAULT_MAX_TO_STRING_FIELDS = 25

You can also add spark.sql.debug.maxToStringFields=100 to spark-defaults.conf

Previous versions of Spark used spark.debug.maxToStringFields instead of spark.sql.debug.maxToStringFields


问:
Getting error while converting json to c#/proto
I am trying to convert my Json file data back to proto and c# object.
I added a proto file and built it and it auto generated my cs file from proto file.
So, I have a Json now which I serialized. Now, I want to read that json and cnvert it back to my proto file and c# object.

But I get an error saying

```<code> JsonParser jsonParser1 = new(JsonParser.Settings.Default);
 MessageDescriptor messageBuilder = MyProtoClass.Descriptor;
 _ = jsonParser1.Parse("myJsonFileData", messageBuilder);
</code>```

Here the error happens at the last line

```<code>'jsonParser1.Parse("myJsonFileData", messageBuilder)' threw an exception of type 'Google.Protobuf.InvalidProtocolBufferException'
    Data: {System.Collections.ListDictionaryInternal}
</code>```

答:
I have worked in C# in the passed, but doing Java mostly now. When programming C# I found that comparing C# and Java was always helpful to me.  I think you'd want a builder at this line, not a descriptor:

```<code>MessageDescriptor messageBuilder = MyProtoClass.Descriptor;
</code>```

Here's a snippet of Java for going from json to ProtoBuf that I know works.

```<code> Builder builder2 = SimpleMessage.newBuilder();
 JsonFormat.parser().ignoringUnknownFields().merge(json, builder2);
 System.out.println("Simple=" + builder2.build());
</code>```

Body:
I am trying to convert my Json file data back to proto and c# object.
I added a proto file and built it and it auto generated my cs file from proto file.
So, I have a Json now which I serialized. Now, I want to read that json and cnvert it back to my proto file and c# object.

But I get an error saying

 JsonParser jsonParser1 = new(JsonParser.Settings.Default);
 MessageDescriptor messageBuilder = MyProtoClass.Descriptor;
 _ = jsonParser1.Parse("myJsonFileData", messageBuilder);

Here the error happens at the last line

'jsonParser1.Parse("myJsonFileData", messageBuilder)' threw an exception of type 'Google.Protobuf.InvalidProtocolBufferException'
    Data: {System.Collections.ListDictionaryInternal}

Body:
I have worked in C# in the passed, but doing Java mostly now. When programming C# I found that comparing C# and Java was always helpful to me.  I think you'd want a builder at this line, not a descriptor:

MessageDescriptor messageBuilder = MyProtoClass.Descriptor;

Here's a snippet of Java for going from json to ProtoBuf that I know works.

 Builder builder2 = SimpleMessage.newBuilder();
 JsonFormat.parser().ignoringUnknownFields().merge(json, builder2);
 System.out.println("Simple=" + builder2.build());


问:
Dockerfile specifying path of API
I'm using docker for the first time. I have a web application in angular and a backend application in python/flask. After some struggle, I finally managed to get everything to work, but docker can't find my API program in my backend:

My server file is at /my_backend/src/server.py

My docker file is at /my_backend/Dockerfile.dockerfile

Content of Dockerfile.dockerfile:

```<code>FROM python:3.7-slim
COPY /src .
WORKDIR /src/app
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]
</code>```

The error message in the command prompt is

```<code>Attaching to backend, frontend
backend   | python: can't open file 'server.py': [Errno 2] No such file or directory
backend exited with code 2
</code>```

Feel free to ask for more information.

I used this tutorial: https://medium.com/analytics-vidhya/dockerizing-angular-application-and-python-backend-bottle-flask-framework-21dcbf8715e5

答:
If your script is in /src directory, don't use WORKDIR /src/app but WORKDIR /src or CMD ["python","../server.py"]

Solution 1:

```<code>FROM python:3.7-slim
COPY /src .
WORKDIR /src    # <- HERE
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]
</code>```

Solution 2:

```<code>FROM python:3.7-slim
COPY /src .
WORKDIR /src/app
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","../server.py"]     # <- HERE, or "/src/server.py"
</code>```

Suggested by @TheFool:

```<code>FROM python:3.7-slim
WORKDIR /src/app  # Swap WORKDIR and COPY
COPY /src .
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]
</code>```

Title: Dockerfile specifying path of API

Body:
I'm using docker for the first time. I have a web application in angular and a backend application in python/flask. After some struggle, I finally managed to get everything to work, but docker can't find my API program in my backend:

My server file is at /my_backend/src/server.py

My docker file is at /my_backend/Dockerfile.dockerfile

Content of Dockerfile.dockerfile:

FROM python:3.7-slim
COPY /src .
WORKDIR /src/app
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]

The error message in the command prompt is

Attaching to backend, frontend
backend   | python: can't open file 'server.py': [Errno 2] No such file or directory
backend exited with code 2

Feel free to ask for more information.

I used this tutorial: https://medium.com/analytics-vidhya/dockerizing-angular-application-and-python-backend-bottle-flask-framework-21dcbf8715e5

Body:
If your script is in /src directory, don't use WORKDIR /src/app but WORKDIR /src or CMD ["python","../server.py"]

Solution 1:

FROM python:3.7-slim
COPY /src .
WORKDIR /src    # <- HERE
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]

Solution 2:

FROM python:3.7-slim
COPY /src .
WORKDIR /src/app
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","../server.py"]     # <- HERE, or "/src/server.py"

Suggested by @TheFool:

FROM python:3.7-slim
WORKDIR /src/app  # Swap WORKDIR and COPY
COPY /src .
ENV PYTHONUNBUFFERED 1
COPY requirements.txt /tmp/requirements.txt
RUN python3 -m pip install -r /tmp/requirements.txt
CMD ["python","server.py"]


问:
How to remove and add class on scroll JavaScript

const scrollList = document.getElementsByClassName('scrollList');

function scrollLeft() {
    scrollList.scrollLeft -= 50
}

function scrollRight() {
    scrollList.scrollLeft += 50
}```
<pre class="snippet-code-css lang-css prettyprint-override"><code>#scrollList {
  display: flex;
  overflow: auto;
  width: 100px;
}

.div {
  display: flex;
}</code>```

  <button onclick="scrollLeft()">Scroll Left</button>

Image

Image

Image

Image

Image

Image

<button  onclick="scrollRight()">Scroll Right</button>

```

I have a Horizontal page scroll on div with arrows and I want to hide the left and right arrow when you can't scroll more. I have an gallery so when there is no more images to scroll to I want to hide the button(arrow). So in the beginning I dont want to show the left arrow, and in the end when there is no more images I want to hide the right arrow. How can I do that with javascript?
Thanks for your help!

答:
You can use window.scroll() function in JavaScript
Let xScroll=window.scrollX
Its return float value and then use conditions to show arrow
" ClassName={xScroll<70 ? 'hide' : 'show'}</p>

Body:
const scrollList = document.getElementsByClassName('scrollList');

function scrollLeft() {
    scrollList.scrollLeft -= 50
}

function scrollRight() {
    scrollList.scrollLeft += 50
}

#scrollList {
  display: flex;
  overflow: auto;
  width: 100px;
}

.div {
  display: flex;
}

  <button onclick="scrollLeft()">Scroll Left</button>

Image

Image

Image

Image

Image

Image

<button  onclick="scrollRight()">Scroll Right</button>

I have a Horizontal page scroll on div with arrows and I want to hide the left and right arrow when you can't scroll more. I have an gallery so when there is no more images to scroll to I want to hide the button(arrow). So in the beginning I dont want to show the left arrow, and in the end when there is no more images I want to hide the right arrow. How can I do that with javascript?
Thanks for your help!


问:
Recommended aplications for software development
In a business context, which applications do you recomend for:

Version Control

Time Management

Knowledge Base

Compared with the possible solutions, what are their main advantages?

答:
Maybe this question is more suited for:

https://softwareengineering.stackexchange.com/


问:
AttributeError: 'DataFrame' object has no attribute 'operation'
I'm trying to write a function that returns the mean, median, standard deviation, max and min values of a dataset.

Below is my code: it should iterate through every item in the "ops" array and pass them into the "statSummary" function.

The function initalises a list as the operation type (e.g mean/median), performs the operation (calculates mean) and prints the result.

```<code>def statSummary(dataset,operation):
    operation = []
    operation.append(df[dataset].operation())

    return print(operation)

#load dataset
df = pd.read_csv('dataset.csv')

columnNames = list(df.columns)

ops = ["mean","median","std","max","min"]

for i in ops:
    statSummary(columnNames,i)
</code>```

However when trying to run this I recieve the error:

```<code>AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_8416/4291564190.py in <module>
     30
     31 for i in ops:
---> 32     statSummary(columnNames,i)
     33
     34

~\AppData\Local\Temp/ipykernel_8416/4291564190.py in statSummary(dataset, operation)
     18 def statSummary(dataset,operation):
     19     operation = []
---> 20     operation.append(df[dataset].operation())
     21
     22     return print(operation)

~\anaconda3\lib\site-packages\pandas\core\generic.py in __getattr__(self, name)
   5485         ):
   5486             return self[name]
-> 5487         return object.__getattribute__(self, name)
   5488
   5489     def __setattr__(self, name: str, value) -> None:

AttributeError: 'DataFrame' object has no attribute 'operation'
</code>```

The error seems to be due to ".operation())" on this line

```<code>operation.append(df[dataset].operation())
</code>```

I want to 'plug' the operation variable into this line so I can iterate through my 'ops' list using "'mean','median',std'" etc.

Is this possible?

答:
If you just want to print out these things you can simply use df.describe().
Or you can use apply filter method using numpy. I have given sample example of this. Else you can use getattr as answered by @AKX</p>
```<code>import pandas as pd
import numpy as np

df = pd.DataFrame({
    'one':np.array([1,2,3,4,5]),
    'two':np.array([2,4,6,8,10]),
})

def statSummary(dataset,ops):
    operation = []
    operation.append(df[dataset].apply(ops))
    return print(operation)

columnNames = list(df.columns)

ops = [np.mean,np.median,np.std,np.max,np.min]

for i in ops:
    statSummary(columnNames,i)
</code>```

Body:
I'm trying to write a function that returns the mean, median, standard deviation, max and min values of a dataset.

Below is my code: it should iterate through every item in the "ops" array and pass them into the "statSummary" function.

The function initalises a list as the operation type (e.g mean/median), performs the operation (calculates mean) and prints the result.

def statSummary(dataset,operation):
    operation = []
    operation.append(df[dataset].operation())

    return print(operation)

#load dataset
df = pd.read_csv('dataset.csv')

columnNames = list(df.columns)

ops = ["mean","median","std","max","min"]

for i in ops:
    statSummary(columnNames,i)

However when trying to run this I recieve the error:

AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_8416/4291564190.py in <module>
     30
     31 for i in ops:
---> 32     statSummary(columnNames,i)
     33
     34

~\AppData\Local\Temp/ipykernel_8416/4291564190.py in statSummary(dataset, operation)
     18 def statSummary(dataset,operation):
     19     operation = []
---> 20     operation.append(df[dataset].operation())
     21
     22     return print(operation)

~\anaconda3\lib\site-packages\pandas\core\generic.py in __getattr__(self, name)
   5485         ):
   5486             return self[name]
-> 5487         return object.__getattribute__(self, name)
   5488
   5489     def __setattr__(self, name: str, value) -> None:

AttributeError: 'DataFrame' object has no attribute 'operation'

The error seems to be due to ".operation())" on this line

operation.append(df[dataset].operation())

I want to 'plug' the operation variable into this line so I can iterate through my 'ops' list using "'mean','median',std'" etc.

Is this possible?

Body:
If you just want to print out these things you can simply use df.describe().
Or you can use apply filter method using numpy. I have given sample example of this. Else you can use getattr as answered by @AKX

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'one':np.array([1,2,3,4,5]),
    'two':np.array([2,4,6,8,10]),
})

def statSummary(dataset,ops):
    operation = []
    operation.append(df[dataset].apply(ops))
    return print(operation)

columnNames = list(df.columns)

ops = [np.mean,np.median,np.std,np.max,np.min]

for i in ops:
    statSummary(columnNames,i)


问:
Bringing a DIV upwards so it sits behind another DIV
I'm confused here... Here's my site that I'm working on: http://s361608839.websitehome.co.uk/marbleenergy/

The div #main is sitting about 10px below #navigation and I've tried bringing it up 10px by adding:

```<code> #main {
 margin-top: -10px;
 }
</code>```

Had no luck there unfortunately. I'm learning CSS here, what is it I need to do?

Thanks

答:
USE

#main {

position relative;
   margin-top:-10px;

}

See Demo: http://jsfiddle.net/rathoreahsan/fSDpJ/

I browse your website in your case you need to use the following css:

```<code>#main {

    position absolute;
    margin:-10px 0 0 12px;
}
</code>```

OR

```<code>#main {

    position relative;
    margin:0 0 0 12px;
    top: -10px;
}
</code>```

Body:
I'm confused here... Here's my site that I'm working on: http://s361608839.websitehome.co.uk/marbleenergy/

The div #main is sitting about 10px below #navigation and I've tried bringing it up 10px by adding:

 #main {
 margin-top: -10px;
 }

Had no luck there unfortunately. I'm learning CSS here, what is it I need to do?

Thanks

Body:
USE

#main {

position relative;
   margin-top:-10px;

}

See Demo: http://jsfiddle.net/rathoreahsan/fSDpJ/

I browse your website in your case you need to use the following css:

#main {

    position absolute;
    margin:-10px 0 0 12px;
}

OR

#main {

    position relative;
    margin:0 0 0 12px;
    top: -10px;
}


问:
How do you delete an embedded document using Mongoengine?
I have a collection that has an EmbeddedDocumentField.  I'm having trouble finding examples of how to delete an embedded document from the collection.  Can someone provide me with an example or reference to one?

Here is what my setup looks like:

Python 2.7.5

Django 1.5.5

MongoEngine 0.8.7

Code:

```<code>class Merchant(Document):
    merchant_id = StringField(max_length=50)
    merchant_name = StringField(max_length=150)
    merchant_name_search_alias = StringField(max_length=150)
    website = StringField(max_length=150)
    location = ListField(EmbeddedDocumentField(Location))
    address = StringField(max_length=50)
    city = StringField(max_length=30)
    state = StringField(max_length=20)
    zipcode = IntField()
    phone_nummber = StringField(max_length=10)
    sub_lat = FloatField()
    sub_lng = FloatField()
    country = StringField(max_length=20)
    promotion = ListField(EmbeddedDocumentField(Promotion))

class Promotion(EmbeddedDocument):
    provider_name = StringField(max_length=50)
    provider_website = URLField()
    promo_name = StringField(max_length=300)
    promo_name_search_alias = StringField(max_length=100)
    retail_price = DecimalField(precision=2, force_string=True)
    discount_price = DecimalField(precision=2, force_string=True)
    deal_url = URLField()
    buy_url = URLField()
    deal_image_url = URLField()
    description = StringField(max_length=1000)
    start_at = DateTimeField()
    end_at = DateTimeField()
    category = StringField(max_length=50)
    dq_category = StringField(max_length=50)
    keywords = StringField(max_length=100)
</code>```

答:
```<code>user = User_.objects(username=username).get()
for ad in user.ads:
    if str(ad["_id"]) == id:
        user.ads.remove(ad)
user.save()
</code>```

this is the only way i found to remove a value from an array of embeded doc

Body:
I have a collection that has an EmbeddedDocumentField.  I'm having trouble finding examples of how to delete an embedded document from the collection.  Can someone provide me with an example or reference to one?

Here is what my setup looks like:

Python 2.7.5

Django 1.5.5

MongoEngine 0.8.7

Code:

class Merchant(Document):
    merchant_id = StringField(max_length=50)
    merchant_name = StringField(max_length=150)
    merchant_name_search_alias = StringField(max_length=150)
    website = StringField(max_length=150)
    location = ListField(EmbeddedDocumentField(Location))
    address = StringField(max_length=50)
    city = StringField(max_length=30)
    state = StringField(max_length=20)
    zipcode = IntField()
    phone_nummber = StringField(max_length=10)
    sub_lat = FloatField()
    sub_lng = FloatField()
    country = StringField(max_length=20)
    promotion = ListField(EmbeddedDocumentField(Promotion))

class Promotion(EmbeddedDocument):
    provider_name = StringField(max_length=50)
    provider_website = URLField()
    promo_name = StringField(max_length=300)
    promo_name_search_alias = StringField(max_length=100)
    retail_price = DecimalField(precision=2, force_string=True)
    discount_price = DecimalField(precision=2, force_string=True)
    deal_url = URLField()
    buy_url = URLField()
    deal_image_url = URLField()
    description = StringField(max_length=1000)
    start_at = DateTimeField()
    end_at = DateTimeField()
    category = StringField(max_length=50)
    dq_category = StringField(max_length=50)
    keywords = StringField(max_length=100)

Body:
user = User_.objects(username=username).get()
for ad in user.ads:
    if str(ad["_id"]) == id:
        user.ads.remove(ad)
user.save()

this is the only way i found to remove a value from an array of embeded doc


问:
Greater than ( >) with ANY equivalent in sqlite
I'm trying to find the id of the customers whose income is greater than any customer who is 22 years old. I have come up with this query which seems correct to me, but it doesn't work in SQLite (I get 'near "SELECT": syntax error'). What is an equivalent query that would work in SQLite?</p>
```<code>SELECT id
FROM Customers
WHERE income > ANY (
  SELECT C.income
  FROM Customers C
  WHERE C.age = 22
)
</code>```

答:
Logically the equivalent condition would be to get the customers whose income is greater than the min income of the customers with age = 22:

```<code>SELECT id
FROM Customers
WHERE income > (SELECT MIN(income) FROM Customers WHERE age = 22);
</code>```

If, in case there are no customers with age = 22, you want your query to return all customers with a salary greater than 0, use also COALESCE():

```<code>SELECT id
FROM Customers
WHERE income > (SELECT COALESCE(MIN(income), 0) FROM Customers WHERE age = 22);
</code>```

Body:
I'm trying to find the id of the customers whose income is greater than any customer who is 22 years old. I have come up with this query which seems correct to me, but it doesn't work in SQLite (I get 'near "SELECT": syntax error'). What is an equivalent query that would work in SQLite?

SELECT id
FROM Customers
WHERE income > ANY (
  SELECT C.income
  FROM Customers C
  WHERE C.age = 22
)

Body:
Logically the equivalent condition would be to get the customers whose income is greater than the min income of the customers with age = 22:

SELECT id
FROM Customers
WHERE income > (SELECT MIN(income) FROM Customers WHERE age = 22);

If, in case there are no customers with age = 22, you want your query to return all customers with a salary greater than 0, use also COALESCE():

SELECT id
FROM Customers
WHERE income > (SELECT COALESCE(MIN(income), 0) FROM Customers WHERE age = 22);


问:
How to remove "chrome is being controlled by automated test software" selenium javascript
Is there anyway to remove the "chrome is being controlled by automated test software" notification in selenium JavaScript. Already tried disable info-bar but turns out chrome patched that couple years back.</p>

答:
For C# :</p>
```<code>var options = new ChromeOptions();
options.AddExcludedArgument("enable-automation");
</code>```

Body:
For C# :

var options = new ChromeOptions();
options.AddExcludedArgument("enable-automation");


问:
Return the rows which are not shipped
I need to return rows which are not shipped based on the request.

if the request comes like i need 3 layers then i need to check shipping status and return layers which are not shipped based on type. Here  types 'o' are total layers and 's' are already shipped

output
1003
1004
1001

答:
You can use an anti-join. For example:

```<code>select a.*
from mytable a
left join mytable b on b.id = a.id and b.type = 's'
where a.type = 'o' and b.type is null
</code>```

Result:

```<code> id  layer  type
 --- ------ ----
 1   1001   o
 2   1002   o
 3   1003   o
</code>```

See running example at db<>fiddle.

Title: Return the rows which are not shipped

Body:
You can use an anti-join. For example:

select a.*
from mytable a
left join mytable b on b.id = a.id and b.type = 's'
where a.type = 'o' and b.type is null

Result:

 id  layer  type
 --- ------ ----
 1   1001   o
 2   1002   o
 3   1003   o

See running example at db<>fiddle.


问:
Test functionality with timeouts in react-testing-library and jest
I'm trying to follow TDD, and I have a span that should appear on screen after 5 seconds. I haven't implemented the span at all, so the test should fail, but currently it passes the test expect(messageSpan).toBeInTheDocument.

Here are my two tests:

```<code>it("doesn't show cta message at first", () => {
    render(<FAB />);
    const messageSpan = screen.queryByText(
      "Considering a career in nursing? Join our team!"
    );
    expect(messageSpan).toBeNull(); // passes, as it should
  });

  it("should show the cta message after 5 secs", () => {
    render(<FAB />);
    setTimeout(() => {
      const messageSpan = screen.getByText( // also tried queryByText instead of get
        "Considering a career in nursing? Join our team!"
      );
      expect(messageSpan).toBeInTheDocument(); // also passes, even though messageSpan should throw an error.
    }, 5000);
  });
</code>```

Here's my FAB component, where you can see there's no message at all:

```<code>export default function FAB() {
  return (
// using styled-components; there's no content in any of these.
    <StyledFABContainer>
      <StyledFABButton>
        <BriefcaseIcon />
      </StyledFABButton>
    </StyledFABContainer>
  );
}
</code>```

To complicate things, I don't plan to have a set function I call for the setTimeout. I will simply set state after a set time of 5 secs. So don't think I can use the suggestions in the timer mocks section of jest docs: https://jestjs.io/docs/timer-mocks

My two questions are:

a) Why would this pass and not throw an error/null?
b) How can I properly test setTimeout functionality in RTL?

UPDATE: Have tried using various combinations of useFakeTimers, act, waitFor etc., but no luck. Here's my current test as it's written out, and throwing two errors - one saying I need to use act when changing state (which I am, but still) and one saying my messageSpan is null:

it("cta message to display after 5 secs", async () => {
    jest.useFakeTimers();
    const el = document.createElement("div");
    act(() => {
      ReactDOM.render(<FAB />, el);
    });
    jest.advanceTimersByTime(5000);
    const messageSpan = screen.queryByText(
      "Considering a career in nursing? Join our team!"
    );
    expect(messageSpan).toBeInTheDocument();
  });
```

答:
You can not use setTimeout like this in your tests. An obvious reason would be that you do not want to wait 5 seconds in your test to then continue. Imagine a component that would change after 10min. You cant wait that long but should use jests mocked timers API instead.
You can progress the nodejs timer so your component changes, then immediately make your assertion.
Other remarks about the tests you wrote:

If you are awaiting changes you should use await waitFor(() => ...) from testing library. It checks the expectations to pass every 50ms by default for a total of 5sec. If all expectations pass it continues. You should make asynchronous assertions there.

"expect(messageSpan).toBeNull()" should be "expect(messageSpan).not.toBeInTheDocument()". Its good to go with the accessibility way testing library provides.

Body:
I'm trying to follow TDD, and I have a span that should appear on screen after 5 seconds. I haven't implemented the span at all, so the test should fail, but currently it passes the test expect(messageSpan).toBeInTheDocument.

Here are my two tests:

it("doesn't show cta message at first", () => {
    render(<FAB />);
    const messageSpan = screen.queryByText(
      "Considering a career in nursing? Join our team!"
    );
    expect(messageSpan).toBeNull(); // passes, as it should
  });

  it("should show the cta message after 5 secs", () => {
    render(<FAB />);
    setTimeout(() => {
      const messageSpan = screen.getByText( // also tried queryByText instead of get
        "Considering a career in nursing? Join our team!"
      );
      expect(messageSpan).toBeInTheDocument(); // also passes, even though messageSpan should throw an error.
    }, 5000);
  });

Here's my FAB component, where you can see there's no message at all:

export default function FAB() {
  return (
// using styled-components; there's no content in any of these.

    </StyledFABContainer>
  );
}

To complicate things, I don't plan to have a set function I call for the setTimeout. I will simply set state after a set time of 5 secs. So don't think I can use the suggestions in the timer mocks section of jest docs: https://jestjs.io/docs/timer-mocks

My two questions are:

a) Why would this pass and not throw an error/null?
b) How can I properly test setTimeout functionality in RTL?

UPDATE: Have tried using various combinations of useFakeTimers, act, waitFor etc., but no luck. Here's my current test as it's written out, and throwing two errors - one saying I need to use act when changing state (which I am, but still) and one saying my messageSpan is null:

it("cta message to display after 5 secs", async () => {
    jest.useFakeTimers();
    const el = document.createElement("div");
    act(() => {
      ReactDOM.render(<FAB />, el);
    });
    jest.advanceTimersByTime(5000);
    const messageSpan = screen.queryByText(
      "Considering a career in nursing? Join our team!"
    );
    expect(messageSpan).toBeInTheDocument();
  });


问:
Need to restart server again and again to make mui styles to work with nextjs
I have followed the examples of Nextjs setup with MUI. And the styles are working for me but when I use makeStyles to override MUI CSS I need to restart the server again and again to see changes with the CSS iam overriding. How do I solve it?</p>

答:
thats because you are using material-ui v5 and using makeStyle
makeStyle is more compatible with v4
for nextjs you can use v4 or their boiler plate on v5 here :
https://github.com/mui/material-ui/tree/master/examples/nextjs
and for styling use sx props instead of makeStyle (check docs in mui.com)</p>


问:
JAVA_HOME directory in Linux
Is there any linux command I could use to find out JAVA_HOME directory? I've tried print out the environment variables ("env") but I can't find the directory.

答:
You can check from the command line by executing this command echo $JAVA_HOME. If Java is installed but the path is not set, you need to identify the path to your java installation. I prefer using sudo update-alternatives --config java which lists all installed versions with current active one marked and provides dialog to switch:

 ```There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.14.0.9-2.fc35.x86_64/bin/java)
   2           java-17-openjdk.x86_64 (/usr/lib/jvm/java-17-openjdk-17.0.2.0.8-1.fc35.x86_64/bin/java)
*+ 3           /usr/java/jdk-17.0.2/bin/java

Enter to keep the current selection[+], or type selection number:
```

from the above list, you can select the version of java you want to be the default. To set the JAVA_HOME to option 3 for instance you can do it this way export JAVA_HOME=/usr/java/jdk-17.0.2

Title: JAVA_HOME directory in Linux

Body:
You can check from the command line by executing this command echo $JAVA_HOME. If Java is installed but the path is not set, you need to identify the path to your java installation. I prefer using sudo update-alternatives --config java which lists all installed versions with current active one marked and provides dialog to switch:

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.14.0.9-2.fc35.x86_64/bin/java)
   2           java-17-openjdk.x86_64 (/usr/lib/jvm/java-17-openjdk-17.0.2.0.8-1.fc35.x86_64/bin/java)
*+ 3           /usr/java/jdk-17.0.2/bin/java

Enter to keep the current selection[+], or type selection number:

from the above list, you can select the version of java you want to be the default. To set the JAVA_HOME to option 3 for instance you can do it this way export JAVA_HOME=/usr/java/jdk-17.0.2


问:
if statement does not work - script doesn't stop when skillpoints are below 0
I have if statement that doesn't work.

I want the script to stop working when the skillpoints are 0 or below.

My code:

```<code>var skillpoints = 5;

        if (skillpoints > 0) {

            // Dodawanie skillpointów

            $('.str .stat-plus img').click(function() {
                var val = $('.str-img div').text();
                var new_val = parseInt($('.str-img div').text(),10) + 1 ;
                $('.str-img div').text(new_val);
                $('.str input').val(new_val);
                skillpoints--;
                $('.skillpoints').text(skillpoints);
                $('.register-skillpoints').val(skillpoints);
            });
    }

</code>```

答:
You have to put the if condition into your click handler, like this:

$('.str .stat-plus img').click(function() {
  if (skillpoints > 0) {
    // etc
  }
}
```

Otherwise, the if condition is checked only once at page load, and never again when clicking.

Body:
I have if statement that doesn't work.

I want the script to stop working when the skillpoints are 0 or below.

My code:

var skillpoints = 5;

        if (skillpoints > 0) {

            // Dodawanie skillpointów

            $('.str .stat-plus img').click(function() {
                var val = $('.str-img div').text();
                var new_val = parseInt($('.str-img div').text(),10) + 1 ;
                $('.str-img div').text(new_val);
                $('.str input').val(new_val);
                skillpoints--;
                $('.skillpoints').text(skillpoints);
                $('.register-skillpoints').val(skillpoints);
            });
    }

Body:
You have to put the if condition into your click handler, like this:

$('.str .stat-plus img').click(function() {
  if (skillpoints > 0) {
    // etc
  }
}

Otherwise, the if condition is checked only once at page load, and never again when clicking.


问:
How to call Angular code server-side, from a .Net Core Service
This is proving to be a difficult thing to research because everyone's first response to similar questions is, "what the heck are you doing?".  Here's exactly what we are trying to do...

We are developing a .Net Core service that will trigger some workflow when a certain Kafka message shows up.  It will go grab some data, then it will need to generate some html from that data, then it will convert that html to a PDF and save that on a file server.  We don't need a browser, at least not a visible one, and we certainly won't have a user.  This should all happen server-side.

The kicker is that we have Angular code, and some hefty third-party modules along with it, that handle the "generate some html" step perfectly.  It would be very elegant to be able to re-use it.  We just need a black box to accept data and spit out html.  It would be a serious pain to have to write that "generate html" black box in C#.

I'm about to start researching Angular Universal and server-side rendering, but I wanted to see if anyone here has done anything like this and can save us some time with the benefit of your experience.  Of particular interest is how to trigger the Angular functionality from .Net.

答:
I think the only thing you need to do is setup SSR indeed

.NET Core 3.1 guide

.NET 6 guide

Then all you have to do is call the angular component url with a query parameter, parse this query parameter, supply it to the ClientApp, inject the value in the component, and render it. Then you can already generate the HTML. You'll still need to covert this html to a pdf though.

Another option (more certain) is to create a razor view to render the html, but then the "other javascript modules" won't be executed...