问:
JQUERY AJAX find filename in response data
Hello im trying to load all file names from a folder.

im using this script and it returns a response object called data. When i use console.log on data it returns me to html below. how can i now collect all the file names out of there and write it into the variable fileNames?

my script:

```<code>var fileNames= "";
var folder = "csvData/";
    $.ajax({
          type: 'GET',
          url:  folder,

          success: function(data){
              console.log(data);
          },
          async:false
      });
</code>```

html inside of data response:
a the bottom you can see that a function is called for each file in the folder.

```<code><!DOCTYPE html>

<html dir="ltr" lang="de">

<head>
<meta charset="utf-8">
<meta name="color-scheme" content="light dark">
<meta name="google" value="notranslate">

<script>
function addRow(name, url, isdir,
    size, size_string, date_modified, date_modified_string) {
  if (name == "." || name == "..")
    return;

  var root = document.location.pathname;
  if (root.substr(-1) !== "/")
    root += "/";

  var tbody = document.getElementById("tbody");
  var row = document.createElement("tr");
  var file_cell = document.createElement("td");
  var link = document.createElement("a");

  link.className = isdir ? "icon dir" : "icon file";

  if (isdir) {
    name = name + "/";
    url = url + "/";
    size = 0;
    size_string = "";
  } else {
    link.draggable = "true";
    link.addEventListener("dragstart", onDragStart, false);
  }
  link.innerText = name;
  link.href = root + url;

  file_cell.dataset.value = name;
  file_cell.appendChild(link);

  row.appendChild(file_cell);
  row.appendChild(createCell(size, size_string));
  row.appendChild(createCell(date_modified, date_modified_string));

  tbody.appendChild(row);
}

function onDragStart(e) {
  var el = e.srcElement;
  var name = el.innerText.replace(":", "");
  var download_url_data = "application/octet-stream:" + name + ":" + el.href;
  e.dataTransfer.setData("DownloadURL", download_url_data);
  e.dataTransfer.effectAllowed = "copy";
}

function createCell(value, text) {
  var cell = document.createElement("td");
  cell.setAttribute("class", "detailsColumn");
  cell.dataset.value = value;
  cell.innerText = text;
  return cell;
}

function start(location) {
  var header = document.getElementById("header");
  header.innerText = header.innerText.replace("LOCATION", location);

  document.getElementById("title").innerText = header.innerText;
}

function onHasParentDirectory() {
  var box = document.getElementById("parentDirLinkBox");
  box.style.display = "block";

  var root = document.location.pathname;
  if (!root.endsWith("/"))
    root += "/";

  var link = document.getElementById("parentDirLink");
  link.href = root + "..";
}

function sortTable(column) {
  var theader = document.getElementById("theader");
  var oldOrder = theader.cells[column].dataset.order || '1';
  oldOrder = parseInt(oldOrder, 10)
  var newOrder = 0 - oldOrder;
  theader.cells[column].dataset.order = newOrder;

  var tbody = document.getElementById("tbody");
  var rows = tbody.rows;
  var list = [], i;
  for (i = 0; i < rows.length; i++) {
    list.push(rows[i]);
  }

  list.sort(function(row1, row2) {
    var a = row1.cells[column].dataset.value;
    var b = row2.cells[column].dataset.value;
    if (column) {
      a = parseInt(a, 10);
      b = parseInt(b, 10);
      return a > b ? newOrder : a < b ? oldOrder : 0;
    }

    // Column 0 is text.
    if (a > b)
      return newOrder;
    if (a < b)
      return oldOrder;
    return 0;
  });

  // Appending an existing child again just moves it.
  for (i = 0; i < list.length; i++) {
    tbody.appendChild(list[i]);
  }
}

// Add event handlers to column headers.
function addHandlers(element, column) {
  element.onclick = (e) => sortTable(column);
  element.onkeydown = (e) => {
    if (e.key == 'Enter' || e.key == ' ') {
      sortTable(column);
      e.preventDefault();
    }
  };
}

function onLoad() {
  addHandlers(document.getElementById('nameColumnHeader'), 0);
  addHandlers(document.getElementById('sizeColumnHeader'), 1);
  addHandlers(document.getElementById('dateColumnHeader'), 2);
}

window.addEventListener('DOMContentLoaded', onLoad);
</script>

<style>

  h1 {
    border-bottom: 1px solid #c0c0c0;
    margin-bottom: 10px;
    padding-bottom: 10px;
    white-space: nowrap;
  }

  table {
    border-collapse: collapse;
  }

  th {
    cursor: pointer;
  }

  td.detailsColumn {
    padding-inline-start: 2em;
    text-align: end;
    white-space: nowrap;
  }

  a.icon {
    padding-inline-start: 1.5em;
    text-decoration: none;
    user-select: auto;
  }

  a.icon:hover {
    text-decoration: underline;
  }

  a.file {
    background : url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAIAAACQkWg2AAAABnRSTlMAAAAAAABupgeRAAABEElEQVR42nRRx3HDMBC846AHZ7sP54BmWAyrsP588qnwlhqw/k4v5ZwWxM1hzmGRgV1cYqrRarXoH2w2m6qqiqKIR6cPtzc3xMSML2Te7XZZlnW7Pe/91/dX47WRBHuA9oyGmRknzGDjab1ePzw8bLfb6WRalmW4ip9FDVpYSWZgOp12Oh3nXJ7nxoJSGEciteP9y+fH52q1euv38WosqA6T2gGOT44vry7BEQtJkMAMMpa6JagAMcUfWYa4hkkzAc7fFlSjwqCoOUYAF5RjHZPVCFBOtSBGfgUDji3c3jpibeEMQhIMh8NwshqyRsBJgvF4jMs/YlVR5KhgNpuBLzk0OcUiR3CMhcPaOzsZiAAA/AjmaB3WZIkAAAAASUVORK5CYII=") left top no-repeat;
  }

  a.dir {
    background : url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAABt0lEQVR42oxStZoWQRCs2cXdHTLcHZ6EjAwnQWIkJyQlRt4Cd3d3d1n5d7q7ju1zv/q+mh6taQsk8fn29kPDRo87SDMQcNAUJgIQkBjdAoRKdXjm2mOH0AqS+PlkP8sfp0h93iu/PDji9s2FzSSJVg5ykZqWgfGRr9rAAAQiDFoB1OfyESZEB7iAI0lHwLREQBcQQKqo8p+gNUCguwCNAAUQAcFOb0NNGjT+BbUC2YsHZpWLhC6/m0chqIoM1LKbQIIBwlTQE1xAo9QDGDPYf6rkTpPc92gCUYVJAZjhyZltJ95f3zuvLYRGWWCUNkDL2333McBh4kaLlxg+aTmyL7c2xTjkN4Bt7oE3DBP/3SRz65R/bkmBRPGzcRNHYuzMjaj+fdnaFoJUEdTSXfaHbe7XNnMPyqryPcmfY+zURaAB7SHk9cXSH4fQ5rojgCAVIuqCNWgRhLYLhJB4k3iZfIPtnQiCpjAzeBIRXMA6emAqoEbQSoDdGxFUrxS1AYcpaNbBgyQBGJEOnYOeENKR/iAd1npusI4C75/c3539+nbUjOgZV5CkAU27df40lH+agUdIuA/EAgDmZnwZlhDc0wAAAABJRU5ErkJggg==") left top no-repeat;
  }

  a.up {
    background : url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACM0lEQVR42myTA+w1RxRHz+zftmrbdlTbtq04qRGrCmvbDWp9tq3a7tPcub8mj9XZ3eHOGQdJAHw77/LbZuvnWy+c/CIAd+91CMf3bo+bgcBiBAGIZKXb19/zodsAkFT+3px+ssYfyHTQW5tr05dCOf3xN49KaVX9+2zy1dX4XMk+5JflN5MBPL30oVsvnvEyp+18Nt3ZAErQMSFOfelCFvw0HcUloDayljZkX+MmamTAMTe+d+ltZ+1wEaRAX/MAnkJdcujzZyErIiVSzCEvIiq4O83AG7LAkwsfIgAnbncag82jfPPdd9RQyhPkpNJvKJWQBKlYFmQA315n4YPNjwMAZYy0TgAweedLmLzTJSTLIxkWDaVCVfAbbiKjytgmm+EGpMBYW0WwwbZ7lL8anox/UxekaOW544HO0ANAshxuORT/RG5YSrjlwZ3lM955tlQqbtVMlWIhjwzkAVFB8Q9EAAA3AFJ+DR3DO/Pnd3NPi7H117rAzWjpEs8vfIqsGZpaweOfEAAFJKuM0v6kf2iC5pZ9+fmLSZfWBVaKfLLNOXj6lYY0V2lfyVCIsVzmcRV9Y0fx02eTaEwhl2PDrXcjFdYRAohQmS8QEFLCLKGYA0AeEakhCCFDXqxsE0AQACgAQp5w96o0lAXuNASeDKWIvADiHwigfBINpWKtAXJvCEKWgSJNbRvxf4SmrnKDpvZavePu1K/zu/due1X/6Nj90MBd/J2Cic7WjBp/jUdIuA8AUtd65M+PzXIAAAAASUVORK5CYII=") left top no-repeat;
  }

  html[dir=rtl] a {
    background-position-x: right;
  }

  #parentDirLinkBox {
    margin-bottom: 10px;
    padding-bottom: 10px;
  }
</style>

<title id="title"></title>

</head>

<body>

<h1 id="header">Index von LOCATION</h1>

<div id="parentDirLinkBox" style="display:none">
  <a id="parentDirLink" class="icon up">
    <span id="parentDirText">[übergeordnetes Verzeichnis]</span>
  </a>
</div>

<table>
  <thead>
    <tr class="header" id="theader">
      <th id="nameColumnHeader" tabindex=0 role="button">Name</th>
      <th id="sizeColumnHeader" class="detailsColumn" tabindex=0 role="button">
        Größe
      </th>
      <th id="dateColumnHeader" class="detailsColumn" tabindex=0 role="button">
        Änderungsdatum
      </th>
    </tr>
  </thead>
  <tbody id="tbody">
  </tbody>
</table>

</body>

</html>
<script>// Copyright (c) 2012 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

/**
 * @fileoverview This file defines a singleton which provides access to all data
 * that is available as soon as the page's resources are loaded (before DOM
 * content has finished loading). This data includes both localized strings and
 * any data that is important to have ready from a very early stage (e.g. things
 * that must be displayed right away).
 *
 * Note that loadTimeData is not guaranteed to be consistent between page
 * refreshes (https://crbug.com/740629) and should not contain values that might
 * change if the page is re-opened later.
 */

/** @type {!LoadTimeData} */
// eslint-disable-next-line no-var
var loadTimeData;

class LoadTimeData {
  constructor() {
    /** @type {?Object} */
    this.data_ = null;
  }

  /**
   * Sets the backing object.
   *
   * Note that there is no getter for |data_| to discourage abuse of the form:
   *
   *     var value = loadTimeData.data()['key'];
   *
   * @param {Object} value The de-serialized page data.
   */
  set data(value) {
    expect(!this.data_, 'Re-setting data.');
    this.data_ = value;
  }

  /**
   * @param {string} id An ID of a value that might exist.
   * @return {boolean} True if |id| is a key in the dictionary.
   */
  valueExists(id) {
    return id in this.data_;
  }

  /**
   * Fetches a value, expecting that it exists.
   * @param {string} id The key that identifies the desired value.
   * @return {*} The corresponding value.
   */
  getValue(id) {
    expect(this.data_, 'No data. Did you remember to include strings.js?');
    const value = this.data_[id];
    expect(typeof value !== 'undefined', 'Could not find value for ' + id);
    return value;
  }

  /**
   * As above, but also makes sure that the value is a string.
   * @param {string} id The key that identifies the desired string.
   * @return {string} The corresponding string value.
   */
  getString(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'string');
    return /** @type {string} */ (value);
  }

  /**
   * Returns a formatted localized string where $1 to $9 are replaced by the
   * second to the tenth argument.
   * @param {string} id The ID of the string we want.
   * @param {...(string|number)} var_args The extra values to include in the
   *     formatted output.
   * @return {string} The formatted string.
   */
  getStringF(id, var_args) {
    const value = this.getString(id);
    if (!value) {
      return '';
    }

    const args = Array.prototype.slice.call(arguments);
    args[0] = value;
    return this.substituteString.apply(this, args);
  }

  /**
   * Returns a formatted localized string where $1 to $9 are replaced by the
   * second to the tenth argument. Any standalone $ signs must be escaped as
   * $$.
   * @param {string} label The label to substitute through.
   *     This is not an resource ID.
   * @param {...(string|number)} var_args The extra values to include in the
   *     formatted output.
   * @return {string} The formatted string.
   */
  substituteString(label, var_args) {
    const varArgs = arguments;
    return label.replace(/\$(.|$|\n)/g, function(m) {
      expect(m.match(/\$[$1-9]/), 'Unescaped $ found in localized string.');
      return m === '$$' ? '$' : varArgs[m[1]];
    });
  }

  /**
   * Returns a formatted string where $1 to $9 are replaced by the second to
   * tenth argument, split apart into a list of pieces describing how the
   * substitution was performed. Any standalone $ signs must be escaped as $$.
   * @param {string} label A localized string to substitute through.
   *     This is not an resource ID.
   * @param {...(string|number)} var_args The extra values to include in the
   *     formatted output.
   * @return {!Array<!{value: string, arg: (null|string)}>} The formatted
   *     string pieces.
   */
  getSubstitutedStringPieces(label, var_args) {
    const varArgs = arguments;
    // Split the string by separately matching all occurrences of $1-9 and of
    // non $1-9 pieces.
    const pieces = (label.match(/(\$[1-9])|(([^$]|\$([^1-9]|$))+)/g) ||
                    []).map(function(p) {
      // Pieces that are not $1-9 should be returned after replacing $$
      // with $.
      if (!p.match(/^\$[1-9]$/)) {
        expect(
            (p.match(/\$/g) || []).length % 2 === 0,
            'Unescaped $ found in localized string.');
        return {value: p.replace(/\$\$/g, '$'), arg: null};
      }

      // Otherwise, return the substitution value.
      return {value: varArgs[p[1]], arg: p};
    });

    return pieces;
  }

  /**
   * As above, but also makes sure that the value is a boolean.
   * @param {string} id The key that identifies the desired boolean.
   * @return {boolean} The corresponding boolean value.
   */
  getBoolean(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'boolean');
    return /** @type {boolean} */ (value);
  }

  /**
   * As above, but also makes sure that the value is an integer.
   * @param {string} id The key that identifies the desired number.
   * @return {number} The corresponding number value.
   */
  getInteger(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'number');
    expect(value === Math.floor(value), 'Number isn\'t integer: ' + value);
    return /** @type {number} */ (value);
  }

  /**
   * Override values in loadTimeData with the values found in |replacements|.
   * @param {Object} replacements The dictionary object of keys to replace.
   */
  overrideValues(replacements) {
    expect(
        typeof replacements === 'object',
        'Replacements must be a dictionary object.');
    for (const key in replacements) {
      this.data_[key] = replacements[key];
    }
  }

  /**
   * Reset loadTimeData's data. Should only be used in tests.
   * @param {?Object} newData The data to restore to, when null restores to
   *    unset state.
   */
  resetForTesting(newData = null) {
    this.data_ = newData;
  }

  /**
   * @return {boolean} Whether loadTimeData.data has been set.
   */
  isInitialized() {
    return this.data_ !== null;
  }
}

  /**
   * Checks condition, throws error message if expectation fails.
   * @param {*} condition The condition to check for truthiness.
   * @param {string} message The message to display if the check fails.
   */
  function expect(condition, message) {
    if (!condition) {
      throw new Error(
          'Unexpected condition on ' + document.location.href + ': ' + message);
    }
  }

  /**
   * Checks that the given value has the given type.
   * @param {string} id The id of the value (only used for error message).
   * @param {*} value The value to check the type on.
   * @param {string} type The type we expect |value| to be.
   */
  function expectIsType(id, value, type) {
    expect(
        typeof value === type, '[' + value + '] (' + id + ') is not a ' + type);
  }

  expect(!loadTimeData, 'should only include this file once');
  loadTimeData = new LoadTimeData;

  // Expose |loadTimeData| directly on |window|, since within a JS module the
  // scope is local and not all files have been updated to import the exported
  // |loadTimeData| explicitly.
  window.loadTimeData = loadTimeData;

  console.warn('crbug/1173575, non-JS module files deprecated.');</script><script>loadTimeData.data = {"header":"Index von LOCATION","headerDateModified":"Änderungsdatum","headerName":"Name","headerSize":"Größe","language":"de","parentDirText":"[übergeordnetes Verzeichnis]","textdirection":"ltr"};</script><script>start("C:\\Users\\MYNAME-\\eclipse-workspace\\htmlCsvtable\\src\\htmlTableGeneration\\csvData\\");</script>
<script>onHasParentDirectory();</script>
<script>addRow("20220201221359.csv","20220201221359.csv",0,2959,"2.9 kB",1644181136,"06.02.22, 21:58:56");</script>
<script>addRow("20220206091030.csv","20220206091030.csv",0,2958,"2.9 kB",1644181136,"06.02.22, 21:58:56");</script>
<script>addRow("20220206135359.csv","20220206135359.csv",0,2959,"2.9 kB",1644181136,"06.02.22, 21:58:56");</script>
</code>```

答:
Since you're using jQuery you can just load the file and convert the response to a jQuery object. This gives you the ability to use jQuery's DOM traversal functions like $.find() etc.

$("#test").load( "csvData/", function(data) {
  console.log($(data));
  // $(data).find("#someselector")
  // or use the regex answer here findFiles(data)
})```

The ID test is just a container to place results in or something like that.

Body:
Hello im trying to load all file names from a folder.

im using this script and it returns a response object called data. When i use console.log on data it returns me to html below. how can i now collect all the file names out of there and write it into the variable fileNames?

my script:

var fileNames= "";
var folder = "csvData/";
    $.ajax({
          type: 'GET',
          url:  folder,

          success: function(data){
              console.log(data);
          },
          async:false
      });

html inside of data response:
a the bottom you can see that a function is called for each file in the folder.

<!DOCTYPE html>

<meta charset="utf-8">
<meta name="color-scheme" content="light dark">
<meta name="google" value="notranslate">

<title id="title"></title>

Index von LOCATION

    [übergeordnetes Verzeichnis]

      Name

        Größe

        Änderungsdatum

Body:
Since you're using jQuery you can just load the file and convert the response to a jQuery object. This gives you the ability to use jQuery's DOM traversal functions like $.find() etc.

$("#test").load( "csvData/", function(data) {
  console.log($(data));
  // $(data).find("#someselector")
  // or use the regex answer here findFiles(data)
})

The ID test is just a container to place results in or something like that.


来源: stackexchange

问:
How can I tell if a date is before a certain time in Java?
For example given a java.util.Date how could I test if the time was before 12.30 PM that day?</p>

答:
Have a look at Calendar.before() Method.</p>

```<code>Calendar now = Calendar.getInstance();
now.set(Calendar.HOUR_OF_DAY, 12);
now.set(Calendar.MINUTE, 30);
Calendar givenDate = Calendar.getInstance();
givenDate.setTime(yourDate);

boolean isBefore = now.before(givenDate);
</code>```

ContentLicense: CC BY-SA 3.0

Body:
Have a look at Calendar.before() Method.

Calendar now = Calendar.getInstance();
now.set(Calendar.HOUR_OF_DAY, 12);
now.set(Calendar.MINUTE, 30);
Calendar givenDate = Calendar.getInstance();
givenDate.setTime(yourDate);

boolean isBefore = now.before(givenDate);


问:
Pyspark - How to get capitalized names?
How to get capitalized names?

```<code>from pyspark.sql import types as T
import pyspark.sql.functions as F
from datetime import datetime
from pyspark.sql.functions import to_timestamp

test = spark.createDataFrame(
[
(1,'2021-10-04 09:05:14', "For the 2nd copy of the ticket, access the link: wa.me/11223332211 (Whats) use ID and our number(1112222333344455). Duvidas, www.abtech.com . AB Tech"),
(2,'2021-10-04 09:10:05', ". MARCIOG, let's try again? Get in touch to rectify your situation. For WhatsApp Link: ab-ab.ab.com/n/12345467. AB Tech"),
(3,'2021-10-04 09:27:27', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/99998-88822 (Whats) ou 0800-999-9999. AB Tech"),
(4,'2021-10-04 14:55:26', "Mr, SUELI. enjoy the holiday with money in your account. AB has great conditions for you. Call now and hire 0800899-9999 (Mon to Fri from 12pm to 6pm)"),
(5,'2021-10-06 09:15:11', ". DEPREZC, let's try again? Get in touch to rectify your situation. For whatsapp Link: csi-csi.abtech.com/n/12345467. AB Tech"),
(6,'2022-02-03 08:00:12', "Mr. SARA. We have great discount options. Regularize your situation with AB! Link: wa.me/25544-8855 (Whats) ou 0800-999-9999. AB."),
(7,'2021-10-04 09:26:00', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/999999999 (Whats) or 0800-999-9999. AB Tech"),
(8,'2018-10-09 12:31:33', "Mr.(a) ANTONI, regularize your situation with the Ammmm Bhhhh. Ligue 0800-729-2406 or access the CHAT www.abtech.com. AB Tech."),
(9,'2018-10-09 15:14:51', "Follow code of bars of your updated deal for today (11111.111111 1111.11111 11111.111111 1 11111111111). Doubts call 0800-999-9999. AB Tech.")
],
T.StructType(
[
T.StructField("id_mt", T.StringType(), True),
T.StructField("date_send", T.StringType(), True),
T.StructField("message", T.StringType(), True),
]
),
)
</code>```

Could you tell me what is the logic to check the uppercase names?

So, there is a column name 'names' which is answer:

enter image description here

答:
We made the Fugue project to port native Python or Pandas code to Spark or Dask. This lets you can keep the logic very readable by expressing it in native Python. Fugue can then port it to Spark for you with one function call.

I think this specific case is hard but Spark but easy in Python. I'll walk through the solution.

First we make a Pandas DataFrame for quick testing:

import pandas as pd
df = pd.DataFrame([(1,'2021-10-04 09:05:14', "For the 2nd copy of the ticket, access the link: wa.me/11223332211 (Whats) use ID and our number(1112222333344455). Duvidas, www.abtech.com . AB Tech"),
(2,'2021-10-04 09:10:05', ". MARCIOG, let's try again? Get in touch to rectify your situation. For WhatsApp Link: ab-ab.ab.com/n/12345467. AB Tech"),
(3,'2021-10-04 09:27:27', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/99998-88822 (Whats) ou 0800-999-9999. AB Tech"),
(4,'2021-10-04 14:55:26', "Mr, SUELI. enjoy the holiday with money in your account. AB has great conditions for you. Call now and hire 0800899-9999 (Mon to Fri from 12pm to 6pm)"),
(5,'2021-10-06 09:15:11', ". DEPREZC, let's try again? Get in touch to rectify your situation. For whatsapp Link: csi-csi.abtech.com/n/12345467. AB Tech"),
(6,'2022-02-03 08:00:12', "Mr. SARA. We have great discount options. Regularize your situation with AB! Link: wa.me/25544-8855 (Whats) ou 0800-999-9999. AB."),
(7,'2021-10-04 09:26:00', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/999999999 (Whats) or 0800-999-9999. AB Tech"),
(8,'2018-10-09 12:31:33', "Mr.(a) ANTONI, regularize your situation with the Ammmm Bhhhh. Ligue 0800-729-2406 or access the CHAT www.abtech.com. AB Tech."),
(9,'2018-10-09 15:14:51', "Follow code of bars of your updated deal for today (11111.111111 1111.11111 11111.111111 1 11111111111). Doubts call 0800-999-9999. AB Tech.")], columns=["id_mt", "date_send","message"])
```
<p>Now we create a native Python function to extract the string. The <code>get_name_for_one_string</code> operates on one string and the <code>get_names</code> will take in the whole DataFrame.</p>
<pre class="lang-py prettyprint-override"><code>from typing import List, Dict, Any
import re

def get_name_for_one_string(message: str) -> str:
    # drop non-alphanumeric
    message = re.sub(r"\s*[^A-Za-z]+\s*", " ", message)
    # string split
    items = message.split(" ")
    # keep all caps and len > 2
    item = [x for x in items if (x.upper() == x and len(x) > 2)]
    if len(item) > 0:
        return item[0]
    else:
        return None

def get_names(df: List[Dict[str,Any]]) -> List[Dict[str,Any]]:
    for row in df:
        row["names"] = get_name_for_one_string(row["message"])
    return df
</code>```

Now we can use this on a Pandas DataFrame using the Fugue transform function and Fugue will handle the conversions

from fugue import transform
transform(df, get_names, schema="*,names:str")
```
<p>This works so now we can bring it to Spark just by specifying the engine.</p>
<pre class="lang-py prettyprint-override"><code>import fugue_spark
transform(df, get_names, schema="*,names:str", engine="spark").show()
</code>```
```<code>+-----+-------------------+--------------------+-------+
|id_mt|          date_send|             message|  names|
+-----+-------------------+--------------------+-------+
|    1|2021-10-04 09:05:14|For the 2nd copy ...|   null|
|    2|2021-10-04 09:10:05|. MARCIOG, let's ...|MARCIOG|
|    3|2021-10-04 09:27:27|, we do not ident...|   null|
|    4|2021-10-04 14:55:26|Mr, SUELI. enjoy ...|  SUELI|
|    5|2021-10-06 09:15:11|. DEPREZC, let's ...|DEPREZC|
|    6|2022-02-03 08:00:12|Mr. SARA. We have...|   SARA|
|    7|2021-10-04 09:26:00|, we do not ident...|   null|
|    8|2018-10-09 12:31:33|Mr.(a) ANTONI, re...| ANTONI|
|    9|2018-10-09 15:14:51|Follow code of ba...|   null|
+-----+-------------------+--------------------+-------+
</code>```

Note you need .show() because Spark evaluates lazily. The transform function can take in both Pandas and Spark DataFrames. If you use the Spark engine, the output will be a Spark DataFrame also.

ContentLicense: CC BY-SA 4.0

Title: Pyspark - How to get capitalized names?

Body:
How to get capitalized names?

from pyspark.sql import types as T
import pyspark.sql.functions as F
from datetime import datetime
from pyspark.sql.functions import to_timestamp

test = spark.createDataFrame(
[
(1,'2021-10-04 09:05:14', "For the 2nd copy of the ticket, access the link: wa.me/11223332211 (Whats) use ID and our number(1112222333344455). Duvidas, www.abtech.com . AB Tech"),
(2,'2021-10-04 09:10:05', ". MARCIOG, let's try again? Get in touch to rectify your situation. For WhatsApp Link: ab-ab.ab.com/n/12345467. AB Tech"),
(3,'2021-10-04 09:27:27', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/99998-88822 (Whats) ou 0800-999-9999. AB Tech"),
(4,'2021-10-04 14:55:26', "Mr, SUELI. enjoy the holiday with money in your account. AB has great conditions for you. Call now and hire 0800899-9999 (Mon to Fri from 12pm to 6pm)"),
(5,'2021-10-06 09:15:11', ". DEPREZC, let's try again? Get in touch to rectify your situation. For whatsapp Link: csi-csi.abtech.com/n/12345467. AB Tech"),
(6,'2022-02-03 08:00:12', "Mr. SARA. We have great discount options. Regularize your situation with AB! Link: wa.me/25544-8855 (Whats) ou 0800-999-9999. AB."),
(7,'2021-10-04 09:26:00', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/999999999 (Whats) or 0800-999-9999. AB Tech"),
(8,'2018-10-09 12:31:33', "Mr.(a) ANTONI, regularize your situation with the Ammmm Bhhhh. Ligue 0800-729-2406 or access the CHAT www.abtech.com. AB Tech."),
(9,'2018-10-09 15:14:51', "Follow code of bars of your updated deal for today (11111.111111 1111.11111 11111.111111 1 11111111111). Doubts call 0800-999-9999. AB Tech.")
],
T.StructType(
[
T.StructField("id_mt", T.StringType(), True),
T.StructField("date_send", T.StringType(), True),
T.StructField("message", T.StringType(), True),
]
),
)

Could you tell me what is the logic to check the uppercase names?

So, there is a column name 'names' which is answer:

enter image description here

Body:
We made the Fugue project to port native Python or Pandas code to Spark or Dask. This lets you can keep the logic very readable by expressing it in native Python. Fugue can then port it to Spark for you with one function call.

I think this specific case is hard but Spark but easy in Python. I'll walk through the solution.

First we make a Pandas DataFrame for quick testing:

import pandas as pd
df = pd.DataFrame([(1,'2021-10-04 09:05:14', "For the 2nd copy of the ticket, access the link: wa.me/11223332211 (Whats) use ID and our number(1112222333344455). Duvidas, www.abtech.com . AB Tech"),
(2,'2021-10-04 09:10:05', ". MARCIOG, let's try again? Get in touch to rectify your situation. For WhatsApp Link: ab-ab.ab.com/n/12345467. AB Tech"),
(3,'2021-10-04 09:27:27', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/99998-88822 (Whats) ou 0800-999-9999. AB Tech"),
(4,'2021-10-04 14:55:26', "Mr, SUELI. enjoy the holiday with money in your account. AB has great conditions for you. Call now and hire 0800899-9999 (Mon to Fri from 12pm to 6pm)"),
(5,'2021-10-06 09:15:11', ". DEPREZC, let's try again? Get in touch to rectify your situation. For whatsapp Link: csi-csi.abtech.com/n/12345467. AB Tech"),
(6,'2022-02-03 08:00:12', "Mr. SARA. We have great discount options. Regularize your situation with AB! Link: wa.me/25544-8855 (Whats) ou 0800-999-9999. AB."),
(7,'2021-10-04 09:26:00', ", we do not identify the payment of the installment of your agreement, if paid disregard. You doubt, link: wa.me/999999999 (Whats) or 0800-999-9999. AB Tech"),
(8,'2018-10-09 12:31:33', "Mr.(a) ANTONI, regularize your situation with the Ammmm Bhhhh. Ligue 0800-729-2406 or access the CHAT www.abtech.com. AB Tech."),
(9,'2018-10-09 15:14:51', "Follow code of bars of your updated deal for today (11111.111111 1111.11111 11111.111111 1 11111111111). Doubts call 0800-999-9999. AB Tech.")], columns=["id_mt", "date_send","message"])

Now we create a native Python function to extract the string. The get_name_for_one_string operates on one string and the get_names will take in the whole DataFrame.

from typing import List, Dict, Any
import re

def get_name_for_one_string(message: str) -> str:
    # drop non-alphanumeric
    message = re.sub(r"\s*[^A-Za-z]+\s*", " ", message)
    # string split
    items = message.split(" ")
    # keep all caps and len > 2
    item = [x for x in items if (x.upper() == x and len(x) > 2)]
    if len(item) > 0:
        return item[0]
    else:
        return None

def get_names(df: List[Dict[str,Any]]) -> List[Dict[str,Any]]:
    for row in df:
        row["names"] = get_name_for_one_string(row["message"])
    return df

Now we can use this on a Pandas DataFrame using the Fugue transform function and Fugue will handle the conversions

from fugue import transform
transform(df, get_names, schema="*,names:str")

This works so now we can bring it to Spark just by specifying the engine.

import fugue_spark
transform(df, get_names, schema="*,names:str", engine="spark").show()

+-----+-------------------+--------------------+-------+
|id_mt|          date_send|             message|  names|
+-----+-------------------+--------------------+-------+
|    1|2021-10-04 09:05:14|For the 2nd copy ...|   null|
|    2|2021-10-04 09:10:05|. MARCIOG, let's ...|MARCIOG|
|    3|2021-10-04 09:27:27|, we do not ident...|   null|
|    4|2021-10-04 14:55:26|Mr, SUELI. enjoy ...|  SUELI|
|    5|2021-10-06 09:15:11|. DEPREZC, let's ...|DEPREZC|
|    6|2022-02-03 08:00:12|Mr. SARA. We have...|   SARA|
|    7|2021-10-04 09:26:00|, we do not ident...|   null|
|    8|2018-10-09 12:31:33|Mr.(a) ANTONI, re...| ANTONI|
|    9|2018-10-09 15:14:51|Follow code of ba...|   null|
+-----+-------------------+--------------------+-------+

Note you need .show() because Spark evaluates lazily. The transform function can take in both Pandas and Spark DataFrames. If you use the Spark engine, the output will be a Spark DataFrame also.


问:
How to solve system of equation with JavaScript
I'm trying to do do some algebra in JavaScript based on some conditions and known variables but I'm not good enough at maths or JavaScript to understand how to write it.

So here's the conditions:

```<code>var w1 = h1/1.98

var w2 = h2/0.6355

h1 = h2

w1 + w2 = 1367
</code>```

I'm not 100% sure but I think there may be enough info here to find out what w1, w2, h1 and h2 are.

Does anyone know how I can create this calculation?

答:
I don't know if anyone's reading here any more, but here you go:

https://sourceforge.net/projects/matgts/files/jsmath/

Access "index.html". Some of the programs require at least Firefox 3 or 4, because of , but the one which solves systems of equations should run in earlier versions (and up to the latest one, of course). Easy to use, download and run offline, like JavaScript programs are always supposed to run. For accessing web sites, NetSurf is the best (https://www.netsurf-browser.org/), no JavaScript. JavaScript is a great language, just not online, where it's just a great nuisance.

A JavaScript function which uses Cholesky's method (the program mentioned above uses Gauss elimination) is included here:

https://sourceforge.net/projects/matgts/files/jswall/

The file is "js/chol.js", the comment above the function says how the elements of the matrix should be organized.

Equivalent C functions are also provided. All GPL version 3.

P.S. I tried to post the message with NetSurf and it didn't work, apparently this site has some serious problems, which I find surprising for a site about programming, I would have expected its creators to know something about the subject.

Body:
I'm trying to do do some algebra in JavaScript based on some conditions and known variables but I'm not good enough at maths or JavaScript to understand how to write it.

So here's the conditions:

var w1 = h1/1.98

var w2 = h2/0.6355

h1 = h2

w1 + w2 = 1367

I'm not 100% sure but I think there may be enough info here to find out what w1, w2, h1 and h2 are.

Does anyone know how I can create this calculation?


问:
Different .htaccess rewrite rules for new directory
My .htaccess currently has the following behaviour using the rewrite rules below:

https://example.com/abcd?lol=true

if file /abcd.php exists, open https://example.com/abcd.php?lol=true

if directory and file /abcd/index.php exist, open https://example.com/abcd/index.php?lol=true

else open https://example.com/index.php?lol=true

-> in all three cases the address bar must still show https://example.com/abcd?lol=true

```<code>RewriteEngine On

RewriteCond %{HTTPS} !=on
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1%{REQUEST_URI} [R=301,QSA,NC,L]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ /$1 [L,R=301]

RewriteCond %{REQUEST_FILENAME}\.php -f
RewriteRule ^(.*)$ $1.php [END]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ / [END]
</code>```

Now to my question: I have a new directory /blog/, where the following must happen:

https://example.com/blog/abcd?lol=true

open https://example.com/blog/index.php?lol=true

-> the address bar must still show https://example.com/blog/abcd?lol=true so I can use php to read the abcd part, which could be an article name.

How can I append my .htaccess code to achieve this?

答:
Aside: You're not actually doing anything for #2 (ie. "if directory and file /abcd/index.php exist...") - you are probably relying on default behaviour by mod_dir. However, if you request /abcd?lol=true and /abcd exists as a physical directory, there will be a 301 external redirect to append the trailing slash (ie.  /abcd/?lol=true ) which will then result in /abcd/index.php being served. So, this does not strictly show /abcd?lol=true in the address bar as you suggest. Is that what you are seeing?

To implement the /blog/abcd?lol=true to /blog/index.php?lol=true rewrite then replace your last rule with the following:

```<code># Prevent further processing if a file or directory is requested directly
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [END]

# Route requests to blog/index.php
RewriteRule ^blog/. blog/index.php [END]

# Everthing else is routed to "/index.php"
RewriteRule . index.php [END]
</code>```

This separates out the filesystem checks into their own rule so you aren't doing this twice.

Your previous "last" rule:

```<code>RewriteRule ^(.*)$ / [END]
</code>```

You were only rewriting to / and then relying on mod_dir issuing an internal subrequest for index.php (the DirectoryIndex). You should instead rewrite directly to index.php as required (no need for the slash prefix). The capturing pattern ^(.*)$ is unnecessary here.

Body:
My .htaccess currently has the following behaviour using the rewrite rules below:

https://example.com/abcd?lol=true

if file /abcd.php exists, open https://example.com/abcd.php?lol=true

if directory and file /abcd/index.php exist, open https://example.com/abcd/index.php?lol=true

else open https://example.com/index.php?lol=true

-> in all three cases the address bar must still show https://example.com/abcd?lol=true

RewriteEngine On

RewriteCond %{HTTPS} !=on
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1%{REQUEST_URI} [R=301,QSA,NC,L]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ /$1 [L,R=301]

RewriteCond %{REQUEST_FILENAME}\.php -f
RewriteRule ^(.*)$ $1.php [END]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ / [END]

Now to my question: I have a new directory /blog/, where the following must happen:

https://example.com/blog/abcd?lol=true

open https://example.com/blog/index.php?lol=true

-> the address bar must still show https://example.com/blog/abcd?lol=true so I can use php to read the abcd part, which could be an article name.

How can I append my .htaccess code to achieve this?

Body:
Aside: You're not actually doing anything for #2 (ie. "if directory and file /abcd/index.php exist...") - you are probably relying on default behaviour by mod_dir. However, if you request /abcd?lol=true and /abcd exists as a physical directory, there will be a 301 external redirect to append the trailing slash (ie.  /abcd/?lol=true ) which will then result in /abcd/index.php being served. So, this does not strictly show /abcd?lol=true in the address bar as you suggest. Is that what you are seeing?

To implement the /blog/abcd?lol=true to /blog/index.php?lol=true rewrite then replace your last rule with the following:

# Prevent further processing if a file or directory is requested directly
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [END]

# Route requests to blog/index.php
RewriteRule ^blog/. blog/index.php [END]

# Everthing else is routed to "/index.php"
RewriteRule . index.php [END]

This separates out the filesystem checks into their own rule so you aren't doing this twice.

Your previous "last" rule:

RewriteRule ^(.*)$ / [END]

You were only rewriting to / and then relying on mod_dir issuing an internal subrequest for index.php (the DirectoryIndex). You should instead rewrite directly to index.php as required (no need for the slash prefix). The capturing pattern ^(.*)$ is unnecessary here.


问:
Image not found using angular material
I get this error when trying to display an image:

http://localhost:4200/daily-process/src/WebUI/Angular/src/images/DailyProcessBlack.png 404 (Not Found)

I put the image in a mat-card and I can see the image when I navigate to the folder path but it doesn't show on the site:

```<code><mat-card class="example-card">
 <mat-card-header>
<div mat-card-avatar class="example-header-image"></div>
 </mat-card-header>
<img mat-card-image src="/daily-process/src/WebUI/Angular/src/images/DailyProcessBlack.png"
alt="Image">
</mat-card>
</code>```

angular.json

```<code> {
 "$schema": "./node_modules/@angular/cli/lib/config/schema.json",
 "version": 1,
 "newProjectRoot": "projects",
 "projects": {
  "daily-process": {
  "projectType": "application",
  "schematics": {},
  "root": "",
  "sourceRoot": "src",
  "prefix": "app",
  "architect": {
    "build": {
      "builder": "@angular-devkit/build-angular:browser",
      "options": {
        "outputPath": "dist/daily-process",
        "index": "src/index.html",
        "main": "src/main.ts",
        "polyfills": "src/polyfills.ts",
        "tsConfig": "tsconfig.app.json",
        "assets": [
          "src/favicon.ico",
          "src/assets"
        ],
        // "styles": [
        //   "src/styles.css"
        // ],
        "scripts": []
      },
      "configurations": {
        "production": {
          "budgets": [
            {
              "type": "initial",
              "maximumWarning": "2mb",
              "maximumError": "5mb"
            },
            {
              "type": "anyComponentStyle",
              "maximumWarning": "6kb",
              "maximumError": "10kb"
            }
          ],
          "fileReplacements": [
            {
              "replace": "src/environments/environment.ts",
              "with": "src/environments/environment.prod.ts"
            }
          ],
          "outputHashing": "all"
        },
        "development": {
          "buildOptimizer": false,
          "optimization": false,
          "vendorChunk": true,
          "extractLicenses": false,
          "sourceMap": true,
          "namedChunks": true
        }
      },
      "defaultConfiguration": "production"
    },
    "serve": {
      "builder": "@angular-devkit/build-angular:dev-server",
      "configurations": {
        "production": {
          "browserTarget": "daily-process:build:production"
        },
        "development": {
          "browserTarget": "daily-process:build:development"
        }
      },
      "defaultConfiguration": "development"
    },
    "extract-i18n": {
      "builder": "@angular-devkit/build-angular:extract-i18n",
      "options": {
        "browserTarget": "daily-process:build"
      }
    },
    "test": {
      "builder": "@angular-devkit/build-angular:karma",
      "options": {
        "main": "src/test.ts",
        "polyfills": "src/polyfills.ts",
        "tsConfig": "tsconfig.spec.json",
        "karmaConfig": "karma.conf.js",
        "assets": [
          "src/favicon.ico",
          "src/assets"
        ],
        // "styles": [
        //   "src/styles.css"
        // ],
         "scripts": []
        }
      }
    }
  }
 },
 "defaultProject": "daily-process"
}
</code>```

Image of folder structure

Can someone please provide a reason why and a solution to what I need to do to get this to work properly.

答:
I recommend you to put that image in assets folder inside your app, and the route you need to add is assets/images/DailyProcessBlack.png
If you have problem with that route try to test local or global route, but you need to add the image inside assets, otherwise you would have to modify the angular.json file</p>

Title: Image not found using angular material

Body:
I get this error when trying to display an image:

http://localhost:4200/daily-process/src/WebUI/Angular/src/images/DailyProcessBlack.png 404 (Not Found)

I put the image in a mat-card and I can see the image when I navigate to the folder path but it doesn't show on the site:

<mat-card class="example-card">
 <mat-card-header>

 </mat-card-header>

</mat-card>

angular.json

 {
 "$schema": "./node_modules/@angular/cli/lib/config/schema.json",
 "version": 1,
 "newProjectRoot": "projects",
 "projects": {
  "daily-process": {
  "projectType": "application",
  "schematics": {},
  "root": "",
  "sourceRoot": "src",
  "prefix": "app",
  "architect": {
    "build": {
      "builder": "@angular-devkit/build-angular:browser",
      "options": {
        "outputPath": "dist/daily-process",
        "index": "src/index.html",
        "main": "src/main.ts",
        "polyfills": "src/polyfills.ts",
        "tsConfig": "tsconfig.app.json",
        "assets": [
          "src/favicon.ico",
          "src/assets"
        ],
        // "styles": [
        //   "src/styles.css"
        // ],
        "scripts": []
      },
      "configurations": {
        "production": {
          "budgets": [
            {
              "type": "initial",
              "maximumWarning": "2mb",
              "maximumError": "5mb"
            },
            {
              "type": "anyComponentStyle",
              "maximumWarning": "6kb",
              "maximumError": "10kb"
            }
          ],
          "fileReplacements": [
            {
              "replace": "src/environments/environment.ts",
              "with": "src/environments/environment.prod.ts"
            }
          ],
          "outputHashing": "all"
        },
        "development": {
          "buildOptimizer": false,
          "optimization": false,
          "vendorChunk": true,
          "extractLicenses": false,
          "sourceMap": true,
          "namedChunks": true
        }
      },
      "defaultConfiguration": "production"
    },
    "serve": {
      "builder": "@angular-devkit/build-angular:dev-server",
      "configurations": {
        "production": {
          "browserTarget": "daily-process:build:production"
        },
        "development": {
          "browserTarget": "daily-process:build:development"
        }
      },
      "defaultConfiguration": "development"
    },
    "extract-i18n": {
      "builder": "@angular-devkit/build-angular:extract-i18n",
      "options": {
        "browserTarget": "daily-process:build"
      }
    },
    "test": {
      "builder": "@angular-devkit/build-angular:karma",
      "options": {
        "main": "src/test.ts",
        "polyfills": "src/polyfills.ts",
        "tsConfig": "tsconfig.spec.json",
        "karmaConfig": "karma.conf.js",
        "assets": [
          "src/favicon.ico",
          "src/assets"
        ],
        // "styles": [
        //   "src/styles.css"
        // ],
         "scripts": []
        }
      }
    }
  }
 },
 "defaultProject": "daily-process"
}

Image of folder structure

Can someone please provide a reason why and a solution to what I need to do to get this to work properly.


问:
SQL 2016 with R - Error HRESULT 0x80004004
I am working through a number of tutorials on using SQL and R. But when I am trying to Run the R Script to obtain the 'ggplot' library I am getting the following error

```<code>Msg 39004, Level 16, State 20, Line 1
A 'R' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 1, Line 1
An external script error occurred:
Error in library("ggplot2") : there is no package called 'ggplot2'
Calls: source -> withVisible -> eval -> eval -> library

Error in ScaleR.  Check the output for more information.
Error in eval(expr, envir, enclos) :
Error in ScaleR.  Check the output for more information.
Calls: source -> withVisible -> eval -> eval -> .Call
Execution halted

(0 row(s) affected)
</code>```

The Original Script is

```<code>INSERT INTO chartBinary (binData)
EXEC sp_execute_external_script
@language = N'R',
@script = N'
library("ggplot2");
img <- inputDataSet;
image_file = tempfile();
png(filename = image_file, width=800, height=600);
print(ggplot(img, aes(x = AirportID, y = WindSpeed)) +
labs(x = "Airport ID", y = "Wind Speed") +
theme(axis.text.x = element_text(angle=90, hjust=1, vjust=0)) +
geom_point(stat = "identity") +
geom_smooth(method = "loess", aes(group = 1)) +
geom_text(aes(label = AirportID), size = 3, vjust = 1.0) +
geom_text(aes(label = round(WindSpeed, digits = 2)), size = 3, vjust = 2.0));
dev.off();
OutputDataset <- data.frame(data=readBin(file(image_file,"rb"),what=raw(),n=1e6));',
@input_data_1 = N'SELECT AirportID, AVG(CONVERT(float, WindSpeed)) as   WindSpeed
FROM
[Weather_Sample] GROUP BY AirportID ORDER BY AirportID;',
@input_data_1_name = N'inputDataSet',
@output_data_1_name = N'OutputDataset';
</code>```

The system has SQL 2016, SSMS 2017, MS R Open 3.4.0
The integration with R works well with Visual Studio 2015 and has no errors. Can download library packages and run the scripts with no errors. Only when I start using SMSS I am unable to download packages

答:
I've been trying to find a solution to this error almost 2 days. I've tried to set up permissions and the result where still the same. There was no dependencies issues. I was trying to run external C# code (i'm aware of SQLCLR).

So after googling and trying all possible proposed solutions, i went back to the Microsoft website and i noted that Language Extension can run only from version of SQL Server CU3 and later. I decided to check which version i was using and it  turns out that i was running an earlier version. So i installed the last cumulative update and the job was done, i get rid of that error.

Bottom line:

Check that you have installed at least SQL Server CU3

set permissions for "ALL APPLICATION PACKAGES" to the <SQL_SERVER_INSTALLATION_PATH><SERVER_FOLDER>\MSSQL

Hope it help's you.

Body:
I am working through a number of tutorials on using SQL and R. But when I am trying to Run the R Script to obtain the 'ggplot' library I am getting the following error

Msg 39004, Level 16, State 20, Line 1
A 'R' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 1, Line 1
An external script error occurred:
Error in library("ggplot2") : there is no package called 'ggplot2'
Calls: source -> withVisible -> eval -> eval -> library

Error in ScaleR.  Check the output for more information.
Error in eval(expr, envir, enclos) :
Error in ScaleR.  Check the output for more information.
Calls: source -> withVisible -> eval -> eval -> .Call
Execution halted

(0 row(s) affected)

The Original Script is

INSERT INTO chartBinary (binData)
EXEC sp_execute_external_script
@language = N'R',
@script = N'
library("ggplot2");
img <- inputDataSet;
image_file = tempfile();
png(filename = image_file, width=800, height=600);
print(ggplot(img, aes(x = AirportID, y = WindSpeed)) +
labs(x = "Airport ID", y = "Wind Speed") +
theme(axis.text.x = element_text(angle=90, hjust=1, vjust=0)) +
geom_point(stat = "identity") +
geom_smooth(method = "loess", aes(group = 1)) +
geom_text(aes(label = AirportID), size = 3, vjust = 1.0) +
geom_text(aes(label = round(WindSpeed, digits = 2)), size = 3, vjust = 2.0));
dev.off();
OutputDataset <- data.frame(data=readBin(file(image_file,"rb"),what=raw(),n=1e6));',
@input_data_1 = N'SELECT AirportID, AVG(CONVERT(float, WindSpeed)) as   WindSpeed
FROM
[Weather_Sample] GROUP BY AirportID ORDER BY AirportID;',
@input_data_1_name = N'inputDataSet',
@output_data_1_name = N'OutputDataset';

The system has SQL 2016, SSMS 2017, MS R Open 3.4.0
The integration with R works well with Visual Studio 2015 and has no errors. Can download library packages and run the scripts with no errors. Only when I start using SMSS I am unable to download packages


问:
how to permanently change the maximum recursion depth in python
I have a command-line python package run in the terminal directly (it was not run inside python environment) that often gives me a maximum recursion error. I know the default setting is 1000, and I know you can use the setrecursionlimit() command to change it. However, every time I change it and exit the python environment, the value resets to the default.

My question is, without changing the python package, is there a way I could change the python default recursion depth permanently. I understand this may not be a good practice, so I plan to change it back to 1000 after I finish using the package, which I probably won't need after a week.

Thanks!

答:
I agree with others that using sys.setrecursionlimit is the way to go. In the end, that is the reason why sys is part of the Python Standard Library.

In an interactive session, you could set your PYTHONSTARTUP global variable to a file with the necessary python commands, however this would still run the exact same code but just a bit more 'hidden' from the user.

I think that if you'd really like to set this to another value, your only option is to dive deep into the C header files of the cpython source code and custom compile python for your needs.


问:
Why does this test fail if someone else runs it at the same time?
I was watching a conference talk (No need to watch it to understand my question but if you're curious it's from 35m28s to 36m28s). The following test was shown:

TEST(Foo, StorageTest){
  StorageServer* server = GetStorageServerHandle();
  auto my_val = rand();
  server -> Store("testKey", my_val);
  EXPECT_EQ(my_val, server->Load("testKey"));
}
```

One of the speakers said: "you can only expect that storing data to a production service works if only one copy of that test is running at a time."

The other speaker said: "Once you add continuous integration in the mix, the test starts failing". I don't know a lot about CI/CD.

I am having some trouble understanding both claims 1. and 2. Any help?

答:
One of the speakers said: "you can only expect that storing data to a production service works if only one copy of that test is running at a time."

Right. Imagine if two instances of this code are running. If both Store operations execute before either Load operation takes place, the one whose Store executed first will load the wrong value.

Consider this pattern where the two instances are called "first" and "second":

First Store executes, stores first random value.

Second Store starts executing, starts storing second random value.

First Load is blocked on the second Store completing due to a lock internal to the database

Second Load is blocked on the Store completing due to a local internal to the database.

Second Store finishes and release the internal lock.

First Load can now execute, it gets second random value.

EXPECT_EQ fails as the first and second random values are different.

The other speaker said: "Once you add continuous integration in the mix, the test starts failing".

If a CI system is testing multiple instances of the code at the same time, race conditions like the example above can occur and cause tests to fail as the multiple instances race with each other.

Body:
I was watching a conference talk (No need to watch it to understand my question but if you're curious it's from 35m28s to 36m28s). The following test was shown:

TEST(Foo, StorageTest){
  StorageServer* server = GetStorageServerHandle();
  auto my_val = rand();
  server -> Store("testKey", my_val);
  EXPECT_EQ(my_val, server->Load("testKey"));
}

One of the speakers said: "you can only expect that storing data to a production service works if only one copy of that test is running at a time."

The other speaker said: "Once you add continuous integration in the mix, the test starts failing". I don't know a lot about CI/CD.

I am having some trouble understanding both claims 1. and 2. Any help?


问:
Convert SOAP XML string to object or just read its contents
Am receiving a SOAP XML response from an endpoint and I am trying to read its' contents. I have tried a couple of methods but it keeps giving me null

```<code>String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"><soap:Body><CashOutPaymentRequestResponse xmlns=\"http://tempuri.org/\"><CashOutPaymentRequestResult><ResponseCode>1002</ResponseCode><ResponseDesc>FAC Code and Amount does not match, Contact your Admin.</ResponseDesc></CashOutPaymentRequestResult></CashOutPaymentRequestResponse></soap:Body></soap:Envelope>";

JAXBContext jaxbContext = JAXBContext.newInstance(cashOutPaymentRequestEnvelope.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();

StringReader reader = new StringReader(xml);
cashOutPaymentRequestEnvelope person = (cashOutPaymentRequestEnvelope) unmarshaller.unmarshal(reader);
</code>```

My method 2

```<code>Element rootdecryptedXml = XmlParseHelper.getStringXml(xml);
String responseUnEscapedXml = StringEscapeUtils.unescapeXml(XmlParseHelper.getString("CashOutPaymentRequestResult", rootdecryptedXml));

String responseEscapedXml = XmlParseHelper.getString("CashOutPaymentRequestResult", rootdecryptedXml);

   System.out.println("step 1.. \n" + responseEscapedXml);

   Element root = XmlParseHelper.getStringXml(responseEscapedXml);

   System.out.println("step 2aa...\n" + XmlParseHelper.getString("ResponseCode", root));
   System.out.println("step 2bb...\n" + XmlParseHelper.getString("ResponseDesc", root));
</code>```

None of the methods seem to work, they all give me null

My getString function

```<code>public static String getString(String tagName, Element element) {
    if (element == null) {
        return " ";
    }

    NodeList list = element.getElementsByTagName(tagName);

    if (list != null && list.getLength() > 0) {
        NodeList subList = list.item(0).getChildNodes();
        if (subList != null && subList.getLength() > 0) {
            return subList.item(0).getNodeValue();
        }
    }
    return " ";
}
</code>```

My get StringXml

```<code>public static Element getStringXml(String responseXml) throws SAXException, IOException, ParserConfigurationException {
    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
    Document document;
    Element rootdecryptedXml = null;
    DocumentBuilder builder;
    builder = factory.newDocumentBuilder();
    document = builder.parse(new InputSource(new StringReader(responseXml)));
    rootdecryptedXml = document.getDocumentElement();
    return rootdecryptedXml;
}
</code>```

答:
I know this is late but if anyone comes here in the future searching/seeking for a solution like I did. I hope this helps you in a way. The other way is converting the XML to JSON using this org.json maven

```<code><dependency>
  <groupId>org.json</groupId>
  <artifactId>json</artifactId>
  <version>20180130</version>
</dependency>
</code>```

And your method that works around the fetch of data will be something like

```<code>private static void cashOutPayment(){
        String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"><soap:Body><CashOutPaymentRequestResponse xmlns=\"http://tempuri.org/\"><CashOutPaymentRequestResult><ResponseCode>1002</ResponseCode><ResponseDesc>FAC Code and Amount does not match, Contact your Admin.</ResponseDesc></CashOutPaymentRequestResult></CashOutPaymentRequestResponse></soap:Body></soap:Envelope>";
        JSONObject jsonObject = XML.toJSONObject(xml);
        String jsonString = jsonObject.toString();
        JSONObject jsonObject1 = ((JSONObject) jsonObject.get("soap:Envelope"))
                .getJSONObject("soap:Body").getJSONObject("CashOutPaymentRequestResponse")
                .getJSONObject("CashOutPaymentRequestResult");
        Integer responseCode = (Integer) jsonObject1.get("ResponseCode");
        String responseDesc = (String) jsonObject1.get("ResponseDesc");
        System.out.println("Response Code => "+responseCode+ " and Response Desc => "+responseDesc);
}
</code>```

Body:
Am receiving a SOAP XML response from an endpoint and I am trying to read its' contents. I have tried a couple of methods but it keeps giving me null

String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"><soap:Body><CashOutPaymentRequestResponse xmlns=\"http://tempuri.org/\"><CashOutPaymentRequestResult><ResponseCode>1002</ResponseCode><ResponseDesc>FAC Code and Amount does not match, Contact your Admin.</ResponseDesc></CashOutPaymentRequestResult></CashOutPaymentRequestResponse></soap:Body></soap:Envelope>";

JAXBContext jaxbContext = JAXBContext.newInstance(cashOutPaymentRequestEnvelope.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();

StringReader reader = new StringReader(xml);
cashOutPaymentRequestEnvelope person = (cashOutPaymentRequestEnvelope) unmarshaller.unmarshal(reader);

My method 2

Element rootdecryptedXml = XmlParseHelper.getStringXml(xml);
String responseUnEscapedXml = StringEscapeUtils.unescapeXml(XmlParseHelper.getString("CashOutPaymentRequestResult", rootdecryptedXml));

String responseEscapedXml = XmlParseHelper.getString("CashOutPaymentRequestResult", rootdecryptedXml);

   System.out.println("step 1.. \n" + responseEscapedXml);

   Element root = XmlParseHelper.getStringXml(responseEscapedXml);

   System.out.println("step 2aa...\n" + XmlParseHelper.getString("ResponseCode", root));
   System.out.println("step 2bb...\n" + XmlParseHelper.getString("ResponseDesc", root));

None of the methods seem to work, they all give me null

My getString function

public static String getString(String tagName, Element element) {
    if (element == null) {
        return " ";
    }

    NodeList list = element.getElementsByTagName(tagName);

    if (list != null && list.getLength() > 0) {
        NodeList subList = list.item(0).getChildNodes();
        if (subList != null && subList.getLength() > 0) {
            return subList.item(0).getNodeValue();
        }
    }
    return " ";
}

My get StringXml

public static Element getStringXml(String responseXml) throws SAXException, IOException, ParserConfigurationException {
    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
    Document document;
    Element rootdecryptedXml = null;
    DocumentBuilder builder;
    builder = factory.newDocumentBuilder();
    document = builder.parse(new InputSource(new StringReader(responseXml)));
    rootdecryptedXml = document.getDocumentElement();
    return rootdecryptedXml;
}

Body:
I know this is late but if anyone comes here in the future searching/seeking for a solution like I did. I hope this helps you in a way. The other way is converting the XML to JSON using this org.json maven

<dependency>
  <groupId>org.json</groupId>
  <artifactId>json</artifactId>
  <version>20180130</version>
</dependency>

And your method that works around the fetch of data will be something like

private static void cashOutPayment(){
        String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"><soap:Body><CashOutPaymentRequestResponse xmlns=\"http://tempuri.org/\"><CashOutPaymentRequestResult><ResponseCode>1002</ResponseCode><ResponseDesc>FAC Code and Amount does not match, Contact your Admin.</ResponseDesc></CashOutPaymentRequestResult></CashOutPaymentRequestResponse></soap:Body></soap:Envelope>";
        JSONObject jsonObject = XML.toJSONObject(xml);
        String jsonString = jsonObject.toString();
        JSONObject jsonObject1 = ((JSONObject) jsonObject.get("soap:Envelope"))
                .getJSONObject("soap:Body").getJSONObject("CashOutPaymentRequestResponse")
                .getJSONObject("CashOutPaymentRequestResult");
        Integer responseCode = (Integer) jsonObject1.get("ResponseCode");
        String responseDesc = (String) jsonObject1.get("ResponseDesc");
        System.out.println("Response Code => "+responseCode+ " and Response Desc => "+responseDesc);
}


问:
What are practical uses of applicative style?
I am a Scala programmer, learning Haskell now. It's easy to find practical use cases and real world examples for OO concepts, such as decorators, strategy pattern etc. Books and interwebs are filled with it.

I came to the realization that this somehow is not the case for functional concepts. Case in point: applicatives.

I am struggling to find practical use cases for applicatives. Almost all of the tutorials and books I have come across so far provide the examples of [] and Maybe. I expected applicatives to be more applicable than that, seeing all the attention they get in the FP community.

I think I understand the conceptual basis for applicatives (maybe I am wrong), and I have waited long for my moment of enlightenment. But it doesn't seem to be happening. Never while programming, have I had a moment when I would shout with a joy, "Eureka! I can use applicative here!" (except again, for [] and Maybe).

Can someone please guide me how applicatives can be used in a day-to-day programming? How do I start spotting the pattern? Thanks!

答:
I finally understood how applicatives can help in day-to-day programming with that presentation:

https://web.archive.org/web/20100818221025/http://applicative-errors-scala.googlecode.com/svn/artifacts/0.6/chunk-html/index.html

The autor shows how applicatives can help for combining validations and handling failures.

The presentation is in Scala, but the author also provides the full code example for Haskell, Java and C#.


问:
Why does fasttext yield <\s> as first entry in VSM?
I am using a large German corpus, which I have cleaned of all special characters/numbers/inter-punctuation signs.
Each line contains one sentence.

Running

```<code>fastText/./fasttext skipgram -input input.txt -output output.txt
-minCount 2 -minn 2 -maxn 8 -dim 300 -ws 5
</code>```

returns a VSM with <\s> as first entry.

From how I understand it, there are white spaces left in the document that are interpreted as a token.

Is that correct?

And how can I get rid of them and/or the <\s> in the VSM?

Thank you.

答:
By convention the fasttext tool converts any newlines in the input file to a pseudoword token '<\s>', to represent an end-of-string ('EOS').

See the discussion in the Python binding Markdown docs:

https://github.com/facebookresearch/fastText/blob/main/python/README.md#important-preprocessing-data--encoding-conventions

The newline character is used to delimit lines of text. In particular,
the EOS token is appended to a line of text if a newline character is
encountered. The only exception is if the number of tokens exceeds the
MAX_LINE_SIZE constant as defined in the Dictionary header. This means
if you have text that is not separate by newlines, such as the fil9
dataset, it will be broken into chunks with MAX_LINE_SIZE of tokens
and the EOS token is not appended.

The length of a token is the number of UTF-8 characters by considering
the leading two bits of a byte to identify subsequent bytes of a
multi-byte sequence. Knowing this is especially important when
choosing the minimum and maximum length of subwords. Further, the EOS
token (as specified in the Dictionary header) is considered a
character and will not be broken into subwords.

(Though only mentioned in that doc about the Python bindings, it's definitely defined/implemented in the core C++ code, especially the dictionary.cc file.)

To eliminate that word-token, you'd have to strip all newlines from your input file.

Body:
I am using a large German corpus, which I have cleaned of all special characters/numbers/inter-punctuation signs.
Each line contains one sentence.

Running

fastText/./fasttext skipgram -input input.txt -output output.txt
-minCount 2 -minn 2 -maxn 8 -dim 300 -ws 5

returns a VSM with <\s> as first entry.

From how I understand it, there are white spaces left in the document that are interpreted as a token.

Is that correct?

And how can I get rid of them and/or the <\s> in the VSM?

Thank you.


问:
Android: bottom of TextInputEditText always scrolling to top of softkeyboard
I'm using a TextInputEditText inside a TextInputLayout, which is inside a NestedScrollView:

```<code><androidx.core.widget.NestedScrollView
    android:id="@+id/nsv"
    android:layout_width="0dp"
    android:layout_height="0dp"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toBottomOf="@id/toolbar">

    <LinearLayout
        android:id="@+id/ll"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:fitsSystemWindows="true"
        android:orientation="vertical">

        <com.google.android.material.textfield.TextInputLayout
            android:id="@+id/textField"
            style="@style/Widget.MaterialComponents.TextInputLayout.OutlinedBox.Dense"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:hint="@{hint}"
            android:padding="@dimen/padding_items"
            app:startIconContentDescription="@{iconDescription}"
            app:startIconDrawable="@{icon}">

            <com.google.android.material.textfield.TextInputEditText
                android:id="@+id/et"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:importantForAutofill="no"
                android:text="@={text}" />
        </com.google.android.material.textfield.TextInputLayout>
    </LinearLayout>
</androidx.core.widget.NestedScrollView>
</code>```

When I enter a long text, so that the TextInputEditText takes more vertical space than is available above the softkeyboard, the upper part of the TextInputEditText scrolls out of view.
This is perfectly fine.

But if I manually scroll up the nestedScrollView so that I can edit the first words of the TextInputEditText, it scrolls back all the time and this happens whenever I click (= set the cursor) into the TextInputEditText or enter a letter.

The bottom edge of the TextInputEditText seems to align with the top edge of the softkeyboard every time I interact with it, which is why the top edge scrolls out of view even when the cursor is placed at the very beginning.

This makes it impossible to edit the beginning of the long string, because you have to scroll up after each letter and see what happens, or you have to type blindly.

How can I work around this?

Steps to reproduce:

enter long text

scroll to the beginning of the text

place cursor at the beginning of the TextInputEditText

scroll again to the beginning of the text so that cursor is visible:
cursor at the beginning of the TextInputEditText

type letters:
cursor no longer visible, because scrolled out of view

答:
I was stumped by this for a long time, too. The key is to add the following to your TextInputEditText

```<code>app:textInputLayoutFocusedRectEnabled="false"
</code>```

as described here: Android TextInputLayout writing focus issue when height is bigger than the screen

Body:
I'm using a TextInputEditText inside a TextInputLayout, which is inside a NestedScrollView:

<androidx.core.widget.NestedScrollView
    android:id="@+id/nsv"
    android:layout_width="0dp"
    android:layout_height="0dp"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toBottomOf="@id/toolbar">

    <LinearLayout
        android:id="@+id/ll"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:fitsSystemWindows="true"
        android:orientation="vertical">

        <com.google.android.material.textfield.TextInputLayout
            android:id="@+id/textField"
            style="@style/Widget.MaterialComponents.TextInputLayout.OutlinedBox.Dense"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:hint="@{hint}"
            android:padding="@dimen/padding_items"
            app:startIconContentDescription="@{iconDescription}"
            app:startIconDrawable="@{icon}">

            <com.google.android.material.textfield.TextInputEditText
                android:id="@+id/et"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:importantForAutofill="no"
                android:text="@={text}" />
        </com.google.android.material.textfield.TextInputLayout>
    </LinearLayout>
</androidx.core.widget.NestedScrollView>

When I enter a long text, so that the TextInputEditText takes more vertical space than is available above the softkeyboard, the upper part of the TextInputEditText scrolls out of view.
This is perfectly fine.

But if I manually scroll up the nestedScrollView so that I can edit the first words of the TextInputEditText, it scrolls back all the time and this happens whenever I click (= set the cursor) into the TextInputEditText or enter a letter.

The bottom edge of the TextInputEditText seems to align with the top edge of the softkeyboard every time I interact with it, which is why the top edge scrolls out of view even when the cursor is placed at the very beginning.

This makes it impossible to edit the beginning of the long string, because you have to scroll up after each letter and see what happens, or you have to type blindly.

How can I work around this?

Steps to reproduce:

enter long text

scroll to the beginning of the text

place cursor at the beginning of the TextInputEditText

scroll again to the beginning of the text so that cursor is visible:
cursor at the beginning of the TextInputEditText

type letters:
cursor no longer visible, because scrolled out of view

Body:
I was stumped by this for a long time, too. The key is to add the following to your TextInputEditText

app:textInputLayoutFocusedRectEnabled="false"

as described here: Android TextInputLayout writing focus issue when height is bigger than the screen


问:
Migration: No DbContext was found in assembly
Using VS Community 2017. I have tried to create initial migration with error message saying:

Both Entity Framework Core and Entity Framework 6 are installed. The Entity Framework Core tools are running. Use 'EntityFramework\Add-Migration' for Entity Framework 6.
  No DbContext was found in assembly 'Test_Project'. Ensure that you're using the correct assembly and that the type is neither abstract nor generic.

... code in my dbcontext:

```<code>protected override void OnModelCreating(DbModelBuilder mb)
{
    base.OnModelCreating(mb);

    mb.Entity<Stuff>().ToTable("Stuff");

}

public DbSet<Stuff> Stuff{ get; set; }
</code>```

答: When I had this issue, I had just renamed an existing project to "EF", then renamed all namespaces to "EF".  What worked for me was I changed the Assembly name to something other than "EF" (Project properties/Application/Assembly Name).  I was using VS Community 2019.</p>

Body:
Using VS Community 2017. I have tried to create initial migration with error message saying:

Both Entity Framework Core and Entity Framework 6 are installed. The Entity Framework Core tools are running. Use 'EntityFramework\Add-Migration' for Entity Framework 6.
  No DbContext was found in assembly 'Test_Project'. Ensure that you're using the correct assembly and that the type is neither abstract nor generic.

... code in my dbcontext:

protected override void OnModelCreating(DbModelBuilder mb)
{
    base.OnModelCreating(mb);

    mb.Entity<Stuff>().ToTable("Stuff");

}

public DbSet<Stuff> Stuff{ get; set; }


问:
Why is R's mean function different than SciPy/NumPy?
Why are these different? I read on the R documentation that mean is actually trimmed mean but I can't figure out how to reproduce this using SciPy/NumPy.

mds_null = pd.read_csv("https://pastebin.com/raw/c1Zwb4pu", index_col=0, sep="\t")
mds_null.mean(axis=0)
# MDS1    4.394633e-16
# MDS2   -2.590520e-16
# dtype: float64
```
```> mds_null = read.table("https://pastebin.com/raw/c1Zwb4pu", sep=sep, row.names=1, header = TRUE, check.names=FALSE)
> apply(mds_null, 2, mean)
         MDS1          MDS2
 4.442518e-16 -2.714842e-16
```

答: Discussion above pointed out that these are due to rounding specs in different languages.</p>

Body:
Why are these different? I read on the R documentation that mean is actually trimmed mean but I can't figure out how to reproduce this using SciPy/NumPy.

mds_null = pd.read_csv("https://pastebin.com/raw/c1Zwb4pu", index_col=0, sep="\t")
mds_null.mean(axis=0)
# MDS1    4.394633e-16
# MDS2   -2.590520e-16
# dtype: float64

> mds_null = read.table("https://pastebin.com/raw/c1Zwb4pu", sep=sep, row.names=1, header = TRUE, check.names=FALSE)
> apply(mds_null, 2, mean)
         MDS1          MDS2
 4.442518e-16 -2.714842e-16


问:
Show data points only for selected date on a slider via Leaflet in R Shiny
I have created a Timeseries visualization map application that updates data points on Leaflet with the progression of the time series animation.

Now I am trying to add another functionality where when the user selects a particular week from the time series, the map will just show the points for that date only. One way I am thinking to do this would be add the ability to define a range with two toggles, so when the user drags the start and end toggle to the same week, the maps will just show the data points for that week. Or maybe there is a better way of doing it.

How can a leaftet be created such that not only does it show the whole data points for the entire time series like an animation (the current  code does this), but also have the ability to show only data points for selected date on the slider?

Code:

```<code># This is a Shiny time series map web application
library(shiny)
library(tidyverse)
library(tidyr)
library(leaflet)
library(xts)

xts_to_tibble <- function(xts_obj) {
  data.frame(index(xts_obj), coredata(xts_obj)) %>%
    set_names(c("date", names(xts_obj))) %>%
    as_tibble()
}

# Create sample data
Date <- c(
  "2014-04-08", "2014-06-04", "2014-04-30",
  "2014-05-30", "2014-05-01"
)
lat <- as.numeric(c(
  "45.53814", "45.51076", "45.43560", "45.54332",
  "45.52234"
))
lon <- as.numeric(c(
  "-73.63672", "-73.61029", "-73.60100",
  "-73.56000 ", "-73.59022"
))
id <- as.numeric(c("1", "2", "3", "4", "5"))

# Create a df from the above columns
df <- data.frame(id, lat, lon, Date)
df$Year <- lubridate::year(df$Date)
df$Month <- lubridate::month(df$Date, label = TRUE, abbr = FALSE)
df$Week <- lubridate::week(df$Date)
df$Date <- as.Date(df$Date)
ui <- fluidPage(

  # Title
  titlePanel("Time Series Visiualization Map"),
  sidebarLayout(

    # Define the sidebar
    sidebarPanel(
      radioButtons(
        inputId = "Frequency",
        label = " Select Timer Series Frequency",
        choices = c(
          "weeks",
          "months",
          "years"
        ),
        selected = "weeks",
        inline = T
      ),
      uiOutput("Time_Series_UI")
    ),
    mainPanel(
      leafletOutput("Time_Series_Map")
    ),
  )
)

# Define server logic required to draw a histogram
server <- function(input, output) {

  # Render slider input depending on data frequency

  observe({
    # Create an xts object
    df_xts <- xts(df, order.by = as.Date(df$Date))

    # All_Dates = unique(df$Start_Date)

    Filtered_Dates <- df_xts[xts::endpoints(
      df_xts,
      on = input$Frequency
    )] %>% xts_to_tibble()

    output$Time_Series_UI <- renderUI({
      sliderInput("Date", "Date:",
        min = pull(slice_min(Filtered_Dates, date), date),
        max = pull(slice_max(Filtered_Dates, date), date),
        value = pull(slice_min(Filtered_Dates, date), date),
        step = 1,
        timeFormat = "%YYYY-%MM-%DD",
        animate = T
      )
    })
  })

  # Filter data for the date selected
  Filtered_Data <- reactive({
    req(input$Date)
    filter(df, Date == input$Date)
  })

  # Create the leaflet map
  output$Time_Series_Map <- renderLeaflet({
    leaflet(df) %>%
      addTiles() %>%
      setView(lat = 0, lng = 0, zoom = 2)
  })

  # Create data markers for selected date
  observe({
    # print(input$Date)

    leafletProxy("Time_Series_Map", data = Filtered_Data()) %>%
      addCircleMarkers(
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })
}

# Run the application
shinyApp(ui = ui, server = server)
</code>```

答:
I think the answer is not too difficult in this case currently your last observer looks like this:

  observe({
    # print(input$Date)

    leafletProxy("Time_Series_Map", data = Filtered_Data())     %>%
      addCircleMarkers(
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })
```
<p>This observer every time adds markers in <code>Filtered_Data()</code> however markers are never removed. By using a group and clearing that group you old markers are removed each time:</p>
<pre class="lang-r prettyprint-override"><code>  observe({
    leafletProxy("Time_Series_Map", data = Filtered_Data())     %>%
      clearGroup("points") %>%
      addCircleMarkers(group='points',
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })
</code>```

Body:
I have created a Timeseries visualization map application that updates data points on Leaflet with the progression of the time series animation.

Now I am trying to add another functionality where when the user selects a particular week from the time series, the map will just show the points for that date only. One way I am thinking to do this would be add the ability to define a range with two toggles, so when the user drags the start and end toggle to the same week, the maps will just show the data points for that week. Or maybe there is a better way of doing it.

How can a leaftet be created such that not only does it show the whole data points for the entire time series like an animation (the current  code does this), but also have the ability to show only data points for selected date on the slider?

Code:

# This is a Shiny time series map web application
library(shiny)
library(tidyverse)
library(tidyr)
library(leaflet)
library(xts)

xts_to_tibble <- function(xts_obj) {
  data.frame(index(xts_obj), coredata(xts_obj)) %>%
    set_names(c("date", names(xts_obj))) %>%
    as_tibble()
}

# Create sample data
Date <- c(
  "2014-04-08", "2014-06-04", "2014-04-30",
  "2014-05-30", "2014-05-01"
)
lat <- as.numeric(c(
  "45.53814", "45.51076", "45.43560", "45.54332",
  "45.52234"
))
lon <- as.numeric(c(
  "-73.63672", "-73.61029", "-73.60100",
  "-73.56000 ", "-73.59022"
))
id <- as.numeric(c("1", "2", "3", "4", "5"))

# Create a df from the above columns
df <- data.frame(id, lat, lon, Date)
df$Year <- lubridate::year(df$Date)
df$Month <- lubridate::month(df$Date, label = TRUE, abbr = FALSE)
df$Week <- lubridate::week(df$Date)
df$Date <- as.Date(df$Date)
ui <- fluidPage(

  # Title
  titlePanel("Time Series Visiualization Map"),
  sidebarLayout(

    # Define the sidebar
    sidebarPanel(
      radioButtons(
        inputId = "Frequency",
        label = " Select Timer Series Frequency",
        choices = c(
          "weeks",
          "months",
          "years"
        ),
        selected = "weeks",
        inline = T
      ),
      uiOutput("Time_Series_UI")
    ),
    mainPanel(
      leafletOutput("Time_Series_Map")
    ),
  )
)

# Define server logic required to draw a histogram
server <- function(input, output) {

  # Render slider input depending on data frequency

  observe({
    # Create an xts object
    df_xts <- xts(df, order.by = as.Date(df$Date))

    # All_Dates = unique(df$Start_Date)

    Filtered_Dates <- df_xts[xts::endpoints(
      df_xts,
      on = input$Frequency
    )] %>% xts_to_tibble()

    output$Time_Series_UI <- renderUI({
      sliderInput("Date", "Date:",
        min = pull(slice_min(Filtered_Dates, date), date),
        max = pull(slice_max(Filtered_Dates, date), date),
        value = pull(slice_min(Filtered_Dates, date), date),
        step = 1,
        timeFormat = "%YYYY-%MM-%DD",
        animate = T
      )
    })
  })

  # Filter data for the date selected
  Filtered_Data <- reactive({
    req(input$Date)
    filter(df, Date == input$Date)
  })

  # Create the leaflet map
  output$Time_Series_Map <- renderLeaflet({
    leaflet(df) %>%
      addTiles() %>%
      setView(lat = 0, lng = 0, zoom = 2)
  })

  # Create data markers for selected date
  observe({
    # print(input$Date)

    leafletProxy("Time_Series_Map", data = Filtered_Data()) %>%
      addCircleMarkers(
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })
}

# Run the application
shinyApp(ui = ui, server = server)

Body:
I think the answer is not too difficult in this case currently your last observer looks like this:

  observe({
    # print(input$Date)

    leafletProxy("Time_Series_Map", data = Filtered_Data())     %>%
      addCircleMarkers(
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })

This observer every time adds markers in Filtered_Data() however markers are never removed. By using a group and clearing that group you old markers are removed each time:

  observe({
    leafletProxy("Time_Series_Map", data = Filtered_Data())     %>%
      clearGroup("points") %>%
      addCircleMarkers(group='points',
        lng = ~lon, lat = ~lat,
        popup = ~id
      )
  })


问:
React useReducer sets the whole state as undefined
I'm having a problem where updating my Context state with a reducer, sets the whole state as undefined... This is my code:

```<code>const [state, dispatch] = useReducer(AccountReducer, initialState);
</code>```

The call to the Dispatch

```<code> dispatch({
  type: SET_TOKEN,
  payload: token
})
</code>```

And the reducer

```<code>  const AccountReducer = (state, action) => {
   switch (action.type) {
    case SET_TOKEN:
      return {
        ...state,
        token: action.payload
      }

    default:
      break
  }
};
</code>```

After I make the dispatch the whole state is set to undefined.... Any ideas on why this is happening?

答:
A reducer must return the new state value each time it is called. In your default case you return nothing (undefined), which will be your next state.

Simply return the last state in the default case:

```<code> const AccountReducer = (state, action) => {
   switch (action.type) {
    case SET_TOKEN:
      return {
        ...state,
        token: action.payload
      }

    default:
      return state; // Return the same value passed in to be used as the next state
  }
};
</code>```

I'm working under the assumption that SET_TOKEN is a variable shared between the dispatch call and the reducer. If they do not match, this will also cause problems.

Body:
I'm having a problem where updating my Context state with a reducer, sets the whole state as undefined... This is my code:

const [state, dispatch] = useReducer(AccountReducer, initialState);

The call to the Dispatch

 dispatch({
  type: SET_TOKEN,
  payload: token
})

And the reducer

  const AccountReducer = (state, action) => {
   switch (action.type) {
    case SET_TOKEN:
      return {
        ...state,
        token: action.payload
      }

    default:
      break
  }
};

After I make the dispatch the whole state is set to undefined.... Any ideas on why this is happening?

Body:
A reducer must return the new state value each time it is called. In your default case you return nothing (undefined), which will be your next state.

Simply return the last state in the default case:

 const AccountReducer = (state, action) => {
   switch (action.type) {
    case SET_TOKEN:
      return {
        ...state,
        token: action.payload
      }

    default:
      return state; // Return the same value passed in to be used as the next state
  }
};

I'm working under the assumption that SET_TOKEN is a variable shared between the dispatch call and the reducer. If they do not match, this will also cause problems.


问:
Scala version issues while using spark runtime configuration on Apache Hop
I have spark cluster running over kubernetes (v3.1.2), which comes with scala jars of v2.12.10

While running any pipeline with Scala provided with HOP results into the following error, using HOP version 1.1.0 (Comes with scala Jars of v2.11.12)

```<code>===[Starting HopRun]=========================================================
2022/01/31 11:33:49 - HopRun - Start of Hop Run
2022/01/31 11:33:49 - HopRun - Enabling project 'samples'
2022/01/31 11:33:49 - HopRun - Starting pipeline: .\config\projects\samples\transforms\calculator-basic.hpl
2022/01/31 11:33:55 - General - Created Apache Beam pipeline with name 'calculator-basic'
2022/01/31 11:33:55 - General - Handled generic transform (TRANSFORM) : Input Data, gets data from 0 previous transform(s), targets=0, infos=0
2022/01/31 11:33:55 - General - Handled generic transform (TRANSFORM) : Calculator, gets data from 1 previous transform(s), targets=0, infos=0
2022/01/31 11:33:55 - General - Executing this pipeline using the Beam Pipeline Engine with run configuration 'Spark'
2022/01/31 11:33:56 - General - ERROR: Error starting the Beam pipeline
2022/01/31 11:33:56 - General - ERROR: org.apache.hop.core.exception.HopException:
2022/01/31 11:33:56 - General - Error executing pipeline with runner Spark
2022/01/31 11:33:56 - General - scala.Product.$init$(Lscala/Product;)V
2022/01/31 11:33:56 - General -
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:244)
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.startThreads(BeamPipelineEngine.java:267)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.runPipeline(HopRun.java:245)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.runPipeline(HopRun.java:201)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.run(HopRun.java:141)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.main(HopRun.java:737)
2022/01/31 11:33:56 - General - Caused by: java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$DeprecatedConfig.<init>(SparkConf.scala:799)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$.<init>(SparkConf.scala:596)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf.contains(SparkConf.scala:455)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.translation.SparkContextFactory.createSparkContext(SparkContextFactory.java:89)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.translation.SparkContextFactory.getSparkContext(SparkContextFactory.java:67)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:220)
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:238)
2022/01/31 11:33:56 - General -         ... 5 more
</code>```

答:
I hope you were able to solve this by now?
If not it will probably be best to align those jars and try again.

As this is a minor version difference you can probably replace the jar in the Hop installation and re-create the fat jar

Body:
I have spark cluster running over kubernetes (v3.1.2), which comes with scala jars of v2.12.10

While running any pipeline with Scala provided with HOP results into the following error, using HOP version 1.1.0 (Comes with scala Jars of v2.11.12)

===[Starting HopRun]=========================================================
2022/01/31 11:33:49 - HopRun - Start of Hop Run
2022/01/31 11:33:49 - HopRun - Enabling project 'samples'
2022/01/31 11:33:49 - HopRun - Starting pipeline: .\config\projects\samples\transforms\calculator-basic.hpl
2022/01/31 11:33:55 - General - Created Apache Beam pipeline with name 'calculator-basic'
2022/01/31 11:33:55 - General - Handled generic transform (TRANSFORM) : Input Data, gets data from 0 previous transform(s), targets=0, infos=0
2022/01/31 11:33:55 - General - Handled generic transform (TRANSFORM) : Calculator, gets data from 1 previous transform(s), targets=0, infos=0
2022/01/31 11:33:55 - General - Executing this pipeline using the Beam Pipeline Engine with run configuration 'Spark'
2022/01/31 11:33:56 - General - ERROR: Error starting the Beam pipeline
2022/01/31 11:33:56 - General - ERROR: org.apache.hop.core.exception.HopException:
2022/01/31 11:33:56 - General - Error executing pipeline with runner Spark
2022/01/31 11:33:56 - General - scala.Product.$init$(Lscala/Product;)V
2022/01/31 11:33:56 - General -
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:244)
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.startThreads(BeamPipelineEngine.java:267)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.runPipeline(HopRun.java:245)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.runPipeline(HopRun.java:201)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.run(HopRun.java:141)
2022/01/31 11:33:56 - General -         at org.apache.hop.run.HopRun.main(HopRun.java:737)
2022/01/31 11:33:56 - General - Caused by: java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$DeprecatedConfig.<init>(SparkConf.scala:799)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$.<init>(SparkConf.scala:596)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala)
2022/01/31 11:33:56 - General -         at org.apache.spark.SparkConf.contains(SparkConf.scala:455)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.translation.SparkContextFactory.createSparkContext(SparkContextFactory.java:89)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.translation.SparkContextFactory.getSparkContext(SparkContextFactory.java:67)
2022/01/31 11:33:56 - General -         at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:220)
2022/01/31 11:33:56 - General -         at org.apache.hop.beam.engines.BeamPipelineEngine.executePipeline(BeamPipelineEngine.java:238)
2022/01/31 11:33:56 - General -         ... 5 more


问:
Error bars look huge in R, but not in Excel
I am having an issue with my error bars. they look gigantic in R, but look much more accurate in Excel. Example in R:

Example in Excel:

Code

```<code>Graph = subset(dat,dat$Condition != "z" & dat$Environment != "a")
    Graph %>%
      group_by(Gate = fct_inorder(Gate), Sound, Block) %>%
      summarize(Accuracy = mean(Correct),
                sd = sd(Correct)) %>%
      ggplot(aes(x = Gate, y = Accuracy, color = Sound, group = Sound)) +
      geom_line() +
      geom_errorbar(aes(ymin = Accuracy - sd, ymax = Accuracy + sd,)) +
      scale_y_continuous(labels = scales::percent)+
      facet_wrap(~Block) +
      theme_minimal()
</code>```

Reproducible sample

```<code>        set.seed(42)
    n <- 100
    dat <- data.frame(Participant=1:n,
                      Environment=rep(LETTERS[1:2], n/2),
                      Condition=rep(LETTERS[25:26], n/2),
                      Gate= sample(1:5, n, replace=TRUE),
                      Block= sample(1:2,n, replace=TRUE),
                      Sound=rep(LETTERS[3:4], n/2),
                      Correct=sample(0:1, n, replace=TRUE)
)
</code>```

答:
They look huge because I was calculating standard deviation instead of standard error. In the above code, I changed

```<code>sd(Correct)
</code>```

to

```<code>sd(Correct)/sqrt(length(Correct))
</code>```

Body:
I am having an issue with my error bars. they look gigantic in R, but look much more accurate in Excel. Example in R:

Example in Excel:

Code

Graph = subset(dat,dat$Condition != "z" & dat$Environment != "a")
    Graph %>%
      group_by(Gate = fct_inorder(Gate), Sound, Block) %>%
      summarize(Accuracy = mean(Correct),
                sd = sd(Correct)) %>%
      ggplot(aes(x = Gate, y = Accuracy, color = Sound, group = Sound)) +
      geom_line() +
      geom_errorbar(aes(ymin = Accuracy - sd, ymax = Accuracy + sd,)) +
      scale_y_continuous(labels = scales::percent)+
      facet_wrap(~Block) +
      theme_minimal()

Reproducible sample

        set.seed(42)
    n <- 100
    dat <- data.frame(Participant=1:n,
                      Environment=rep(LETTERS[1:2], n/2),
                      Condition=rep(LETTERS[25:26], n/2),
                      Gate= sample(1:5, n, replace=TRUE),
                      Block= sample(1:2,n, replace=TRUE),
                      Sound=rep(LETTERS[3:4], n/2),
                      Correct=sample(0:1, n, replace=TRUE)
)

Body:
They look huge because I was calculating standard deviation instead of standard error. In the above code, I changed

sd(Correct)

to

sd(Correct)/sqrt(length(Correct))