问:
Android HttpUrlConneion upload a large file cause 413, but getResponseCode can't return and throw another exception
In my application I use HttpUrlConnection to upload file to server, when I want to upload a large file, nginx give me a 413 exception code, but HttpUrlConnection's getResponseCode can't catch the exception, I got another exception message.
Here is the exception message:

```<code>error asyncExecute:Write error: ssl=0x7ec7b80008: I/O error during system call, Connection reset by peer
</code>```

Here is my code:

```<code>    URL originUrl = new URL(url);
    String protocol = originUrl.getProtocol();
    if (TextUtils.equals(protocol, "http")) {
        port = 80;
    }
    mURL = new URL(protocol, originUrl.getHost(), port, originUrl.getFile());
    mConn = (HttpURLConnection) mURL.openConnection();

    if (mConn instanceof HttpsURLConnection) {
        selfSignedCertificate = true;
        SSLCustomSocketFactory factory = new SSLCustomSocketFactory(selfSignedCertificate ? SSLCustomSocketFactory.getSocketFactory()
                : (SSLSocketFactory) SSLSocketFactory.getDefault());
        ((HttpsURLConnection) mConn).setSSLSocketFactory(factory);
        ((HttpsURLConnection) mConn).setHostnameVerifier(new HostnameVerifier() {
            @Override
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        });
    }
    mConn.setRequestMethod("POST");
    mConn.setDoOutput(true);
    mConn.setDoInput(true);
    mConn.setUseCaches(false);
    mConn.setConnectTimeout(30000);
    mConn.setReadTimeout(30000);
    mConn.setRequestProperty("User-agent", "xxxx");//xxxx涉及到项目中信息不展示
    mConn.setRequestProperty("Connection", "Keep-Alive");
    mConn.setRequestProperty("Charset", "UTF-8");
    mConn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + BOUNDARY);
    mConn.setRequestProperty("Expect", "100-Continue");
    mConn.setChunkedStreamingMode(0);
    //request headers
    for (Map.Entry<String, String> item : headers.entrySet()) {
          mConn.setRequestProperty(item.getKey(), item.getValue());
     }
    mConn.connect();
    //get outputStream
    DataOutputStream out = new DataOutputStream(connect.getOutputStream());
    //write params
    out.write(params.getBytes());
    out.flush();
    StringBuffer strBuf = new StringBuffer();
    for (String key : paramsMap.keySet()){
        strBuf.append(TWO_HYPHENS);
        strBuf.append(BOUNDARY);
        strBuf.append(LINE_END);
        strBuf.append("Content-Disposition: form-data; name=\"" + key + "\"");
        strBuf.append(LINE_END);

        strBuf.append("Content-Type: " + "text/plain" );
        strBuf.append(LINE_END);
        strBuf.append("Content-Length: "+paramsMap.get(key).length());
        strBuf.append(LINE_END);
        strBuf.append(LINE_END);
        strBuf.append(paramsMap.get(key));
        strBuf.append(LINE_END);
    }
    String paramsString = strBuf.toString();
    out.write(paramsString.getBytes());
    out.flush();
    String fileName = UriUtils.getFileNameByUri(mContext, fileUri);
    String mimeType = UriUtils.getMimeType(mContext, fileUri);
    long fileLength = UriUtils.getFileLength(mContext, fileUri);
    if(!TextUtils.isEmpty(filename)) {
        fileName = filename;
    }
    //add file headers
    out.write(getFileHeaderParamsString(fileKey, fileName, mimeType, fileLength).getBytes());
    //add file body
    String filePath = UriUtils.getFilePath(fileUri);
    InputStream in = null;
    try {
        if (!TextUtils.isEmpty(filePath) && new File(filePath).exists()) {
            in = new FileInputStream(new File(filePath));
        } else {
            in = mContext.getContentResolver().openInputStream(fileUri);
        }
        byte[] tmp = new byte[2048];
        int l;
        long sum = 0;
        while ((l = in.read(tmp)) != -1) {
            out.write(tmp, 0, l);
            sum += l;
            if (callback != null) {
                callback.onProgress(fileLength, sum);
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (in != null) {
            in.close();
        }
    }
    //add file end
    out.write(getFileEndString().getBytes());
    out.flush();
    HttpResponse response = new HttpResponse();
    //can't get response code here
    response.code = mConn.getResponseCode();
    if(response.code == HttpURLConnection.HTTP_OK) {
        response.contentLength = mConn.getContentLength();
        response.inputStream = mConn.getInputStream();
        response.content = parseStream(response.inputStream);
    }else {
        response.errorStream = mConn.getErrorStream();
        response.content = parseStream(response.errorStream);
    }
    ......

 public static class SSLCustomSocketFactory extends SSLSocketFactory {
    private static final String TAG = "SSLCustomSocketFactory";
    private static final String[] TLS_SUPPORT_VERSION = {"TLSv1.1", "TLSv1.2"};
    private static final String KEY_PASS = "";
    final SSLSocketFactory delegate;
    static SSLContext  sslContext;
    static SSLSocketFactory mSocketFactory;

    public SSLCustomSocketFactory(SSLSocketFactory base) {
        delegate = base;
    }

    @Override
    public String[] getDefaultCipherSuites() {
        return delegate.getDefaultCipherSuites();
    }

    @Override
    public String[] getSupportedCipherSuites() {
        return delegate.getSupportedCipherSuites();
    }

    @Override
    public Socket createSocket(Socket s, String host, int port, boolean autoClose) throws IOException {
        return patch(delegate.createSocket(s, host, port, autoClose));
    }

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        return patch(delegate.createSocket(host, port));
    }

    @Override
    public Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException, UnknownHostException {
        return patch(delegate.createSocket(host, port, localHost, localPort));
    }

    @Override
    public Socket createSocket(InetAddress host, int port) throws IOException {
        return patch(delegate.createSocket(host, port));
    }

    @Override
    public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
        return patch(delegate.createSocket(address, port, localAddress, localPort));
    }

    private Socket patch(Socket s) {
        if(s instanceof SSLSocket) {
            ((SSLSocket) s).setEnabledProtocols(TLS_SUPPORT_VERSION);
        }
        return s;
    }

    public static synchronized SSLSocketFactory getSocketFactory() {
        if(mSocketFactory == null) {
            try {
                CertificateFactory cf = CertificateFactory.getInstance("X.509");
                AssetManager am = context().getAssets();
                String[] certsPaths = am.list("xxx_certs");
                String keyStoreType = KeyStore.getDefaultType();
                KeyStore keyStore = KeyStore.getInstance(keyStoreType);
                keyStore.load(null, null);
                for (int i = 0; i < certsPaths.length; i++) {
                    String certPath = certsPaths[i];
                    InputStream caInput = null;
                    try {
                        caInput = am.open("xxxx_certs/" + certPath);
                        Certificate ca = cf.generateCertificate(caInput);
                        System.out.println("ca=" + ((X509Certificate) ca).getSubjectDN());
                        keyStore.setCertificateEntry("ca" + i, ca);
                    } catch (Exception e) {
                        e.printStackTrace();
                    } finally {
                        if (caInput != null) {
                            caInput.close();
                        }
                    }
                }

                sslContext = SSLContext.getInstance("TLS");
                TrustManagerFactory tmf = TrustManagerFactory.
                        getInstance(TrustManagerFactory.getDefaultAlgorithm());
                tmf.init(keyStore);

                sslContext.init(null, tmf.getTrustManagers(), null);
                mSocketFactory = sslContext.getSocketFactory();
            } catch (Throwable e) {
                e.printStackTrace();
            }
        }
        return mSocketFactory;
    }
}
</code>```

Here is the nagix return:

```<code><html>

   <head>
      <title>413 Request Entity Too Large</title>
   </head>

   <body>
      <center>
          <h1>413 Request Entity Too Large</h1>
      </center>
      <hr>
      <center>nginx/1.17.5</center>
   </body>

 <html>
</code>```

How I can get the right exception code and exception message?

答:
This is a payload limit determined by the server, there is nothing you can do in your application to accept this payload. You must verify that the format and the amount of data sent is correct.
If the server is yours, you must change the limit. Look at this: How to edit nginx.conf to increase file size upload</p>

Body:
In my application I use HttpUrlConnection to upload file to server, when I want to upload a large file, nginx give me a 413 exception code, but HttpUrlConnection's getResponseCode can't catch the exception, I got another exception message.
Here is the exception message:

error asyncExecute:Write error: ssl=0x7ec7b80008: I/O error during system call, Connection reset by peer

Here is my code:

    URL originUrl = new URL(url);
    String protocol = originUrl.getProtocol();
    if (TextUtils.equals(protocol, "http")) {
        port = 80;
    }
    mURL = new URL(protocol, originUrl.getHost(), port, originUrl.getFile());
    mConn = (HttpURLConnection) mURL.openConnection();

    if (mConn instanceof HttpsURLConnection) {
        selfSignedCertificate = true;
        SSLCustomSocketFactory factory = new SSLCustomSocketFactory(selfSignedCertificate ? SSLCustomSocketFactory.getSocketFactory()
                : (SSLSocketFactory) SSLSocketFactory.getDefault());
        ((HttpsURLConnection) mConn).setSSLSocketFactory(factory);
        ((HttpsURLConnection) mConn).setHostnameVerifier(new HostnameVerifier() {
            @Override
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        });
    }
    mConn.setRequestMethod("POST");
    mConn.setDoOutput(true);
    mConn.setDoInput(true);
    mConn.setUseCaches(false);
    mConn.setConnectTimeout(30000);
    mConn.setReadTimeout(30000);
    mConn.setRequestProperty("User-agent", "xxxx");//xxxx涉及到项目中信息不展示
    mConn.setRequestProperty("Connection", "Keep-Alive");
    mConn.setRequestProperty("Charset", "UTF-8");
    mConn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + BOUNDARY);
    mConn.setRequestProperty("Expect", "100-Continue");
    mConn.setChunkedStreamingMode(0);
    //request headers
    for (Map.Entry<String, String> item : headers.entrySet()) {
          mConn.setRequestProperty(item.getKey(), item.getValue());
     }
    mConn.connect();
    //get outputStream
    DataOutputStream out = new DataOutputStream(connect.getOutputStream());
    //write params
    out.write(params.getBytes());
    out.flush();
    StringBuffer strBuf = new StringBuffer();
    for (String key : paramsMap.keySet()){
        strBuf.append(TWO_HYPHENS);
        strBuf.append(BOUNDARY);
        strBuf.append(LINE_END);
        strBuf.append("Content-Disposition: form-data; name=\"" + key + "\"");
        strBuf.append(LINE_END);

        strBuf.append("Content-Type: " + "text/plain" );
        strBuf.append(LINE_END);
        strBuf.append("Content-Length: "+paramsMap.get(key).length());
        strBuf.append(LINE_END);
        strBuf.append(LINE_END);
        strBuf.append(paramsMap.get(key));
        strBuf.append(LINE_END);
    }
    String paramsString = strBuf.toString();
    out.write(paramsString.getBytes());
    out.flush();
    String fileName = UriUtils.getFileNameByUri(mContext, fileUri);
    String mimeType = UriUtils.getMimeType(mContext, fileUri);
    long fileLength = UriUtils.getFileLength(mContext, fileUri);
    if(!TextUtils.isEmpty(filename)) {
        fileName = filename;
    }
    //add file headers
    out.write(getFileHeaderParamsString(fileKey, fileName, mimeType, fileLength).getBytes());
    //add file body
    String filePath = UriUtils.getFilePath(fileUri);
    InputStream in = null;
    try {
        if (!TextUtils.isEmpty(filePath) && new File(filePath).exists()) {
            in = new FileInputStream(new File(filePath));
        } else {
            in = mContext.getContentResolver().openInputStream(fileUri);
        }
        byte[] tmp = new byte[2048];
        int l;
        long sum = 0;
        while ((l = in.read(tmp)) != -1) {
            out.write(tmp, 0, l);
            sum += l;
            if (callback != null) {
                callback.onProgress(fileLength, sum);
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (in != null) {
            in.close();
        }
    }
    //add file end
    out.write(getFileEndString().getBytes());
    out.flush();
    HttpResponse response = new HttpResponse();
    //can't get response code here
    response.code = mConn.getResponseCode();
    if(response.code == HttpURLConnection.HTTP_OK) {
        response.contentLength = mConn.getContentLength();
        response.inputStream = mConn.getInputStream();
        response.content = parseStream(response.inputStream);
    }else {
        response.errorStream = mConn.getErrorStream();
        response.content = parseStream(response.errorStream);
    }
    ......

 public static class SSLCustomSocketFactory extends SSLSocketFactory {
    private static final String TAG = "SSLCustomSocketFactory";
    private static final String[] TLS_SUPPORT_VERSION = {"TLSv1.1", "TLSv1.2"};
    private static final String KEY_PASS = "";
    final SSLSocketFactory delegate;
    static SSLContext  sslContext;
    static SSLSocketFactory mSocketFactory;

    public SSLCustomSocketFactory(SSLSocketFactory base) {
        delegate = base;
    }

    @Override
    public String[] getDefaultCipherSuites() {
        return delegate.getDefaultCipherSuites();
    }

    @Override
    public String[] getSupportedCipherSuites() {
        return delegate.getSupportedCipherSuites();
    }

    @Override
    public Socket createSocket(Socket s, String host, int port, boolean autoClose) throws IOException {
        return patch(delegate.createSocket(s, host, port, autoClose));
    }

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        return patch(delegate.createSocket(host, port));
    }

    @Override
    public Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException, UnknownHostException {
        return patch(delegate.createSocket(host, port, localHost, localPort));
    }

    @Override
    public Socket createSocket(InetAddress host, int port) throws IOException {
        return patch(delegate.createSocket(host, port));
    }

    @Override
    public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
        return patch(delegate.createSocket(address, port, localAddress, localPort));
    }

    private Socket patch(Socket s) {
        if(s instanceof SSLSocket) {
            ((SSLSocket) s).setEnabledProtocols(TLS_SUPPORT_VERSION);
        }
        return s;
    }

    public static synchronized SSLSocketFactory getSocketFactory() {
        if(mSocketFactory == null) {
            try {
                CertificateFactory cf = CertificateFactory.getInstance("X.509");
                AssetManager am = context().getAssets();
                String[] certsPaths = am.list("xxx_certs");
                String keyStoreType = KeyStore.getDefaultType();
                KeyStore keyStore = KeyStore.getInstance(keyStoreType);
                keyStore.load(null, null);
                for (int i = 0; i < certsPaths.length; i++) {
                    String certPath = certsPaths[i];
                    InputStream caInput = null;
                    try {
                        caInput = am.open("xxxx_certs/" + certPath);
                        Certificate ca = cf.generateCertificate(caInput);
                        System.out.println("ca=" + ((X509Certificate) ca).getSubjectDN());
                        keyStore.setCertificateEntry("ca" + i, ca);
                    } catch (Exception e) {
                        e.printStackTrace();
                    } finally {
                        if (caInput != null) {
                            caInput.close();
                        }
                    }
                }

                sslContext = SSLContext.getInstance("TLS");
                TrustManagerFactory tmf = TrustManagerFactory.
                        getInstance(TrustManagerFactory.getDefaultAlgorithm());
                tmf.init(keyStore);

                sslContext.init(null, tmf.getTrustManagers(), null);
                mSocketFactory = sslContext.getSocketFactory();
            } catch (Throwable e) {
                e.printStackTrace();
            }
        }
        return mSocketFactory;
    }
}

Here is the nagix return:

      <title>413 Request Entity Too Large</title>

413 Request Entity Too Large

      nginx/1.17.5

How I can get the right exception code and exception message?


来源: stackexchange

问:
Firebase functions: sync with Algolia doesn't work
I am currently trying to sync my firestore documents with algolia upon a new document creation or the update of a document. The path to the collection in firestore is videos/video. The function seems to be triggering fine, however after triggering, the firebase function does not seem to relay any of the information to algolia (no records are being created). I am not getting any errors in the log. (I also double checked the rules and made sure the node could be read by default, and yes I am on the blaze plan). Does anyone know how to sync a firestore node and algolia? Thanks for all your help!</p>

```<code>"use strict";
const functions = require("firebase-functions");
const admin = require("firebase-admin");
const algoliasearch_1 = require("algoliasearch");
// Set up Firestore.
admin.initializeApp();
const env = functions.config();
// Set up Algolia.
// The app id and API key are coming from the cloud functions environment, as we set up in Part 1,
const algoliaClient = algoliasearch_1.default(env.algolia.appid, env.algolia.apikey);
// Since I'm using develop and production environments, I'm automatically defining
// the index name according to which environment is running. functions.config().projectId is a default property set by Cloud Functions.
const collectionindexvideo = algoliaClient.initIndex('videos');

exports.collectionvideoOnCreate = functions.firestore.document('videos/{uid}').onCreate(async(snapshot, context) => {
  await savevideo(snapshot);
});
exports.collectionvideoOnUpdate = functions.firestore.document('videos/{uid}').onUpdate(async(change, context) => {
  await updatevideo(change);
});
exports.collectionvideoOnDelete = functions.firestore.document('videos/{uid}').onDelete(async(snapshot, context) => {
  await deletevideo(snapshot);
});
async function savevideo(snapshot) {
  if (snapshot.exists) {
    const document = snapshot.data();
    // Essentially, you want your records to contain any information that facilitates search,
    // display, filtering, or relevance. Otherwise, you can leave it out.
    const record = {
      objectID: snapshot.id,
      uid: document.uid,
      title: document.title,
      thumbnailurl: document.thumbnailurl,
      date: document.date,
      description: document.description,
      genre: document.genre,
      recipe: document.recipe
    };
    if (record) { // Removes the possibility of snapshot.data() being undefined.
      if (document.isIncomplete === false) {
        // In this example, we are including all properties of the Firestore document
        // in the Algolia record, but do remember to evaluate if they are all necessary.
        // More on that in Part 2, Step 2 above.
        await collectionindexvideo.saveObject(record); // Adds or replaces a specific object.
      }
    }
  }
}
async function updatevideo(change) {
  const docBeforeChange = change.before.data();
  const docAfterChange = change.after.data();
  if (docBeforeChange && docAfterChange) {
    if (docAfterChange.isIncomplete && !docBeforeChange.isIncomplete) {
      // If the doc was COMPLETE and is now INCOMPLETE, it was
      // previously indexed in algolia and must now be removed.
      await deletevideo(change.after);
    } else if (docAfterChange.isIncomplete === false) {
      await savevideo(change.after);
    }
  }
}
async function deletevideo(snapshot) {
  if (snapshot.exists) {
    const objectID = snapshot.id;
    await collectionindexvideo.deleteObject(objectID);
  }
}
</code>```

答:
Still don't know what I did wrong, however if anyone else is stuck in this situation, this repository is a great resource: https://github.com/nayfin/algolia-firestore-sync. I used it and was able to properly sync firebase and algolia. Cheers!</p>

```<code>// Takes an the Algolia index and key of document to be deleted
const removeObject = (index, key) => {
  // then it deletes the document
  return index.deleteObject(key, (err) => {
    if (err) throw err
    console.log('Key Removed from Algolia Index', key)
  })
}
// Takes an the Algolia index and data to be added or updated to
const upsertObject = (index, data) => {
  // then it adds or updates it
  return index.saveObject(data, (err, content) => {
    if (err) throw err
    console.log(`Document ${data.objectID} Updated in Algolia Index `)
  })
}

exports.syncAlgoliaWithFirestore = (index, change, context) => {
  const data = change.after.exists ? change.after.data() : null;
  const key = context.params.id; // gets the id of the document changed
  // If no data then it was a delete event
  if (!data) {
    // so delete the document from Algolia index
    return removeObject(index, key);
  }
  data['objectID'] = key;
  // upsert the data to the Algolia index
  return upsertObject(index, data);
};
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I am currently trying to sync my firestore documents with algolia upon a new document creation or the update of a document. The path to the collection in firestore is videos/video. The function seems to be triggering fine, however after triggering, the firebase function does not seem to relay any of the information to algolia (no records are being created). I am not getting any errors in the log. (I also double checked the rules and made sure the node could be read by default, and yes I am on the blaze plan). Does anyone know how to sync a firestore node and algolia? Thanks for all your help!

"use strict";
const functions = require("firebase-functions");
const admin = require("firebase-admin");
const algoliasearch_1 = require("algoliasearch");
// Set up Firestore.
admin.initializeApp();
const env = functions.config();
// Set up Algolia.
// The app id and API key are coming from the cloud functions environment, as we set up in Part 1,
const algoliaClient = algoliasearch_1.default(env.algolia.appid, env.algolia.apikey);
// Since I'm using develop and production environments, I'm automatically defining
// the index name according to which environment is running. functions.config().projectId is a default property set by Cloud Functions.
const collectionindexvideo = algoliaClient.initIndex('videos');

exports.collectionvideoOnCreate = functions.firestore.document('videos/{uid}').onCreate(async(snapshot, context) => {
  await savevideo(snapshot);
});
exports.collectionvideoOnUpdate = functions.firestore.document('videos/{uid}').onUpdate(async(change, context) => {
  await updatevideo(change);
});
exports.collectionvideoOnDelete = functions.firestore.document('videos/{uid}').onDelete(async(snapshot, context) => {
  await deletevideo(snapshot);
});
async function savevideo(snapshot) {
  if (snapshot.exists) {
    const document = snapshot.data();
    // Essentially, you want your records to contain any information that facilitates search,
    // display, filtering, or relevance. Otherwise, you can leave it out.
    const record = {
      objectID: snapshot.id,
      uid: document.uid,
      title: document.title,
      thumbnailurl: document.thumbnailurl,
      date: document.date,
      description: document.description,
      genre: document.genre,
      recipe: document.recipe
    };
    if (record) { // Removes the possibility of snapshot.data() being undefined.
      if (document.isIncomplete === false) {
        // In this example, we are including all properties of the Firestore document
        // in the Algolia record, but do remember to evaluate if they are all necessary.
        // More on that in Part 2, Step 2 above.
        await collectionindexvideo.saveObject(record); // Adds or replaces a specific object.
      }
    }
  }
}
async function updatevideo(change) {
  const docBeforeChange = change.before.data();
  const docAfterChange = change.after.data();
  if (docBeforeChange && docAfterChange) {
    if (docAfterChange.isIncomplete && !docBeforeChange.isIncomplete) {
      // If the doc was COMPLETE and is now INCOMPLETE, it was
      // previously indexed in algolia and must now be removed.
      await deletevideo(change.after);
    } else if (docAfterChange.isIncomplete === false) {
      await savevideo(change.after);
    }
  }
}
async function deletevideo(snapshot) {
  if (snapshot.exists) {
    const objectID = snapshot.id;
    await collectionindexvideo.deleteObject(objectID);
  }
}

Body:
Still don't know what I did wrong, however if anyone else is stuck in this situation, this repository is a great resource: https://github.com/nayfin/algolia-firestore-sync. I used it and was able to properly sync firebase and algolia. Cheers!

// Takes an the Algolia index and key of document to be deleted
const removeObject = (index, key) => {
  // then it deletes the document
  return index.deleteObject(key, (err) => {
    if (err) throw err
    console.log('Key Removed from Algolia Index', key)
  })
}
// Takes an the Algolia index and data to be added or updated to
const upsertObject = (index, data) => {
  // then it adds or updates it
  return index.saveObject(data, (err, content) => {
    if (err) throw err
    console.log(`Document ${data.objectID} Updated in Algolia Index `)
  })
}

exports.syncAlgoliaWithFirestore = (index, change, context) => {
  const data = change.after.exists ? change.after.data() : null;
  const key = context.params.id; // gets the id of the document changed
  // If no data then it was a delete event
  if (!data) {
    // so delete the document from Algolia index
    return removeObject(index, key);
  }
  data['objectID'] = key;
  // upsert the data to the Algolia index
  return upsertObject(index, data);
};


问:
Can not decode QR code using pyzbar with python3
I need to decode this QR Code using python3, opencv and pyzbar, but I'm not able to get any good result: the script is not able to detect/decode the QR Code.

Here is the code I've used. Is there any good way to improve the recognition?

```<code>import cv2
from pyzbar import pyzbar

# Image load
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required=True, help="path to input image")
args = vars(ap.parse_args())
image = cv2.imread(args["image"])

# Read QR
qr_code = pyzbar.decode(image)
print(qr_code)
</code>```

Before applying the QR decoder I also tried to:

Change brightness/contrast:

```<code># Change brightness/contrast
image = cv2.convertScaleAbs(image, alpha=6, beta=0)
</code>```

Make a binary image:

```<code># Make binary image
image[image > 19] = 255
image[image <= 19] = 0
</code>```

Rotate image (using this function):

```<code>def rotate_image(image, angle):
    image_center = tuple(np.array(image.shape[1::-1]) / 2)
    rot_mat = cv2.getRotationMatrix2D(image_center, angle, 1.0)
    result = cv2.warpAffine(image, rot_mat, image.shape[1::-1], flags=cv2.INTER_LINEAR)
    return result

image = rotate_image(image, 7)
</code>```

答: Ok, I discovered that the image I was trying to decode is not a QR Code, but a Data Matrix Barcode. Now using pylibdmtx I can decode it.</p>

Body:
I need to decode this QR Code using python3, opencv and pyzbar, but I'm not able to get any good result: the script is not able to detect/decode the QR Code.

Here is the code I've used. Is there any good way to improve the recognition?

import cv2
from pyzbar import pyzbar

# Image load
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required=True, help="path to input image")
args = vars(ap.parse_args())
image = cv2.imread(args["image"])

# Read QR
qr_code = pyzbar.decode(image)
print(qr_code)

Before applying the QR decoder I also tried to:

Change brightness/contrast:

# Change brightness/contrast
image = cv2.convertScaleAbs(image, alpha=6, beta=0)

Make a binary image:

# Make binary image
image[image > 19] = 255
image[image <= 19] = 0

Rotate image (using this function):

def rotate_image(image, angle):
    image_center = tuple(np.array(image.shape[1::-1]) / 2)
    rot_mat = cv2.getRotationMatrix2D(image_center, angle, 1.0)
    result = cv2.warpAffine(image, rot_mat, image.shape[1::-1], flags=cv2.INTER_LINEAR)
    return result

image = rotate_image(image, 7)


问:
Altering Web.Config SessionTimeout value from within application
I am trying to work out a way to alter the session timeout dynamically, as my customer has requested that this be a value they can easily change (as though it were a database transaction). Here is a truncated version of the value I am trying to change:

```<code><appSettings>
   <add key="SessionTimeout" value="20"/>
</appsettings>
</code>```

I'm working with C#/.Net 3.5 with all of this.

For the record, I am aware that altering the web.config file will cause a restart of the application. What I am going for is to give the administrator of the program the ability to alter the timeout on the fly. If, for example, one of the forms they are requiring people to fill out is taking longer than 20 minutes, and they need to bump the time up to a half hour, or if they want it shortened for some other reason. Since we want to keep the users (including admins) out of the code files, the goal is to create this as part of the site's admin page.

The problem is, I have no idea how to do this, or if it's even possible (much less a good idea). I've given google the run around and come up with nothing.

Is there a way to make these alterations, or even potentially to dynamically change the timeout on the fly (overriding the contents of the web.config)?

Thanks for the help.

答:
Editing your web.config at runtime is possible, but as you have already pointed out, I'm not sure if it is a good idea. There are multiple ways you can go about actually doing the editing of this file, however you have to make sure that the account you are running your application under has the appropriate file system permissions to be able to write to the application directory (eg edit and save the web.config file). If you are in a clustered environment, I would strongly advise not going down this path (though still possible).

Once your permissions etc have been sorted out, below is a quick way of doing it.

```<code>public void EditMyWebConfig(string appSettingsKey, string appSettingsValue, string         filePath)
    {
        XmlDocument xml = new XmlDocument();
        //Load current config into memory
        xml.Load(filePath);
        //Overwrite config
        using(FileStream fs = new FileStream(filePath, FileMode.Create,FileAccess.ReadWrite))
        {
            //select appSetting you want to edit
            XmlNode xmlN = xml.SelectSingleNode("//configuration/appSettings/add[@key = '" + appSettingsKey + "']");
            //assign new value
            xmlN.Attributes["value"].Value = appSettingsValue;
            //Create new web config
            xml.Save(fs);
        }
    }
</code>```

I hope this helps, but I think you should just pick a large session time would be easier and cause less problems in the future.

ContentLicense: CC BY-SA 3.0

Body:
I am trying to work out a way to alter the session timeout dynamically, as my customer has requested that this be a value they can easily change (as though it were a database transaction). Here is a truncated version of the value I am trying to change:

<appSettings>
   <add key="SessionTimeout" value="20"/>
</appsettings>

I'm working with C#/.Net 3.5 with all of this.

For the record, I am aware that altering the web.config file will cause a restart of the application. What I am going for is to give the administrator of the program the ability to alter the timeout on the fly. If, for example, one of the forms they are requiring people to fill out is taking longer than 20 minutes, and they need to bump the time up to a half hour, or if they want it shortened for some other reason. Since we want to keep the users (including admins) out of the code files, the goal is to create this as part of the site's admin page.

The problem is, I have no idea how to do this, or if it's even possible (much less a good idea). I've given google the run around and come up with nothing.

Is there a way to make these alterations, or even potentially to dynamically change the timeout on the fly (overriding the contents of the web.config)?

Thanks for the help.

Body:
Editing your web.config at runtime is possible, but as you have already pointed out, I'm not sure if it is a good idea. There are multiple ways you can go about actually doing the editing of this file, however you have to make sure that the account you are running your application under has the appropriate file system permissions to be able to write to the application directory (eg edit and save the web.config file). If you are in a clustered environment, I would strongly advise not going down this path (though still possible).

Once your permissions etc have been sorted out, below is a quick way of doing it.

public void EditMyWebConfig(string appSettingsKey, string appSettingsValue, string         filePath)
    {
        XmlDocument xml = new XmlDocument();
        //Load current config into memory
        xml.Load(filePath);
        //Overwrite config
        using(FileStream fs = new FileStream(filePath, FileMode.Create,FileAccess.ReadWrite))
        {
            //select appSetting you want to edit
            XmlNode xmlN = xml.SelectSingleNode("//configuration/appSettings/add[@key = '" + appSettingsKey + "']");
            //assign new value
            xmlN.Attributes["value"].Value = appSettingsValue;
            //Create new web config
            xml.Save(fs);
        }
    }

I hope this helps, but I think you should just pick a large session time would be easier and cause less problems in the future.


问:
How to sort a composite aggregation on the basis of a sub aggregation ? Below is the query
```<code>
GET myIndex/_search
{
  "from": 0,
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "user_id": {
              "value": "a88604b0",
              "boost": 1
            }
          }
        },
        {
          "term": {
            "entity_status.keyword": {
              "value": "ACTIVE",
              "boost": 1
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "sort": [
    {
      "txn_date": {
        "order": "desc"
      }
    }
  ],
  "aggs": {
    "my_buckets": {
      "composite": {
        "sources": [
          {
            "group_by": {
              "terms": {
                "field": "category"
              }
            }
          }
        ]
      },
      "aggs": {
        "total_amount": {
          "sum": {
            "field": "amount"
          }
        }
      }
    }
  }
}
</code>```

I am executing above query but I want the aggregations to be sorted by sub-aggregation
total_amount in descending order. Any modification or other ways to achieve this ?

Here is the result of the above query.

```<code>{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 4,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 22,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "my_buckets" : {
      "after_key" : {
        "group_by" : "Travel"
      },
      "buckets" : [
        {
          "key" : {
            "group_by" : "Bills"
          },
          "doc_count" : 2,
          "total_amount" : {
            "value" : 86710.44
          }
        },
        {
          "key" : {
            "group_by" : "Grocery"
          },
          "doc_count" : 1,
          "total_amount" : {
            "value" : 43355.22
          }
        },
        {
          "key" : {
            "group_by" : "Fashion"
          },
          "doc_count" : 5,
          "total_amount" : {
            "value" : 216776.1
          }
        },
        {
          "key" : {
            "group_by" : "Recharge"
          },
          "doc_count" : 7,
          "total_amount" : {
            "value" : 303486.54
          }
        },
        {
          "key" : {
            "group_by" : "Shopping"
          },
          "doc_count" : 2,
          "total_amount" : {
            "value" : 86710.44
          }
        },
        {
          "key" : {
            "group_by" : "Travel"
          },
          "doc_count" : 5,
          "total_amount" : {
            "value" : 216776.1
          }
        }
      ]
    }
  }
}
</code>```

I want to have the aggregations to be in sorted manner according to total_amount.

答:
Unfortunately, this is not possible right now. Each source can be ordered in ascending or descending order, but that's pretty much it.

Ordering by a sub-aggregation would require gathering all compound keys and computing the total amount for each bucket, which would be very costly in terms of memory and exactly the opposite of what the composite aggregation is trying to achieve, i.e. a way to paginate through buckets with a very low memory footprint

Also note that if you have a low cardinality of categories (<1000), you don't really need the composite aggregation, you can achieve what you need with the terms aggregation, like this:

```<code>{
  ...
  "aggs": {
    "group_by": {
      "terms": {
        "field": "category",
        "size": 100,
        "order": {
          "total_amount": "desc"
        }
      },
      "aggs": {
        "total_amount": {
          "sum": {
            "field": "amount"
          }
        }
      }
    }
  }
}
</code>```

Body:
GET myIndex/_search
{
  "from": 0,
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "user_id": {
              "value": "a88604b0",
              "boost": 1
            }
          }
        },
        {
          "term": {
            "entity_status.keyword": {
              "value": "ACTIVE",
              "boost": 1
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "sort": [
    {
      "txn_date": {
        "order": "desc"
      }
    }
  ],
  "aggs": {
    "my_buckets": {
      "composite": {
        "sources": [
          {
            "group_by": {
              "terms": {
                "field": "category"
              }
            }
          }
        ]
      },
      "aggs": {
        "total_amount": {
          "sum": {
            "field": "amount"
          }
        }
      }
    }
  }
}

I am executing above query but I want the aggregations to be sorted by sub-aggregation
total_amount in descending order. Any modification or other ways to achieve this ?

Here is the result of the above query.

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 4,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 22,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "my_buckets" : {
      "after_key" : {
        "group_by" : "Travel"
      },
      "buckets" : [
        {
          "key" : {
            "group_by" : "Bills"
          },
          "doc_count" : 2,
          "total_amount" : {
            "value" : 86710.44
          }
        },
        {
          "key" : {
            "group_by" : "Grocery"
          },
          "doc_count" : 1,
          "total_amount" : {
            "value" : 43355.22
          }
        },
        {
          "key" : {
            "group_by" : "Fashion"
          },
          "doc_count" : 5,
          "total_amount" : {
            "value" : 216776.1
          }
        },
        {
          "key" : {
            "group_by" : "Recharge"
          },
          "doc_count" : 7,
          "total_amount" : {
            "value" : 303486.54
          }
        },
        {
          "key" : {
            "group_by" : "Shopping"
          },
          "doc_count" : 2,
          "total_amount" : {
            "value" : 86710.44
          }
        },
        {
          "key" : {
            "group_by" : "Travel"
          },
          "doc_count" : 5,
          "total_amount" : {
            "value" : 216776.1
          }
        }
      ]
    }
  }
}

I want to have the aggregations to be in sorted manner according to total_amount.

Body:
Unfortunately, this is not possible right now. Each source can be ordered in ascending or descending order, but that's pretty much it.

Ordering by a sub-aggregation would require gathering all compound keys and computing the total amount for each bucket, which would be very costly in terms of memory and exactly the opposite of what the composite aggregation is trying to achieve, i.e. a way to paginate through buckets with a very low memory footprint

Also note that if you have a low cardinality of categories (<1000), you don't really need the composite aggregation, you can achieve what you need with the terms aggregation, like this:

{
  ...
  "aggs": {
    "group_by": {
      "terms": {
        "field": "category",
        "size": 100,
        "order": {
          "total_amount": "desc"
        }
      },
      "aggs": {
        "total_amount": {
          "sum": {
            "field": "amount"
          }
        }
      }
    }
  }
}


问:
How to save multiple plots as seperate png files with names in Python?
I have code which looks something like this:

```<code>a = datetime(2019, 1, 22) #Key dates area
b = datetime(2019, 1, 23)

for _, d in df.set_index('Date').groupby('Country'):
    fig, ax = plt.subplots()
    d['Counts'].plot()
    plt.axhline(y=d['Counts'].mean(), color='r', linestyle='--')
    plt.xticks(rotation=90)
    plt.title(f"Weekly Sim Count for {d['Country'].iat[0]}")
    plt.xlabel('Week')

    plt.axvspan(a, b, color='gray', alpha=0.2, lw=0)
    plt.legend()
    plt.show()
</code>```

I'm hoping to save every graph returned from this loop as separate png files where the name of the image is based on the country being represented within the graph. Is there a way to do this?

Thank you :)

答:
What you are looking for is plt.savefig which saves the current figure. You can add it into the loop before showing or instead of showing the plot:

```<code># ...
plt.legend()
plt.savefig(f"{d['Country'].iat[0]}_plot.png")
# ...
</code>```

and it should save in the current directory the plots generated in the loop.

Body:
I have code which looks something like this:

a = datetime(2019, 1, 22) #Key dates area
b = datetime(2019, 1, 23)

for _, d in df.set_index('Date').groupby('Country'):
    fig, ax = plt.subplots()
    d['Counts'].plot()
    plt.axhline(y=d['Counts'].mean(), color='r', linestyle='--')
    plt.xticks(rotation=90)
    plt.title(f"Weekly Sim Count for {d['Country'].iat[0]}")
    plt.xlabel('Week')

    plt.axvspan(a, b, color='gray', alpha=0.2, lw=0)
    plt.legend()
    plt.show()

I'm hoping to save every graph returned from this loop as separate png files where the name of the image is based on the country being represented within the graph. Is there a way to do this?

Thank you :)

Body:
What you are looking for is plt.savefig which saves the current figure. You can add it into the loop before showing or instead of showing the plot:

# ...
plt.legend()
plt.savefig(f"{d['Country'].iat[0]}_plot.png")
# ...

and it should save in the current directory the plots generated in the loop.


问:
Idling and ExternalEvent are not raised while an Element is selected
I'm building a Revit AddIn with WPF modeless dialogs and I want to use an ExternalEvent to retrieve Elements selected by the user. Is what I am doing viable and what do I need to change for it to work?

Since I don't have a valid API document context, I raise an ExternalEvent when a button is clicked to retrieve UniqueId of Elements that are currently selected.

Here are the relevant classes (I tried to reduce the code as much as I could) :

public class App : IExternalApplication {
  internal static App _app = null;
  public static App Instance => _app;

  public Result OnStartup(UIControlledApplication application) {
    _app = this;
    return Result.Succeeded;
  }

  public void ShowWin(UIApplication ui_app) {
    var eventHandler = new CustomEventHandler();
    var externalEvent = ExternalEvent.Create(eventHandler);
    var window = new WPFWindow(eventHandler, externalEvent);
    Process proc = Process.GetCurrentProcess();
    WindowInteropHelper helper = new WindowInteropHelper(window) {
      Owner = proc.MainWindowHandle
    };
    window.Show();
  }
}

public class AddIn : IExternalCommand {
  public Result Execute(ExternalCommandData commandData, ref string message, ElementSet elements) {
    App.Instance.ShowWin(commandData.Application);
    return Result.Succeeded;
  }
}

public class CustomEventHandler : IExternalEventHandler {
  public event Action<List<string>> CustomEventHandlerDone;

  public void Execute(UIApplication ui_app) {
    UIDocument ui_doc = ui_app.ActiveUIDocument;
    if (ui_doc == null) {
      return;
    }
    Document doc = ui_doc.Document;
    List<string> element_ids = null;
    var ui_view = ui_doc.GetOpenUIViews().Where(x => x.ViewId == doc.ActiveView.Id).FirstOrDefault();
    if (doc.ActiveView is View3D view3d && ui_view != null) {
      using (Transaction tx = new Transaction(doc)) {
        tx.Start();
        element_ids = ui_doc.Selection.GetElementIds().Select(x => doc.GetElement(x)?.UniqueId).Where(x => x != null).ToList();
        tx.Commit();
      }
    }
    this.CustomEventHandlerDone?.Invoke(element_ids);
  }
}

public partial class WPFWindow {
  private CustomEventHandler _eventHandler;
  private ExternalEvent _externalEvent;

  public WPFWindow(CustomEventHandler eventHandler, ExternalEvent externalEvent) {
    this._eventHandler = eventHandler;
    this._eventHandler.CustomEventHandlerDone += this.WPFWindow_CustomEventDone;
    this._externalEvent = externalEvent;
  }

  private void Button_Click(object sender, RoutedEventArgs e) {
    this._externalEvent.Raise();
  }

  private void WPFWindow_CustomEventDone(List<string> element_ids) {
    // this point is never reached while an element is selected
  }
}
```

When an element is selected, the ExternalEvent is marked as pending but is only executed when the selection is cleared by the user.

The same happens with UIControlledApplication.Idling.

I would like for it to be executed even when elements are selected, or an alternative way to do it, and not involving PickObject.

答:
I ran into the same problem.

I was able to determine that the problem occurs if elements of the same family are selected. Moreover, there is a certain threshold value, somewhere from 10 to 20 or more, at which this is manifested.

I was able to get around this by canceling the selection of elements UIDocument.Selection.SetElementIds(new List<ElementId>()) before calling ExternalEvent.Raise(). And then at the end return the selection, if necessary.

Body:
I'm building a Revit AddIn with WPF modeless dialogs and I want to use an ExternalEvent to retrieve Elements selected by the user. Is what I am doing viable and what do I need to change for it to work?

Since I don't have a valid API document context, I raise an ExternalEvent when a button is clicked to retrieve UniqueId of Elements that are currently selected.

Here are the relevant classes (I tried to reduce the code as much as I could) :

public class App : IExternalApplication {
  internal static App _app = null;
  public static App Instance => _app;

  public Result OnStartup(UIControlledApplication application) {
    _app = this;
    return Result.Succeeded;
  }

  public void ShowWin(UIApplication ui_app) {
    var eventHandler = new CustomEventHandler();
    var externalEvent = ExternalEvent.Create(eventHandler);
    var window = new WPFWindow(eventHandler, externalEvent);
    Process proc = Process.GetCurrentProcess();
    WindowInteropHelper helper = new WindowInteropHelper(window) {
      Owner = proc.MainWindowHandle
    };
    window.Show();
  }
}

public class AddIn : IExternalCommand {
  public Result Execute(ExternalCommandData commandData, ref string message, ElementSet elements) {
    App.Instance.ShowWin(commandData.Application);
    return Result.Succeeded;
  }
}

public class CustomEventHandler : IExternalEventHandler {
  public event Action<List<string>> CustomEventHandlerDone;

  public void Execute(UIApplication ui_app) {
    UIDocument ui_doc = ui_app.ActiveUIDocument;
    if (ui_doc == null) {
      return;
    }
    Document doc = ui_doc.Document;
    List<string> element_ids = null;
    var ui_view = ui_doc.GetOpenUIViews().Where(x => x.ViewId == doc.ActiveView.Id).FirstOrDefault();
    if (doc.ActiveView is View3D view3d && ui_view != null) {
      using (Transaction tx = new Transaction(doc)) {
        tx.Start();
        element_ids = ui_doc.Selection.GetElementIds().Select(x => doc.GetElement(x)?.UniqueId).Where(x => x != null).ToList();
        tx.Commit();
      }
    }
    this.CustomEventHandlerDone?.Invoke(element_ids);
  }
}

public partial class WPFWindow {
  private CustomEventHandler _eventHandler;
  private ExternalEvent _externalEvent;

  public WPFWindow(CustomEventHandler eventHandler, ExternalEvent externalEvent) {
    this._eventHandler = eventHandler;
    this._eventHandler.CustomEventHandlerDone += this.WPFWindow_CustomEventDone;
    this._externalEvent = externalEvent;
  }

  private void Button_Click(object sender, RoutedEventArgs e) {
    this._externalEvent.Raise();
  }

  private void WPFWindow_CustomEventDone(List<string> element_ids) {
    // this point is never reached while an element is selected
  }
}

When an element is selected, the ExternalEvent is marked as pending but is only executed when the selection is cleared by the user.

The same happens with UIControlledApplication.Idling.

I would like for it to be executed even when elements are selected, or an alternative way to do it, and not involving PickObject.


问:
MongoDB - Most Recent 20 unique
I'm trying to get the most recent unique value from a collection in MongoDB, however, the $group seems to be breaking the $sort somehow. Ie, in this case, im trying to get the most recent activity for a user (but only returning 20 activities total).

```<code>db.activities.aggregate([{ $sort: { timestamp:-1, firstname: 1 } }, { $limit: 20 }])
</code>```

If i run the above, i get 20 results back and in the correct order, however, i have multiple doduments with the same firstname as the user has submitted multiple activities.

If i run the below, the top user from the first query is no longer in the list.  I get a completely different set of people back.

```<code>db.activities.aggregate([{ $sort: { timestamp:-1, firstname: 1 } },  { $group: { _id: "$firstname"} } , { $limit: 20 }])
</code>```

How can i get the most recent activity per person, but limiting it to only 20 activities?

答:
You need to first find unique values, then apply limit and in the end just sort however you want. Its your solution with right ordering:

```<code>db.activities.aggregate([
     { $group: { _id: "$firstname"} } ,
     { $limit: 20 },
     { $sort: { timestamp:-1, firstname: 1 } },
])
</code>```

Edit: Above code will not return correct result because from docs $group doesn't respect ordering. Instead you should $sort them before selecting first 20 like this:

```<code>db.activities.aggregate([
     { $group: { _id: "$firstname"} } ,
     { $sort: { timestamp:-1, firstname: 1 } },
     { $limit: 20 },
])
</code>```

This will return unique docs but if you need only recent docs without unique constraint then just remove the $group stage as its redundant and will consume performance exponentially. If you still need unique docs with better performance then you need to make time-related decision i.e. $match the docs greater than last 1 hour, $group them, $sort and $limit first 20.

Title: MongoDB - Most Recent 20 unique

Body:
I'm trying to get the most recent unique value from a collection in MongoDB, however, the $group seems to be breaking the $sort somehow. Ie, in this case, im trying to get the most recent activity for a user (but only returning 20 activities total).

db.activities.aggregate([{ $sort: { timestamp:-1, firstname: 1 } }, { $limit: 20 }])

If i run the above, i get 20 results back and in the correct order, however, i have multiple doduments with the same firstname as the user has submitted multiple activities.

If i run the below, the top user from the first query is no longer in the list.  I get a completely different set of people back.

db.activities.aggregate([{ $sort: { timestamp:-1, firstname: 1 } },  { $group: { _id: "$firstname"} } , { $limit: 20 }])

How can i get the most recent activity per person, but limiting it to only 20 activities?

Body:
You need to first find unique values, then apply limit and in the end just sort however you want. Its your solution with right ordering:

db.activities.aggregate([
     { $group: { _id: "$firstname"} } ,
     { $limit: 20 },
     { $sort: { timestamp:-1, firstname: 1 } },
])

Edit: Above code will not return correct result because from docs $group doesn't respect ordering. Instead you should $sort them before selecting first 20 like this:

db.activities.aggregate([
     { $group: { _id: "$firstname"} } ,
     { $sort: { timestamp:-1, firstname: 1 } },
     { $limit: 20 },
])

This will return unique docs but if you need only recent docs without unique constraint then just remove the $group stage as its redundant and will consume performance exponentially. If you still need unique docs with better performance then you need to make time-related decision i.e. $match the docs greater than last 1 hour, $group them, $sort and $limit first 20.


问:
Sightly/HTL listChildren on different than current path
In Apache Sling, I can do something like this with HTL/sightly:

```<code><div data-sly-list="${resource.listChildren}">
        <p>title: ${item.title}</p>
</div>
</code>```

Variable resource points to the current path of, say, /a/b/c. How can I get a resource at /x/y/z and list its children? That's through the resolver global variable but I am unclear about the syntax to be used.

答:
You can use data-sly-use to grab a Resource object for a given path:

```<code><sly data-sly-use.test="/content/core-components-examples"></sly>
<ul data-sly-list="${test.listChildren}">
    <li>${item.path}</li>
</ul>
</code>```

Body:
In Apache Sling, I can do something like this with HTL/sightly:

title: ${item.title}

Variable resource points to the current path of, say, /a/b/c. How can I get a resource at /x/y/z and list its children? That's through the resolver global variable but I am unclear about the syntax to be used.

Body:
You can use data-sly-use to grab a Resource object for a given path:

<sly data-sly-use.test="/content/core-components-examples"></sly>
<ul data-sly-list="${test.listChildren}">
    <li>${item.path}</li>
</ul>


问:
Android to Drupal cookie transfer
Can someone please enlighten me on how I might send Drupal login cookie information from my Android app back to my Drupal site?

Below is the code I am using in my attempts:

```<code>HttpResponse response;
HttpClient httpClient   =   new DefaultHttpClient();
HttpPost httpPost       =   new HttpPost("http://test2.icerge.com/testpoint/node/");
**httpPost.addHeader("Cookie: " + USERPREFERENCES.getString(COOKIE_NAME, ""), " "+USERPREFERENCES.getString(COOKIE_VALUE, ""));**
Toast.LENGTH_LONG).show();
// TODO Auto-generated method stub
try{
    List<NameValuePair> nameValuePairs  =   new ArrayList<NameValuePair>();
    nameValuePairs.add( new BasicNameValuePair("node[title]", "sample node from app") );
    nameValuePairs.add( new BasicNameValuePair("node[type]", "story") );
    nameValuePairs.add( new BasicNameValuePair("node[body]", "sample app node body content") );
    httpPost.setEntity( new UrlEncodedFormEntity(nameValuePairs));

    response    =   httpClient.execute(httpPost);

    Log.i("SEEMS TO WORK", response.toString());
    Log.v("CODE", httpPost.getRequestLine().toString() + " - " + response.toString());

}catch(Exception e){
    Log.e("HTTP-ERROR: node creation", e.toString());
}
</code>```

I am using the "httpPost addHeader" line to send my cookie, but nojoy.

Can someone please direct me on this issue, please?

答:
I do not believe that the HttpClient manages cookies by default. So you need to set up a cookie store, bind it the HTTP Context, and then use the context in POSTs, like this

```<code>    mHttpContext = new BasicHttpContext();
    mCookieStore = new CookieStore();
    mHttpContext.setAttribute(ClientContext.COOKIE_STORE, mCookieStore);
    ...
    HttpResponse response = mHttpClient.execute(mRequest, mHttpContext);
</code>```

Once that is in place any cookies that sent from the Drupal site on responses will be automatically included on subsequent requests.

If you are interested to see what cookies are held by the CookieStore you can always do

```<code>List<Cookie> cookies = mCookieStore.getCookies();
</code>```

Title: Android to Drupal cookie transfer

Body:
Can someone please enlighten me on how I might send Drupal login cookie information from my Android app back to my Drupal site?

Below is the code I am using in my attempts:

HttpResponse response;
HttpClient httpClient   =   new DefaultHttpClient();
HttpPost httpPost       =   new HttpPost("http://test2.icerge.com/testpoint/node/");
**httpPost.addHeader("Cookie: " + USERPREFERENCES.getString(COOKIE_NAME, ""), " "+USERPREFERENCES.getString(COOKIE_VALUE, ""));**
Toast.LENGTH_LONG).show();
// TODO Auto-generated method stub
try{
    List<NameValuePair> nameValuePairs  =   new ArrayList<NameValuePair>();
    nameValuePairs.add( new BasicNameValuePair("node[title]", "sample node from app") );
    nameValuePairs.add( new BasicNameValuePair("node[type]", "story") );
    nameValuePairs.add( new BasicNameValuePair("node[body]", "sample app node body content") );
    httpPost.setEntity( new UrlEncodedFormEntity(nameValuePairs));

    response    =   httpClient.execute(httpPost);

    Log.i("SEEMS TO WORK", response.toString());
    Log.v("CODE", httpPost.getRequestLine().toString() + " - " + response.toString());

}catch(Exception e){
    Log.e("HTTP-ERROR: node creation", e.toString());
}

I am using the "httpPost addHeader" line to send my cookie, but nojoy.

Can someone please direct me on this issue, please?

Body:
I do not believe that the HttpClient manages cookies by default. So you need to set up a cookie store, bind it the HTTP Context, and then use the context in POSTs, like this

    mHttpContext = new BasicHttpContext();
    mCookieStore = new CookieStore();
    mHttpContext.setAttribute(ClientContext.COOKIE_STORE, mCookieStore);
    ...
    HttpResponse response = mHttpClient.execute(mRequest, mHttpContext);

Once that is in place any cookies that sent from the Drupal site on responses will be automatically included on subsequent requests.

If you are interested to see what cookies are held by the CookieStore you can always do

List<Cookie> cookies = mCookieStore.getCookies();


答:
Try the below method.</p>

```<code>FarmGeoJson farmGeoJson = com.alibaba.fastjson.JSONObject.parseObject(locationStringJson, FarmGeoJson.class);
</code>```

Title: String to JSONObject troubles

Body:
Is the first time I have to make this and I think I'm going to the wrong side. I trying to pass my String to a JSONObject but it's nothing working. My code is this:

String locationStringJson = farm.getCommonAreas().get(0).getLocation().replaceAll("\\\\", "");
                                    try {
                                        JSONObject locationObj = new JSONObject(locationStringJson);
                                        Log.d("My App", locationObj.toString());

                                    } catch (Throwable t) {
                                        Log.e("My App", "Could not parse malformed JSON: \"" + locationStringJson + "\"");
                                    }

The locationObj value after:

{"type":"FeatureCollection","features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-46.19679305516182,-21.3557873035952,875.7922817609483],[-46.19509985061597,-21.35461892353634,851.7199664001099],[-46.19352440512642,-21.35465468553253,830.544107896781],[-46.19210971746411,-21.35533178647862,834.4995579747846],[-46.19246633589201,-21.35325655623514,818.0053650355306],[-46.19056278024465,-21.35223771617781,814.4561932463444],[-46.19485426724801,-21.34927033959156,820.8368637750491],[-46.19406925297564,-21.34411169376047,835.869714331503],[-46.19383528445265,-21.34257184056689,846.0504330417629],[-46.19484350479828,-21.34182697538793,856.2311517520229],[-46.1962878784062,-21.34151208337897,853.2060549203649],[-46.19720802861893,-21.34212719882432,863.5767764476165],[-46.19799767515288,-21.34318122502575,872.7086576897467],[-46.19824820477749,-21.34470459858373,883.3796232608263],[-46.19909510449183,-21.34436858896095,878.2872975189589],[-46.19956076082133,-21.34528105975711,869.8407434658429],[-46.20088312493932,-21.3453798405374,859.443203762798],[-46.2002428481787,-21.34812665616198,844.5221403864139],[-46.1987871829488,-21.34973816454015,824.467785524764],[-46.2044190023153,-21.35358726214673,896.3662262563042],[-46.20506039198465,-21.35408860964525,897.8948471808142],[-46.20532057690928,-21.35511888769405,899.4234681053241],[-46.20541331912865,-21.35620272644546,900.8867965073521],[-46.20461209029964,-21.3574176131669,902.0945075082227],[-46.20399199591689,-21.35775448256089,892.599969822067],[-46.1984691069506,-21.35606738619609,878.2315580520376],[-46.19679305516182,-21.3557873035952,875.7922817609483]]]},"properties":{"name":"Areado01","styleUrl":"#__managed_style_07990BBE71133E4DE97F","styleMapHash":{"normal":"#__managed_style_1F6694D867133E4DE97F","highlight":"#__managed_style_0817ADA964133E4DE982"}},"id":"03F80C78A7133E4C22CA"}]}

(The code continue like a String and I don't know why)

The class I'm going to use to map later is this

public class FarmGeoJson {

    @SerializedName("type")
    @Expose
    private String type;
    @SerializedName("features")
    @Expose
    private List<Feature> features = null;

    public FarmGeoJson() {
        super();
    }

    public FarmGeoJson(String type, List<Feature> features) {
        this.type = type;
        this.features = features;
    }

    public String getType() {
        return type;
    }

    public void setType(String type) {
        this.type = type;
    }

    public List<Feature> getFeatures() {
        return features;
    }

    public void setFeatures(List<Feature> features) {
        this.features = features;
    }

    @Override
    public String toString() {
        return "FarmGeoJson{" +
                "type='" + type + '\'' +
                ", features=" + features +
                '}';
    }
}

How can I convert my String locationStringJson to a FarmGeoJson.class?

Body:
Try the below method.

FarmGeoJson farmGeoJson = com.alibaba.fastjson.JSONObject.parseObject(locationStringJson, FarmGeoJson.class);


问:
Positioning children widgets at the center and at the end of a Column parent widget
I have to position 2 children widgets inside a Column parent widget. I want one of them to be positioned in the center and the other at the end of the column. What is the best way to do that?

import 'package:flutter/material.dart';

class Screen extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Screen'),
      ),
      body: Column(
        mainAxisAlignment: MainAxisAlignment.center, // this is wrong, it positions both the widgets in the center
        children: <Widget>[
          Text('This text should be at the center of the column'),
          Text('This text should be at the end of the column'),
        ],
      ),
    );
  }
}
```

答:
you can use the Spacer widget like this: </p>

```<code>Column(
        mainAxisAlignment: MainAxisAlignment.center,
        children: <Widget>[
          Spacer(),
          Text('This text should be at the center of the column'),
          Spacer(),
          Text('This text should be at the end of the column'),
        ],
      ),
    );
</code>```

Body:
you can use the Spacer widget like this:

Column(
        mainAxisAlignment: MainAxisAlignment.center,
        children: <Widget>[
          Spacer(),
          Text('This text should be at the center of the column'),
          Spacer(),
          Text('This text should be at the end of the column'),
        ],
      ),
    );


问:
How best to rechunk a NetCDF file collection to Zarr dataset
I'm trying to rechunk a NetCDF file collection and create a Zarr dataset on AWS S3. I have 168 original NetCDF4 classic files with arrays of dimension time: 1, y: 3840, x: 4608 chunked as chunks={'time':1, 'y':768, 'x':922}.

I want to write this output to Zarr, and I want to optimize for time series extraction, so include more time records in my chunks.  I figured I would use xarray to help get the job done, since I have a lot of processors available to take advantage of Dask, and xarray has both xr.open_mfdataset and ds.to_zarr.

I first tried rechunking to chunks={'time':24, 'y':768, 'x':922} to match the input NetCDF4 chunking in x and y, but when I tried to write to Zarr it complained because it needed uniform chunk sizes in both x and y, only allowing non-uniform size in the last chunk along the time dimension (and unfortunately in the x dimension, the total size 4608 is not a multiple of chunk size 922.

So then I tried chunks={'time':168, 'y':384, 'x':288} and that started working, and proceeded very quickly for a few minutes, then got slower and slower.  Eventually after 50 minutes, the cluster died with:

```<code>4072 distributed.core - INFO - Event loop was unresponsive in Worker for 1.41s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
4073 slurmstepd: error: Step 3294889.0 exceeded memory limit (25346188 > 25165824), being killed
</code>```

Here's the code I'm using:

```<code>from dask.distributed import Client

import pandas as pd
import xarray as xr
import s3fs
import zarr

client = Client(scheduler_file='/home/rsignell/scheduler.json')
client
</code>```

```<code>root = '/lustre/projects/hazards/cmgp/woodshole/rsignell/nwm/forcing_short_range/'

bucket_endpoint='https://s3.us-west-1.amazonaws.com/'

f_zarr = 'rsignell/nwm/test_week4'

dates = pd.date_range(start='2018-04-01T00:00', end='2018-04-07T23:00', freq='H')

urls = ['{}{}/nwm.t{}z.short_range.forcing.f001.conus.nc'.format(root,a.strftime('%Y%m%d'),a.strftime('%H')) for a in dates]

ds = xr.open_mfdataset(urls, concat_dim='time', chunks={'time':1, 'y':768, 'x':922})
ds = ds.drop(['ProjectionCoordinateSystem','time_bounds'])
ds = ds.chunk(chunks={'time':168, 'y':384, 'x':288}).persist()
ds
</code>```

producing

```<code><xarray.Dataset>
Dimensions:         (reference_time: 168, time: 168, x: 4608, y: 3840)
Coordinates:
  * reference_time  (reference_time) datetime64[ns] 2018-04-01 ...
  * x               (x) float64 -2.304e+06 -2.303e+06 -2.302e+06 -2.301e+06 ...
  * y               (y) float64 -1.92e+06 -1.919e+06 -1.918e+06 -1.917e+06 ...
  * time            (time) datetime64[ns] 2018-04-01T01:00:00 ...
Data variables:
    T2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    LWDOWN          (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    Q2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    U2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    V2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    PSFC            (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    RAINRATE        (time, y, x) float32 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    SWDOWN          (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
</code>```

Then I call

```<code>fs = s3fs.S3FileSystem(anon=False, client_kwargs=dict(endpoint_url=bucket_endpoint))
d = s3fs.S3Map(f_zarr, s3=fs)

compressor = zarr.Blosc(cname='zstd', clevel=3, shuffle=2)
encoding = {vname: {'compressor': compressor} for vname in ds.data_vars}

delayed_store = ds.to_zarr(store=d, mode='w', encoding=encoding, compute=False)
persist_store = delayed_store.persist(retries=100)
</code>```

and just before it dies, the Dask Daskboard looks like this:

The total size of the NetCDF4 files is 20GB, so it seems a bit crazy that I've got over 500GB showing in the Dask Dashboard, and that 30 processors each with 60GB RAM is not sufficient for the job.

What am I doing wrong, or what would be a better solution?

答:
I notice that you say you want to increase the number of chunks in the time dimension. Or maybe I've misunderstood.

You start with chunks specified as chunks={'time':1, 'y':768, 'x':922}, but then try chunks={'time':168, 'y':384, 'x':288} and find that the second one uses a large amount of memory.

The problem is that the chunks keyword specifies the size of the chunks, not the number of chunks!

In the first case the size of each chunk is 1*768*922 ~ 7e5, whereas in the second case the size of each chunk is 168*384*288 ~ 2e7.

The maximum number of chunks in time is achieved by chunks={'time': 1}.

Body:
I'm trying to rechunk a NetCDF file collection and create a Zarr dataset on AWS S3. I have 168 original NetCDF4 classic files with arrays of dimension time: 1, y: 3840, x: 4608 chunked as chunks={'time':1, 'y':768, 'x':922}.

I want to write this output to Zarr, and I want to optimize for time series extraction, so include more time records in my chunks.  I figured I would use xarray to help get the job done, since I have a lot of processors available to take advantage of Dask, and xarray has both xr.open_mfdataset and ds.to_zarr.

I first tried rechunking to chunks={'time':24, 'y':768, 'x':922} to match the input NetCDF4 chunking in x and y, but when I tried to write to Zarr it complained because it needed uniform chunk sizes in both x and y, only allowing non-uniform size in the last chunk along the time dimension (and unfortunately in the x dimension, the total size 4608 is not a multiple of chunk size 922.

So then I tried chunks={'time':168, 'y':384, 'x':288} and that started working, and proceeded very quickly for a few minutes, then got slower and slower.  Eventually after 50 minutes, the cluster died with:

4072 distributed.core - INFO - Event loop was unresponsive in Worker for 1.41s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
4073 slurmstepd: error: Step 3294889.0 exceeded memory limit (25346188 > 25165824), being killed

Here's the code I'm using:

from dask.distributed import Client

import pandas as pd
import xarray as xr
import s3fs
import zarr

client = Client(scheduler_file='/home/rsignell/scheduler.json')
client

root = '/lustre/projects/hazards/cmgp/woodshole/rsignell/nwm/forcing_short_range/'

bucket_endpoint='https://s3.us-west-1.amazonaws.com/'

f_zarr = 'rsignell/nwm/test_week4'

dates = pd.date_range(start='2018-04-01T00:00', end='2018-04-07T23:00', freq='H')

urls = ['{}{}/nwm.t{}z.short_range.forcing.f001.conus.nc'.format(root,a.strftime('%Y%m%d'),a.strftime('%H')) for a in dates]

ds = xr.open_mfdataset(urls, concat_dim='time', chunks={'time':1, 'y':768, 'x':922})
ds = ds.drop(['ProjectionCoordinateSystem','time_bounds'])
ds = ds.chunk(chunks={'time':168, 'y':384, 'x':288}).persist()
ds

producing

<xarray.Dataset>
Dimensions:         (reference_time: 168, time: 168, x: 4608, y: 3840)
Coordinates:
  * reference_time  (reference_time) datetime64[ns] 2018-04-01 ...
  * x               (x) float64 -2.304e+06 -2.303e+06 -2.302e+06 -2.301e+06 ...
  * y               (y) float64 -1.92e+06 -1.919e+06 -1.918e+06 -1.917e+06 ...
  * time            (time) datetime64[ns] 2018-04-01T01:00:00 ...
Data variables:
    T2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    LWDOWN          (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    Q2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    U2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    V2D             (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    PSFC            (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    RAINRATE        (time, y, x) float32 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>
    SWDOWN          (time, y, x) float64 dask.array<shape=(168, 3840, 4608), chunksize=(168, 384, 288)>

Then I call

fs = s3fs.S3FileSystem(anon=False, client_kwargs=dict(endpoint_url=bucket_endpoint))
d = s3fs.S3Map(f_zarr, s3=fs)

compressor = zarr.Blosc(cname='zstd', clevel=3, shuffle=2)
encoding = {vname: {'compressor': compressor} for vname in ds.data_vars}

delayed_store = ds.to_zarr(store=d, mode='w', encoding=encoding, compute=False)
persist_store = delayed_store.persist(retries=100)

and just before it dies, the Dask Daskboard looks like this:

The total size of the NetCDF4 files is 20GB, so it seems a bit crazy that I've got over 500GB showing in the Dask Dashboard, and that 30 processors each with 60GB RAM is not sufficient for the job.

What am I doing wrong, or what would be a better solution?


问:
Why can @decorator not decorate a staticmethod or a classmethod?
Why can decorator not decorate a staticmethod or a classmethod?

```<code>from decorator import decorator

@decorator
def print_function_name(function, *args):
    print '%s was called.' % function.func_name
    return function(*args)

class My_class(object):
    @print_function_name
    @classmethod
    def get_dir(cls):
        return dir(cls)

    @print_function_name
    @staticmethod
    def get_a():
        return 'a'
</code>```

Both get_dir and get_a result in AttributeError: <'classmethod' or 'staticmethod'>,  object has no attribute '__name__'.

Why does decorator rely on the attribute __name__ instead of the attribute func_name? (Afaik all functions, including classmethods and staticmethods, have the func_name attribute.)

Edit: I'm using Python 2.6.

答:
classmethod and staticmethod return descriptor objects, not functions.  Most decorators are not designed to accept descriptors.

Normally, then, you must apply classmethod and staticmethod last when using multiple decorators.  And since decorators are applied in "bottom up" order, classmethod and staticmethod normally should be top-most in your source.

Like this:

```<code>class My_class(object):
    @classmethod
    @print_function_name
    def get_dir(cls):
        return dir(cls)

    @staticmethod
    @print_function_name
    def get_a():
        return 'a'
</code>```

Body:
Why can decorator not decorate a staticmethod or a classmethod?

from decorator import decorator

@decorator
def print_function_name(function, *args):
    print '%s was called.' % function.func_name
    return function(*args)

class My_class(object):
    @print_function_name
    @classmethod
    def get_dir(cls):
        return dir(cls)

    @print_function_name
    @staticmethod
    def get_a():
        return 'a'

Both get_dir and get_a result in AttributeError: <'classmethod' or 'staticmethod'>,  object has no attribute '__name__'.

Why does decorator rely on the attribute __name__ instead of the attribute func_name? (Afaik all functions, including classmethods and staticmethods, have the func_name attribute.)

Edit: I'm using Python 2.6.

Body:
classmethod and staticmethod return descriptor objects, not functions.  Most decorators are not designed to accept descriptors.

Normally, then, you must apply classmethod and staticmethod last when using multiple decorators.  And since decorators are applied in "bottom up" order, classmethod and staticmethod normally should be top-most in your source.

Like this:

class My_class(object):
    @classmethod
    @print_function_name
    def get_dir(cls):
        return dir(cls)

    @staticmethod
    @print_function_name
    def get_a():
        return 'a'


问:
I have a Turkish character problem when I use innerHTML in js
This is the result when I enter a Turkish character in span with innerHTML.The result is like this, "�r�n Favorilere Eklendi"</p>

```<code><a href="javascript:;" onclick="Main.AddRemoveFavorite(this)" id="true" data-favoriteid="@Model.product.FavoriteId">
<i id="icon" style="font-size:1.2rem; color:#798499;" class="icon-star"></i>
<span id="text" style="font-size:1rem; font-weight:400; color:#798499;cursor:pointer;">Ürünü Favorilerime Ekle</span>
</a>
</code>```

```<code>document.getElementById("icon").className = "icon-star_alt"
document.getElementById("text").innerHTML = 'Ürün Favorilere Eklendi';
$(btn).data('favoriteid', data)
</code>```

答:
Are you using meta tag inside your head

```<code><meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
</code>```

You can also check that HTTP server headers are there, because if they are present then the meta tag is ignored by browsers

the header:

```<code>Content-Type: text/html; charset=utf-8
</code>```

Body:
This is the result when I enter a Turkish character in span with innerHTML.The result is like this, "�r�n Favorilere Eklendi"

Ürünü Favorilerime Ekle

document.getElementById("icon").className = "icon-star_alt"
document.getElementById("text").innerHTML = 'Ürün Favorilere Eklendi';
$(btn).data('favoriteid', data)

Body:
Are you using meta tag inside your head

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

You can also check that HTTP server headers are there, because if they are present then the meta tag is ignored by browsers

the header:

Content-Type: text/html; charset=utf-8


问:
Use a captured value as row identifier
I need to parse this raw data in order to process it:

```<code>Port        Align-Err     FCS-Err    Xmit-Err     Rcv-Err  UnderSize  OutDiscards
Gi0/1               1           2           3           4          5            6
Gi0/2               11          12          13          14         15           16

Port      Single-Col  Multi-Col   Late-Col  Excess-Col  Carri-Sen      Runts     Giants
Gi0/1              1          2          3           4          5          6          7
Gi0/2              111        122        133         144        155        166        177
</code>```

To do this, I'm using TextFSM.

I would want this output:

```<code>['Gi0/1', '1', '2', '3', '4', '5', '6', '1', '2', '3', '4', '5', '6', '7']
['Gi0/2', '11', '12', '13', '14', '15', '16', '111', '112', '113', '114', '115', '116', '117']
</code>```

The first template I wrote is this one:

```<code>Value PORT (\S+(/\d+)?)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)

Start
  ^Port\s+Align-Err.* -> FIRST
  ^Port\s+Single-Col.* -> SECOND

FIRST
  ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS} -> Continue.Record

SECOND
  ^${PORT}\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS} -> Record
</code>```

However, the output is not right:

```<code>['Gi0/1', '1', '2', '3', '4', '5', '6', '', '', '', '', '', '', '']
['Gi0/2', '11', '12', '13', '14', '15', '16', '', '', '', '', '', '', '']
['Gi0/1', '1', '2', '3', '4', '5', '6', '', '', '', '', '', '', '']
['Gi0/2', '111', '122', '133', '144', '155', '166', '', '', '', '', '', '', '']
</code>```

I found a post on the forum giving a solution in pure Regex: TextFSM logic - Avoid capturing same data twice

When I adapt it to my needs, I have a match for what I need: https://regex101.com/r/DY0Meb/6

However, I'm unable to translate it in a TextFSM template, it fails.
Here is my template:

```<code>Value PORT (\S+(/\d+)?)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)

Start
  ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS}(?=.*\1\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS}) -> Record
</code>```

Any clue about why I don't have any matches?
I'm a beginner in Regex, and I can't seem to find the solution...

Any help would be very welcome :).
Thanks in advance!

答:
I finally managed to do what I want.

My team wished to use Ansible for the formatting, so I had to improvise a bit.

I used ntc-ansible for that.

With the help of members of the NTC Slack, I finally got it working. Here's what I came up with:

A functionality that is very poorly documented on the TextFSM repo is that you can, in an index file, combine two templates that share a common "Key" attribute.

So I created two templates:

```<code>Value Key PORT (\S+(/\d+)+)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Start
  ^Port\s+Align-Err.* -> Begin
Begin
 ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS} -> Record
      ^Port\s+Single-Col.* -> End
</code>```

And:

```<code>Value Key PORT (\S+(/\d+)+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)
Start
  ^Port\s+Single-Col.* -> Begin
Begin
^${PORT}\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS} -> Record
</code>```

Then, I created an index file containing this:

```<code>Template, Hostname, Vendor, Command
show_int_counter_errors1.template:show_int_counter_errors2.template, .*, cisco_ios, sh[[ow]] int[[erfaces]] cou[[nter]] er[[rors]]
</code>```

You can test it in Python with this little script:

```<code>import textfsm
import sys
from textfsm import clitable
# Define Input Data
input_data = sys.stdin.read()
# Initialise CliTable with the content of the 'index' file.
cli_table = clitable.CliTable('index', '.')
# Firstly we will use attributes to match a 'show version' command on a Cisco device.
attributes = {'Command': sys.argv[1], 'Vendor': 'cisco_ios'}
# Parse Data
cli_table.ParseCmd(input_data, attributes)
print(cli_table)
</code>```

To launch it, just use this command:

```<code>python3 test_table.py 'show interface counter errors' < show_int_counter_errors.txt
</code>```

To use it in Ansible, after installing ntc-ansible, create a 'templates' directory, put the index file and the template files in it, and specify the directory path in the playbook:

```<code>- hosts: all
  connection: local
  gather_facts: no

  tasks:
    - name: Formatting the errors
      ntc_show_command:
        connection: ssh
        platform: cisco_ios
        command: 'show interfaces counter errors'
        use_templates: True
        template_dir: 'PATH/TO/TEMPLATES/DIRECTORY'
        host: "{{ ansible_host }}"
        username: "{{ ansible_user }}"
        password: "{{ ansible_password }}"
 register: interface_errors

    - name: Display the registered variable
      debug:
        var: interface_errors
</code>```

Hope this can help anybody :).

Title: Use a captured value as row identifier

Body:
I need to parse this raw data in order to process it:

Port        Align-Err     FCS-Err    Xmit-Err     Rcv-Err  UnderSize  OutDiscards
Gi0/1               1           2           3           4          5            6
Gi0/2               11          12          13          14         15           16

Port      Single-Col  Multi-Col   Late-Col  Excess-Col  Carri-Sen      Runts     Giants
Gi0/1              1          2          3           4          5          6          7
Gi0/2              111        122        133         144        155        166        177

To do this, I'm using TextFSM.

I would want this output:

['Gi0/1', '1', '2', '3', '4', '5', '6', '1', '2', '3', '4', '5', '6', '7']
['Gi0/2', '11', '12', '13', '14', '15', '16', '111', '112', '113', '114', '115', '116', '117']

The first template I wrote is this one:

Value PORT (\S+(/\d+)?)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)

Start
  ^Port\s+Align-Err.* -> FIRST
  ^Port\s+Single-Col.* -> SECOND

FIRST
  ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS} -> Continue.Record

SECOND
  ^${PORT}\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS} -> Record

However, the output is not right:

['Gi0/1', '1', '2', '3', '4', '5', '6', '', '', '', '', '', '', '']
['Gi0/2', '11', '12', '13', '14', '15', '16', '', '', '', '', '', '', '']
['Gi0/1', '1', '2', '3', '4', '5', '6', '', '', '', '', '', '', '']
['Gi0/2', '111', '122', '133', '144', '155', '166', '', '', '', '', '', '', '']

I found a post on the forum giving a solution in pure Regex: TextFSM logic - Avoid capturing same data twice

When I adapt it to my needs, I have a match for what I need: https://regex101.com/r/DY0Meb/6

However, I'm unable to translate it in a TextFSM template, it fails.
Here is my template:

Value PORT (\S+(/\d+)?)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)

Start
  ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS}(?=.*\1\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS}) -> Record

Any clue about why I don't have any matches?
I'm a beginner in Regex, and I can't seem to find the solution...

Any help would be very welcome :).
Thanks in advance!

Body:
I finally managed to do what I want.

My team wished to use Ansible for the formatting, so I had to improvise a bit.

I used ntc-ansible for that.

With the help of members of the NTC Slack, I finally got it working. Here's what I came up with:

A functionality that is very poorly documented on the TextFSM repo is that you can, in an index file, combine two templates that share a common "Key" attribute.

So I created two templates:

Value Key PORT (\S+(/\d+)+)
Value ALIGNERR (\d+)
Value FCSERR (\d+)
Value XMITERR (\d+)
Value RCVERR (\d+)
Value UNDERSIZE (\d+)
Value OUTDISCARDS (\d+)
Start
  ^Port\s+Align-Err.* -> Begin
Begin
 ^${PORT}\s+${ALIGNERR}\s+${FCSERR}\s+${XMITERR}\s+${RCVERR}\s+${UNDERSIZE}\s+${OUTDISCARDS} -> Record
      ^Port\s+Single-Col.* -> End

And:

Value Key PORT (\S+(/\d+)+)
Value SINGLECOL (\d+)
Value MULTICOL (\d+)
Value LATECOL (\d+)
Value EXCESSCOL (\d+)
Value CARRISEN (\d+)
Value RUNTS (\d+)
Value GIANTS (\d+)
Start
  ^Port\s+Single-Col.* -> Begin
Begin
^${PORT}\s+${SINGLECOL}\s+${MULTICOL}\s+${LATECOL}\s+${EXCESSCOL}\s+${CARRISEN}\s+${RUNTS}\s+${GIANTS} -> Record

Then, I created an index file containing this:

Template, Hostname, Vendor, Command
show_int_counter_errors1.template:show_int_counter_errors2.template, .*, cisco_ios, sh[[ow]] int[[erfaces]] cou[[nter]] er[[rors]]

You can test it in Python with this little script:

import textfsm
import sys
from textfsm import clitable
# Define Input Data
input_data = sys.stdin.read()
# Initialise CliTable with the content of the 'index' file.
cli_table = clitable.CliTable('index', '.')
# Firstly we will use attributes to match a 'show version' command on a Cisco device.
attributes = {'Command': sys.argv[1], 'Vendor': 'cisco_ios'}
# Parse Data
cli_table.ParseCmd(input_data, attributes)
print(cli_table)

To launch it, just use this command:

python3 test_table.py 'show interface counter errors' < show_int_counter_errors.txt

To use it in Ansible, after installing ntc-ansible, create a 'templates' directory, put the index file and the template files in it, and specify the directory path in the playbook:

- hosts: all
  connection: local
  gather_facts: no

  tasks:
    - name: Formatting the errors
      ntc_show_command:
        connection: ssh
        platform: cisco_ios
        command: 'show interfaces counter errors'
        use_templates: True
        template_dir: 'PATH/TO/TEMPLATES/DIRECTORY'
        host: "{{ ansible_host }}"
        username: "{{ ansible_user }}"
        password: "{{ ansible_password }}"
 register: interface_errors

    - name: Display the registered variable
      debug:
        var: interface_errors

Hope this can help anybody :).


问:
Getting others to use my apps with Visual C++
I am learning how to build games in Visual C++ and when I upload them so friends can check them out, they all end up with messages saying it can not run. I did some research and found that it is because I am compiling against a Dynamic library instead of a static library. Correct me if I am wrong anywhere please. Upon further research, I found that a lot of people do not advise going this direction but instead include the files needed by my game. </p>

<p>How would I go about distributing my games to friends and make it real easy for them to just open up my .exe and play the game? </p>

答:
There are essentially two options: Keep everything as-is and provide them with the runtime files (also named Microsoft Visual C++ * Redistributable Package or similar; the * has to be replaced with your version, e.g. 2005, 2008 or 2010). Downloads can be found on Microsoft's download site as well as in your Visual Studio installation folder (look for a folder called "Redist").

Alternative solution: In your project settings you're able to select the runtime environment (under linker options). Change your release build to use "Multithreaded" instead of "Multithreaded-DLL".

Body:
I am learning how to build games in Visual C++ and when I upload them so friends can check them out, they all end up with messages saying it can not run. I did some research and found that it is because I am compiling against a Dynamic library instead of a static library. Correct me if I am wrong anywhere please. Upon further research, I found that a lot of people do not advise going this direction but instead include the files needed by my game.

How would I go about distributing my games to friends and make it real easy for them to just open up my .exe and play the game?


问:
print shape values for all dimesions for a given input to a Keras model
I have a simple Keras model -

```<code>def print_tensor_shape(x):
    print(x.shape)
    return x

def stem(stem_width=32):
    inputs = keras.layers.Input(shape=[None, None, 3])
    X = keras.layers.ZeroPadding2D((1,1))(inputs)
    X = keras.layers.Conv2D(stem_width, kernel_size = 3, strides = 2, activation='relu')(X)
    keras.layers.Lambda(print_tensor_shape)(X)
    return keras.models.Model(inputs=inputs, outputs=X)
</code>```

I want to view the exact shape of the tensor for a specific input tensor after the convolution layer. However when I run -

```<code>inputs = tf.zeros([32,512,512,3])
out = stem()(inputs)
</code>```

The output I get is

```<code>(None, None, None, 32)
</code>```

I want to know the exact values for the first 3 dimensions for the input of shape [32,512,512,3] and not just [None, None, None].  Anyone can please help me to do this ?

答:
the print output you see (None, None, None, 32) isn't related to the input you passed. it refers generally to the layer dimension. if you would like to check the print you required, take care to connect the Lambda layer in your model. this enables to call the function every time you pass a new input</p>

```<code>def print_tensor_shape(x):
    print(x.shape)
    return x

def stem(stem_width=32):
    inputs = keras.layers.Input(shape=[None, None, 3])
    X = keras.layers.ZeroPadding2D((1,1))(inputs)
    X = keras.layers.Conv2D(stem_width, kernel_size = 3, strides = 2, activation='relu')(X)
    X = keras.layers.Lambda(print_tensor_shape)(X)
    return keras.models.Model(inputs=inputs, outputs=X)

inputs = tf.zeros([32,512,512,3])
out = stem()(inputs)
</code>```

Body:
I have a simple Keras model -

def print_tensor_shape(x):
    print(x.shape)
    return x

def stem(stem_width=32):
    inputs = keras.layers.Input(shape=[None, None, 3])
    X = keras.layers.ZeroPadding2D((1,1))(inputs)
    X = keras.layers.Conv2D(stem_width, kernel_size = 3, strides = 2, activation='relu')(X)
    keras.layers.Lambda(print_tensor_shape)(X)
    return keras.models.Model(inputs=inputs, outputs=X)

I want to view the exact shape of the tensor for a specific input tensor after the convolution layer. However when I run -

inputs = tf.zeros([32,512,512,3])
out = stem()(inputs)

The output I get is

(None, None, None, 32)

I want to know the exact values for the first 3 dimensions for the input of shape [32,512,512,3] and not just [None, None, None].  Anyone can please help me to do this ?

Body:
the print output you see (None, None, None, 32) isn't related to the input you passed. it refers generally to the layer dimension. if you would like to check the print you required, take care to connect the Lambda layer in your model. this enables to call the function every time you pass a new input

def print_tensor_shape(x):
    print(x.shape)
    return x

def stem(stem_width=32):
    inputs = keras.layers.Input(shape=[None, None, 3])
    X = keras.layers.ZeroPadding2D((1,1))(inputs)
    X = keras.layers.Conv2D(stem_width, kernel_size = 3, strides = 2, activation='relu')(X)
    X = keras.layers.Lambda(print_tensor_shape)(X)
    return keras.models.Model(inputs=inputs, outputs=X)

inputs = tf.zeros([32,512,512,3])
out = stem()(inputs)


答:
One of the way could be using Expanded widget along with alignment as for the need

```<code> Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Screen'),
      ),
      body: Column(
        // mainAxisAlignment: MainAxisAlignment.center, // this is wrong, it positions both the widgets in the center
        children: <Widget>[

          Expanded(child: Container(alignment:Alignment.bottomLeft,child: Text('This text should be at the center of the column'))),
          Expanded(child: Container(alignment: Alignment.bottomLeft, child: Text('This text should be at the end of the column'))),
        ],
      ),
    );

  }
</code>```

Title: Positioning children widgets at the center and at the end of a Column parent widget

Body:
I have to position 2 children widgets inside a Column parent widget. I want one of them to be positioned in the center and the other at the end of the column. What is the best way to do that?

import 'package:flutter/material.dart';

class Screen extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Screen'),
      ),
      body: Column(
        mainAxisAlignment: MainAxisAlignment.center, // this is wrong, it positions both the widgets in the center
        children: <Widget>[
          Text('This text should be at the center of the column'),
          Text('This text should be at the end of the column'),
        ],
      ),
    );
  }
}

Body:
One of the way could be using Expanded widget along with alignment as for the need

 Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Screen'),
      ),
      body: Column(
        // mainAxisAlignment: MainAxisAlignment.center, // this is wrong, it positions both the widgets in the center
        children: <Widget>[

          Expanded(child: Container(alignment:Alignment.bottomLeft,child: Text('This text should be at the center of the column'))),
          Expanded(child: Container(alignment: Alignment.bottomLeft, child: Text('This text should be at the end of the column'))),
        ],
      ),
    );

  }