问:
Pygal render SVG in Jupyter Notebooks
Im currently using Jupyter Notebooks and I would like to render SVG's via HTML without saving them as a local SVG and display them afterwards.

Im using this code:

```<code>%matplotlib inline
from IPython.display import SVG, HTML
html_pygal = """
<!DOCTYPE html>
<html>
  <head>
  <script type="text/javascript" src="http://kozea.github.com/pygal.js/javascripts/svg.jquery.js"></script>
  <script type="text/javascript" src="http://kozea.github.com/pygal.js/javascripts/pygal-tooltips.js"></script>
    <!-- ... -->
  </head>
  <body>
    <figure>
      {pygal_render}
    </figure>
  </body>
</html>
"""

import pygal
line_chart = pygal.Line()
line_chart.title = 'Browser usage evolution (in %)'
line_chart.x_labels = map(str, range(2002, 2013))
line_chart.add('Firefox', [None, None,    0, 16.6,   25,   31, 36.4, 45.5, 46.3, 42.8, 37.1])
line_chart.add('Chrome',  [None, None, None, None, None, None,    0,  3.9, 10.8, 23.8, 35.3])
line_chart.add('IE',      [85.8, 84.6, 84.7, 74.5,   66, 58.6, 54.7, 44.8, 36.2, 26.6, 20.1])
line_chart.add('Others',  [14.2, 15.4, 15.3,  8.9,    9, 10.4,  8.9,  5.8,  6.7,  6.8,  7.5])
HTML(html_pygal.format(pygal_render=line_chart.render()))
</code>```

Here is the problem: While Jupyter manages to render the SVG file it also displays a Javascript error which looks like follows:

b'\n

*SVG picture comes here

'
Javascript error adding output!
SyntaxError: illegal character
See your browser Javascript console for more details.

When checking the browser console this is the error message:

```<code>loaded custom.js  2017-04-20_analytics.ipynb:121:15
Default extension for cell metadata editing loaded.  main.min.js:24261:9
Raw Cell Format toolbar preset loaded.  main.min.js:24347:5
Slideshow extension for metadata editing loaded.  main.min.js:24395:9
actions jupyter-notebook:find-and-replace does not exist, still binding it in case it will be defined later...  main.min.js:29204:17
load_extensions Arguments { 0: "jupyter-js-widgets/extension", 2 weitere… }  main.min.js:12231:9
Session: kernel_created (b51341fe-f4d8-42be-a9b8-46ac47476a99)  main.min.js:23547:13
Starting WebSockets: ws://localhost:8888/api/kernels/1852f19e-52f2-4ce2-91d8-f6276ea95b40  main.min.js:22833:9
Loading extension: jupyter-js-widgets/extension  main.min.js:12208:21
Kernel: kernel_connected (1852f19e-52f2-4ce2-91d8-f6276ea95b40)  main.min.js:22480:13
Kernel: kernel_ready (1852f19e-52f2-4ce2-91d8-f6276ea95b40)  main.min.js:22480:13
loaded widgets  extension.js:134:14
Widget backend and frontend versions are compatible  extension.js:32280:22
SyntaxError: illegal character
Stack-Trace:
globalEval@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:4231
domManip@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:21389
append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:18980
OutputArea.prototype._safe_append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20345:13
OutputArea.prototype.append_execute_result@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20372:9
OutputArea.prototype.append_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20218:17
OutputArea.prototype.handle_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20165:9
output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:21820:21
Kernel.prototype._handle_output_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23440:13
i@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:5486
Kernel.prototype._handle_iopub_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23467:20
Kernel.prototype._finish_ws_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23304:24
Kernel.prototype._handle_ws_message/this._msg_queue<@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23295:39
  main.min.js:20347:13
SyntaxError: illegal character
Stack-Trace:
globalEval@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:4231
domManip@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:21389
append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:18980
OutputArea.prototype._safe_append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20345:13
OutputArea.prototype.append_execute_result@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20372:9
OutputArea.prototype.append_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20218:17
OutputArea.prototype.handle_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20165:9
output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:21820:21
Kernel.prototype._handle_output_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23440:13
i@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:5486
Kernel.prototype._handle_iopub_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23467:20
Kernel.prototype._finish_ws_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23304:24
Kernel.prototype._handle_ws_message/this._msg_queue<@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23295:39
</code>```

My Python Version:
3.6.0
OS: Windows 7 64 Bit
Pygal Version: 2.3.1

Note: my current workaround which works but is not the solution which I'm looking for is to create a local SVG and then display it with the library import: from IPython.display import display

答:
Even better:</p>

```<code>bar_chart = pygal.Bar()
bar_chart.add('Fibonacci', [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55])
display({'image/svg+xml': bar_chart.render()}, raw=True)
</code>```

Title: Pygal render SVG in Jupyter Notebooks

Body:
Im currently using Jupyter Notebooks and I would like to render SVG's via HTML without saving them as a local SVG and display them afterwards.

Im using this code:

%matplotlib inline
from IPython.display import SVG, HTML
html_pygal = """
<!DOCTYPE html>

      {pygal_render}

"""

import pygal
line_chart = pygal.Line()
line_chart.title = 'Browser usage evolution (in %)'
line_chart.x_labels = map(str, range(2002, 2013))
line_chart.add('Firefox', [None, None,    0, 16.6,   25,   31, 36.4, 45.5, 46.3, 42.8, 37.1])
line_chart.add('Chrome',  [None, None, None, None, None, None,    0,  3.9, 10.8, 23.8, 35.3])
line_chart.add('IE',      [85.8, 84.6, 84.7, 74.5,   66, 58.6, 54.7, 44.8, 36.2, 26.6, 20.1])
line_chart.add('Others',  [14.2, 15.4, 15.3,  8.9,    9, 10.4,  8.9,  5.8,  6.7,  6.8,  7.5])
HTML(html_pygal.format(pygal_render=line_chart.render()))

Here is the problem: While Jupyter manages to render the SVG file it also displays a Javascript error which looks like follows:

b'\n

*SVG picture comes here

'
Javascript error adding output!
SyntaxError: illegal character
See your browser Javascript console for more details.

When checking the browser console this is the error message:

loaded custom.js  2017-04-20_analytics.ipynb:121:15
Default extension for cell metadata editing loaded.  main.min.js:24261:9
Raw Cell Format toolbar preset loaded.  main.min.js:24347:5
Slideshow extension for metadata editing loaded.  main.min.js:24395:9
actions jupyter-notebook:find-and-replace does not exist, still binding it in case it will be defined later...  main.min.js:29204:17
load_extensions Arguments { 0: "jupyter-js-widgets/extension", 2 weitere… }  main.min.js:12231:9
Session: kernel_created (b51341fe-f4d8-42be-a9b8-46ac47476a99)  main.min.js:23547:13
Starting WebSockets: ws://localhost:8888/api/kernels/1852f19e-52f2-4ce2-91d8-f6276ea95b40  main.min.js:22833:9
Loading extension: jupyter-js-widgets/extension  main.min.js:12208:21
Kernel: kernel_connected (1852f19e-52f2-4ce2-91d8-f6276ea95b40)  main.min.js:22480:13
Kernel: kernel_ready (1852f19e-52f2-4ce2-91d8-f6276ea95b40)  main.min.js:22480:13
loaded widgets  extension.js:134:14
Widget backend and frontend versions are compatible  extension.js:32280:22
SyntaxError: illegal character
Stack-Trace:
globalEval@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:4231
domManip@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:21389
append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:18980
OutputArea.prototype._safe_append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20345:13
OutputArea.prototype.append_execute_result@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20372:9
OutputArea.prototype.append_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20218:17
OutputArea.prototype.handle_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20165:9
output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:21820:21
Kernel.prototype._handle_output_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23440:13
i@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:5486
Kernel.prototype._handle_iopub_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23467:20
Kernel.prototype._finish_ws_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23304:24
Kernel.prototype._handle_ws_message/this._msg_queue<@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23295:39
  main.min.js:20347:13
SyntaxError: illegal character
Stack-Trace:
globalEval@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:4231
domManip@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:21389
append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:88:18980
OutputArea.prototype._safe_append@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20345:13
OutputArea.prototype.append_execute_result@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20372:9
OutputArea.prototype.append_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20218:17
OutputArea.prototype.handle_output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:20165:9
output@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:21820:21
Kernel.prototype._handle_output_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23440:13
i@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:87:5486
Kernel.prototype._handle_iopub_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23467:20
Kernel.prototype._finish_ws_message@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23304:24
Kernel.prototype._handle_ws_message/this._msg_queue<@http://localhost:8888/static/notebook/js/main.min.js?v=edbc43d9bb99b1be2585a121ed720314:23295:39

My Python Version:
3.6.0
OS: Windows 7 64 Bit
Pygal Version: 2.3.1

Note: my current workaround which works but is not the solution which I'm looking for is to create a local SVG and then display it with the library import: from IPython.display import display

Body:
Even better:

bar_chart = pygal.Bar()
bar_chart.add('Fibonacci', [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55])
display({'image/svg+xml': bar_chart.render()}, raw=True)


来源: stackexchange

问:
Update Data Connection to latest DB version on Visual Studio 17
I'm currently using Visual Studio to develop an ASP.NET/C# web application that relies on a SQL Server database for the data behind. From the moment I originally created the Data Connection through VS, I've gone back and changed certain things in the DB (certain column types, added test rows, etc) through SQL Server Management Studio. However, none of these changes are reflected in the VS application, which is a pain because I am constantly getting errors because of the mismatch. For example, I tried to insert a string into a column that was originally an int and I get an error when trying to process the query through my code behind, even though the actual DB lets me run the query just fine through SQL Server Manager.

This mismatch is so bad that if I run the same query (SELECT * FROM Project) on both SQL Server Manager and Visual Studio I get completely different results: SQL Server Management Studio vs Visual Studio

I've tried refreshing my connection on Server Explorer to no end and even created a new connection string on Web.config to see if this would somehow fix it, but nothing seems to work. I found an article that explains that VS creates a local version of the DB during the build, but it seems like the solution it provides only works for non-ASP.NET projects because I don't have a bin/Debug DB.

Any suggestions to get Visual Studio to reflect the latest version of my schema/data? Thank you!

答:
From your example files, you are working on two different databases.

Execute the following command in SQL Server Management Studio and in Visual Studio.  You will likely see that the filenames listed are different.

```<code>exec sp_helpfile
</code>```

If so, then you are not connecting to the same server/database in both environments.  You can click on the database in Visual Studio to see the "Connection string" in the Properties window.  Check it against your web.config connection string.

ContentLicense: CC BY-SA 3.0

Body:
From your example files, you are working on two different databases.

Execute the following command in SQL Server Management Studio and in Visual Studio.  You will likely see that the filenames listed are different.

exec sp_helpfile

If so, then you are not connecting to the same server/database in both environments.  You can click on the database in Visual Studio to see the "Connection string" in the Properties window.  Check it against your web.config connection string.


问:
how to execute celery tasks conditionally python
I am new to celery.  I have a celery task that needs to be executed when a condition is met.  Otherwise retry after few minutes.  From the below code, I am stuck at how to retry the same task in else condition?  Appreciate your help.</p>

```<code>@app.task(bind=True,soft_time_limit=4 * 3600)
def task_message_queue(id, name=None, tid=None, src=None, dest=None, queue="MessageQueue"):
   ThreadLocalStore().set_data({"id": id, "tid": tid, "name": name,"src": src, "dest":dest})
   num_files = os.popen("find %s -type f | wc -l" % dest).read().strip().split('\n')[0]
   if num_files < 20:
     #Move files from src to destination
   else:
     #wait for 2 minutes and retry the task
</code>```

答:
Instead of relying on retry you could also just trigger the task again.</p>

```<code>from celery.task import task
@app.task(bind=True,soft_time_limit=4 * 3600)
def task_message_queue(self, id, name=None, tid=None, src=None, dest=None, queue="MessageQueue"):
   ThreadLocalStore().set_data({"id": id, "tid": tid, "name": name,"src": src, "dest":dest})
   num_files = os.popen("find %s -type f | wc -l" % dest).read().strip().split('\n')[0]

   if num_files < 20:
       #Move files from src to destination
   else:
       # Trigger the task again in 120 seconds.
       task_message_queue.apply_async(countdown=120)
</code>```

Body:
I am new to celery.  I have a celery task that needs to be executed when a condition is met.  Otherwise retry after few minutes.  From the below code, I am stuck at how to retry the same task in else condition?  Appreciate your help.

@app.task(bind=True,soft_time_limit=4 * 3600)
def task_message_queue(id, name=None, tid=None, src=None, dest=None, queue="MessageQueue"):
   ThreadLocalStore().set_data({"id": id, "tid": tid, "name": name,"src": src, "dest":dest})
   num_files = os.popen("find %s -type f | wc -l" % dest).read().strip().split('\n')[0]
   if num_files < 20:
     #Move files from src to destination
   else:
     #wait for 2 minutes and retry the task

Body:
Instead of relying on retry you could also just trigger the task again.

from celery.task import task
@app.task(bind=True,soft_time_limit=4 * 3600)
def task_message_queue(self, id, name=None, tid=None, src=None, dest=None, queue="MessageQueue"):
   ThreadLocalStore().set_data({"id": id, "tid": tid, "name": name,"src": src, "dest":dest})
   num_files = os.popen("find %s -type f | wc -l" % dest).read().strip().split('\n')[0]

   if num_files < 20:
       #Move files from src to destination
   else:
       # Trigger the task again in 120 seconds.
       task_message_queue.apply_async(countdown=120)


问:
MMORPG protocol encryption
Anyone know what the trend is with MMORPG developers encryption their client/server protocols these days?

The pro's and con's are as follows.

Encrypting protocol:

protects trade secrets regarding client/server protocol to a degree?

Botting isn't stopped, it is only changed because people will create bots which read screen states and trigger mouse+keyboard events will then be developed, merely mitigating the development of bots as opposed to stopping them in anyway. Still though bot development is less in supply somewhat, but the demand is still there, they merely get higher bids on rentacoder.com

Regular plain text:

more advanced bots since the developers are able to inject their own datagrams into the client/server protocol. (ie: running through walls, teleporting... which the server side has to now check, which in turn leads to a development contest between out patching exploits via injection (time consuming)

答: I was under the impression that none of the major MMOs used encrypted data transport. As Samuel said, you don't really get any security out of it since the client has to be able to encrypt and decrypt the data too.</p>

OwnerDisplayName: Zombies

ContentLicense: CC BY-SA 2.5

Title: MMORPG protocol encryption


问:
Appending predicted residuals to pandas dataframe - by groups
This question extends this question, but now I want to add the residuals for separate groups.

So, how to append the residuals when you want to run regressions for separate groups?

Here is the data frame:

```<code>df = pd.DataFrame({'gp': [1,1,1,1,1,2,2,2,2,2],
               'x1': [3.17, 4.76, 4.17, 8.70, 11.45, 3.17, 4.76, 4.17, 8.70, 11.45],
               'x2': [23, 26, 73, 72, 16, 26, 73, 72, 16, 25],
               'y': [880.37, 716.20, 974.79, 322.80, 1054.25, 980.37, 816.20, 1074.79, 522.80, 1254.25]},
               index=np.arange(10, 30, 2))
</code>```

Now, I want to run separate regression for the two groups (gp), and append the residuals in a separate column.

I tried this code, but it only fills the residuals of the last regression group (gp=2):

```<code>import numpy as np
import pandas as pd
import statsmodels.formula.api as sm

def groupreg(df, regmodel):
    groups = df.groupby('gp')
    for item, group in groups:
        df['residual'] = sm.ols(formula=regmodel, data=group).fit().resid
    return (df)

regmodel = 'y ~ x1 + x2'

df = groupreg(df,regmodel)
</code>```

I found one way to crack this problem, but the code is long en looks inefficient:

```<code>def groupreg2(df, regmodel):
    groups = df.groupby('gp')
    i = 0
    resname='residual'
    for item, group in groups:
        data = group.copy()
        data[resname] = sm.ols(formula=regmodel, data=data).fit().resid
        if i == 0:
            i = 1
            dout = data[resname].copy()
        else:
            dout = dout.append(data[resname].copy())

    df = pd.concat([df,dout],axis=1)
    return (df)

    df = groupreg2(df,regmodel)
</code>```

Any suggestions to improve?

Body:
This question extends this question, but now I want to add the residuals for separate groups.

So, how to append the residuals when you want to run regressions for separate groups?

Here is the data frame:

df = pd.DataFrame({'gp': [1,1,1,1,1,2,2,2,2,2],
               'x1': [3.17, 4.76, 4.17, 8.70, 11.45, 3.17, 4.76, 4.17, 8.70, 11.45],
               'x2': [23, 26, 73, 72, 16, 26, 73, 72, 16, 25],
               'y': [880.37, 716.20, 974.79, 322.80, 1054.25, 980.37, 816.20, 1074.79, 522.80, 1254.25]},
               index=np.arange(10, 30, 2))

Now, I want to run separate regression for the two groups (gp), and append the residuals in a separate column.

I tried this code, but it only fills the residuals of the last regression group (gp=2):

import numpy as np
import pandas as pd
import statsmodels.formula.api as sm

def groupreg(df, regmodel):
    groups = df.groupby('gp')
    for item, group in groups:
        df['residual'] = sm.ols(formula=regmodel, data=group).fit().resid
    return (df)

regmodel = 'y ~ x1 + x2'

df = groupreg(df,regmodel)

I found one way to crack this problem, but the code is long en looks inefficient:

def groupreg2(df, regmodel):
    groups = df.groupby('gp')
    i = 0
    resname='residual'
    for item, group in groups:
        data = group.copy()
        data[resname] = sm.ols(formula=regmodel, data=data).fit().resid
        if i == 0:
            i = 1
            dout = data[resname].copy()
        else:
            dout = dout.append(data[resname].copy())

    df = pd.concat([df,dout],axis=1)
    return (df)

    df = groupreg2(df,regmodel)

Any suggestions to improve?

Body:
Simply turn your defined method into a groupby.apply() where you pass in each gp:

def groupreg(g):
    g['residual'] = sm.ols(formula=regmodel, data=g).fit().resid
    return g

df = df.groupby('gp').apply(groupreg)
print(df)

#     gp     x1  x2        y    residual
# 10   1   3.17  23   880.37  -43.579309
# 12   1   4.76  26   716.20 -174.532201
# 14   1   4.17  73   974.79  318.634921
# 16   1   8.70  72   322.80 -287.710952
# 18   1  11.45  16  1054.25  187.187542
# 20   2   3.17  26   980.37  174.295283
# 22   2   4.76  73   816.20 -173.045597
# 24   2   4.17  72  1074.79  101.623955
# 26   2   8.70  16   522.80 -372.840833
# 28   2  11.45  25  1254.25  269.967192


问:
Combine HTML and JS from Fiddle
A similar question was asked in this forum post, but I am still unable to convert the code from JSfiddle to HTML.

The JSfiddle example can be found here.

I tried to use the technique suggested in the forum post previously mentioned, that is:

```<code><html>
<head>
<style type="text/css">
    // CSS Content
    </style>
</head>
<body ng-app="myApp">
 <!-- some html elements -->
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.4.5/angular.min.js"></script>
    <script language="JavaScript"  type="text/javascript">
        // more js here.
    </script>
</body>
</code>```

As I am a complete noob, I simply copied the HTML and Javascript into the some html elements and // more js here sections. Without changing the API key, the final code looked like this:

```<code><html>
<head>
<style type="text/css">
    // CSS Content
    </style>
</head>
<body ng-app="myApp">
<div ng-app="myapp" ng-controller="WeatherCtrl">
    <h2>Weather in Salzburg, Austria</h2>
    <weather-icon cloudiness="{{ weather.clouds }}"></weather-icon>
    <h3>Current: {{ weather.temp.current | temp:2 }}</h3>
    min: {{ weather.temp.min | temp }}, max: {{ weather.temp.max | temp }}
</div>
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.4.5/angular.min.js"></script>
    <script language="JavaScript"  type="text/javascript">

'use strict';

var myapp = angular.module('myapp', []);

myapp.factory('weatherService', function($http) {
    return {
      getWeather: function() {
        var weather = { temp: {}, clouds: null };
        $http.jsonp('http://api.openweathermap.org/data/2.5/weather?q=Salzburg,at&units=metric&callback=JSON_CALLBACK&APPID=f9dbd911bc01df1d9ce563b2ba4d3209').success(function(data) {
            if (data) {
                if (data.main) {
                    weather.temp.current = data.main.temp;
                    weather.temp.min = data.main.temp_min;
                    weather.temp.max = data.main.temp_max;
                }
                weather.clouds = data.clouds ? data.clouds.all : undefined;
            }
        });

        return weather;
      }
    };
});

myapp.filter('temp', function($filter) {
    return function(input, precision) {
        if (!precision) {
            precision = 1;
        }
        var numberFilter = $filter('number');
        return numberFilter(input, precision) + '\u00B0C';
    };
});

myapp.controller('WeatherCtrl', function ($scope, weatherService) {
    $scope.weather = weatherService.getWeather();
});

myapp.directive('weatherIcon', function() {
    return {
        restrict: 'E', replace: true,
        scope: {
            cloudiness: '@'
        },
        controller: function($scope) {
            $scope.imgurl = function() {
                var baseUrl = 'https://ssl.gstatic.com/onebox/weather/128/';
                if ($scope.cloudiness < 20) {
                    return baseUrl + 'sunny.png';
                } else if ($scope.cloudiness < 90) {
                   return baseUrl + 'partly_cloudy.png';
                } else {
                    return baseUrl + 'cloudy.png';
                }
            };
        },
        template: '<div style="float:left"><img ng-src="{{ imgurl() }}"></div>'
    };
});

    </script>
</body>
</code>```

My output looks like this:

Could anyone please show me how to do this?

答: The problem is that you are declaring two apps remove "ng-app="myApp" from the body  tag.

Title: Combine HTML and JS from Fiddle

Body:
A similar question was asked in this forum post, but I am still unable to convert the code from JSfiddle to HTML.

The JSfiddle example can be found here.

I tried to use the technique suggested in the forum post previously mentioned, that is:

As I am a complete noob, I simply copied the HTML and Javascript into the some html elements and // more js here sections. Without changing the API key, the final code looked like this:

Weather in Salzburg, Austria

    <weather-icon cloudiness="{{ weather.clouds }}"></weather-icon>

Current: {{ weather.temp.current | temp:2 }}

    min: {{ weather.temp.min | temp }}, max: {{ weather.temp.max | temp }}

My output looks like this:

Could anyone please show me how to do this?

Body: The problem is that you are declaring two apps remove "ng-app="myApp" from the body <body> tag.


问:
for loop prints the same thing in each iteration, should only print it once
I am having trouble in which my for loop prints the same vector 17 times from 17 elements instead of printing 1 time and drawing from 17 elements. What is going wrong?

Also, I am trying to add the mean value at the end of the inverted vector but it is saying that the dimensions are off. (Second function works but I included it for reference as it is inside ProcessSpike).

```<code>function [] = ProcessSpike(dataset,element,cluster)
%UNTITLED2 Summary of this function goes here
% Detailed explanation goes here
result = []
for a = 1:element
    for b = 1:cluster
        result = [result AvSpike(dataset, a, b)];
        mean = nanmean(result)
        r = [result]'
        r(end+1) = num2str(mean)
    end
end

function [result] = AvSpike(dataset,element,cluster)
%UNTITLED Summary of this function goes here
% Detailed explanation goes here
Trans1 = dataset.Trans1;
Before_Trans1 = Trans1-600;
Firing_Time1 = dataset(cluster).time(dataset(cluster).time>Before_Trans1(element)&dataset(cluster).time<Trans1(element));
ISI1 = diff(Firing_Time1);
result = numel(ISI1)/600
result(result == 0) = NaN
end
</code>```

答:
The prints are caused by lines that are missing an ending ;, your editor should draw an orange line under these lines (warnings).
Regarding the mismatching dimensions, you are attempting to add a string (char array) to an existing array (r(end+1) = num2str(mean)). If the length of that char array does not match the length of the other elements in r, this will cause such an error. I would advise to not use num2str() here and just push a single value instead of the string representation of the value.

Body:
I am having trouble in which my for loop prints the same vector 17 times from 17 elements instead of printing 1 time and drawing from 17 elements. What is going wrong?

Also, I am trying to add the mean value at the end of the inverted vector but it is saying that the dimensions are off. (Second function works but I included it for reference as it is inside ProcessSpike).

function [] = ProcessSpike(dataset,element,cluster)
%UNTITLED2 Summary of this function goes here
% Detailed explanation goes here
result = []
for a = 1:element
    for b = 1:cluster
        result = [result AvSpike(dataset, a, b)];
        mean = nanmean(result)
        r = [result]'
        r(end+1) = num2str(mean)
    end
end

function [result] = AvSpike(dataset,element,cluster)
%UNTITLED Summary of this function goes here
% Detailed explanation goes here
Trans1 = dataset.Trans1;
Before_Trans1 = Trans1-600;
Firing_Time1 = dataset(cluster).time(dataset(cluster).time>Before_Trans1(element)&dataset(cluster).time<Trans1(element));
ISI1 = diff(Firing_Time1);
result = numel(ISI1)/600
result(result == 0) = NaN
end


问:
Count cells greater than 0 by group
I am trying to count the number of cells greater than 0 by group, and I need help as to how to approach this.

My data looks like this:

```<code>Group   Number of Shoes
    1                 0
    1                 1
    1                 2
    2                 0
    2                NA
    2                 1
    3                 1
    3                 2
    3                 2
</code>```

And I want this:

```<code>Group       Shoe owners
    1                 2
    2                 1
    3                 3
</code>```

答:
we can also use data.table

```<code>library(data.table)
setDT(df1)[Number.of.Shoes >0, .(Shoe.Owners = .N), Group]
#      Group Shoe.Owners
#1:     1           2
#2:     2           1
#3:     3           3
</code>```

Title: Count cells greater than 0 by group

Body:
I am trying to count the number of cells greater than 0 by group, and I need help as to how to approach this.

My data looks like this:

Group   Number of Shoes
    1                 0
    1                 1
    1                 2
    2                 0
    2                NA
    2                 1
    3                 1
    3                 2
    3                 2

And I want this:

Group       Shoe owners
    1                 2
    2                 1
    3                 3

Body:
we can also use data.table

library(data.table)
setDT(df1)[Number.of.Shoes >0, .(Shoe.Owners = .N), Group]
#      Group Shoe.Owners
#1:     1           2
#2:     2           1
#3:     3           3


问:
How do I make something show only on Firefox and Explorer?
I have a link on my page that adds the page to the user's favorites/bookmarks. The problem is that it only works on Explorer or Firefox so I want to add a condition so that it doesn't appear if the user is using Chrome. How do I do it?

This is the code:

```<code><a href="#" rel="sidebar" onclick="if(document.all && !window.opera){ window.external.AddFavorite(location.href, document.title);
return false; }else{ this.title = document.title; }" title="bookmark this page">Agregá esta página a favoritos</a>
</code>```

Thanks

答:
there are a plethora of ways to do this via css but this is the easiest

turn off webkit
/** saf3+, chrome1+ **/
@media screen and (-webkit-min-device-pixel-ratio:0) { a[rel="sidebar"]{display:none} }

Body:
I have a link on my page that adds the page to the user's favorites/bookmarks. The problem is that it only works on Explorer or Firefox so I want to add a condition so that it doesn't appear if the user is using Chrome. How do I do it?

This is the code:

Agregá esta página a favoritos

Thanks


问:
Python: remove row when a row element consists of string within row name
I have dataframe:

```<code>        Col1   Col2
Rowab1   3     5
Rowbc2   4     6
Rowxy3   7     2
</code>```

I want to search the dataframe and only keep the row where the row name contains "bc" like this while deleting rest of the rows:

```<code>        Col1   Col2
Rowab1   3     5
Rowxy3   7     2
</code>```

I have the following piece of code:

```<code>df.loc[df.index.isin(['bc'])]
</code>```

However, this does not search for "bc" within a string but it searches for an entire independent string "bc". Is there any "like" operator that I can possibly incorporate such as

```<code>df.loc[df.index.isin(['%bc%'])]  ?
</code>```

Thank you!

答:
You said in your post you wanted every row not containing "bc" to be deleted. Here are a variety of solutions:

Using a simple string-membership test:

```<code>>>> df.ix[[i for i in df.index if 'bc' in i]]
        Col1  Col2
Rowbc2     4     6
</code>```

Using regular expressions and the re module:

```<code>>>> df.ix[[i for i in df.index if re.match('.+bc.+', i)]]
        Col1  Col2
Rowbc2     4     6
</code>```

You could also use pandas.Series.str.match, as I alluded to in the comments:

```<code>>>> df[pd.Series(data=df.index, index=df.index).str.match('.+bc.+')]
        Col1  Col2
Rowbc2     4     6
</code>```

You can't directly use pandas.Series.str.match on a pandas.core.indexes.base.Index object, so I just converted it to a pandas.Series object with effectively garbage as the values. The you can use the str accessor that comes with Series objects.

Body:
I have dataframe:

        Col1   Col2
Rowab1   3     5
Rowbc2   4     6
Rowxy3   7     2

I want to search the dataframe and only keep the row where the row name contains "bc" like this while deleting rest of the rows:

        Col1   Col2
Rowab1   3     5
Rowxy3   7     2

I have the following piece of code:

df.loc[df.index.isin(['bc'])]

However, this does not search for "bc" within a string but it searches for an entire independent string "bc". Is there any "like" operator that I can possibly incorporate such as

df.loc[df.index.isin(['%bc%'])]  ?

Thank you!

Body:
You said in your post you wanted every row not containing "bc" to be deleted. Here are a variety of solutions:

Using a simple string-membership test:

>>> df.ix[[i for i in df.index if 'bc' in i]]
        Col1  Col2
Rowbc2     4     6

Using regular expressions and the re module:

>>> df.ix[[i for i in df.index if re.match('.+bc.+', i)]]
        Col1  Col2
Rowbc2     4     6

You could also use pandas.Series.str.match, as I alluded to in the comments:

>>> df[pd.Series(data=df.index, index=df.index).str.match('.+bc.+')]
        Col1  Col2
Rowbc2     4     6

You can't directly use pandas.Series.str.match on a pandas.core.indexes.base.Index object, so I just converted it to a pandas.Series object with effectively garbage as the values. The you can use the str accessor that comes with Series objects.


问:
how to retrieve hashMap data as a list and display in list
I am saving date to firebase with HashMap, I watched many tutorials, but facing problem in getting that data into list and display as List. Any help please thanks.</p>

答: pretty much you are gonna traversal hashMap and add the useful data in to a arrayList</p>


问:
Grabbing specific sections of a txt file via Powershell
I am new to Powershell scripting, but I feel I am overlooking a simple answer, hopefully some of you can help.

My company exports files from all of our computers with a section around the middle of Mapped Network Printers. It looks like this:

```<code>-------------------------------------------------------------------------
Mapped Network Printers:
NetworkAddress\HP425DN    [DEFAULT PRINTER]
-------------------------------------------------------------------------
Local Printers:
</code>```

What I have been asked to do is copy just the Mapped Network Printers to a new text file.

I tried using Select-String with a context parameter, but I have no way of knowing how many network printers there are, so I can't guess.

I also tried using the following code which I found on this site, but it returns nothing:

```<code>$MapPrint  = gc C:\Users\User1\Documents\Config.txt

$from =  ($MapPrint | Select-String -pattern "Mapped Network Printers:" |
Select-Object LineNumber).LineNumber
$to =  ($MapPrint  | Select-String -pattern "-------------------------------
--------------------------------------------" | Select-Object
LineNumber).LineNumber

$i = 0
$array = @()
foreach ($line in $MapPrint)
{
foreach-object { $i++ }
    if (($i -gt $from) -and ($i -lt $to))
    {
    $array += $line
    }
}
$array
</code>```

I basically want to start the search at "Mapped Network Printers" and end it at the next row of "------"

Any help would be greatly appreciated.

答:
Select-String has no feature for extracting a range of lines based on content.

The simplest approach is to read the file as a whole and use the -replace operator to extract the range via a regular expression (regex):

```<code>$file = 'C:\Users\User1\Documents\Config.txt'
$regex = '(?sm).*^Mapped Network Printers:\r?\n(.*?)\r?\n---------------------.*'
(Get-Content -Raw $file) -replace $regex, '$1'
</code>```

Reading an input file as a whole can be problematic with files too large to fit into memory, but that's probably not a concern for you.

On the plus side, this approach is much faster than processing the lines in a loop.

Get-Content -Raw (PSv3+) reads the input file as a whole.

Inline regex options (?sm) turn on both the multi-line and the single-line option:

m means that ^ and $ match the start and end of each line rather than the input string as a whole.

s means that metacharacter . matches \n characters too, so that an expression such as .* can be used to match across lines.

\r?\n matches a single line break, both the CRLF and the LF variety.

(.*?) is the capture group that (non-greedily) captures everything between the bounding lines.

Note that the regex matches the entire input string, and then replaces it with just the substring (range) of interest, captured in the 1st (and only) capture group ($1).

Assuming that $file contains:

```<code>-------------------------------------------------------------------------
Mapped Network Printers:
NetworkAddress\HP425DN    [DEFAULT PRINTER]
NetworkAddress\HP426DN
-------------------------------------------------------------------------
Local Printers:
</code>```

the above yields:

```<code>NetworkAddress\HP425DN    [DEFAULT PRINTER]
NetworkAddress\HP426DN
</code>```

Body:
I am new to Powershell scripting, but I feel I am overlooking a simple answer, hopefully some of you can help.

My company exports files from all of our computers with a section around the middle of Mapped Network Printers. It looks like this:

-------------------------------------------------------------------------
Mapped Network Printers:
NetworkAddress\HP425DN    [DEFAULT PRINTER]
-------------------------------------------------------------------------
Local Printers:

What I have been asked to do is copy just the Mapped Network Printers to a new text file.

I tried using Select-String with a context parameter, but I have no way of knowing how many network printers there are, so I can't guess.

I also tried using the following code which I found on this site, but it returns nothing:

$MapPrint  = gc C:\Users\User1\Documents\Config.txt

$from =  ($MapPrint | Select-String -pattern "Mapped Network Printers:" |
Select-Object LineNumber).LineNumber
$to =  ($MapPrint  | Select-String -pattern "-------------------------------
--------------------------------------------" | Select-Object
LineNumber).LineNumber

$i = 0
$array = @()
foreach ($line in $MapPrint)
{
foreach-object { $i++ }
    if (($i -gt $from) -and ($i -lt $to))
    {
    $array += $line
    }
}
$array

I basically want to start the search at "Mapped Network Printers" and end it at the next row of "------"

Any help would be greatly appreciated.

Body:
Select-String has no feature for extracting a range of lines based on content.

The simplest approach is to read the file as a whole and use the -replace operator to extract the range via a regular expression (regex):

$file = 'C:\Users\User1\Documents\Config.txt'
$regex = '(?sm).*^Mapped Network Printers:\r?\n(.*?)\r?\n---------------------.*'
(Get-Content -Raw $file) -replace $regex, '$1'

Reading an input file as a whole can be problematic with files too large to fit into memory, but that's probably not a concern for you.

On the plus side, this approach is much faster than processing the lines in a loop.

Get-Content -Raw (PSv3+) reads the input file as a whole.

Inline regex options (?sm) turn on both the multi-line and the single-line option:

m means that ^ and $ match the start and end of each line rather than the input string as a whole.

s means that metacharacter . matches \n characters too, so that an expression such as .* can be used to match across lines.

\r?\n matches a single line break, both the CRLF and the LF variety.

(.*?) is the capture group that (non-greedily) captures everything between the bounding lines.

Note that the regex matches the entire input string, and then replaces it with just the substring (range) of interest, captured in the 1st (and only) capture group ($1).

Assuming that $file contains:

-------------------------------------------------------------------------
Mapped Network Printers:
NetworkAddress\HP425DN    [DEFAULT PRINTER]
NetworkAddress\HP426DN
-------------------------------------------------------------------------
Local Printers:

the above yields:

NetworkAddress\HP425DN    [DEFAULT PRINTER]
NetworkAddress\HP426DN


问:
PHP Class on Class, get Parent
Let's say I have three classes. One being the parent and two being "childs". However, I'm not using  class - extends -:

```<code>class root
{
   function root(){
      $this->son = new son();
      $this->daughter = new daughter();
   }
}
class son
{
 ...
}
class daughter
{
  ...
}
</code>```

How could I call a function of son from a function of daughter? In other words, how could I reference the class root from son/daughter so that I could call functions of each other, from each other?

答:
Since son and daughter aren't actually subclasses of root, the only way to call each other's functions is to get an instance and call from there.  Unless the functions are declared static, in which case you can call them via son::my_func() / daughter::my_func, etc.

I'm not sure what the goal is, but maybe this will help: http://php.net/manual/en/language.oop5.patterns.php

Title: PHP Class on Class, get Parent

Body:
Let's say I have three classes. One being the parent and two being "childs". However, I'm not using  class - extends -:

class root
{
   function root(){
      $this->son = new son();
      $this->daughter = new daughter();
   }
}
class son
{
 ...
}
class daughter
{
  ...
}

How could I call a function of son from a function of daughter? In other words, how could I reference the class root from son/daughter so that I could call functions of each other, from each other?


问:
How to figure out a pod's namespace from within a pod?
My service X is running in a pod P with Namespace N. how do I find out namespace info from withing Service X using Kubernetes go client API?</p>

答:
I'm not sure what is meant by go client API. The pod's namespace is in /var/run/secrets/kubernetes.io/serviceaccount/namespace so this "service X" just needs to read this file.

Alternatively, you can also use the Downward API.


问:
h5py not sticking to chunking specification?
Problem:
I have existing netCDF4 files (about 5000 of them), (typically in shape 96x3712x3712) datapoints (float32). These are files with the first dimension being time (1 file per day), the second and third spatial dimensions.
Currently, making a slice over the first dimension (even a partial slice), would take a lot of time because of the following reasons:

the netCDF files are chunked with a chunksize of 1x3712x3712. Slicing over the time dimension basically would read the entire file.

looping (even in multiple processes) over all of the smaller files would take a lot of time as well.

My goal:

create monthly files (about 2900x3712x3712) data points

optimize them for slicing in time dimension (chunksize of 2900x1x1 or slightly bigger in the spatial dimensions)

Other requirements:

the files should be appendable by a single timestamp (1x3712x3712), and this update process should take less than 15min

the query should be fast enough: a full slice over time in less than one second (that is 2900x1x1)==> not so much data in fact...

preferably the files should be accessible for read by multiple processes while being updated

processing the historical data (the other 5000 daily files) should take less than a couple of weeks preferably.

I tried already multiple approaches:

concatenating netcdf files and rechunking them==> takes too much memory and too much time...

writing them from pandas to an hdf file (using pytables)==> creates a wide table with a huge index. This will eventually take too much time as well to read and requires the dataset to be tiled over the spatial dimensions due to meta data constraints.

my last approach was writing them to an hdf5 file using h5py:

Here's the code to create a single monthly file:

```<code>import h5py
import pandas as pd
import numpy as np

def create_h5(fps):
    timestamps=pd.date_range("20050101",periods=31*96,freq='15T') #Reference time period
    output_fp = r'/data/test.h5'
    try:
        f = h5py.File(output_fp, 'a',libver='latest')
        shape = 96*nodays, 3712, 3712
        d = f.create_dataset('variable', shape=(1,3712,3712), maxshape=(None,3712,3712),dtype='f', compression='gzip', compression_opts=9,chunks=(1,29,29))
        f.swmr_mode = True
        for fp in fps:
            try:
                nc=Dataset(fp)
                times = num2date(nc.variables['time'][:], nc.variables['time'].units)
                indices=np.searchsorted(timestamps, times)
                for j,time in enumerate(times):
                    logger.debug("File: {}, timestamp: {:%Y%m%d %H:%M}, pos: {}, new_pos: {}".format(os.path.basename(fp),time,j,indices[j]))
                    d.resize((indices[j]+1,shape[1],shape[2]))
                    d[indices[j]]=nc.variables['variable'][j:j+1]
                    f.flush()
            finally:
                nc.close()
    finally:
        f.close()
    return output_fp
</code>```

I'm using the last version of HDF5 to have the SWMR option. The fps argument is a list of file paths of the daily netCDF4 files. It creates the file (on an ssd, but I see that creating the file is mainly CPU bound) in about 2 hours, which is acceptable.

I have compression set up to keep the file size within limits. I did earlier tests without, and saw that the creation without is a bit faster but the slicing takes not so much longer with compression. H5py automatically chunks the dataset in 1x116x116 chunks.

Now the problem: slicing on a NAS with RAID 6 setup, takes about 20seconds to slice the time dimension, even though it is in a single chunk...

I figure, that even though it is in a single chunk in the file, because I wrote all of the values in a loop, it must be fragmented some how (don't know how this process works though). This is why I tried to do a h5repack using the CML tools of HDF5 into a new file, with the same chunks but hopefully reordering the values so that the query is able to read the values in a more sequential order, but no luck. Even though this process took 6h to run, it didn't do a thing on the query speed.

If I do my calculations right, reading one chunk (2976x32x32) is only a few MB big (11MB uncompressed, only a bit more than 1MB compressed I think). How can this take so long? What am I doing wrong? I would be glad if someone can shine a light on what is actually going on behind the scenes...

答:
The influence of chunk size

In a worst case scenario reading and writing one chunk can be considered as random read/write operation. The main advantage of a SSD is the speed of reading or writing small chunks of data. A HDD is much slower at this task (a factor 100 can be observed), a NAS can even be much slower than a HDD.

So the solution of the problem will be a larger chunk size. Some benchmarks on my system (Core i5-4690).

Exampe_1 (chunk size (1,29,29)=3,4 kB):

```<code>import numpy as np
import tables #needed for blosc
import h5py as h5
import time
import h5py_cache as h5c

def original_chunk_size():
    File_Name_HDF5='some_Path'
    #Array=np.zeros((1,3712,3712),dtype=np.float32)
    Array=np.random.rand(96,3712,3712)

    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    nodays=1

    shape = 96*nodays, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(1,29,29),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays):
        d[i:i+1,:,:]=Array

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    d=f['variable']

    for i in xrange(0,3712,29):
        for j in xrange(0,3712,29):
            A=np.copy(d[:,i:i+29,j:j+29])

    print(time.time()-t1)
</code>```

Results (write/read):

SSD: 38s/54s

HDD: 40s/57s

NAS: 252s/823s

In the second example I will use h5py_chache because I wan't to maintain providing chunks of (1,3712,3712). The standard chunk-chache-size is only one MB so it has to be changed, to avoid multiple read/write operations on chunks.
https://pypi.python.org/pypi/h5py-cache/1.0

Exampe_2 (chunk size (96,58,58)=1,3 MB):

```<code>import numpy as np
import tables #needed for blosc
import h5py as h5
import time
import h5py_cache as h5c

def modified_chunk_size():
    File_Name_HDF5='some_Path'
    Array=np.random.rand(1,3712,3712)

    f = h5c.File(File_Name_HDF5, 'a',libver='latest',
    chunk_cache_mem_size=6*1024**3)
    f.swmr_mode = True
    nodays=1

    shape = 96*nodays, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays):
        d[i:i+1,:,:]=Array

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5c.File(File_Name_HDF5, 'a',libver='latest', chunk_cache_mem_size=6*1024**3) #6 GB chunk chache
    f.swmr_mode = True
    d=f['variable']

    for i in xrange(0,3712,58):
        for j in xrange(0,3712,58):
            A=np.copy(d[:,i:i+58,j:j+58])

    print(time.time()-t1)
</code>```

Results (write/read):

SSD: 10s/16s

HDD: 10s/16s

NAS: 13s/20s

The read/write speed can further be improved by mininimizing the api calls (reading and writing larger chunk blocks).

I also want't to mention her the compression method. Blosc can achieve up to 1GB/s throughput (CPU bottlenecking) gzip is slower, but provides better compression ratios.

```<code>d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=3)
</code>```

20s/30s file size: 101 MB

d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=6)

50s/58s file size: 87 MB

d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=9)

50s/60s file size: 64 MB

And now a benchmark of a whole month (30 days). The writing is a bit optimized and is written with (96,3712, 3712).

```<code>def modified_chunk_size():
    File_Name_HDF5='some_Path'

    Array_R=np.random.rand(1,3712,3712)
    Array=np.zeros((96,3712,3712),dtype=np.float32)
    for j in xrange(0,96):
        Array[j,:,:]=Array_R

    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    nodays=30

    shape = 96, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays,96):
        d[i:i+96,:,:]=Array
        d.resize((d.shape[0]+96,shape[1],shape[2]))

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    d=f['variable']
    for i in xrange(0,3712,58):
        for j in xrange(0,3712,58):
            A=np.copy(d[:,i:i+58,j:j+58])

    print(time.time()-t1)
</code>```

133s/301s with blosc

432s/684s with gzip compression_opts=3

I had the same problems when accessing data on a NAS. I hope this helps...

Body:
Problem:
I have existing netCDF4 files (about 5000 of them), (typically in shape 96x3712x3712) datapoints (float32). These are files with the first dimension being time (1 file per day), the second and third spatial dimensions.
Currently, making a slice over the first dimension (even a partial slice), would take a lot of time because of the following reasons:

the netCDF files are chunked with a chunksize of 1x3712x3712. Slicing over the time dimension basically would read the entire file.

looping (even in multiple processes) over all of the smaller files would take a lot of time as well.

My goal:

create monthly files (about 2900x3712x3712) data points

optimize them for slicing in time dimension (chunksize of 2900x1x1 or slightly bigger in the spatial dimensions)

Other requirements:

the files should be appendable by a single timestamp (1x3712x3712), and this update process should take less than 15min

the query should be fast enough: a full slice over time in less than one second (that is 2900x1x1)==> not so much data in fact...

preferably the files should be accessible for read by multiple processes while being updated

processing the historical data (the other 5000 daily files) should take less than a couple of weeks preferably.

I tried already multiple approaches:

concatenating netcdf files and rechunking them==> takes too much memory and too much time...

writing them from pandas to an hdf file (using pytables)==> creates a wide table with a huge index. This will eventually take too much time as well to read and requires the dataset to be tiled over the spatial dimensions due to meta data constraints.

my last approach was writing them to an hdf5 file using h5py:

Here's the code to create a single monthly file:

import h5py
import pandas as pd
import numpy as np

def create_h5(fps):
    timestamps=pd.date_range("20050101",periods=31*96,freq='15T') #Reference time period
    output_fp = r'/data/test.h5'
    try:
        f = h5py.File(output_fp, 'a',libver='latest')
        shape = 96*nodays, 3712, 3712
        d = f.create_dataset('variable', shape=(1,3712,3712), maxshape=(None,3712,3712),dtype='f', compression='gzip', compression_opts=9,chunks=(1,29,29))
        f.swmr_mode = True
        for fp in fps:
            try:
                nc=Dataset(fp)
                times = num2date(nc.variables['time'][:], nc.variables['time'].units)
                indices=np.searchsorted(timestamps, times)
                for j,time in enumerate(times):
                    logger.debug("File: {}, timestamp: {:%Y%m%d %H:%M}, pos: {}, new_pos: {}".format(os.path.basename(fp),time,j,indices[j]))
                    d.resize((indices[j]+1,shape[1],shape[2]))
                    d[indices[j]]=nc.variables['variable'][j:j+1]
                    f.flush()
            finally:
                nc.close()
    finally:
        f.close()
    return output_fp

I'm using the last version of HDF5 to have the SWMR option. The fps argument is a list of file paths of the daily netCDF4 files. It creates the file (on an ssd, but I see that creating the file is mainly CPU bound) in about 2 hours, which is acceptable.

I have compression set up to keep the file size within limits. I did earlier tests without, and saw that the creation without is a bit faster but the slicing takes not so much longer with compression. H5py automatically chunks the dataset in 1x116x116 chunks.

Now the problem: slicing on a NAS with RAID 6 setup, takes about 20seconds to slice the time dimension, even though it is in a single chunk...

I figure, that even though it is in a single chunk in the file, because I wrote all of the values in a loop, it must be fragmented some how (don't know how this process works though). This is why I tried to do a h5repack using the CML tools of HDF5 into a new file, with the same chunks but hopefully reordering the values so that the query is able to read the values in a more sequential order, but no luck. Even though this process took 6h to run, it didn't do a thing on the query speed.

If I do my calculations right, reading one chunk (2976x32x32) is only a few MB big (11MB uncompressed, only a bit more than 1MB compressed I think). How can this take so long? What am I doing wrong? I would be glad if someone can shine a light on what is actually going on behind the scenes...

Body:
The influence of chunk size

In a worst case scenario reading and writing one chunk can be considered as random read/write operation. The main advantage of a SSD is the speed of reading or writing small chunks of data. A HDD is much slower at this task (a factor 100 can be observed), a NAS can even be much slower than a HDD.

So the solution of the problem will be a larger chunk size. Some benchmarks on my system (Core i5-4690).

Exampe_1 (chunk size (1,29,29)=3,4 kB):

import numpy as np
import tables #needed for blosc
import h5py as h5
import time
import h5py_cache as h5c

def original_chunk_size():
    File_Name_HDF5='some_Path'
    #Array=np.zeros((1,3712,3712),dtype=np.float32)
    Array=np.random.rand(96,3712,3712)

    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    nodays=1

    shape = 96*nodays, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(1,29,29),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays):
        d[i:i+1,:,:]=Array

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    d=f['variable']

    for i in xrange(0,3712,29):
        for j in xrange(0,3712,29):
            A=np.copy(d[:,i:i+29,j:j+29])

    print(time.time()-t1)

Results (write/read):

SSD: 38s/54s

HDD: 40s/57s

NAS: 252s/823s

In the second example I will use h5py_chache because I wan't to maintain providing chunks of (1,3712,3712). The standard chunk-chache-size is only one MB so it has to be changed, to avoid multiple read/write operations on chunks.
https://pypi.python.org/pypi/h5py-cache/1.0

Exampe_2 (chunk size (96,58,58)=1,3 MB):

import numpy as np
import tables #needed for blosc
import h5py as h5
import time
import h5py_cache as h5c

def modified_chunk_size():
    File_Name_HDF5='some_Path'
    Array=np.random.rand(1,3712,3712)

    f = h5c.File(File_Name_HDF5, 'a',libver='latest',
    chunk_cache_mem_size=6*1024**3)
    f.swmr_mode = True
    nodays=1

    shape = 96*nodays, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays):
        d[i:i+1,:,:]=Array

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5c.File(File_Name_HDF5, 'a',libver='latest', chunk_cache_mem_size=6*1024**3) #6 GB chunk chache
    f.swmr_mode = True
    d=f['variable']

    for i in xrange(0,3712,58):
        for j in xrange(0,3712,58):
            A=np.copy(d[:,i:i+58,j:j+58])

    print(time.time()-t1)

Results (write/read):

SSD: 10s/16s

HDD: 10s/16s

NAS: 13s/20s

The read/write speed can further be improved by mininimizing the api calls (reading and writing larger chunk blocks).

I also want't to mention her the compression method. Blosc can achieve up to 1GB/s throughput (CPU bottlenecking) gzip is slower, but provides better compression ratios.

d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=3)

20s/30s file size: 101 MB

d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=6)

50s/58s file size: 87 MB

d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression='gzip', compression_opts=9)

50s/60s file size: 64 MB

And now a benchmark of a whole month (30 days). The writing is a bit optimized and is written with (96,3712, 3712).

def modified_chunk_size():
    File_Name_HDF5='some_Path'

    Array_R=np.random.rand(1,3712,3712)
    Array=np.zeros((96,3712,3712),dtype=np.float32)
    for j in xrange(0,96):
        Array[j,:,:]=Array_R

    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    nodays=30

    shape = 96, 3712, 3712
    d = f.create_dataset('variable', shape, maxshape=(None,3712,3712),dtype='f',chunks=(96,58,58),compression=32001,compression_opts=(0, 0, 0, 0, 9, 1, 1), shuffle=False)

    #Writing
    t1=time.time()
    for i in xrange(0,96*nodays,96):
        d[i:i+96,:,:]=Array
        d.resize((d.shape[0]+96,shape[1],shape[2]))

    f.close()
    print(time.time()-t1)

    #Reading
    f = h5.File(File_Name_HDF5, 'a',libver='latest')
    f.swmr_mode = True
    d=f['variable']
    for i in xrange(0,3712,58):
        for j in xrange(0,3712,58):
            A=np.copy(d[:,i:i+58,j:j+58])

    print(time.time()-t1)

133s/301s with blosc

432s/684s with gzip compression_opts=3

I had the same problems when accessing data on a NAS. I hope this helps...


问:
Run cron job at 12 am in every time zone
So I think there is around 30 minutes between each time zone. I want to run my script cron.php for at 12 am (midnight) for every time zone. How can I do this?

I was looking at this code:

```<code>TZ=UTC
* 7 * * * root date | mail root
TZ=CEST
* 7 * * * root date | mail root
TZ=PCT
* 7 * * * root date | mail root
</code>```

Does this work?

What does any of the code mean?

Any better solution?

答:
Most time zones are 1 hour apart, with a couple of exceptions.
The above code will run at 7 AM in each of the time zones listed.

It might be better to have a script that runs every hour and just kick that off once in your own time zone

Body:
So I think there is around 30 minutes between each time zone. I want to run my script cron.php for at 12 am (midnight) for every time zone. How can I do this?

I was looking at this code:

TZ=UTC
* 7 * * * root date | mail root
TZ=CEST
* 7 * * * root date | mail root
TZ=PCT
* 7 * * * root date | mail root

Does this work?

What does any of the code mean?

Any better solution?


问:
Copy remote directory recursively with Fabric
Is there a way to copy a remote directory recursively with Fabric? I am currently using fabric to build a list of directories to download then using local("scp ...") to download them. The documentation on get() is spares and seems only for single files.</p>

答:
Use fabric.operations.get:

```<code>from fabric.operations import get as download
lcd('/path/to/local-directory-to-save-to')
download('/path/to/remote_directory', '.')
</code>```

Body:
Use fabric.operations.get:

from fabric.operations import get as download
lcd('/path/to/local-directory-to-save-to')
download('/path/to/remote_directory', '.')


问:
Python OpenCV: Detecting a general direction of movement?
I'm still hacking together a book scanning script, and for now, all I need is to be able to automagically detect a page turn. The book fills up 90% of the screen (I'm using a cruddy webcam for the motion detection), so when I turn a page, the direction of motion is basically in that same direction.

I have modified a motion-tracking script, but derivatives are getting me nowhere:

```<code>#!/usr/bin/env python

import cv, numpy

class Target:
    def __init__(self):
        self.capture = cv.CaptureFromCAM(0)
        cv.NamedWindow("Target", 1)

    def run(self):
        # Capture first frame to get size
        frame = cv.QueryFrame(self.capture)
        frame_size = cv.GetSize(frame)
        grey_image = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_8U, 1)
        moving_average = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_32F, 3)
        difference = None
        movement = []

        while True:
            # Capture frame from webcam
            color_image = cv.QueryFrame(self.capture)

            # Smooth to get rid of false positives
            cv.Smooth(color_image, color_image, cv.CV_GAUSSIAN, 3, 0)

            if not difference:
                # Initialize
                difference = cv.CloneImage(color_image)
                temp = cv.CloneImage(color_image)
                cv.ConvertScale(color_image, moving_average, 1.0, 0.0)
            else:
                cv.RunningAvg(color_image, moving_average, 0.020, None)

            # Convert the scale of the moving average.
            cv.ConvertScale(moving_average, temp, 1.0, 0.0)

            # Minus the current frame from the moving average.
            cv.AbsDiff(color_image, temp, difference)

            # Convert the image to grayscale.
            cv.CvtColor(difference, grey_image, cv.CV_RGB2GRAY)

            # Convert the image to black and white.
            cv.Threshold(grey_image, grey_image, 70, 255, cv.CV_THRESH_BINARY)

            # Dilate and erode to get object blobs
            cv.Dilate(grey_image, grey_image, None, 18)
            cv.Erode(grey_image, grey_image, None, 10)

            # Calculate movements
            storage = cv.CreateMemStorage(0)
            contour = cv.FindContours(grey_image, storage, cv.CV_RETR_CCOMP, cv.CV_CHAIN_APPROX_SIMPLE)
            points = []

            while contour:
                # Draw rectangles
                bound_rect = cv.BoundingRect(list(contour))
                contour = contour.h_next()

                pt1 = (bound_rect[0], bound_rect[1])
                pt2 = (bound_rect[0] + bound_rect[2], bound_rect[1] + bound_rect[3])
                points.append(pt1)
                points.append(pt2)
                cv.Rectangle(color_image, pt1, pt2, cv.CV_RGB(255,0,0), 1)

            num_points = len(points)

            if num_points:
                x = 0
                for point in points:
                    x += point[0]
                x /= num_points

                movement.append(x)

            if len(movement) > 0 and numpy.average(numpy.diff(movement[-30:-1])) > 0:
              print 'Left'
            else:
              print 'Right'

            # Display frame to user
            cv.ShowImage("Target", color_image)

            # Listen for ESC or ENTER key
            c = cv.WaitKey(7) % 0x100
            if c == 27 or c == 10:
                break

if __name__=="__main__":
    t = Target()
    t.run()
</code>```

It detects the average motion of the average center of all of the boxes, which is extremely inefficient. How would I go about detecting such motions quickly and accurately (i.e. within a threshold)?

I'm using Python, and I plan to stick with it, as my whole framework is based on Python.

And help is appreciated, so thank you all in advance. Cheers.

答:
I haven't used OpenCV in Python before, just a bit in C++ with openframeworks.

For this I presume OpticalFlow's velx,vely properties would work.

For more on how Optical Flow works check out this paper.

HTH

Body:
I'm still hacking together a book scanning script, and for now, all I need is to be able to automagically detect a page turn. The book fills up 90% of the screen (I'm using a cruddy webcam for the motion detection), so when I turn a page, the direction of motion is basically in that same direction.

I have modified a motion-tracking script, but derivatives are getting me nowhere:

#!/usr/bin/env python

import cv, numpy

class Target:
    def __init__(self):
        self.capture = cv.CaptureFromCAM(0)
        cv.NamedWindow("Target", 1)

    def run(self):
        # Capture first frame to get size
        frame = cv.QueryFrame(self.capture)
        frame_size = cv.GetSize(frame)
        grey_image = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_8U, 1)
        moving_average = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_32F, 3)
        difference = None
        movement = []

        while True:
            # Capture frame from webcam
            color_image = cv.QueryFrame(self.capture)

            # Smooth to get rid of false positives
            cv.Smooth(color_image, color_image, cv.CV_GAUSSIAN, 3, 0)

            if not difference:
                # Initialize
                difference = cv.CloneImage(color_image)
                temp = cv.CloneImage(color_image)
                cv.ConvertScale(color_image, moving_average, 1.0, 0.0)
            else:
                cv.RunningAvg(color_image, moving_average, 0.020, None)

            # Convert the scale of the moving average.
            cv.ConvertScale(moving_average, temp, 1.0, 0.0)

            # Minus the current frame from the moving average.
            cv.AbsDiff(color_image, temp, difference)

            # Convert the image to grayscale.
            cv.CvtColor(difference, grey_image, cv.CV_RGB2GRAY)

            # Convert the image to black and white.
            cv.Threshold(grey_image, grey_image, 70, 255, cv.CV_THRESH_BINARY)

            # Dilate and erode to get object blobs
            cv.Dilate(grey_image, grey_image, None, 18)
            cv.Erode(grey_image, grey_image, None, 10)

            # Calculate movements
            storage = cv.CreateMemStorage(0)
            contour = cv.FindContours(grey_image, storage, cv.CV_RETR_CCOMP, cv.CV_CHAIN_APPROX_SIMPLE)
            points = []

            while contour:
                # Draw rectangles
                bound_rect = cv.BoundingRect(list(contour))
                contour = contour.h_next()

                pt1 = (bound_rect[0], bound_rect[1])
                pt2 = (bound_rect[0] + bound_rect[2], bound_rect[1] + bound_rect[3])
                points.append(pt1)
                points.append(pt2)
                cv.Rectangle(color_image, pt1, pt2, cv.CV_RGB(255,0,0), 1)

            num_points = len(points)

            if num_points:
                x = 0
                for point in points:
                    x += point[0]
                x /= num_points

                movement.append(x)

            if len(movement) > 0 and numpy.average(numpy.diff(movement[-30:-1])) > 0:
              print 'Left'
            else:
              print 'Right'

            # Display frame to user
            cv.ShowImage("Target", color_image)

            # Listen for ESC or ENTER key
            c = cv.WaitKey(7) % 0x100
            if c == 27 or c == 10:
                break

if __name__=="__main__":
    t = Target()
    t.run()

It detects the average motion of the average center of all of the boxes, which is extremely inefficient. How would I go about detecting such motions quickly and accurately (i.e. within a threshold)?

I'm using Python, and I plan to stick with it, as my whole framework is based on Python.

And help is appreciated, so thank you all in advance. Cheers.


问:
What is the main difference between RSS, RPS and RFS?
As known, there are: https://www.kernel.org/doc/Documentation/networking/scaling.txt

RSS: Receive Side Scaling

RPS: Receive Packet Steering

RFS: Receive Flow Steering

Does it meant that:

RSS - allows us to use many CPU-Cores to process Soft-irq from Ethernet (one CPU-Core for each Ethernet queue)

RPS - allows us to process Soft-irq for all packets from one the same connection on one and the same CPU-Core

RFS - allows us to process Soft-irq for all packets from one the same connection on one and the same CPU-Core on which thread of our Application procces this connection

Is that correct?

答:
Quotes are from https://www.kernel.org/doc/Documentation/networking/scaling.txt.

RSS: Receive Side Scaling - is hardware implemented and hashes some bytes of packets ("hash function over the network and/or transport layer headers-- for example, a 4-tuple hash over IP addresses and TCP ports of a packet"). Implementations are different, some may not filter most useful bytes or may be limited in other ways. This filtering and queue distribution is fast (only several additional cycles are needed in hw to classify packet), but not portable between some network cards or can't be used with tunneled packets or some rare protocols. And sometimes your hardware have no support of number of queues enough to get one queue per logical CPU core.

RSS should be enabled when latency is a concern or whenever receive
  interrupt processing forms a bottleneck. Spreading load between CPUs
  decreases queue length.

Receive Packet Steering (RPS) "is logically a software implementation of
RSS. Being in software, it is necessarily called later in the datapath.". So, this is software alternative to hardware RSS (still parses some bytes to hash them into queue id), when you use hardware without RSS or want to classify based on more complex rule than hw can or have protocol which can't be parsed in HW RSS classifier. But with RPS more CPU resources are used and there is additional inter-CPU traffic.

RPS has some advantages over RSS: 1) it can be used with any NIC,
  2) software filters can easily be added to hash over new protocols,
  3) it does not increase hardware device interrupt rate (although it does
  introduce inter-processor interrupts (IPIs)).

RFS: Receive Flow Steering is like RSS (software mechanism with more CPU overhead), but it not just hashing into pseudo-random queue id, but takes "into account application locality." (so, packet processing will probably be faster due to good locality). Queues are tracked to be more local to the thread which will process received data, and packets are delivered to correct CPU core.

The goal of RFS is to increase datacache hitrate by steering
  kernel processing of packets to the CPU where the application thread
  consuming the packet is running. RFS relies on the same RPS mechanisms
  to enqueue packets onto the backlog of another CPU and to wake up that
  CPU. ... In RFS, packets are not forwarded directly by the value of their hash,
  but the hash is used as index into a flow lookup table. This table maps
  flows to the CPUs where those flows are being processed.

Accelerated RFS - RFS with hw support. (Check your network driver for ndo_rx_flow_steer) "Accelerated RFS is to RFS what RSS is to RPS: a hardware-accelerated load balancing mechanism that uses soft state to steer flows based on where the application thread consuming the packets of each flow is running.".

Similar method for packet transmitting (but packet is already generated and ready to be send, just select best queue to send it with - and to easier post-processing like freeing skb)

XPS: Transmit Packet Steering: "a mapping from CPU to hardware queue(s) is
recorded. The goal of this mapping is usually to assign queues
exclusively to a subset of CPUs, where the transmit completions for
these queues are processed on a CPU within this set"


问:
C# OutOfMemory Exception when reading Post Response
I'm currently working on a simple application that utilizes JSON objects to POST to an API and gets the responding data.  However, when I run the POST method, the POST response is so large that I encounter the OutOfMemory Exception.

I'm currently using WebClient and a CookieContainer for the process:

```<code>string jsonObject ="...."; //Example JSON string - It's very small

using (var client = new WebClient())
{
     var auth = new NameValueCollection();
       values["username"] = "username";
       values["password"] = "password";

     client.uploadValues(endpoint,auth);

     // This is causing the OutOfMemory Exception
     var response = client.uploadString(endpoint, jsonObject);

}
</code>```

I have looked into the issue, and have set the property AllowStreamBuffering to be false.

```<code> client.AllowStreamBuffering() = false;
</code>```

However, I am still encountering the issue, and do not know how to control the POST response.

Update: 7/5/2017

Thanks to @Tim suggestions, I've moved to a stream of the response, however I'm encountering issues regarding the actual response.  After writing the JSON (as a string) to the end point with a POST method, the script gets stuck at trying to read the response.

```<code>    String endPoint = @"http://example.com/v1/api/";
    String json = @"....";

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
    request.Method = "POST";
    request.KeepAlive = false;
    request.AllowReadStreamBuffering = false;

    /* Pretend this middle part does the Authorization with username and password. */
    /* I have actually authenticated using the above method, and passed a key to the request */

    //This part POST the JSON to the API
    using (StreamWriter writeStream = new StreamWriter(request.GetRequestStream()))
        {
            writeStream.Write(json);
            writeStream.Flush();
            writeStream.Close();
        }

    //This bottom part opens up a console, but never reads or loads the data
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    StreamReader reader = new StreamReader(response.GetResponseStream());
</code>```

I'm wondering if the JSON is not encoded probably.

(Side Note: I have looked at writing the response, line by line onto a file, but it's the response that is causing the issue -- http://cc.davelozinski.com/c-sharp/fastest-way-to-read-text-files)

答:
I was able to solve my own problem.  For the header, I forgot to encode the JSON, and set the content-type properly for the API.

Here is exactly the same code above for the stream, but with an updated header and bufferedStream for increased efficiency with handling data and memory.

```<code>String endPoint = @"http://example.com/v1/api/";
String json = @"....";//Example JSON string

Byte[] jsonData = Encoding.UTF8.GetBytes(json);

StreamWriter file = new StreamWriter(@"File Location");

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "POST";
request.ContentType = "text/plain";
request.KeepAlive = false;
request.AllowReadStreamBuffering = false;
request.ContentLength = jsonData.Length;

/* Pretend this middle part does the Authorization with username and password. */
/* I have actually authenticated using the above method, and passed a key to the request */

 //This part POST the JSON to the API
 Stream writeStream = request.GetRequestStream();
 writeStream.Write(jsonData, 0, jsonData.Length);

 //This conducts the reading/writing into the file
 HttpWebResponse response = (HttpWebResponse)request.GetResponse();
 using (Stream receivedStream = response.GetResponseStream())
 using (BufferedStream bs = new BufferedStream(receivedStream))
 using (StreamReader sr = new StreamReader(bs))
 {
       String s;
       while ((s = sr.ReadLine()) != null)
       {
            file.WriteLine(s);
        }

 }
</code>```

A lot of the methods here come from Dave Lozinski's blog post regarding memory handling, stream reading, and stream writing (although he uses files instead of streams).  It's very useful to go through the post above for help.

Body:
I'm currently working on a simple application that utilizes JSON objects to POST to an API and gets the responding data.  However, when I run the POST method, the POST response is so large that I encounter the OutOfMemory Exception.

I'm currently using WebClient and a CookieContainer for the process:

string jsonObject ="...."; //Example JSON string - It's very small

using (var client = new WebClient())
{
     var auth = new NameValueCollection();
       values["username"] = "username";
       values["password"] = "password";

     client.uploadValues(endpoint,auth);

     // This is causing the OutOfMemory Exception
     var response = client.uploadString(endpoint, jsonObject);

}

I have looked into the issue, and have set the property AllowStreamBuffering to be false.

 client.AllowStreamBuffering() = false;

However, I am still encountering the issue, and do not know how to control the POST response.

Update: 7/5/2017

Thanks to @Tim suggestions, I've moved to a stream of the response, however I'm encountering issues regarding the actual response.  After writing the JSON (as a string) to the end point with a POST method, the script gets stuck at trying to read the response.

    String endPoint = @"http://example.com/v1/api/";
    String json = @"....";

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
    request.Method = "POST";
    request.KeepAlive = false;
    request.AllowReadStreamBuffering = false;

    /* Pretend this middle part does the Authorization with username and password. */
    /* I have actually authenticated using the above method, and passed a key to the request */

    //This part POST the JSON to the API
    using (StreamWriter writeStream = new StreamWriter(request.GetRequestStream()))
        {
            writeStream.Write(json);
            writeStream.Flush();
            writeStream.Close();
        }

    //This bottom part opens up a console, but never reads or loads the data
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    StreamReader reader = new StreamReader(response.GetResponseStream());

I'm wondering if the JSON is not encoded probably.

(Side Note: I have looked at writing the response, line by line onto a file, but it's the response that is causing the issue -- http://cc.davelozinski.com/c-sharp/fastest-way-to-read-text-files)

Body:
I was able to solve my own problem.  For the header, I forgot to encode the JSON, and set the content-type properly for the API.

Here is exactly the same code above for the stream, but with an updated header and bufferedStream for increased efficiency with handling data and memory.

String endPoint = @"http://example.com/v1/api/";
String json = @"....";//Example JSON string

Byte[] jsonData = Encoding.UTF8.GetBytes(json);

StreamWriter file = new StreamWriter(@"File Location");

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "POST";
request.ContentType = "text/plain";
request.KeepAlive = false;
request.AllowReadStreamBuffering = false;
request.ContentLength = jsonData.Length;

/* Pretend this middle part does the Authorization with username and password. */
/* I have actually authenticated using the above method, and passed a key to the request */

 //This part POST the JSON to the API
 Stream writeStream = request.GetRequestStream();
 writeStream.Write(jsonData, 0, jsonData.Length);

 //This conducts the reading/writing into the file
 HttpWebResponse response = (HttpWebResponse)request.GetResponse();
 using (Stream receivedStream = response.GetResponseStream())
 using (BufferedStream bs = new BufferedStream(receivedStream))
 using (StreamReader sr = new StreamReader(bs))
 {
       String s;
       while ((s = sr.ReadLine()) != null)
       {
            file.WriteLine(s);
        }

 }

A lot of the methods here come from Dave Lozinski's blog post regarding memory handling, stream reading, and stream writing (although he uses files instead of streams).  It's very useful to go through the post above for help.


问:
text in index.html is bottom aligned, I want it to be top alined
I have a html file that when renders the text is bottom aligned. When the web-browser is open wide, the whole text is very far from the top as it can been seen here:

Now if I make the web-browser window narrower the text goes up like this:

Here is what I have in the html file:

```<code><div class="post">

        <h1>Title of Sample Work 2</h1>

        <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam dapibus id nisl ut suscipit. Nullam vel justo tellus. Suspendisse vehicula rhoncus nunc sed accumsan. In hac habitasse platea dictumst. Mauris vel dolor velit. Phasellus finibus massa mauris, at interdum nisl luctus at. Etiam porttitor, metus non dapibus pretium, orci arcu pretium nulla, eget congue augue libero at lectus. Mauris pretium urna tristique, laoreet enim rhoncus, euismod tortor.</p>

        <p>Integer dolor nibh, blandit et suscipit in, egestas eu diam. Maecenas imperdiet posuere odio, et tempus sapien efficitur et. Aliquam nec augue efficitur, consectetur elit at, efficitur purus. Fusce sem augue, congue ac metus et, molestie tempus velit. Quisque vitae arcu porttitor, porta augue mollis, consectetur magna. Aenean ante orci, sagittis quis vehicula sit amet, finibus in risus. Sed eu elementum urna. Curabitur quis consequat nunc. Sed sollicitudin purus ac leo volutpat, sit amet rhoncus neque malesuada. Phasellus enim ante, ornare non sapien non, mattis sollicitudin lorem. Phasellus suscipit magna felis, sed ullamcorper tellus molestie ut. Phasellus tincidunt fringilla imperdiet. Fusce rhoncus ultrices sapien, vitae dignissim nunc tincidunt in.</p>
</code>```

and here is what I have in the css:

```<code>.post {
    margin-top: 5em;
    margin-left: 2em;
    margin-right: 2em;
}

.post p{
    max-width: 1200px;
    padding: .5em;
    text-align: justify;
    margin: 0;
    top: 1.25em;
}

.post h1{
    text-align: center;
}
</code>```

I tried to add position: absolute; to .post{ like this:

```<code>.post {
    margin-top: 5em;
    margin-left: 2em;
    margin-right: 2em;
    position: absolute;
}
</code>```

but then it messes up left and right margins as it can be seen here:

As can be seen the right margin now is much smaller than left margin.

Any suggestion on how I align the text to the top without messing up left and right margin is really appreciated.

答: Try to use px instead of em in the margin attribute</p>

Body:
I have a html file that when renders the text is bottom aligned. When the web-browser is open wide, the whole text is very far from the top as it can been seen here:

Now if I make the web-browser window narrower the text goes up like this:

Here is what I have in the html file:

Title of Sample Work 2

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam dapibus id nisl ut suscipit. Nullam vel justo tellus. Suspendisse vehicula rhoncus nunc sed accumsan. In hac habitasse platea dictumst. Mauris vel dolor velit. Phasellus finibus massa mauris, at interdum nisl luctus at. Etiam porttitor, metus non dapibus pretium, orci arcu pretium nulla, eget congue augue libero at lectus. Mauris pretium urna tristique, laoreet enim rhoncus, euismod tortor.

Integer dolor nibh, blandit et suscipit in, egestas eu diam. Maecenas imperdiet posuere odio, et tempus sapien efficitur et. Aliquam nec augue efficitur, consectetur elit at, efficitur purus. Fusce sem augue, congue ac metus et, molestie tempus velit. Quisque vitae arcu porttitor, porta augue mollis, consectetur magna. Aenean ante orci, sagittis quis vehicula sit amet, finibus in risus. Sed eu elementum urna. Curabitur quis consequat nunc. Sed sollicitudin purus ac leo volutpat, sit amet rhoncus neque malesuada. Phasellus enim ante, ornare non sapien non, mattis sollicitudin lorem. Phasellus suscipit magna felis, sed ullamcorper tellus molestie ut. Phasellus tincidunt fringilla imperdiet. Fusce rhoncus ultrices sapien, vitae dignissim nunc tincidunt in.

and here is what I have in the css:

.post {
    margin-top: 5em;
    margin-left: 2em;
    margin-right: 2em;
}

.post p{
    max-width: 1200px;
    padding: .5em;
    text-align: justify;
    margin: 0;
    top: 1.25em;
}

.post h1{
    text-align: center;
}

I tried to add position: absolute; to .post{ like this:

.post {
    margin-top: 5em;
    margin-left: 2em;
    margin-right: 2em;
    position: absolute;
}

but then it messes up left and right margins as it can be seen here:

As can be seen the right margin now is much smaller than left margin.

Any suggestion on how I align the text to the top without messing up left and right margin is really appreciated.


问:
Explicitly setting colorbar bins in plt.scatter
I have created a scatter plot over a basemap of the U.S. using pyplot.scatter in which the bins of the color bar are evenly spaced over the range of the input data.  However, I would like to explicitly control the bins of the color bar in order to create uneven bins, e.g. [-20, -10, -5,-4,-3,-2,-1,0,1,2,3,4,5,50] instead of evenly spaced bins between -20 and 50. How can this be done?

答:
To set your own ticks in a colorbar in plt.scatter you can change ticks parameter in fig.colorbar to whatever list of values you desire:

```<code>import matplotlib.pyplot as plt
fig, ax = plt.subplots()
cm = plt.cm.get_cmap('RdYlBu')
xy = range(20)
z = xy
sc = plt.scatter(xy, xy, c=z, vmin=0, vmax=20, s=35, cmap=cm)

cbar = fig.colorbar(sc, ticks=[0, 1, 15])
plt.show()
</code>```

Body:
To set your own ticks in a colorbar in plt.scatter you can change ticks parameter in fig.colorbar to whatever list of values you desire:

import matplotlib.pyplot as plt
fig, ax = plt.subplots()
cm = plt.cm.get_cmap('RdYlBu')
xy = range(20)
z = xy
sc = plt.scatter(xy, xy, c=z, vmin=0, vmax=20, s=35, cmap=cm)

cbar = fig.colorbar(sc, ticks=[0, 1, 15])
plt.show()


问:
Replacing N-level for loops with recursion in C++
I have been trying for some time now to come up with a way to compute all the various combinations of strings of words for some time now. Unlike most methods for combining on the web though, the algorithm must produce every combination, including those in which all the combined elements aren't in a single combination. ie, if I am combining 'Hello', 'New' and 'World', the combinations I am looking for are:

```<code>HelloNewWorld
HelloNew
HelloWorld
Hello
NewWorld
New
World
</code>```

A professor from my college did come up with a quick and dirty solution for doing just that, but it is using nested for loops.

```<code>#include <iostream>
#include <vector>
#include <array>
#include <string>

int main()
{
std::vector<std::array<std::string, 2>> vec(3);
vec[0] = {"Hello", ""};
vec[1] = {"New", ""};
vec[2] = {"World", ""};
for (int i = 0; i < 2; i++)
    for (int j = 0; j < 2; j++)
        for (int k = 0; k < 2; k++)
            std::cout << vec[0][i] + vec[1][j] + vec[2][k] << std::endl;
}
</code>```

As you might imagine, I desired a way to make this actually somewhat usable and portable. I know that this is possible with recursion, I just don't know how to implement it. Optimally, I would like to make this tail-recursive if at all possible, as the plan is to compute very large combinations. What would be the best way to do this recursively, and would it be easy to make tail-recursive?

答:
If i understand this correctly you want to generate all combinations of a string. in that case you can use a BFS along with a set and a queue to generate the combinations, I will try to explain.

Say your string is ABCD. You have a queue to which you add ABCD and a set to which you add ABCD now

```<code>while the queue is not empty
1) you pop the top element
2) you generate substrings of that popped element
  a) if that substring is not in the set add it to the queue
</code>```

to generate substrings in step 2 you do the following

```<code>for(int i =0;i<string.length();i++)
{
  string substr1 = string.substr(0,i);
  string substr2 = string.substr(i,string.length()-1);
  string substring = substr1+substr2;
}
</code>```

doing this on ABCD(the input string) will generate BCD,ACD and ABD and ABC. now add those 3 to the set and the queue

now, you add BCD,ACD and ABD to the set. Lets say BCD is queue.front(). You pop that and generate CD,BD and BC and add them to the set and queue. when you pop ACD next, you generate CD,AD and AC but now you wont add CD to the queue because it is in the set.

EDIT:

I see your issue, my answer works for a string but you can use the same principle on a vector<string> to generate all combinations ABCD would just beHello(A)World(B)...

Body:
I have been trying for some time now to come up with a way to compute all the various combinations of strings of words for some time now. Unlike most methods for combining on the web though, the algorithm must produce every combination, including those in which all the combined elements aren't in a single combination. ie, if I am combining 'Hello', 'New' and 'World', the combinations I am looking for are:

HelloNewWorld
HelloNew
HelloWorld
Hello
NewWorld
New
World

A professor from my college did come up with a quick and dirty solution for doing just that, but it is using nested for loops.

#include <iostream>
#include <vector>
#include <array>
#include <string>

int main()
{
std::vector<std::array<std::string, 2>> vec(3);
vec[0] = {"Hello", ""};
vec[1] = {"New", ""};
vec[2] = {"World", ""};
for (int i = 0; i < 2; i++)
    for (int j = 0; j < 2; j++)
        for (int k = 0; k < 2; k++)
            std::cout << vec[0][i] + vec[1][j] + vec[2][k] << std::endl;
}

As you might imagine, I desired a way to make this actually somewhat usable and portable. I know that this is possible with recursion, I just don't know how to implement it. Optimally, I would like to make this tail-recursive if at all possible, as the plan is to compute very large combinations. What would be the best way to do this recursively, and would it be easy to make tail-recursive?

Body:
If i understand this correctly you want to generate all combinations of a string. in that case you can use a BFS along with a set and a queue to generate the combinations, I will try to explain.

Say your string is ABCD. You have a queue to which you add ABCD and a set to which you add ABCD now

while the queue is not empty
1) you pop the top element
2) you generate substrings of that popped element
  a) if that substring is not in the set add it to the queue

to generate substrings in step 2 you do the following

for(int i =0;i<string.length();i++)
{
  string substr1 = string.substr(0,i);
  string substr2 = string.substr(i,string.length()-1);
  string substring = substr1+substr2;
}

doing this on ABCD(the input string) will generate BCD,ACD and ABD and ABC. now add those 3 to the set and the queue

now, you add BCD,ACD and ABD to the set. Lets say BCD is queue.front(). You pop that and generate CD,BD and BC and add them to the set and queue. when you pop ACD next, you generate CD,AD and AC but now you wont add CD to the queue because it is in the set.

EDIT:

I see your issue, my answer works for a string but you can use the same principle on a vector<string> to generate all combinations ABCD would just beHello(A)World(B)...


答: You will have to pass an instance of root to son or daughter if you want to call methods between them. Or if it is something that does not depend on the instance, use static methods.</p>

Title: PHP Class on Class, get Parent


问:
Tables and how to manage data
I have a table:

    1
    2
    3
    4

```
</div>
</div>
</p>

<p>I want the cells to break at a certain width so that it displays:</p>

```1 2
3 4
```

I tried using td:nth-child(), but couldn't get past changing the color of the td or having them all vertical.

Thanks for any help

答:
Adjusting screen size to your need, currently set to 768px

HTML:

```<code><table>
  <tr class="row1">
    <td>1</td>
    <td>2</td>
  </tr>
  <tr class="row2">
    <td>3</td>
    <td>4</td>
  </tr>
</table>
</code>```

CSS:

```<code>@media (min-width: 768px) {
  tr {
    display: inline-block;
  }
}

@media (max-width: 767px) {
  tr {
    display: block;
  }
}
</code>```

Title: Tables and how to manage data

Body:
I have a table:

    1
    2
    3
    4

I want the cells to break at a certain width so that it displays:

1 2
3 4

I tried using td:nth-child(), but couldn't get past changing the color of the td or having them all vertical.

Thanks for any help

Body:
Adjusting screen size to your need, currently set to 768px

HTML:

<table>
  <tr class="row1">
    <td>1</td>
    <td>2</td>
  </tr>
  <tr class="row2">
    <td>3</td>
    <td>4</td>
  </tr>
</table>

CSS:

@media (min-width: 768px) {
  tr {
    display: inline-block;
  }
}

@media (max-width: 767px) {
  tr {
    display: block;
  }
}


问:
How to fetch and represent data produced by python in react web app
I am currently working on a small project to build a web application to visualize some local data produced by some Python scripts I have already finished, and the front-end data visualization is written in react. I am kinda confused how could I fetch the data to represent on my web app. I saw some suggestion of using fetch() instead of jQuery. But everything I have done yet is just locally hosted, so how could I make the request to Python scripts in React and how should I retrieve params from the fetch() request in Python?  Thank you!

import React, { Component } from 'react';
import logo from './logo.svg';
import './App.css';
import RadarChart from './components/RadarChart';
import LineChart from './components/LineChart';

class App extends Component {
  constructor() {
    super();
    this.state = {
      bioData: {},
      appData: {}
    }
  }

  componentWillMount() {
    this.getChartData();
  }

  static lastSyncTime = "2017-06-28";
  getChartData() {
    // TODO: AJAX calls/fetch() request here to retrieve data as well as pass a param: lastTimeSync to Python scripts
    this.setState({

    });
  }

  render() {
    return (

Daily Report

        <LineChart  chartData={this.state.bioData} />

        <button onClick = {this.handleRefreshLine}>Refresh</button>

        <RadarChart chartData={this.state.appData} />

    );
  }
}

export default App;```

答:
Something like this may help you:</p>

```<code>getChartData() {
  $.getJSON("https://yourserver.com/your/python/api", { lastTimeSync: App.lastSyncTime }).then(results => {
    this.setState({ appData: results.appData, bioData: results.bioData })
  })
}
</code>```

Body:
Something like this may help you:

getChartData() {
  $.getJSON("https://yourserver.com/your/python/api", { lastTimeSync: App.lastSyncTime }).then(results => {
    this.setState({ appData: results.appData, bioData: results.bioData })
  })
}