问:
Using python to parse a large set of filenames concatenated from inconsistent object names
/tldr Looking to parse a large set of filenames that are a concatenation of two names (container + child) for the original two names where nomenclature is inconsistent.  Python library suggestions or any other guidance appreciated.

I am looking for a way to parse strings for information where the nomenclature and formatting of information within those strings will likely be inconsistent to some degree.

Background
Industry: Automation controls

Problem to be solved:

Time series data is exported from an automation system with a single data point being saved to a single .csv file.  (example:  If the controls system were an environmental controls system the point might be the measured temperature of a room taken at 15 minute intervals.)  It is possible to have an environment where there are a few dozen points that export to CSV files or several thousand points that export to CSV files.  The structure that the points are normally stored in is as follows: points are contained within a controller, controllers are integrated under a management system and occasionally management systems could be integrated into another management system.  The resulting structure is a simple hierarchical tree.

The filenames associated with the CSV files are assembled from the path structure of each point as follows:  Directories are created for the management systems (nested if necessary) and under those directories are the CSV files where the filename is a concatenation of the controller name and the point name.

I have written a python script that processes a monthly export of the CSV files (currently about 5500 of them [growing]) into a structured data store and another that assembles spreadsheets for others to review.  Currently, I am using some really ugly regular expressions and even uglier string.find()s with a list of static string values that I have hand entered to parse out control names and point names for each file so that they can be inserted into the structured data store.

Unfortunately, as mentioned above, the nomenclature used in these environments are rarely consistent.  Point names vary widely.  The point referenced above might be known as ROOMTEMP, RM_T, RM-T, ROOM-T, ZN_T, ZNT, RMT or several other possibilities.  This applies to almost any point contained within a controller.  Controller names are also somewhat inconsistent where they may be named for what type of device they are controlling, the geographic location of the device or even an asset number associated with the device.

I would very much like to get out of the business of hand writing regular expressions to parse file names every time a new location is added.  I would like to write code that reads in filenames and looks for patterns across the filenames and then makes a recommendation for parsing the controller and point name out of each filename.  I already have an interface where I can assign controller name and point name to each point object by hand so if there are errors with the parse I can modify the results.  Ideally, the patterns created by the existing objects would influence the suggested names of new files being parsed.

Some examples of filenames are as follows:
UNIT1254_SAT.csv, UNIT1254_RMT.csv, UNIT1254_fil.csv, AHU_5311_CLG_O.csv, QE239-01_DISCH_STPT.csv, HX_E2_CHW_Return.csv, Plant_RM221_CHW_Sys_Enable.csv, TU_E7_Actual Clg Setpoint.csv, 1725_ROOMTEMP.csv, 1725_DA_T.csv, 1725_RA_T.csv

The order will always be consistent where it is a concatenation of controller name and then point name.  There will most likely be a consistent character used to separate controller name from point name (normally an underscore, but occasionally a dash or some other character.)

Does anyone have any recommendations on how to get started with parsing these file names?  I’ve thought through a few ideas, but keep shelving them before trying them prior to implementation because I keep finding the potential for performance issues or identifying failure points.  The rest of my code is working pretty much the way I need it to, I just haven’t figured out an efficient or useful way to pull the correct names out of the filename.  Unfortunately, It is not an option to modify the names on the control system side to be consistent.

答:
I don't know if the following code will help you, but I hope it'll give you at least some idea.

Considering that a filename as "QE239-01_STPT_1725_ROOMTEMP_DA" can contain following names

 'QE239-01'

 'QE239-01_STPT'

 'QE239-01_STPT_1725'

 'QE239-01_STPT_1725_ROOMTEMP'

 'QE239-01_STPT_1725_ROOMTEMP_DA'

 'STPT'

 'STPT_1725'

 'STPT_1725_ROOMTEMP'

 'STPT_1725_ROOMTEMP_DA'

 '1725'

 '1725_ROOMTEMP'

 '1725_ROOMTEMP_DA'

 'ROOMTEMP'

 'ROOMTEMP_DA'

 'DA'

as being possible elements (container name or point name) of the filename,
I defined the function treat() to return this list from the name.

Then the code treats all the filenames to find all the possible elements of filenames.
The function is based on the idea that in the chosen example the element ROOMTEMP can't follow the element STPT because STPT_ROOMTEMP isn't a possible container name in this example string since there is 1725 between these two elements.

And then, with the help of a function in difflib module, I try to discriminate elements that may have some similarity, in order to try to detect patterns under which several elements of names can be gathered.

You must play with the value passed as argument to cutoff parameter to choose what could be the best to give interesting results for you.

It's far from being good, certainly, but I didn't understood all aspects of your problem.

```<code>s =\
"""UNIT1254_SAT
UNIT1254_RMT
UNIT1254_fil
AHU_5311_CLG_O
QE239-01_DISCH_STPT,
HX_E2_CHW_Return
Plant_RM221_CHW_Sys_Enable
TU_E7_Actual Clg Setpoint
1725_ROOMTEMP
1725_DA_T
1725_RA_T
UNT147_ROOMTEMP
TRU_EZ_RM_T
HXX_V2_RM-T
RHXX_V2_ROOM-T
SIX8_ZN_T
Plint_RP228_ZNT
SOHO79_EZ_RMT"""

li = s.split('\n')
print(li)
print('- - - - - - - - - - - - - - - - - ')

import difflib
from pprint import pprint

def treat(name):
    lu = name.split('_')
    W = []
    while lu:
        W.extend('_'.join(lu[0:x]) for x in range(1,len(lu)+1))
        lu.pop(0)
    return W

if 0:
    q = "QE239-01_STPT_1725_ROOMTEMP_DA"
    pprint(treat(q))
    print('==========================================')

WALL = []
for t in li:
    WALL.extend(treat(t))
pprint(WALL)

for x in WALL:
    j = set(difflib.get_close_matches(x, WALL, n=9000000, cutoff=0.7 ))
    if len(j)>1:
        print(j,'\n')
</code>```

Body:
I don't know if the following code will help you, but I hope it'll give you at least some idea.

Considering that a filename as "QE239-01_STPT_1725_ROOMTEMP_DA" can contain following names

 'QE239-01'

 'QE239-01_STPT'

 'QE239-01_STPT_1725'

 'QE239-01_STPT_1725_ROOMTEMP'

 'QE239-01_STPT_1725_ROOMTEMP_DA'

 'STPT'

 'STPT_1725'

 'STPT_1725_ROOMTEMP'

 'STPT_1725_ROOMTEMP_DA'

 '1725'

 '1725_ROOMTEMP'

 '1725_ROOMTEMP_DA'

 'ROOMTEMP'

 'ROOMTEMP_DA'

 'DA'

as being possible elements (container name or point name) of the filename,
I defined the function treat() to return this list from the name.

Then the code treats all the filenames to find all the possible elements of filenames.
The function is based on the idea that in the chosen example the element ROOMTEMP can't follow the element STPT because STPT_ROOMTEMP isn't a possible container name in this example string since there is 1725 between these two elements.

And then, with the help of a function in difflib module, I try to discriminate elements that may have some similarity, in order to try to detect patterns under which several elements of names can be gathered.

You must play with the value passed as argument to cutoff parameter to choose what could be the best to give interesting results for you.

It's far from being good, certainly, but I didn't understood all aspects of your problem.

s =\
"""UNIT1254_SAT
UNIT1254_RMT
UNIT1254_fil
AHU_5311_CLG_O
QE239-01_DISCH_STPT,
HX_E2_CHW_Return
Plant_RM221_CHW_Sys_Enable
TU_E7_Actual Clg Setpoint
1725_ROOMTEMP
1725_DA_T
1725_RA_T
UNT147_ROOMTEMP
TRU_EZ_RM_T
HXX_V2_RM-T
RHXX_V2_ROOM-T
SIX8_ZN_T
Plint_RP228_ZNT
SOHO79_EZ_RMT"""

li = s.split('\n')
print(li)
print('- - - - - - - - - - - - - - - - - ')

import difflib
from pprint import pprint

def treat(name):
    lu = name.split('_')
    W = []
    while lu:
        W.extend('_'.join(lu[0:x]) for x in range(1,len(lu)+1))
        lu.pop(0)
    return W

if 0:
    q = "QE239-01_STPT_1725_ROOMTEMP_DA"
    pprint(treat(q))
    print('==========================================')

WALL = []
for t in li:
    WALL.extend(treat(t))
pprint(WALL)

for x in WALL:
    j = set(difflib.get_close_matches(x, WALL, n=9000000, cutoff=0.7 ))
    if len(j)>1:
        print(j,'\n')


来源: stackexchange

问:
tidyjson could not find function "spread_all"
I wanted to follow the example here but I got this message:

Error in function_list[k] : could not find function "spread_all"

Why? What am I doing wrong? Here's my session info
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS
locale:

```<code>R version 3.3.1 (2016-06-21)
</code>```

Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

答:
The problem is that the new version of tidyjson has unfortunately not yet been released to CRAN.  That is something that I hope will happen soon!

In any case, you can always use the most recent development version (the version that includes spread_all and the version that supports the example that you found) with:

```<code>devtools::install_github('jeremystan/tidyjson')
library(tidyjson)

'{"a":1,"b":2,"c":3}' %>% spread_all()
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I wanted to follow the example here but I got this message:

Error in function_list[k] : could not find function "spread_all"

Why? What am I doing wrong? Here's my session info
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS
locale:

R version 3.3.1 (2016-06-21)

Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

Body:
The problem is that the new version of tidyjson has unfortunately not yet been released to CRAN.  That is something that I hope will happen soon!

In any case, you can always use the most recent development version (the version that includes spread_all and the version that supports the example that you found) with:

devtools::install_github('jeremystan/tidyjson')
library(tidyjson)

'{"a":1,"b":2,"c":3}' %>% spread_all()


问:
C++ unordered_map< pair<uint, uint>, UserClass > insertion
I am attempting to use the unordered_map to with keys as a pair of uint and values of the class Tile.

```<code>#include <iostream>
#include <unordered_map>
#include <boost/functional/hash.hpp>

class Tile {
public:
  std::pair<uint, uint> coordinate;

  bool operator==(const Tile &t) {
    return this->coordinate.first == t.coordinate.first
      && this->coordinate.second == t.coordinate.second;
  }

  Tile (uint x, uint y) {
    this->coordinate.first = x;
    this->coordinate.second = y;
  }

  std::pair<uint, uint> GetCoor() const
  {return this->coordinate;}

  uint GetX() const
  {return coordinate.first;}

  uint GetY() const
  {return coordinate.second;}
};

struct TileHash {
  std::size_t operator()(const Tile &t) const
  {
    size_t seed = 0;
    boost::hash_combine (seed, t.GetX());
    boost::hash_combine (seed, t.GetY());
    return seed;
  }
};

int main ()
{
  std::unordered_map<std::pair<uint, uint>, Tile, TileHash> board;
  Tile t1 = Tile (0, 0);

  board.insert (std::make_pair (t1.GetCoor(), t1));
  //board[t1.GetCoor()] = t1; // This causes a different error at compile time...
}
</code>```

I get following error followed by some notes that I did not paste here

```<code>/usr/include/c++/v1/unordered_map:400:17: error: no matching function for call to object of type 'const TileHash'
        {return static_cast<const _Hash&>(*this)(__x.__cc.first);}
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/v1/__hash_table:1712:21: note: in instantiation of member function 'std::__1::__unordered_map_hasher<std::__1::pair<unsigned int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>,
      Tile>, TileHash, true>::operator()' requested here
    size_t __hash = hash_function()(__x);
                    ^
/usr/include/c++/v1/__hash_table:1692:12: note: in instantiation of member function 'std::__1::__hash_table<std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>,
      std::__1::__unordered_map_hasher<std::__1::pair<unsigned int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>, TileHash, true>, std::__1::__unordered_map_equal<std::__1::pair<unsigned
      int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>, std::__1::equal_to<std::__1::pair<unsigned int, unsigned int> >, true>,
      std::__1::allocator<std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile> > >::__insert_unique_value' requested here
    return __insert_unique_value(__x);
</code>```

I know that my instantiation of the unordered_map is ok, when I compile without a call to insert all is well.

答: Your hash function hashes the type Tile. Instead, you should specify a hash function over the key of type std::pair< uint,uint >, so that the map keys could be hashed.</p>

Body:
I am attempting to use the unordered_map to with keys as a pair of uint and values of the class Tile.

#include <iostream>
#include <unordered_map>
#include <boost/functional/hash.hpp>

class Tile {
public:
  std::pair<uint, uint> coordinate;

  bool operator==(const Tile &t) {
    return this->coordinate.first == t.coordinate.first
      && this->coordinate.second == t.coordinate.second;
  }

  Tile (uint x, uint y) {
    this->coordinate.first = x;
    this->coordinate.second = y;
  }

  std::pair<uint, uint> GetCoor() const
  {return this->coordinate;}

  uint GetX() const
  {return coordinate.first;}

  uint GetY() const
  {return coordinate.second;}
};

struct TileHash {
  std::size_t operator()(const Tile &t) const
  {
    size_t seed = 0;
    boost::hash_combine (seed, t.GetX());
    boost::hash_combine (seed, t.GetY());
    return seed;
  }
};

int main ()
{
  std::unordered_map<std::pair<uint, uint>, Tile, TileHash> board;
  Tile t1 = Tile (0, 0);

  board.insert (std::make_pair (t1.GetCoor(), t1));
  //board[t1.GetCoor()] = t1; // This causes a different error at compile time...
}

I get following error followed by some notes that I did not paste here

/usr/include/c++/v1/unordered_map:400:17: error: no matching function for call to object of type 'const TileHash'
        {return static_cast<const _Hash&>(*this)(__x.__cc.first);}
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/v1/__hash_table:1712:21: note: in instantiation of member function 'std::__1::__unordered_map_hasher<std::__1::pair<unsigned int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>,
      Tile>, TileHash, true>::operator()' requested here
    size_t __hash = hash_function()(__x);
                    ^
/usr/include/c++/v1/__hash_table:1692:12: note: in instantiation of member function 'std::__1::__hash_table<std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>,
      std::__1::__unordered_map_hasher<std::__1::pair<unsigned int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>, TileHash, true>, std::__1::__unordered_map_equal<std::__1::pair<unsigned
      int, unsigned int>, std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile>, std::__1::equal_to<std::__1::pair<unsigned int, unsigned int> >, true>,
      std::__1::allocator<std::__1::__hash_value_type<std::__1::pair<unsigned int, unsigned int>, Tile> > >::__insert_unique_value' requested here
    return __insert_unique_value(__x);

I know that my instantiation of the unordered_map is ok, when I compile without a call to insert all is well.


问:
Create Multiple Plots for time series using dplyr in R
I have multiple sites that I sample multiple times (different dates) with which I run time-series experiments to obtain a value for each time point in that series (for each site there are 4 time points). I would like to graph each time series (Time..hours.) (x-axis) versus their values (Sample.N2O.gas...ppm.)(y-axis) for each site for its respective date and have multiple plots to view all this data at once.

I'm new to R, so I'm not sure if attaching the image of my data is helpful or not. Data

What I've found so far is:

```<code>  plots <- data %>% group_by(Site, Date, Treatment, Rep) %>% do(plots=ggplot(data=.)) +
  aes(x=Time..hours., y=Sample.N2O.gas...ppm.) + geom_point()
</code>```

but when I run plots, I just get an output table '#A Tibble 149x5".

Here's what I've copied from my dput output:

```<code>   data = structure(list(Site = structure(c(4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("QA1", "QA1 ",
"QA2", "W1", "W2", "W3", "W4"), class = "factor"), Date = structure(c(1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000), class = c("POSIXct", "POSIXt"), tzone = ""), Treatment = structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c("Control",
"DEA", "Denit"), class = "factor"), Rep = c(1L, 1L, 1L, 1L, 2L,
2L, 2L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L), Time..hours. = c(0.333333333,
1.166666667, 2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333,
0.333333333, 1.166666667, 2, 2.833333333, 0.333333333, 1.166666667,
2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333, 0.333333333,
1.166666667, 2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333,
0.333333333, 1.166666667, 2, 2.833333333), Sample.N2O.gas...ppm. = c(0.02536993437111,
0.02539270198174, 0.0258252865837, 0.02236460976796, 0.01136785383374,
0.00886341666446, 0.01291605135657, 0.01583030551719, 0.05517273668559,
0.15960776664476, 0.12206397671612, 0.10999714308229, 0.08618222236346,
0.062344534034, 0.0527366023482, 0.05845127261629, 0.03570642959706,
0.05093796110844, 0.04784156606278, 0.05763163863362, 24.1815481999959,
43.9535598867979, 76.0087551511654, 97.5847143861169, 26.1044322512172,
42.2811119155545, 82.8088875655402, 109.127381921808, 24.9257144556248,
46.2630281688243, 84.5042194191463, 102.597585589484)), .Names = c("Site",
"Date", "Treatment", "Rep", "Time..hours.", "Sample.N2O.gas...ppm."
), row.names = c(NA, 32L), class = "data.frame")
</code>```

答:
That's because you've created a list of plots (technically a tibble of plots).  To view the first plot in the list, say, you'll need to do plots[[1]]

Addendum:  Actually, your code in its current form doesn't quite work.  Did you mean to do the following?

```<code>plots <- data %>% group_by(Site, Date, Treatment, Rep) %>% do(plots=ggplot(data=.) +
    aes(x=Time..hours., y=Sample.N2O.gas...ppm.) + geom_point())
</code>```

If so, then (strangely), the plots are buried in a list called plots.  You can access the first one with plots$plots[[1]], and show all of them by just going plots$plots.

ContentLicense: CC BY-SA 4.0

Body:
I have multiple sites that I sample multiple times (different dates) with which I run time-series experiments to obtain a value for each time point in that series (for each site there are 4 time points). I would like to graph each time series (Time..hours.) (x-axis) versus their values (Sample.N2O.gas...ppm.)(y-axis) for each site for its respective date and have multiple plots to view all this data at once.

I'm new to R, so I'm not sure if attaching the image of my data is helpful or not. Data

What I've found so far is:

  plots <- data %>% group_by(Site, Date, Treatment, Rep) %>% do(plots=ggplot(data=.)) +
  aes(x=Time..hours., y=Sample.N2O.gas...ppm.) + geom_point()

but when I run plots, I just get an output table '#A Tibble 149x5".

Here's what I've copied from my dput output:

   data = structure(list(Site = structure(c(4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("QA1", "QA1 ",
"QA2", "W1", "W2", "W3", "W4"), class = "factor"), Date = structure(c(1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000, 1462860000, 1462860000, 1462860000, 1462860000, 1462860000,
1462860000), class = c("POSIXct", "POSIXt"), tzone = ""), Treatment = structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c("Control",
"DEA", "Denit"), class = "factor"), Rep = c(1L, 1L, 1L, 1L, 2L,
2L, 2L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L), Time..hours. = c(0.333333333,
1.166666667, 2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333,
0.333333333, 1.166666667, 2, 2.833333333, 0.333333333, 1.166666667,
2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333, 0.333333333,
1.166666667, 2, 2.833333333, 0.333333333, 1.166666667, 2, 2.833333333,
0.333333333, 1.166666667, 2, 2.833333333), Sample.N2O.gas...ppm. = c(0.02536993437111,
0.02539270198174, 0.0258252865837, 0.02236460976796, 0.01136785383374,
0.00886341666446, 0.01291605135657, 0.01583030551719, 0.05517273668559,
0.15960776664476, 0.12206397671612, 0.10999714308229, 0.08618222236346,
0.062344534034, 0.0527366023482, 0.05845127261629, 0.03570642959706,
0.05093796110844, 0.04784156606278, 0.05763163863362, 24.1815481999959,
43.9535598867979, 76.0087551511654, 97.5847143861169, 26.1044322512172,
42.2811119155545, 82.8088875655402, 109.127381921808, 24.9257144556248,
46.2630281688243, 84.5042194191463, 102.597585589484)), .Names = c("Site",
"Date", "Treatment", "Rep", "Time..hours.", "Sample.N2O.gas...ppm."
), row.names = c(NA, 32L), class = "data.frame")

Body:
That's because you've created a list of plots (technically a tibble of plots).  To view the first plot in the list, say, you'll need to do plots[[1]]

Addendum:  Actually, your code in its current form doesn't quite work.  Did you mean to do the following?

plots <- data %>% group_by(Site, Date, Treatment, Rep) %>% do(plots=ggplot(data=.) +
    aes(x=Time..hours., y=Sample.N2O.gas...ppm.) + geom_point())

If so, then (strangely), the plots are buried in a list called plots.  You can access the first one with plots$plots[[1]], and show all of them by just going plots$plots.


问:
Why does loading dataset from Cassandra fail with NullPointerException?
I am trying to read/write to Cassandra from Spark and using these dependencies:

```<code>"com.datastax.spark" % "spark-cassandra-connector-unshaded_2.11" % "2.0.0-M3",
"com.datastax.cassandra" % "cassandra-driver-core" % "3.0.0"
</code>```

And this is the code :

```<code>import com.datastax.spark.connector._
val sparkConf: SparkConf = new SparkConf().setAppName(appName)
      .set("spark.cassandra.connection.host", hostname)
      .set("spark.cassandra.auth.username",user)
      .set("spark.cassandra.auth.password",password)

 val spark = SparkSession.builder().config(sparkConf).getOrCreate()
val df = spark
      .read
      .format("org.apache.spark.sql.cassandra")
      .options(Map( "table" -> s"$TABLE", "keyspace" -> s"$KEYSPACE"))
      .load() // This Dataset will use a spark.cassandra.input.size of 128
</code>```

However on trying to spark-submit, I am getting this on the df ...load() line above

```<code>Exception in thread "main" java.lang.NullPointerException
    at com.datastax.driver.core.Cluster$Manager.close(Cluster.java:1516)
    at com.datastax.driver.core.Cluster$Manager.access$200(Cluster.java:1237)
    at com.datastax.driver.core.Cluster.closeAsync(Cluster.java:540)
    at com.datastax.driver.core.Cluster.close(Cluster.java:551)
    at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:162)
    at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
    at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
    at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
    at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56)
    at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:82)
    at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:110)
    at com.datastax.spark.connector.rdd.partitioner.dht.TokenFactory$.forSystemLocalPartitioner(TokenFactory.scala:98)
    at org.apache.spark.sql.cassandra.CassandraSourceRelation$.apply(CassandraSourceRelation.scala:255)
    at org.apache.spark.sql.cassandra.DefaultSource.createRelation(DefaultSource.scala:55)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:345)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:122)
</code>```

答:
M3 is a milestone release and you should really use the real release 2.0.2 which is currently the latest.

https://github.com/datastax/spark-cassandra-connector#most-recent-release-scala-docs

You should not be including the java driver in the same project with the Cassandra Connector. Unless you are explicitly re-shading inside your project and that is for experts only. See the FAQ for more details.

I would recommend using only the shaded artifact and following the examples as posted here

```<code>  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-hive" % sparkVersion % "provided",
  "com.datastax.spark" %% "spark-cassandra-connector" % connectorVersion % "provided"
</code>```

And launching uses Spark Packages or an assembly

//Assembly

https://github.com/datastax/SparkBuildExamples#sbt

// Packages

https://spark-packages.org/package/datastax/spark-cassandra-connector

Body:
I am trying to read/write to Cassandra from Spark and using these dependencies:

"com.datastax.spark" % "spark-cassandra-connector-unshaded_2.11" % "2.0.0-M3",
"com.datastax.cassandra" % "cassandra-driver-core" % "3.0.0"

And this is the code :

import com.datastax.spark.connector._
val sparkConf: SparkConf = new SparkConf().setAppName(appName)
      .set("spark.cassandra.connection.host", hostname)
      .set("spark.cassandra.auth.username",user)
      .set("spark.cassandra.auth.password",password)

 val spark = SparkSession.builder().config(sparkConf).getOrCreate()
val df = spark
      .read
      .format("org.apache.spark.sql.cassandra")
      .options(Map( "table" -> s"$TABLE", "keyspace" -> s"$KEYSPACE"))
      .load() // This Dataset will use a spark.cassandra.input.size of 128

However on trying to spark-submit, I am getting this on the df ...load() line above

Exception in thread "main" java.lang.NullPointerException
    at com.datastax.driver.core.Cluster$Manager.close(Cluster.java:1516)
    at com.datastax.driver.core.Cluster$Manager.access$200(Cluster.java:1237)
    at com.datastax.driver.core.Cluster.closeAsync(Cluster.java:540)
    at com.datastax.driver.core.Cluster.close(Cluster.java:551)
    at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:162)
    at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
    at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
    at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
    at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56)
    at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:82)
    at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:110)
    at com.datastax.spark.connector.rdd.partitioner.dht.TokenFactory$.forSystemLocalPartitioner(TokenFactory.scala:98)
    at org.apache.spark.sql.cassandra.CassandraSourceRelation$.apply(CassandraSourceRelation.scala:255)
    at org.apache.spark.sql.cassandra.DefaultSource.createRelation(DefaultSource.scala:55)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:345)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:122)

Body:
M3 is a milestone release and you should really use the real release 2.0.2 which is currently the latest.

https://github.com/datastax/spark-cassandra-connector#most-recent-release-scala-docs

You should not be including the java driver in the same project with the Cassandra Connector. Unless you are explicitly re-shading inside your project and that is for experts only. See the FAQ for more details.

I would recommend using only the shaded artifact and following the examples as posted here

  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-hive" % sparkVersion % "provided",
  "com.datastax.spark" %% "spark-cassandra-connector" % connectorVersion % "provided"

And launching uses Spark Packages or an assembly

//Assembly

https://github.com/datastax/SparkBuildExamples#sbt

// Packages

https://spark-packages.org/package/datastax/spark-cassandra-connector


问:
attribute d: Expected number, "MNaN,NaNLNaN,NaNL…". react with d3
I am trying to create a basic d3 bar chart with dates in the data using react,typescript. I am completely new to d3 and typescript.

Here is my data

```<code>[ { x: 180, y: 2017-06-08},
{ x: 250, y: 2017-06-09},
{ x: 150, y: 2017-06-10},
{ x: 496, y: 2017-06-11},
{ x: 140, y: 2017-06-12},
{ x: 380, y: 2017-06-13},
{ x: 100, y: 2017-06-14},
{ x: 150, y: 2017-06-15}]
</code>```

I am returning this data from api

and using this code to generate the chart

```<code>     public render(): JSX.Element {

            let ldata = this.getState().loadedData;

            if (ldata != null) {

                let margin = { top: 5, right: 50, bottom: 20, left: 50 },
                    w = 800 - (margin.left + margin.right),
                    h = 300 - (margin.top + margin.bottom);

                let transform = 'translate(' + margin.left + ',' + margin.top + ')';

                let x = d3.scaleTime()
                    .domain(d3.extent(ldata, function (d: any) { return d.x })).rangeRound([0, w]);

                let y = d3.scaleTime()
                    .domain([0, d3.max(ldata, function (d: any) { return d.y + 100; })]).range([h, 0]);

                let line: any = d3.line().x(function (d: any) { return x(d.x); }).y(function (d: any) { return x(d.y); });

                let d = line(ldata);
                return (
                    <div className="container body-content">
                        <div className="ms-Grid">
                            <div className="ms-Grid-row">

                                <div className="ms-Grid-col ms-u-lg6">
                                    <h3 className="ms-font-xl">CIA</h3>
                                    <table className="ms-font-m table table-bordered table-striped" style={{ marginBottom: "0px" }}>
                                        <thead className="thead-vsts">
                                            <tr>
                                                <th>Column1</th>
                                                <th>Column2</th>
                                                <th>Column3</th>
                                                <th>Column4</th>
                                                <th>Graph</th>
                                            </tr>
                                        </thead>
                                        <tbody><tr>
                                            <td>Value 1</td>
                                            <td>Value 2</td>
                                            <td>Value 3</td>
                                            <td>Value 4</td>
                                            <td>   <svg width="800" height="300">
                                                <g transform={transform}>
                                                    <path className="line shadow" d={d} style={{ stroke: "red" }} />
                                                </g>
                                            </svg></td>
                                        </tr>
                                        </tbody>
                                    </table>
                                </div>
                            </div>
                        </div>
                    </div>);
            }
        }

    }
</code>```

I am getting the error attribute d: Expected number, "MNaN,NaNLNaN,NaNL…"

if i use a number instead of date it is working properly.

Can any one help me to fix this?

答:
if i use a number instead of date it is working properly.

From your data { x: 180, y: 2017-06-08}, The value of y i.e. 2017-06-08 is not a valid literal. Perhaps you meant string "2017-06-08".

MNaN,NaNLNaN,NaNL…

NaNs appear when you do invalid arithmetic e.g.

```<code>console.log("2017-06-08" - 4); // NaN
</code>```

Fix

Parse the date into a number when doing Number arithmetic.

Body:
I am trying to create a basic d3 bar chart with dates in the data using react,typescript. I am completely new to d3 and typescript.

Here is my data

[ { x: 180, y: 2017-06-08},
{ x: 250, y: 2017-06-09},
{ x: 150, y: 2017-06-10},
{ x: 496, y: 2017-06-11},
{ x: 140, y: 2017-06-12},
{ x: 380, y: 2017-06-13},
{ x: 100, y: 2017-06-14},
{ x: 150, y: 2017-06-15}]

I am returning this data from api

and using this code to generate the chart

     public render(): JSX.Element {

            let ldata = this.getState().loadedData;

            if (ldata != null) {

                let margin = { top: 5, right: 50, bottom: 20, left: 50 },
                    w = 800 - (margin.left + margin.right),
                    h = 300 - (margin.top + margin.bottom);

                let transform = 'translate(' + margin.left + ',' + margin.top + ')';

                let x = d3.scaleTime()
                    .domain(d3.extent(ldata, function (d: any) { return d.x })).rangeRound([0, w]);

                let y = d3.scaleTime()
                    .domain([0, d3.max(ldata, function (d: any) { return d.y + 100; })]).range([h, 0]);

                let line: any = d3.line().x(function (d: any) { return x(d.x); }).y(function (d: any) { return x(d.y); });

                let d = line(ldata);
                return (

CIA

                                                Column1
                                                Column2
                                                Column3
                                                Column4
                                                Graph

                                            Value 1
                                            Value 2
                                            Value 3
                                            Value 4
                                               <svg width="800" height="300">
                                                <g transform={transform}>
                                                    <path className="line shadow" d={d} style={{ stroke: "red" }} />
                                                </g>
                                            </svg>

);
            }
        }

    }

I am getting the error attribute d: Expected number, "MNaN,NaNLNaN,NaNL…"

if i use a number instead of date it is working properly.

Can any one help me to fix this?

Body:
if i use a number instead of date it is working properly.

From your data { x: 180, y: 2017-06-08}, The value of y i.e. 2017-06-08 is not a valid literal. Perhaps you meant string "2017-06-08".

MNaN,NaNLNaN,NaNL…

NaNs appear when you do invalid arithmetic e.g.

console.log("2017-06-08" - 4); // NaN

Fix

Parse the date into a number when doing Number arithmetic.


问:
How to get text after or before certain tags using Nokogiri
I have an HTML document, something like this:

```<code><root><template>title</template>
<h level="3" i="3">Something</h>
<template element="1"><title>test</title></template>
# one
# two
# three
# four
<h level="4" i="5">something1</h>
some random test
<template element="1"><title>test</title></template>
# first
# second
# third
# fourth
<template element="2"><title>testing</title></template>
</code>```

I want to extract:

```<code># one
# two
# three
# four
# first
# second
# third
# fourth
</root>
</code>```

In other words, I want "all text after <template element="1"><title>test</title></template> and before the next tag that starts after that."

I can get all text between root using '//root/text()' but how do I get all text before and after certain tags?

答:
I'm pretty sure krusty.ar is right that there's not a built-in method for achieving this.  You can just remove all the tags inside the root tag one by one if you'd like.  It's a hack, but it works:

```<code>doc = Nokogiri::HTML(open(url)) # or Nokogiri::HTML.parse(File.open(file_path))
doc.xpath('//template').remove
doc.xpath('//h').remove
doc
</code>```

That gives the result you're looking for with the HTML you posted.

Body:
I have an HTML document, something like this:

<root><template>title</template>
<h level="3" i="3">Something</h>
<template element="1"><title>test</title></template>
# one
# two
# three
# four
<h level="4" i="5">something1</h>
some random test
<template element="1"><title>test</title></template>
# first
# second
# third
# fourth
<template element="2"><title>testing</title></template>

I want to extract:

# one
# two
# three
# four
# first
# second
# third
# fourth
</root>

In other words, I want "all text after <template element="1"><title>test</title></template> and before the next tag that starts after that."

I can get all text between root using '//root/text()' but how do I get all text before and after certain tags?

Body:
I'm pretty sure krusty.ar is right that there's not a built-in method for achieving this.  You can just remove all the tags inside the root tag one by one if you'd like.  It's a hack, but it works:

doc = Nokogiri::HTML(open(url)) # or Nokogiri::HTML.parse(File.open(file_path))
doc.xpath('//template').remove
doc.xpath('//h').remove
doc

That gives the result you're looking for with the HTML you posted.

ContentLicense: CC BY-SA 2.5


问:
SSLPeerUnverifiedException: peer not authenticated with self signed certificates in netbeans 8.2 on localhost
I have been googling this error for hours and nothing worked so far and tbh I have no idea why...I generate my certificates like this:

client:

keytool -genkey -alias babicaprivate -keystore babica.private -storetype JKS -keyalg rsa -dname "CN=Babica" -storepass babicapwd -keypass babicapwd -validity 365
keytool -export -alias babicaprivate -keystore babica.private -file temp.key -storepass babicapwd
keytool -import -noprompt -alias babicapublic -keystore client.public -file temp.key -storepass public
del temp.key

server:

keytool -genkey -alias serverprivate -keystore server.private -storetype JKS -keyalg rsa -dname "CN=localhost" -storepass serverpwd -keypass serverpwd -validity 365
keytool -export -alias serverprivate -keystore server.private -file temp.key -storepass serverpwd
keytool -import -noprompt -alias serverpublic -keystore server.public -file temp.key -storepass public
del temp.key

this gives me public and private certificates for server and client

then I start the server and client and when I try to sign on the server and try to read the "Common name" from the client on server with

Socket newClientSocket = ss.accept();
 ((SSLSocket)newClientSocket).startHandshake();
 String username = ((SSLSocket) newClientSocket).getSession().getPeerPrincipal().getName();

and it's the username fetching that gives me the error and I have no idea why. Is there something wrong with how I make the certificates?

Here is my full code for server:

```<code>import java.io.*;
import java.net.*;
import java.util.*;
import javax.net.ssl.*;
import java.security.*;

public class ChatServer {

protected List<Socket> clients = new ArrayList<Socket>(); // list of clients

private String[][] portUserPair = new String[10][2];
int i = 0;

private String passphrase = "serverpwd";
private int port = 1234;

public static void main(String[] args) throws Exception {
    new ChatServer();
}

public ChatServer() throws Exception {

    KeyStore clientKeyStore = KeyStore.getInstance("JKS");
    clientKeyStore.load(new FileInputStream("client.public"), "public".toCharArray());

    KeyStore serverKeyStore = KeyStore.getInstance("JKS");
    serverKeyStore.load(new FileInputStream("server.private"), passphrase.toCharArray());

    TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
    tmf.init(clientKeyStore);
    KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
    kmf.init(serverKeyStore, passphrase.toCharArray());
    SSLContext sslContext = SSLContext.getInstance("TLS");
    SSLContext.setDefault(sslContext);
    sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), (new SecureRandom()));

    SSLServerSocketFactory factory = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
    SSLServerSocket ss = (SSLServerSocket) factory.createServerSocket(port);
    ss.setNeedClientAuth(false);
    ss.setEnabledCipherSuites(new String[] {"TLS_RSA_WITH_AES_128_CBC_SHA"});

    // start listening for new connections
    System.out.println("[system] listening ...");
    try {
        while (true) {
            Socket newClientSocket = ss.accept(); // wait for a new client connection
            ((SSLSocket)newClientSocket).startHandshake();
            String username = ((SSLSocket) newClientSocket).getSession().getPeerPrincipal().getName();
            System.out.println("Established SSL connection with: " + username);
            synchronized(this) {
                portUserPair[i][0] = Integer.toString(newClientSocket.getPort());
                clients.add(newClientSocket); // add client to the list of clients
            }
            ChatServerConnector conn = new ChatServerConnector(this, newClientSocket); // create a new thread for communication with the new client
            conn.start(); // run the new thread
        }
    } catch (Exception e) {
        System.err.println("[error] Accept failed.");
        e.printStackTrace(System.err);
        System.exit(1);
    }

    // close socket
    System.out.println("[system] closing server socket ...");
    try {
        ss.close();
    } catch (IOException e) {
        e.printStackTrace(System.err);
        System.exit(1);
    }
}

// send a message to all clients connected to the server
public void sendToAllClients(String message) throws Exception {
    Iterator<Socket> i = clients.iterator();
    while (i.hasNext()) { // iterate through the client list
        Socket socket = (Socket) i.next(); // get the socket for communicating with this client
        try {
            DataOutputStream out = new DataOutputStream(socket.getOutputStream()); // create output stream for sending messages to the client
            out.writeUTF(message); // send message to the client
        } catch (Exception e) {
            System.err.println("[system] could not send message to a client");
            e.printStackTrace(System.err);
        }
    }
}

public void sendToAClient(String message, String userInfo) throws Exception {
    Socket userSocket = null;
    String userPort = "";
    Iterator<Socket> i = clients.iterator();
    for (int k = 0; k < portUserPair.length; k++) {
        if(portUserPair[k][1].equals(userInfo)) {
            userPort = portUserPair[k][0];
            break;
        }
    }

    System.out.println(userPort);

    while (i.hasNext()) { // iterate through the client list
        Socket socket = (Socket) i.next(); // get the socket for communicating with this client
        if(Integer.toString(socket.getPort()).equals(userPort)) {
            userSocket = socket;
            break;
        }
    }

    message = message.substring(userInfo.length()+2);

    try {
        DataOutputStream out = new DataOutputStream(userSocket.getOutputStream()); // create output stream for sending messages to the client
        out.writeUTF(message); // send message to the client
    } catch (Exception e) {
        System.err.println("[system] could not send message to the client");
        e.printStackTrace(System.err);
    }
}

public void removeClient(Socket socket) {
    synchronized(this) {
        clients.remove(socket);
    }
}

public void saveUserInfo(String userInfo) {
    portUserPair[i][1] = userInfo;
    System.out.println(portUserPair[i][0] + " : " + portUserPair[i][1]);
    i++;
}
}

class ChatServerConnector extends Thread {
private ChatServer server;
private Socket socket;

public ChatServerConnector(ChatServer server, Socket socket) {
    this.server = server;
    this.socket = socket;
}

public void run() {
    System.out.println("[system] connected with " + this.socket.getInetAddress().getHostName() + ":" + this.socket.getPort());

    DataInputStream in;
    try {
        in = new DataInputStream(this.socket.getInputStream()); // create input stream for listening for incoming messages
    } catch (IOException e) {
        System.err.println("[system] could not open input stream!");
        e.printStackTrace(System.err);
        this.server.removeClient(socket);
        return;
    }

    while (true) { // infinite loop in which this thread waits for incoming messages and processes them
        String msg_received;
        try {
            msg_received = in.readUTF(); // read the message from the client
        } catch (Exception e) {
            System.err.println("[system] there was a problem while reading message client on port " + this.socket.getPort());
            e.printStackTrace(System.err);
            this.server.removeClient(this.socket);
            return;
        }

        if (msg_received.length() == 0) // invalid message
            continue;

        String[] msg = msg_received.split(" ");
        System.out.println(msg_received);

        if (msg[1].equals("system")) {
//            try {
//                server.saveUserInfo(((SSLSocket) socket).getSession().getPeerPrincipal().getName());
//            } catch (SSLPeerUnverifiedException ex) {
//                Logger.getLogger(ChatServerConnector.class.getName()).log(Level.SEVERE, null, ex);
//            }
            continue;
        }

        System.out.println(msg_received); // print the incoming message in the console

        String msg_send = msg_received.toUpperCase();

        if (msg[3].charAt(0) == '/') {
            String[] privateMsg = msg[3].split(" ");
            try {
                this.server.sendToAClient(msg_send, privateMsg[0].replace("/", "")); // send message to the client
            } catch (Exception e) {
                System.err.println("[system] there was a problem while sending the message to the client");
                e.printStackTrace(System.err);
                continue;
            }
            continue;
        }

        try {
            this.server.sendToAllClients(msg_send); // send message to all clients
        } catch (Exception e) {
            System.err.println("[system] there was a problem while sending the message to all clients");
            e.printStackTrace(System.err);
            continue;
        }
    }
}
}
</code>```

and here is my cleint:

```<code>import java.io.*;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Scanner;
import javax.net.ssl.*;
import java.security.*;

public class ChatClient extends Thread {

    private String userInfo = "";

    private String passphrase = "pwd";
    private int port = 1234;

    public static void main(String[] args) throws Exception {
        new ChatClient();
    }

    public ChatClient() throws Exception {

        KeyStore serverKeyStore = KeyStore.getInstance("JKS");
        serverKeyStore.load(new FileInputStream("server.public"), "public".toCharArray());

        KeyStore clientKeyStore = KeyStore.getInstance("JKS");
        Scanner s = new Scanner(System.in);
        String username = s.nextLine();
        clientKeyStore.load(new FileInputStream(username + ".private"), (username + passphrase).toCharArray());

        TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
        tmf.init(serverKeyStore);
        KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
        kmf.init(clientKeyStore, (username + passphrase).toCharArray());
        SSLContext sslContext = SSLContext.getInstance("TLS");
        SSLContext.setDefault(sslContext);
        sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());

        SSLSocketFactory sf = sslContext.getSocketFactory();
        SSLSocket socket = (SSLSocket) sf.createSocket("localhost", port);
        socket.setEnabledCipherSuites(new String[] {"TLS_RSA_WITH_AES_128_CBC_SHA"});
        socket.startHandshake();

        DataInputStream in = null;
        DataOutputStream out = null;

        // connect to the chat server
        try {
            System.out.println("[system] connecting to chat server ...");
            System.out.println("[system] you are signed in as: " + username);

            in = new DataInputStream(socket.getInputStream()); // create input stream for listening for incoming messages
            out = new DataOutputStream(socket.getOutputStream()); // create output stream for sending messages

            this.sendMessage(username, out, "system");

            System.out.println("[system] connected");
            System.out.println("[system] to send a private message type \"/<NameOfUser>\" and then type the message");

            ChatClientMessageReceiver message_receiver = new ChatClientMessageReceiver(in); // create a separate thread for listening to messages from the chat server
            message_receiver.start(); // run the new thread
        } catch (Exception e) {
            e.printStackTrace(System.err);
            System.exit(1);
        }

        // read from STDIN and send messages to the chat server
        BufferedReader std_in = new BufferedReader(new InputStreamReader(System.in));
        String userInput;
        while ((userInput = std_in.readLine()) != null) { // read a line from the console
            this.sendMessage(userInput, out, userInfo); // send the message to the chat server
        }

        // cleanup
        out.close();
        in.close();
        std_in.close();
        socket.close();
    }

    private void sendMessage(String message, DataOutputStream out, String userInfo) {
        message = "[" + new SimpleDateFormat("HH:mm:ss").format(new Date()) + "] " + userInfo + " : " + message;
        try {
            out.writeUTF(message); // send the message to the chat server
            out.flush(); // ensure the message has been sent
        } catch (IOException e) {
            System.err.println("[system] could not send message");
            e.printStackTrace(System.err);
        }
    }
}

// wait for messages from the chat server and print the out
class ChatClientMessageReceiver extends Thread {
    private DataInputStream in;

    public ChatClientMessageReceiver(DataInputStream in) {
        this.in = in;
    }

    public void run() {
        try {
            String message;
            while ((message = this.in.readUTF()) != null) { // read new message
                System.out.println(message); // print the message to the console
            }
        } catch (Exception e) {
            System.err.println("[system] could not read message");
            e.printStackTrace(System.err);
        }
    }
}
</code>```

the whole thing is supposed to be a "chat" that uses SSL to encrypt messages and should get the username from the certificate...so I have everything figured out except the "getting the usename out of the certificate's 'Common name' field"

Help, I'm losing my mind over this! :'(

答:
```<code>ss.setNeedClientAuth(false);
</code>```

The problem is here. You never asked for the client certificate, so it never got sent. Change this to true.

Body:
I have been googling this error for hours and nothing worked so far and tbh I have no idea why...I generate my certificates like this:

client:

keytool -genkey -alias babicaprivate -keystore babica.private -storetype JKS -keyalg rsa -dname "CN=Babica" -storepass babicapwd -keypass babicapwd -validity 365
keytool -export -alias babicaprivate -keystore babica.private -file temp.key -storepass babicapwd
keytool -import -noprompt -alias babicapublic -keystore client.public -file temp.key -storepass public
del temp.key

server:

keytool -genkey -alias serverprivate -keystore server.private -storetype JKS -keyalg rsa -dname "CN=localhost" -storepass serverpwd -keypass serverpwd -validity 365
keytool -export -alias serverprivate -keystore server.private -file temp.key -storepass serverpwd
keytool -import -noprompt -alias serverpublic -keystore server.public -file temp.key -storepass public
del temp.key

this gives me public and private certificates for server and client

then I start the server and client and when I try to sign on the server and try to read the "Common name" from the client on server with

Socket newClientSocket = ss.accept();
 ((SSLSocket)newClientSocket).startHandshake();
 String username = ((SSLSocket) newClientSocket).getSession().getPeerPrincipal().getName();

and it's the username fetching that gives me the error and I have no idea why. Is there something wrong with how I make the certificates?

Here is my full code for server:

import java.io.*;
import java.net.*;
import java.util.*;
import javax.net.ssl.*;
import java.security.*;

public class ChatServer {

protected List<Socket> clients = new ArrayList<Socket>(); // list of clients

private String[][] portUserPair = new String[10][2];
int i = 0;

private String passphrase = "serverpwd";
private int port = 1234;

public static void main(String[] args) throws Exception {
    new ChatServer();
}

public ChatServer() throws Exception {

    KeyStore clientKeyStore = KeyStore.getInstance("JKS");
    clientKeyStore.load(new FileInputStream("client.public"), "public".toCharArray());

    KeyStore serverKeyStore = KeyStore.getInstance("JKS");
    serverKeyStore.load(new FileInputStream("server.private"), passphrase.toCharArray());

    TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
    tmf.init(clientKeyStore);
    KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
    kmf.init(serverKeyStore, passphrase.toCharArray());
    SSLContext sslContext = SSLContext.getInstance("TLS");
    SSLContext.setDefault(sslContext);
    sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), (new SecureRandom()));

    SSLServerSocketFactory factory = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
    SSLServerSocket ss = (SSLServerSocket) factory.createServerSocket(port);
    ss.setNeedClientAuth(false);
    ss.setEnabledCipherSuites(new String[] {"TLS_RSA_WITH_AES_128_CBC_SHA"});

    // start listening for new connections
    System.out.println("[system] listening ...");
    try {
        while (true) {
            Socket newClientSocket = ss.accept(); // wait for a new client connection
            ((SSLSocket)newClientSocket).startHandshake();
            String username = ((SSLSocket) newClientSocket).getSession().getPeerPrincipal().getName();
            System.out.println("Established SSL connection with: " + username);
            synchronized(this) {
                portUserPair[i][0] = Integer.toString(newClientSocket.getPort());
                clients.add(newClientSocket); // add client to the list of clients
            }
            ChatServerConnector conn = new ChatServerConnector(this, newClientSocket); // create a new thread for communication with the new client
            conn.start(); // run the new thread
        }
    } catch (Exception e) {
        System.err.println("[error] Accept failed.");
        e.printStackTrace(System.err);
        System.exit(1);
    }

    // close socket
    System.out.println("[system] closing server socket ...");
    try {
        ss.close();
    } catch (IOException e) {
        e.printStackTrace(System.err);
        System.exit(1);
    }
}

// send a message to all clients connected to the server
public void sendToAllClients(String message) throws Exception {
    Iterator<Socket> i = clients.iterator();
    while (i.hasNext()) { // iterate through the client list
        Socket socket = (Socket) i.next(); // get the socket for communicating with this client
        try {
            DataOutputStream out = new DataOutputStream(socket.getOutputStream()); // create output stream for sending messages to the client
            out.writeUTF(message); // send message to the client
        } catch (Exception e) {
            System.err.println("[system] could not send message to a client");
            e.printStackTrace(System.err);
        }
    }
}

public void sendToAClient(String message, String userInfo) throws Exception {
    Socket userSocket = null;
    String userPort = "";
    Iterator<Socket> i = clients.iterator();
    for (int k = 0; k < portUserPair.length; k++) {
        if(portUserPair[k][1].equals(userInfo)) {
            userPort = portUserPair[k][0];
            break;
        }
    }

    System.out.println(userPort);

    while (i.hasNext()) { // iterate through the client list
        Socket socket = (Socket) i.next(); // get the socket for communicating with this client
        if(Integer.toString(socket.getPort()).equals(userPort)) {
            userSocket = socket;
            break;
        }
    }

    message = message.substring(userInfo.length()+2);

    try {
        DataOutputStream out = new DataOutputStream(userSocket.getOutputStream()); // create output stream for sending messages to the client
        out.writeUTF(message); // send message to the client
    } catch (Exception e) {
        System.err.println("[system] could not send message to the client");
        e.printStackTrace(System.err);
    }
}

public void removeClient(Socket socket) {
    synchronized(this) {
        clients.remove(socket);
    }
}

public void saveUserInfo(String userInfo) {
    portUserPair[i][1] = userInfo;
    System.out.println(portUserPair[i][0] + " : " + portUserPair[i][1]);
    i++;
}
}

class ChatServerConnector extends Thread {
private ChatServer server;
private Socket socket;

public ChatServerConnector(ChatServer server, Socket socket) {
    this.server = server;
    this.socket = socket;
}

public void run() {
    System.out.println("[system] connected with " + this.socket.getInetAddress().getHostName() + ":" + this.socket.getPort());

    DataInputStream in;
    try {
        in = new DataInputStream(this.socket.getInputStream()); // create input stream for listening for incoming messages
    } catch (IOException e) {
        System.err.println("[system] could not open input stream!");
        e.printStackTrace(System.err);
        this.server.removeClient(socket);
        return;
    }

    while (true) { // infinite loop in which this thread waits for incoming messages and processes them
        String msg_received;
        try {
            msg_received = in.readUTF(); // read the message from the client
        } catch (Exception e) {
            System.err.println("[system] there was a problem while reading message client on port " + this.socket.getPort());
            e.printStackTrace(System.err);
            this.server.removeClient(this.socket);
            return;
        }

        if (msg_received.length() == 0) // invalid message
            continue;

        String[] msg = msg_received.split(" ");
        System.out.println(msg_received);

        if (msg[1].equals("system")) {
//            try {
//                server.saveUserInfo(((SSLSocket) socket).getSession().getPeerPrincipal().getName());
//            } catch (SSLPeerUnverifiedException ex) {
//                Logger.getLogger(ChatServerConnector.class.getName()).log(Level.SEVERE, null, ex);
//            }
            continue;
        }

        System.out.println(msg_received); // print the incoming message in the console

        String msg_send = msg_received.toUpperCase();

        if (msg[3].charAt(0) == '/') {
            String[] privateMsg = msg[3].split(" ");
            try {
                this.server.sendToAClient(msg_send, privateMsg[0].replace("/", "")); // send message to the client
            } catch (Exception e) {
                System.err.println("[system] there was a problem while sending the message to the client");
                e.printStackTrace(System.err);
                continue;
            }
            continue;
        }

        try {
            this.server.sendToAllClients(msg_send); // send message to all clients
        } catch (Exception e) {
            System.err.println("[system] there was a problem while sending the message to all clients");
            e.printStackTrace(System.err);
            continue;
        }
    }
}
}

and here is my cleint:

import java.io.*;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Scanner;
import javax.net.ssl.*;
import java.security.*;

public class ChatClient extends Thread {

    private String userInfo = "";

    private String passphrase = "pwd";
    private int port = 1234;

    public static void main(String[] args) throws Exception {
        new ChatClient();
    }

    public ChatClient() throws Exception {

        KeyStore serverKeyStore = KeyStore.getInstance("JKS");
        serverKeyStore.load(new FileInputStream("server.public"), "public".toCharArray());

        KeyStore clientKeyStore = KeyStore.getInstance("JKS");
        Scanner s = new Scanner(System.in);
        String username = s.nextLine();
        clientKeyStore.load(new FileInputStream(username + ".private"), (username + passphrase).toCharArray());

        TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
        tmf.init(serverKeyStore);
        KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
        kmf.init(clientKeyStore, (username + passphrase).toCharArray());
        SSLContext sslContext = SSLContext.getInstance("TLS");
        SSLContext.setDefault(sslContext);
        sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());

        SSLSocketFactory sf = sslContext.getSocketFactory();
        SSLSocket socket = (SSLSocket) sf.createSocket("localhost", port);
        socket.setEnabledCipherSuites(new String[] {"TLS_RSA_WITH_AES_128_CBC_SHA"});
        socket.startHandshake();

        DataInputStream in = null;
        DataOutputStream out = null;

        // connect to the chat server
        try {
            System.out.println("[system] connecting to chat server ...");
            System.out.println("[system] you are signed in as: " + username);

            in = new DataInputStream(socket.getInputStream()); // create input stream for listening for incoming messages
            out = new DataOutputStream(socket.getOutputStream()); // create output stream for sending messages

            this.sendMessage(username, out, "system");

            System.out.println("[system] connected");
            System.out.println("[system] to send a private message type \"/<NameOfUser>\" and then type the message");

            ChatClientMessageReceiver message_receiver = new ChatClientMessageReceiver(in); // create a separate thread for listening to messages from the chat server
            message_receiver.start(); // run the new thread
        } catch (Exception e) {
            e.printStackTrace(System.err);
            System.exit(1);
        }

        // read from STDIN and send messages to the chat server
        BufferedReader std_in = new BufferedReader(new InputStreamReader(System.in));
        String userInput;
        while ((userInput = std_in.readLine()) != null) { // read a line from the console
            this.sendMessage(userInput, out, userInfo); // send the message to the chat server
        }

        // cleanup
        out.close();
        in.close();
        std_in.close();
        socket.close();
    }

    private void sendMessage(String message, DataOutputStream out, String userInfo) {
        message = "[" + new SimpleDateFormat("HH:mm:ss").format(new Date()) + "] " + userInfo + " : " + message;
        try {
            out.writeUTF(message); // send the message to the chat server
            out.flush(); // ensure the message has been sent
        } catch (IOException e) {
            System.err.println("[system] could not send message");
            e.printStackTrace(System.err);
        }
    }
}

// wait for messages from the chat server and print the out
class ChatClientMessageReceiver extends Thread {
    private DataInputStream in;

    public ChatClientMessageReceiver(DataInputStream in) {
        this.in = in;
    }

    public void run() {
        try {
            String message;
            while ((message = this.in.readUTF()) != null) { // read new message
                System.out.println(message); // print the message to the console
            }
        } catch (Exception e) {
            System.err.println("[system] could not read message");
            e.printStackTrace(System.err);
        }
    }
}

the whole thing is supposed to be a "chat" that uses SSL to encrypt messages and should get the username from the certificate...so I have everything figured out except the "getting the usename out of the certificate's 'Common name' field"

Help, I'm losing my mind over this! :'(

Body:
ss.setNeedClientAuth(false);

The problem is here. You never asked for the client certificate, so it never got sent. Change this to true.


问:
Best way to prevent duplicate data on copy csv postgresql
This is more of a conceptual question because I'm planning how best to achieve our goals here.

I have a postgresql/postgis table with 5 columns.  I'll be inserting/appending data into the database from a csv file every 10 minutes or so via the copy command. There will likely be some duplicate rows of data, so I'd like to copy the data from the csv file to the postgresql table but prevent any duplicate entries from getting into the table from the csv file.  There are three columns, where if they are all equal, that will mean the entry is a duplicate.  They are "latitude", "longitude" and "time".  Should I make a composite key from all three columns?  If I do that, will it just throw an error upon trying to copy the csv file into the database?  I'm going to be copying the csv file automatically so I would want it to go ahead and copy the rest of the file that aren't duplicates and not copy the duplicates.  Is there a way to do this?

Also, I of course want it to look for duplicates in the most efficient way.  I don't need to look through the whole table (which will be quite large) for duplicates...just the past 20 minutes or so via the timestamp on the row.  And I've indexed the db with the time column.

Thanks for any help!

答:
Upsert

The Answer by Linoff is correct but can simplified a bit by Postgres 9.5 new ”UPSERT“ feature (a.k.a. MERGE). That new feature is implemented in Postgres as INSERT ON CONFLICT syntax.

Rather than explicitly check for violation of the unique index, we can let the ON CONFLICT clause detect the violation. Then we DO NOTHING, meaning we abandon the effort to INSERT without bothering to attempt an UPDATE. So if we cannot insert, we just move on to next row.

We get the same results as Linoff’s code but lose the WHERE clause.

```<code>INSERT INTO bigtable(col1, … )
    SELECT col1, …
    FROM stagingtable st
ON CONFLICT idx_bigtable_col1_col2_col
DO NOTHING
;
</code>```

Body:
Upsert

The Answer by Linoff is correct but can simplified a bit by Postgres 9.5 new ”UPSERT“ feature (a.k.a. MERGE). That new feature is implemented in Postgres as INSERT ON CONFLICT syntax.

Rather than explicitly check for violation of the unique index, we can let the ON CONFLICT clause detect the violation. Then we DO NOTHING, meaning we abandon the effort to INSERT without bothering to attempt an UPDATE. So if we cannot insert, we just move on to next row.

We get the same results as Linoff’s code but lose the WHERE clause.

INSERT INTO bigtable(col1, … )
    SELECT col1, …
    FROM stagingtable st
ON CONFLICT idx_bigtable_col1_col2_col
DO NOTHING
;


问:
How do I use previous token's label as feature in my CRF?
I am looking for a way to use features conditioned with attributes and label bigrams in mallet. I am still trying to understand how would one be able to use the label of a token just generated as a feature for determining the label of the next token? Are the feature vectors for tokens generated as the labels for previous tokens obtained?

Have I misunderstood that CRF allows the usage of predicted previous label as a feature for the next token?

Thanks in advance!

答:
Are the feature vectors for tokens generated as the labels for previous tokens obtained?

No, CRF optimizes loss jointly, there is no left-to-right processing like in MEMM where you predict a label and then use it. CRF takes in account all possible previous labels and finds a most likely sequence.

Have I misunderstood that CRF allows the usage of predicted previous label as a feature for the next token?

CRF allows to use previous labels as features; most likely it already happens automatically in your case. I don't have experience with Mallet, but in most out-of-box linear-chain CRF packages there are 2 kinds of features:

"state features". These are per-token features user defines; they can use any information from the input sequence (e.g. current and previous token, last 3 letters of the current token, etc.) Each state feature is usually conditioned on the current output label.

"transition features". In the most common 1-st order linear-chain CRF it is the current label conditioned on a previous label. Usually these features are generated automatically, for all possible label pairs.

Sometimes you can also condition (2) transition features on user-defined features which depend on a current token. It seems this is what you're looking for, but I'm not sure. Some packages implement this (e.g. wapiti), some don't (e.g. crfsuite). Some packages allow to define arbitrary CRFs, and use arbitrary features (e.g. pystruct, factorie, GRMM(?)). Sorry, I don't have experience with Mallet, so it is not really an answer :)


问:
Why is my page reloading even though I have event.preventDefault (form handling jquery + node.js)
I am using jQuery to asynchronously post my data to my db using node.js and express.js

Here is roughly what I am doing:

index.html:

```<code><form id='mailer' action='/mail/new' method='post'>
  <input name='mycontent'/>
  <input type='submit'/>
</form>
</code>```

script.js:

```<code>$("#mailer").submit(function(e){
  e.preventDefault();
  $.post('/mailer/new',{mycontent:mycontent});
});
</code>```

app.js:

```<code>app.post("/mailer/new",function(){
  //insert into database
}
</code>```

So it is essentially submitting to my database but ignoring the e.preventDefault because my page shows the loader wheel when I press submit

答:
As a commenter pointed out, you need to remove the hash # from the id attribute of your form:

<form id='mailer'...

You need quotes around the element you're submitting. A JavaScript error occurs (undefined variable, most likely) and the form submits anyway before the preventDefault() runs.

$(#mailer).submit(function(e){...

Should be:

$('#mailer').submit(function(e){

Body:
I am using jQuery to asynchronously post my data to my db using node.js and express.js

Here is roughly what I am doing:

index.html:

<form id='mailer' action='/mail/new' method='post'>
  <input name='mycontent'/>
  <input type='submit'/>
</form>

script.js:

$("#mailer").submit(function(e){
  e.preventDefault();
  $.post('/mailer/new',{mycontent:mycontent});
});

app.js:

app.post("/mailer/new",function(){
  //insert into database
}

So it is essentially submitting to my database but ignoring the e.preventDefault because my page shows the loader wheel when I press submit


问:
Why do my components not updating reactively in this vuejs application?
I'm currently shooting into the dark at the moment. The code below (and dev environment) are nothing to be proud of. I have just now started to learn vuejs using a small project as guinea pig. I am advancing, but I have a problem regarding vuex.

I'm still fooling about, so please disregard some weird code (especially the choice and structure of route/team assignments). That will be refactored later, and shouldn't be related to the issue.

Quick Problem Description (details follow below)

I have a vuex store

One view (the route list) contains a list of RouteBlock components. Each RouteBlock contains a list of teams which are assigned to that route, and a list of teams assigned to no route.

When clicking the "assign" button, a previously unassigned group will become assigned to one route, and will disappear from the "unassigned" list of each RouteBlock

This currently only works when refreshing the complete state from the server. From what I understand this should not be necessary thanks to vuex. I'm trying to find out what I need to change to make this work without a full refresh.

If you don't want to bother running the code, you can scroll down to the bottom to the "Following the Flow" section.

The code / Running Example / Demo

I added a tag to keep a reference to the code at the time of this writing
Here's a link to a snapshot of the code at the time of this writing. The folder frontend contains the vuejs code, whereas the powonline folder contains the Python backend.

Getting the Python part up and running should be as easy as:

```<code>git clone https://github.com/exhuma/powonline.git
cd powonline
git checkout stackoverflow-2017-06-09
pyvenv env
./env/bin/pip install -e .
./env/bin/python autoapp.py
</code>```

Everything here in this prototype runs in-memory. There is no persistent storage. If the Python process restarts, the data is reset!

The vuejs part is located in the frontend folder. I've had some issues getting npm set-up and ended up creating two docker containers in the frontend root, with two helper scripts (1 and 2). Those script contain hard-coded folders which certainly must be changed! When building the containers, they should have the names exhuma/npm and exhuma/vue as appropriate, so they can be run using the helper scripts. It is likely that files created (if any) by one of those containers are owned by "root", which also needs to be fixed via a chown (but normally, no files will be created, I think).

With everything in place, and using docker, the frontend should be runnable using:

```<code>cd powonline/frontend
./npm run dev
</code>```

Preparing Example Data for Testing

Access the frontend

Click on "Teams"

Add two teams ("Team 1" and "Team 2")

Click on "Routes"

Add two routes ("Route 1" and "Route 2")

You then should see something like the following:

Following the Flow, and my Thoughts

A user clicking on "Assign" calls assign which dispatches to the vuex action assignTeamToRouteRemote

This triggers an async remote action. Once this returns the new data is committed using the assignTeamToRoute mutation. This is what I expect to trigger a reactive update, but which doesn't!

I've added a dispatch to action refreshRemote because the reactive change above does not happen. I want to remove that (if possible)!

The view in turn uses a computed property (assignedTeams), which delegates to the vuex getter assignedTeams. I would expect this to update after the previous changes. Is it possible that this is cached (as per the docs?) and that vuejs does not detect the change of the getter as a dependency and hence does not update the cache? I've tried to move it over to a method instead of a property, but then I ran into a weird CORS problem...

答:
With Vuex, it's best to treat the state properties (in mutations) as immutable.

Your assignTeamToRoute mutation is potentially adding object properties which is not reactive (see https://v2.vuejs.org/v2/guide/reactivity.html#Change-Detection-Caveats)

Try the following using Vue.set...

```<code>assignTeamToRoute (state, payload) {
  const current = state.route_team_map[payload.routeName] || [];
  Vue.set(state.route_team_map, payload.routeName, current.concat(payload.team.name))
}
</code>```

You'll probably need to do something similar with the mutations that remove teams from routes.

Body:
I'm currently shooting into the dark at the moment. The code below (and dev environment) are nothing to be proud of. I have just now started to learn vuejs using a small project as guinea pig. I am advancing, but I have a problem regarding vuex.

I'm still fooling about, so please disregard some weird code (especially the choice and structure of route/team assignments). That will be refactored later, and shouldn't be related to the issue.

Quick Problem Description (details follow below)

I have a vuex store

One view (the route list) contains a list of RouteBlock components. Each RouteBlock contains a list of teams which are assigned to that route, and a list of teams assigned to no route.

When clicking the "assign" button, a previously unassigned group will become assigned to one route, and will disappear from the "unassigned" list of each RouteBlock

This currently only works when refreshing the complete state from the server. From what I understand this should not be necessary thanks to vuex. I'm trying to find out what I need to change to make this work without a full refresh.

If you don't want to bother running the code, you can scroll down to the bottom to the "Following the Flow" section.

The code / Running Example / Demo

I added a tag to keep a reference to the code at the time of this writing
Here's a link to a snapshot of the code at the time of this writing. The folder frontend contains the vuejs code, whereas the powonline folder contains the Python backend.

Getting the Python part up and running should be as easy as:

git clone https://github.com/exhuma/powonline.git
cd powonline
git checkout stackoverflow-2017-06-09
pyvenv env
./env/bin/pip install -e .
./env/bin/python autoapp.py

Everything here in this prototype runs in-memory. There is no persistent storage. If the Python process restarts, the data is reset!

The vuejs part is located in the frontend folder. I've had some issues getting npm set-up and ended up creating two docker containers in the frontend root, with two helper scripts (1 and 2). Those script contain hard-coded folders which certainly must be changed! When building the containers, they should have the names exhuma/npm and exhuma/vue as appropriate, so they can be run using the helper scripts. It is likely that files created (if any) by one of those containers are owned by "root", which also needs to be fixed via a chown (but normally, no files will be created, I think).

With everything in place, and using docker, the frontend should be runnable using:

cd powonline/frontend
./npm run dev

Preparing Example Data for Testing

Access the frontend

Click on "Teams"

Add two teams ("Team 1" and "Team 2")

Click on "Routes"

Add two routes ("Route 1" and "Route 2")

You then should see something like the following:

Following the Flow, and my Thoughts

A user clicking on "Assign" calls assign which dispatches to the vuex action assignTeamToRouteRemote

This triggers an async remote action. Once this returns the new data is committed using the assignTeamToRoute mutation. This is what I expect to trigger a reactive update, but which doesn't!

I've added a dispatch to action refreshRemote because the reactive change above does not happen. I want to remove that (if possible)!

The view in turn uses a computed property (assignedTeams), which delegates to the vuex getter assignedTeams. I would expect this to update after the previous changes. Is it possible that this is cached (as per the docs?) and that vuejs does not detect the change of the getter as a dependency and hence does not update the cache? I've tried to move it over to a method instead of a property, but then I ran into a weird CORS problem...

Body:
With Vuex, it's best to treat the state properties (in mutations) as immutable.

Your assignTeamToRoute mutation is potentially adding object properties which is not reactive (see https://v2.vuejs.org/v2/guide/reactivity.html#Change-Detection-Caveats)

Try the following using Vue.set...

assignTeamToRoute (state, payload) {
  const current = state.route_team_map[payload.routeName] || [];
  Vue.set(state.route_team_map, payload.routeName, current.concat(payload.team.name))
}

You'll probably need to do something similar with the mutations that remove teams from routes.


问:
get data from XML file using jQuery
I am trying to get an attribute value from an XML file using jQuery, but it returns undefined and the image doesn't load.

In my HTML file I have a
 with id answer and one <button> with id showInfo.

```<code>xml code:
<?xml version="1.0"?>
    <student studentNum="5678">
        <image>images/mypic.jpg</image>
    </student>
</code>```

my jQuery code:

```<code>$(document).ready(function(){
  $.ajax( {
   url:'stu.xml',
   data:{},
   type:'GET',
   dataType:'xml',
   success: function(resp){

$('#showInfo').click(function () {

   var myInfo = $(resp).find('student');
   myInfo.each(function(index, obj) {

    var myNum = ($(obj).find('student').attr('studentNum'));
    var myImg = $(obj).find('image').text();

    $('#answer').append("<p> " + "Student Number:  " +myNum+
    "<img src='" + myImg + "'/><br/>" );
});
});
});
</code>```

答:
Just check once with your xml response, i did some minor changes to response xml. Hope that's work for you.

var resp = '<?xml version="1.0"?><students><student studentNum="5678"><images>images/mypic.jpg</images></student><student studentNum="4576"><images>images/mypic.jpg</images></student></students>';

var myInfo = $(resp).find('student');
myInfo.each(function(index, obj) {
    var myNum = ($(obj).attr('studentNum'));
    var myImg = $(obj).find('images').text();
    $('#answer').append("
 " + "Student Number:  " + myNum + "
" );
});```
<pre class="snippet-code-html lang-html prettyprint-override"><code><script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div id='answer'></div></code>```

Title: get data from XML file using jQuery

Body:
I am trying to get an attribute value from an XML file using jQuery, but it returns undefined and the image doesn't load.

In my HTML file I have a
 with id answer and one <button> with id showInfo.

xml code:
<?xml version="1.0"?>
    <student studentNum="5678">
        <image>images/mypic.jpg</image>
    </student>

my jQuery code:

$(document).ready(function(){
  $.ajax( {
   url:'stu.xml',
   data:{},
   type:'GET',
   dataType:'xml',
   success: function(resp){

$('#showInfo').click(function () {

   var myInfo = $(resp).find('student');
   myInfo.each(function(index, obj) {

    var myNum = ($(obj).find('student').attr('studentNum'));
    var myImg = $(obj).find('image').text();

    $('#answer').append("
 " + "Student Number:  " +myNum+
    "
" );
});
});
});

Body:
Just check once with your xml response, i did some minor changes to response xml. Hope that's work for you.

var resp = '<?xml version="1.0"?><students><student studentNum="5678"><images>images/mypic.jpg</images></student><student studentNum="4576"><images>images/mypic.jpg</images></student></students>';

var myInfo = $(resp).find('student');
myInfo.each(function(index, obj) {
    var myNum = ($(obj).attr('studentNum'));
    var myImg = $(obj).find('images').text();
    $('#answer').append("<p> " + "Student Number:  " + myNum + "<img src='" + myImg + "'/><br/>" );
});

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div id='answer'></div>


问:
Bootstrap Column Sidebar Height
I have kind of an issue with the bootstrap grid system which confuses me,
i have a 2 column layout that becomes a one column (col-xs-12) layout on mobile devices.

The issue i have is that on large screens, if my sidebar is higher than my "header div" it takes up whitespace.

```<code><div class="container">
    <div class="row">
        <div class="col-lg-8 col-md-8 col-sm-8 col-xs-12 header">header</div>
        <div class="col-lg-4 col-md-4 col-sm-4 col-xs-12 sidebar">sidebar</div>
        <div class="col-lg-8 col-sm-8 col-xs-12 main_content">main_content</div>
    </div>
</div>
</code>```

Heres a jsbin :
https://jsbin.com/helizemefa/edit?html,css,output

Maybe im just standing on the line right now..

Thanks for your help!

答:
.header{
  background-color:red;
}

.sidebar {
  background-color:yellow;

}

.main_content {
  background-color:green;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>JS Bin</title>
</head>
<body>
<script src="https://code.jquery.com/jquery.min.js"></script>
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" rel="stylesheet" type="text/css" />
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>

  <div class="container">
    <div class="row">
      <div class="col-lg-8 col-md-8 col-sm-8 col-xs-12 header">header</div>
      <div class="col-lg-4 col-md-4 col-sm-4 col-xs-12 pull-right sidebar">sidebar</div>
      <div class="col-lg-8 col-lg-8 col-sm-8 col-xs-12 main_content">main_content</div>
    </div>
  </div>
</body>
</html></code>```

you can add pull-right on your sidebar, like this

```<code>      <div class="col-lg-4 col-md-4 col-sm-4 col-xs-12 sidebar pull-right">sidebar</div>
</code>```

Title: Bootstrap Column Sidebar Height

Body:
I have kind of an issue with the bootstrap grid system which confuses me,
i have a 2 column layout that becomes a one column (col-xs-12) layout on mobile devices.

The issue i have is that on large screens, if my sidebar is higher than my "header div" it takes up whitespace.

header

sidebar

main_content

Heres a jsbin :
https://jsbin.com/helizemefa/edit?html,css,output

Maybe im just standing on the line right now..

Thanks for your help!

Body:
.header{
  background-color:red;
}

.sidebar {
  background-color:yellow;

}

.main_content {
  background-color:green;
}

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>JS Bin</title>
</head>
<body>
<script src="https://code.jquery.com/jquery.min.js"></script>
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" rel="stylesheet" type="text/css" />
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>

  <div class="container">
    <div class="row">
      <div class="col-lg-8 col-md-8 col-sm-8 col-xs-12 header">header</div>
      <div class="col-lg-4 col-md-4 col-sm-4 col-xs-12 pull-right sidebar">sidebar</div>
      <div class="col-lg-8 col-lg-8 col-sm-8 col-xs-12 main_content">main_content</div>
    </div>
  </div>
</body>
</html>

you can add pull-right on your sidebar, like this

      <div class="col-lg-4 col-md-4 col-sm-4 col-xs-12 sidebar pull-right">sidebar</div>


问:
Why are my instances changing?
I'm making a class of polynomial objects, represented by dictionaries:

3*x^2+x+2 == {2:3, 1:1, 0:2}

This is the part of my code that's relevant for the question:

```<code>class Sparse_polynomial():
    def __init__(self, coeffs_dict):
        self.coeffs_dict = coeffs_dict

    def __repr__(self):
        terms = [" + ("+str(self.coeffs_dict[k])+"*x^" + str(k)+")" \
                 for k in sorted(self.coeffs_dict.keys(), reverse=True)]
        terms = "".join(terms)
        return terms[3:]

    def __neg__(self):
        neg_pol= self.coeffs_dict
        for key in self.coeffs_dict:
            neg_pol[key]= -self.coeffs_dict[key]
        return Sparse_polynomial(neg_pol)
</code>```

Whenever I try to use the __neg__ method, the original object changes.  For example:

```<code>>>> p1= Sparse_polynomial({1:3,5:1})
>>> p1
(1*x^5) + (3*x^1)
>>> -p1
(-1*x^5) + (-3*x^1)
>>> p1
(-1*x^5) + (-3*x^1)
>>>
</code>```

I really can't understand why the original p1 changes. I made no direct changes to it, only accessed its fields.

Can anyone clarify so I can fix this?

答:
As the others pointed out the:

```<code>neg_pol = self.coeffs_dict
</code>```

doesn't create a new dict, neg_pol will be just another reference for the same dictionary and any changes (like neg_pol[key] = -self.coeffs_dict[key]) will be visible for both variables.

Personally I prefer dict-comprehensions to create new dicts instead of just using copy:

```<code>class Sparse_polynomial():

    # ... your other stuff

    def __neg__(self):
        return Sparse_polynomial({key: -value for key, value in self.coeffs_dict.items()})
</code>```

Title: Why are my instances changing?

Body:
I'm making a class of polynomial objects, represented by dictionaries:

3*x^2+x+2 == {2:3, 1:1, 0:2}

This is the part of my code that's relevant for the question:

class Sparse_polynomial():
    def __init__(self, coeffs_dict):
        self.coeffs_dict = coeffs_dict

    def __repr__(self):
        terms = [" + ("+str(self.coeffs_dict[k])+"*x^" + str(k)+")" \
                 for k in sorted(self.coeffs_dict.keys(), reverse=True)]
        terms = "".join(terms)
        return terms[3:]

    def __neg__(self):
        neg_pol= self.coeffs_dict
        for key in self.coeffs_dict:
            neg_pol[key]= -self.coeffs_dict[key]
        return Sparse_polynomial(neg_pol)

Whenever I try to use the __neg__ method, the original object changes.  For example:

>>> p1= Sparse_polynomial({1:3,5:1})
>>> p1
(1*x^5) + (3*x^1)
>>> -p1
(-1*x^5) + (-3*x^1)
>>> p1
(-1*x^5) + (-3*x^1)
>>>

I really can't understand why the original p1 changes. I made no direct changes to it, only accessed its fields.

Can anyone clarify so I can fix this?

Body:
As the others pointed out the:

neg_pol = self.coeffs_dict

doesn't create a new dict, neg_pol will be just another reference for the same dictionary and any changes (like neg_pol[key] = -self.coeffs_dict[key]) will be visible for both variables.

Personally I prefer dict-comprehensions to create new dicts instead of just using copy:

class Sparse_polynomial():

    # ... your other stuff

    def __neg__(self):
        return Sparse_polynomial({key: -value for key, value in self.coeffs_dict.items()})