问:
Updating a big table in oracle without drooping the base table
I am trying to update a table in sql query. My table contains half a million records and it is taking around 10 minutes of time. It is my dev environment.

When i will go for UAT or prod, the table there will contain a lot more data than the dev environment. updating the whole table and then putting a commit will consume a lot of memory. Can someone suggest me a good way of updating a table in batches.

I have gone through net and the most common solution i found is creating a new table by "create new_table as select (update ) from old_table." But in my case i can not use this as in my prod environment i can not drop  and create table.

Thanks
Ankit.

答:
CTAS remains the fastest option. Bear in mind you don't need to drop the original table; you can just rename and retain it to keep your management happy.

Otherwise you have some options depending on your circumstances.

If:

the concern is just about elapsed time; and

you have the Enterprise Edition license; and

Production has CPUs to spare

Then you could consider Parallel DML to solve the problem.

```<code>alter session enable parallel dml;

update /*+ parallel (your_table) */ your_table
set ..
</code>```

If you have only Standard Edition but are on 11gR2 or higher you can use DBMS_PARALLEL_EXECUTE package to run a PL/SQL job in parallel. The caveats about CPU still apply.

Beyond that, you should look at the efficiency of the WHERE clause. An UPDATE is a scan too and can be tuned like any other query.

Splitting the update into batches will make the total elapsed time longer. Of course it will, you're doing more work. The main advantage of batching over a single update statement is that by committing chunks of records we don't lose work if the task falls over halfway through. This is only an advantage if there is a mechanism for distinguishing the updated records ( a status or date column which is updated by this task and no  other). If you don't have  a flag like that you run the risk of corrupting the table absolutely.

The simplest way of batching is a simple PL/SQL loop. Assuming you have a million rows to update:

```<code>begin
    for idx in 1..100 loop
         update your_table
         set whatever = whatever * 1.1
                    , status = 'touched'
         where status != 'touched'
          and rownum <= 10000;
          commit;
    end loop;
end;
</code>```

Obviously you'll need some error handling, perhaps logging, etc. But resist the lure of bulk collect, forall etc unless your update rules are sufficiently complicated that they demand procedural logic.

Body:
CTAS remains the fastest option. Bear in mind you don't need to drop the original table; you can just rename and retain it to keep your management happy.

Otherwise you have some options depending on your circumstances.

If:

the concern is just about elapsed time; and

you have the Enterprise Edition license; and

Production has CPUs to spare

Then you could consider Parallel DML to solve the problem.

alter session enable parallel dml;

update /*+ parallel (your_table) */ your_table
set ..

If you have only Standard Edition but are on 11gR2 or higher you can use DBMS_PARALLEL_EXECUTE package to run a PL/SQL job in parallel. The caveats about CPU still apply.

Beyond that, you should look at the efficiency of the WHERE clause. An UPDATE is a scan too and can be tuned like any other query.

Splitting the update into batches will make the total elapsed time longer. Of course it will, you're doing more work. The main advantage of batching over a single update statement is that by committing chunks of records we don't lose work if the task falls over halfway through. This is only an advantage if there is a mechanism for distinguishing the updated records ( a status or date column which is updated by this task and no  other). If you don't have  a flag like that you run the risk of corrupting the table absolutely.

The simplest way of batching is a simple PL/SQL loop. Assuming you have a million rows to update:

begin
    for idx in 1..100 loop
         update your_table
         set whatever = whatever * 1.1
                    , status = 'touched'
         where status != 'touched'
          and rownum <= 10000;
          commit;
    end loop;
end;

Obviously you'll need some error handling, perhaps logging, etc. But resist the lure of bulk collect, forall etc unless your update rules are sufficiently complicated that they demand procedural logic.


来源: stackexchange

问:
Serving pharo seaside 3.0 with lighttpd
I am trying to serve my app with lighttpd. With seaside 2.8 the following worked in my lighttpd.conf:

```<code>proxy.server = ( "/pharo" => (
    ( "host" => "127.0.0.1", "port" => 8080, "check-local" => "disable"))
)
</code>```

but with seaside 3.0 it rewrites the url to say http://localhost/pharo when accessing it over a network.

Anyone been able to serve pharo and seaside 3 with lighttpd?

答: In the seaside config of your app, put your IP address in the Server Hostname field</p>

ContentLicense: CC BY-SA 2.5

Title: Serving pharo seaside 3.0 with lighttpd

Body:
I am trying to serve my app with lighttpd. With seaside 2.8 the following worked in my lighttpd.conf:

proxy.server = ( "/pharo" => (
    ( "host" => "127.0.0.1", "port" => 8080, "check-local" => "disable"))
)

but with seaside 3.0 it rewrites the url to say http://localhost/pharo when accessing it over a network.

Anyone been able to serve pharo and seaside 3 with lighttpd?


问:
Casperjs: How to manipulate javascript paging( links overloaded by javascript)
Using casperjs, I'm working on scraping some informaion from a web site, in this page there is a paging created by overloading a link

```<code><a id="ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2"
href="javascript:;//ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2"><img
src="themes/images/fleche-suivant.gif" alt="Aller à la page suivante"></a>
</code>```

there are two methods to access to the next page:

to click on the next button
as shown in this image

or to change the value of the field as shown in this pic and click the enter key on the keyboard

I've tried both methods but didn't work, could someone please help me to code one of them.

```<code>var mouse = require("mouse").create(casper);
var url = '';
var url2 = '';
var tst;
casper.test.begin('Scraping start', function(test) {
casper.start(url, function() {
    this.test.pass('Opened 1st page');
})
.thenOpen(url2, function(){
    this.test.pass('Opened 2nd page')
})
.then(function(){

 //these are the tow methodes I try to click on the next button
//this.mouse.click("#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2");
this.thenClick(' div.liens a#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2');
})

.then(function() {
    tst = this.evaluate(function() {
    return
__utils__.getFieldValue(
'[name="ctl0$CONTENU_PAGE$resultSearch$numPageTop"]');
});
})
.run(function() {
    console.log(tst);
    test.done();
});
});
</code>```

the version of casperjs is 1.1.4 and I'm using the phantom browser

答:
This normally navigates to 2nd page on my end. Try it, also you can save screenshot for more debugging purposes.

```<code>var casper = require('casper').create();
var url = 'https://www.marchespublics.gov.ma/index.php5?page=entreprise.EntrepriseAdvancedSearch&AllCons&EnCours&domaineActivite=1.15';

casper
    .start(url, function() {
        this.echo('Opened page ' + this.evaluate(function() {
            return document.title;
        }), 'INFO');
    })
    .then(function() {
        if (this.exists('a[id="ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2"]')) {
            this.echo('the heading exists');
        } else {
            this.echo('Does not exist');
        }
    })
    .thenClick('div.liens a#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2')
    .wait(5000)
    .then(function() {
        this.capture('exit.png');
    })
    .run();
</code>```

Hope that helps!

ContentLicense: CC BY-SA 3.0

Body:
Using casperjs, I'm working on scraping some informaion from a web site, in this page there is a paging created by overloading a link

there are two methods to access to the next page:

to click on the next button
as shown in this image

or to change the value of the field as shown in this pic and click the enter key on the keyboard

I've tried both methods but didn't work, could someone please help me to code one of them.

var mouse = require("mouse").create(casper);
var url = '';
var url2 = '';
var tst;
casper.test.begin('Scraping start', function(test) {
casper.start(url, function() {
    this.test.pass('Opened 1st page');
})
.thenOpen(url2, function(){
    this.test.pass('Opened 2nd page')
})
.then(function(){

 //these are the tow methodes I try to click on the next button
//this.mouse.click("#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2");
this.thenClick(' div.liens a#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2');
})

.then(function() {
    tst = this.evaluate(function() {
    return
__utils__.getFieldValue(
'[name="ctl0$CONTENU_PAGE$resultSearch$numPageTop"]');
});
})
.run(function() {
    console.log(tst);
    test.done();
});
});

the version of casperjs is 1.1.4 and I'm using the phantom browser

Body:
This normally navigates to 2nd page on my end. Try it, also you can save screenshot for more debugging purposes.

var casper = require('casper').create();
var url = 'https://www.marchespublics.gov.ma/index.php5?page=entreprise.EntrepriseAdvancedSearch&AllCons&EnCours&domaineActivite=1.15';

casper
    .start(url, function() {
        this.echo('Opened page ' + this.evaluate(function() {
            return document.title;
        }), 'INFO');
    })
    .then(function() {
        if (this.exists('a[id="ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2"]')) {
            this.echo('the heading exists');
        } else {
            this.echo('Does not exist');
        }
    })
    .thenClick('div.liens a#ctl0_CONTENU_PAGE_resultSearch_PagerTop_ctl2')
    .wait(5000)
    .then(function() {
        this.capture('exit.png');
    })
    .run();

Hope that helps!


问:
DoctrineCacheBundle: Flush cache via SYmfony route
Over my Symfony project I use the DoctrineCacheBundle and I want when I visit http://example.com/api/cache/flush I want to uncache (flush) any cached key.

The sole reason is because I have applications that visit the url above in order to remove any cached result.

As far I searched the DoctrineCacheBundle uses a command in order to uncache the cached results (as you can see via php ./bin/console list doctrine:cache command):

```<code>Symfony 3.3.12 (kernel: app, env: dev, debug: true)

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -e, --env=ENV         The environment name [default: "dev"]
      --no-debug        Switches off debug mode
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands for the "doctrine:cache" namespace:
  doctrine:cache:clear     Flush a given cache
  doctrine:cache:contains  Check if a cache entry exists
  doctrine:cache:delete    Delete a cache entry
  doctrine:cache:flush     [doctrine:cache:clear] Flush a given cache
  doctrine:cache:stats     Get stats on a given cache provider
</code>```

But how can I do this programmatically?

答:
The best way is to make your own Cache adapter by following one of theese 2 approaches:

Approach 1: Use dedicated manager for uncaching:

```<code>namespace AppBundle\CacheManagers;

use Doctrine\Common\Cache\FlushableCache;

class PurgeAllcachesManager
{

    /**
     * @var FlushableCache
     */
    private $purgeCachingHandler=null;

    public function __construct(FlushableCache $purgeCachingHandler)
    {
        $this->purgeCachingHandler=$purgeCachingHandler;
    }

    /**
     * Method that does all the dirty job to uncache all the keys
     */
    public function uncache()
    {
        $this->purgeCachingHandler->flushAll();
    }
}
</code>```

Approach2: Do as Doctrine does:

```<code>namespace AppBundle\CacheManagers;

use Doctrine\Common\Cache\Cache as CacheHandler;

class PurgeAllcachesManager
{

    /**
     * @var CacheHandler
     */
    private $cacheHandler=null;

    public function __construct(CacheHandler $cacheHandler)
    {
        $this->cacheHandler=$cacheHandler;
    }

    /**
     * Method that does all the dirty job to uncache all the keys
     * @throws Exception
     */
    public function uncacheAllKeys()
    {
        if(!method_exists($this->purgeCachingHandler) ){
          throw new Exception("You cannot empty the cache");
        }
        $this->purgeCachingHandler->flushAll();
    }

    //Yet another methods to handle the cache
}
</code>```

Also have a look on this question for extra info on how to use it.

Body:
Over my Symfony project I use the DoctrineCacheBundle and I want when I visit http://example.com/api/cache/flush I want to uncache (flush) any cached key.

The sole reason is because I have applications that visit the url above in order to remove any cached result.

As far I searched the DoctrineCacheBundle uses a command in order to uncache the cached results (as you can see via php ./bin/console list doctrine:cache command):

Symfony 3.3.12 (kernel: app, env: dev, debug: true)

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -e, --env=ENV         The environment name [default: "dev"]
      --no-debug        Switches off debug mode
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands for the "doctrine:cache" namespace:
  doctrine:cache:clear     Flush a given cache
  doctrine:cache:contains  Check if a cache entry exists
  doctrine:cache:delete    Delete a cache entry
  doctrine:cache:flush     [doctrine:cache:clear] Flush a given cache
  doctrine:cache:stats     Get stats on a given cache provider

But how can I do this programmatically?

Body:
The best way is to make your own Cache adapter by following one of theese 2 approaches:

Approach 1: Use dedicated manager for uncaching:

namespace AppBundle\CacheManagers;

use Doctrine\Common\Cache\FlushableCache;

class PurgeAllcachesManager
{

    /**
     * @var FlushableCache
     */
    private $purgeCachingHandler=null;

    public function __construct(FlushableCache $purgeCachingHandler)
    {
        $this->purgeCachingHandler=$purgeCachingHandler;
    }

    /**
     * Method that does all the dirty job to uncache all the keys
     */
    public function uncache()
    {
        $this->purgeCachingHandler->flushAll();
    }
}

Approach2: Do as Doctrine does:

namespace AppBundle\CacheManagers;

use Doctrine\Common\Cache\Cache as CacheHandler;

class PurgeAllcachesManager
{

    /**
     * @var CacheHandler
     */
    private $cacheHandler=null;

    public function __construct(CacheHandler $cacheHandler)
    {
        $this->cacheHandler=$cacheHandler;
    }

    /**
     * Method that does all the dirty job to uncache all the keys
     * @throws Exception
     */
    public function uncacheAllKeys()
    {
        if(!method_exists($this->purgeCachingHandler) ){
          throw new Exception("You cannot empty the cache");
        }
        $this->purgeCachingHandler->flushAll();
    }

    //Yet another methods to handle the cache
}

Also have a look on this question for extra info on how to use it.


问:
Is URL Authorization enough secure in ASP.NET?
I have categorized my pages in different folders based on some roles and using URL authorization , For example I use the following code in web.config for limiting access to that administrative stuff :

```<code><configuration>
  <system.web>
    <authorization>
      <allow roles="Admin" />
      <allow roles="SuperAdmin" />
      <deny roles="Member" />
      <deny users="?"/>
    </authorization>
  </system.web>
</configuration>
</code>```

My question is : Is that enough secure ?
Or I need to do some other things like : in every events which user want to do administrative stuff  , I should check If they are in appropriate role ?

Edit:
I use "Forms Authentication Provider" and "ASP.NET Membership Provider" for authenticating users and Membership , And as far as i know it's secure

答:
This does only verify, no user is getting any page delivered, which she is not authorized for. Since you are doing a lot of business stuff in your business layer, you want to check the autorization there also.

Most the time you cant avoid having to check several time inside the page also, since the pages often deliver different content, depending if a low right user or a high right user requests them.

In general you will have to decide on your own for every single information you are giving away - is it ok or not? If it not in general a public information: think about, how could you restrict it effectively? The authorization scheme based on the ASP.NET membership classes is a good base to do so.

Body:
I have categorized my pages in different folders based on some roles and using URL authorization , For example I use the following code in web.config for limiting access to that administrative stuff :

<configuration>
  <system.web>
    <authorization>
      <allow roles="Admin" />
      <allow roles="SuperAdmin" />
      <deny roles="Member" />
      <deny users="?"/>
    </authorization>
  </system.web>
</configuration>

My question is : Is that enough secure ?
Or I need to do some other things like : in every events which user want to do administrative stuff  , I should check If they are in appropriate role ?

Edit:
I use "Forms Authentication Provider" and "ASP.NET Membership Provider" for authenticating users and Membership , And as far as i know it's secure


问:
How to use standard evaluation in dplyr for nested functions
In a data frame I want to have a column with a number extracted from another column which name is only given as a variable:

```<code>> name <- 'x'
> df <- data.frame(x = '01abc78')
> mutate_(df, number = as.numeric(substr(name, 6, 7)))
</code>```

But it puts 'NA' values in the new column:

```<code>        x number
1 01abc78     NA
</code>```

Curiously enough

```<code>> mutate_(df, number = substr(name, 6, 7))
</code>```

throws an error:

```<code>Error in parse(text = x)[[1]] : subscript out of bounds
</code>```

Only

```<code>> mutate_(df, number = name)
</code>```

works but does a different thing. Also get() is no solution:

```<code>> mutate(df, number = as.numeric(substr(get(name), 6, 7)))
Error: object 'x' not found
</code>```

Of course I could do

```<code>mutate_(df, number = name) %>% mutate(number = as.numeric(substr(number, 6, 7)
</code>```

but it sure isn't pretty.

How can I fix this to use either only mutate_ or get()?

答:
Using paste can help</p>

```<code>dots = paste0("as.numeric(substr(",name,", 6, 7) )")
mutate_(df, .dots = setNames(dots,"number"))
</code>```

Body:
In a data frame I want to have a column with a number extracted from another column which name is only given as a variable:

> name <- 'x'
> df <- data.frame(x = '01abc78')
> mutate_(df, number = as.numeric(substr(name, 6, 7)))

But it puts 'NA' values in the new column:

        x number
1 01abc78     NA

Curiously enough

> mutate_(df, number = substr(name, 6, 7))

throws an error:

Error in parse(text = x)[[1]] : subscript out of bounds

Only

> mutate_(df, number = name)

works but does a different thing. Also get() is no solution:

> mutate(df, number = as.numeric(substr(get(name), 6, 7)))
Error: object 'x' not found

Of course I could do

mutate_(df, number = name) %>% mutate(number = as.numeric(substr(number, 6, 7)

but it sure isn't pretty.

How can I fix this to use either only mutate_ or get()?

Body:
Using paste can help

dots = paste0("as.numeric(substr(",name,", 6, 7) )")
mutate_(df, .dots = setNames(dots,"number"))


问:
HOw to use Jmeter Variable in REgex?
My regular expression is looks like this "\u003e(\d+)\u003c/td\u003e\u003ctd colName=\"FacilityName\" style=\"text-align:left\"\u003eFAC${JMETERVARIABLE}"

This is not working in Jmeter.Somebody please help me on this

Thanks in Advance

答:
I had the same problem of getting regex to work when the returned text contains '\u003e' etc instead of the normal characters, and in the end I inserted a Beanshell sampler to put the variable ${whatever} together with the rest of the regex string, then passed the entire string as a variable into the regular expression extractor (REE). My jmeter script is on a different machine so please forgive the lack of screenshots.

I got this from another SO answer which i can't now find (do please someone insert the reference if you know it), but in brief it looks something like this (in the large 'Script' window of the Beanshell sampler)

```<code>String s = "your regex phrase ${jmeter variable}";
vars.put("RegPhrase",s);
</code>```

and then in the REE for 'Regular Expression' field you just put ${RegPhrase}

It also has the advantage that the final RegPhrase appears as a variable in the debug PostProcessor, so you can copy and paste it into the RegEx field of the http request output in the view results tree, to test the phrase works with your data, then tweak as required.

Title: HOw to use Jmeter Variable in REgex?

Body:
I had the same problem of getting regex to work when the returned text contains '\u003e' etc instead of the normal characters, and in the end I inserted a Beanshell sampler to put the variable ${whatever} together with the rest of the regex string, then passed the entire string as a variable into the regular expression extractor (REE). My jmeter script is on a different machine so please forgive the lack of screenshots.

I got this from another SO answer which i can't now find (do please someone insert the reference if you know it), but in brief it looks something like this (in the large 'Script' window of the Beanshell sampler)

String s = "your regex phrase ${jmeter variable}";
vars.put("RegPhrase",s);

and then in the REE for 'Regular Expression' field you just put ${RegPhrase}

It also has the advantage that the final RegPhrase appears as a variable in the debug PostProcessor, so you can copy and paste it into the RegEx field of the http request output in the view results tree, to test the phrase works with your data, then tweak as required.


问:
iOS 11 navigationItem.titleView Width Not Set
Seeing a behavior on iOS11 with a navigationItem.titleView where the width of the titleView is not the full width of the screen.

I have a custom view that I set as the titleView. Previous to iOS11 the view would fill the navigation bar area. But iOS 11 it is not resizing to fill the width of the screen.

I've tried setting the frame of the view before setting titleView but no luck. I've tried to force the titleViews superview to layout constraints as well but no luck.

Screenshots attached:

iOS10:

iOS11:

Anyone else experience this?

答:
return UILayoutFittingExpandedSize not helped me, because view was added vertically few more times to fill layout.

The solution was to override intrinsicContentSize in custom view setting width to max screen width:

```<code> - (CGSize)intrinsicContentSize {
    //fills empty space. View will be resized to be smaller, but if it is too small - then it stays too small
    CGRect frame = self.frame;
    frame.size.width = MAX(SCREEN_WIDTH, SCREEN_HEIGHT);
    return frame.size;
}
</code>```

Body:
return UILayoutFittingExpandedSize not helped me, because view was added vertically few more times to fill layout.

The solution was to override intrinsicContentSize in custom view setting width to max screen width:

 - (CGSize)intrinsicContentSize {
    //fills empty space. View will be resized to be smaller, but if it is too small - then it stays too small
    CGRect frame = self.frame;
    frame.size.width = MAX(SCREEN_WIDTH, SCREEN_HEIGHT);
    return frame.size;
}


问:
How can I GROUP_CONCAT values in mySQL with a SEPERATOR "
" but without breaking lines?
When I use in mySQL request...

```<code>GROUP_CONCAT(text SEPARATOR " <br> ") AS texts
</code>```

...my result in my html table is:

```<code>My name is
Fred
My age is 24
My friend's
name is Toni
My job is
teacher
</code>```

But I would need a kind of nowrap result for each element:

```<code>My name is Fred
My age is 24
My friend's name is Toni
My job is teacher
</code>```

I need a solution directly inside the mySQL request:

答:
Ordinarily when using a
 or  element that's too small for its contents, one uses css to enlarge the element or maybe the table column. That seems to be difficult in this case.

If your text is too wide for your column you should change "
 " to "
" to eliminate the extra spacing.

You could try swapping out all the space characters for non-breaking spaces, like so.

```<code>REPLACE(GROUP_CONCAT(text SEPARATOR '<br>'), ' ', ' ' AS texts
</code>```

Title: How can I GROUP_CONCAT values in mySQL with a SEPERATOR "<br>" but without breaking lines?

Body:
When I use in mySQL request...

GROUP_CONCAT(text SEPARATOR "
 ") AS texts

...my result in my html table is:

My name is
Fred
My age is 24
My friend's
name is Toni
My job is
teacher

But I would need a kind of nowrap result for each element:

My name is Fred
My age is 24
My friend's name is Toni
My job is teacher

I need a solution directly inside the mySQL request:

Body:
Ordinarily when using a <div> or <td> element that's too small for its contents, one uses css to enlarge the element or maybe the table column. That seems to be difficult in this case.

If your text is too wide for your column you should change " <br> " to "<br>" to eliminate the extra spacing.

You could try swapping out all the space characters for non-breaking spaces, like so.

REPLACE(GROUP_CONCAT(text SEPARATOR '<br>'), ' ', '&nbsp;' AS texts


问:
Getting out of memory error while reading parquet file in spark submit job
```<code>[Stage 0:>                                                          (0 + 0) / 8]SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[Stage 1:=====================================================>   (43 + 3) / 46]17/11/16 13:11:18 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 54)
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
17/11/16 13:11:18 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[Executor task launch worker-4,5,main]
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
17/11/16 13:11:18 WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 54, localhost): java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
</code>```

This is my code-

```<code>val sqlContext = new SQLContext(sc)
    //sqlContext.setConf("spark.sql.inMemoryColumnarStorage.compressed", "true")
    log.setLevel(Level.INFO)
    val config = HBaseConfiguration.create()
    val newDataDF = sqlContext.read.parquet(file)
    newDataDF.registerTempTable("newDataDF")
    //sqlContext.cacheTable("newDataDF")
    val result = sqlContext.sql("SELECT rec FROM newDataDF")
    val rows = result.map(t => t(0)).collect()
    //val rows = result.map(t => t.getAs[String]("rec"))
</code>```

It throws out of memory at below line
//val rows = result.map(t => t(0)).collect()

Have tried all options of memory tuning and increasing executor/driver memory increase, but nothing seems to work.
Any advise would be greatly appreciated.

答:
Well, by calling collect on your DataFrame, you tell Spark to gather ALL data onto the driver. For larger datasets this will indeed drown the driver and cause OOMs.

Spark is a framework for distributed computing intended to be used on large dataset that will not fit on a single machine. Only in very few cases do you ever want to call collect on a DataFrame and that is when you do debugging (on small datasets) or you know that you dataset has been reduced vastly in size due to some filtering or aggregation transformations.

Body:
[Stage 0:>                                                          (0 + 0) / 8]SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[Stage 1:=====================================================>   (43 + 3) / 46]17/11/16 13:11:18 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 54)
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
17/11/16 13:11:18 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[Executor task launch worker-4,5,main]
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
17/11/16 13:11:18 WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 54, localhost): java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
    at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:44)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:240)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

This is my code-

val sqlContext = new SQLContext(sc)
    //sqlContext.setConf("spark.sql.inMemoryColumnarStorage.compressed", "true")
    log.setLevel(Level.INFO)
    val config = HBaseConfiguration.create()
    val newDataDF = sqlContext.read.parquet(file)
    newDataDF.registerTempTable("newDataDF")
    //sqlContext.cacheTable("newDataDF")
    val result = sqlContext.sql("SELECT rec FROM newDataDF")
    val rows = result.map(t => t(0)).collect()
    //val rows = result.map(t => t.getAs[String]("rec"))

It throws out of memory at below line
//val rows = result.map(t => t(0)).collect()

Have tried all options of memory tuning and increasing executor/driver memory increase, but nothing seems to work.
Any advise would be greatly appreciated.


问:
Native C Dll calling C++/CLI Mixed Mode Dll - Unhandled Exception
I have a Native C Dll that is dynamically loaded by a legacy application.  The intent of this dll is to allow overriding of application behavior based on certain application events.  I have a C# dll that contains functions that I call from the Native C dll through the mixed mode C++/CLI dll to enhance these application events.  The application using this architecture works flawlessly on Windows 2000.

The application also runs on Windows XP but unfortunately the application crashes(Unhandled Exception) once the Native C dll is loaded at application startup.  It appears that it crashes when it is attempting to load the mixed mode dll.  I have removed all dependencies to the mixed mode dll from the Native C dll and the application boots up fine.  But once the dependency is added, the crash occurs.  The code on Windows 2000 is the same as what is used on Windows XP. I don't have access to the application code but do have access to the Native C dll code but can't get the debugger to stop as the crash occurs before intialization has completed. I suspect it is something to do with CLR initialization and OS loader differences but am not certain.  I'm looking for suggestions on how to go about resolving this. I am using VS2005 using the 2.0 Framework.  I appreciate any help you could give.

The exception and stack trace is not real helpful:

Unhandled exception at 0x775125f6 in MyApplication.exe: 0xC0000005: Access violation reading location 0x775125f6.

```<code>775125f6()
user32.dll!7e418734()
[Frames below may be incorrect and/or missing, no symbols loaded for user32.dll]
user32.dll!7e418816()
user32.dll!7e428ea0()
user32.dll!7e42ce7c()
ntdll.dll!7c90e473()
user32.dll!7e42e389()
</code>```

...

答:
Seems to me the symbol is not setup correctly.  You might want to do the following first:

install Debugging Tools for Windows
http://www.microsoft.com/whdc/devtools/debugging/default.mspx

open command prompt, change to the folder, run WinDBG -I

run your program, let it crash, WinDBG will be triggered

type .symfix

type .sympath+

type .loadby sos clr

type !CLRStack -a

type kb

Step 7 and 8 should get you a much more meaningful stack to debug with

Body:
I have a Native C Dll that is dynamically loaded by a legacy application.  The intent of this dll is to allow overriding of application behavior based on certain application events.  I have a C# dll that contains functions that I call from the Native C dll through the mixed mode C++/CLI dll to enhance these application events.  The application using this architecture works flawlessly on Windows 2000.

The application also runs on Windows XP but unfortunately the application crashes(Unhandled Exception) once the Native C dll is loaded at application startup.  It appears that it crashes when it is attempting to load the mixed mode dll.  I have removed all dependencies to the mixed mode dll from the Native C dll and the application boots up fine.  But once the dependency is added, the crash occurs.  The code on Windows 2000 is the same as what is used on Windows XP. I don't have access to the application code but do have access to the Native C dll code but can't get the debugger to stop as the crash occurs before intialization has completed. I suspect it is something to do with CLR initialization and OS loader differences but am not certain.  I'm looking for suggestions on how to go about resolving this. I am using VS2005 using the 2.0 Framework.  I appreciate any help you could give.

The exception and stack trace is not real helpful:

Unhandled exception at 0x775125f6 in MyApplication.exe: 0xC0000005: Access violation reading location 0x775125f6.

775125f6()
user32.dll!7e418734()
[Frames below may be incorrect and/or missing, no symbols loaded for user32.dll]
user32.dll!7e418816()
user32.dll!7e428ea0()
user32.dll!7e42ce7c()
ntdll.dll!7c90e473()
user32.dll!7e42e389()

...


问:
How to see picture of a graph?
does any body know how to show a graph?
by a graph I mean connectivity graph and by showing it I mean see a picture of it while you have it's adj list.

UPD 1 : by see ing a picuter of it I mean does any body know any kjnd of website or app that it get graph adj list as an input and give us picture of that graph as an output ?!

答:
tikz http://graphviz.org/ example http://graphs.grevian.org/example

dot http://www.texample.net/

graph-easy http://search.cpan.org/~tels/Graph-Easy/lib/Graph/Easy.pm

Many other programs
or normal in any script language

Title: How to see picture of a graph?


问:
Where storage node on zookeeper
I would checks value of node which storage on Zookeeper.
Where storage nodes?
How i can see value of specific node?

I use ZooInspector for checks node name which storage on Zookeeper and use save node, but node don't save.

答:
May be it's help for someone.
You can checks node info on log files, they are locate on {zookeekeper_repository}/data/version-2/ in log4j format.

And for checks value of node exists util for Zookeeper - ZooInspector(https://github.com/apache/zookeeper/tree/b79af153d0f98a4f3f3516910ed47234d7b3d74e/src/contrib/zooinspector).

Title: Where storage node on zookeeper


问:
Do children entities count towards the total size of the parent entity in Google Datastore?
Thinking about using Google Datastore in a project since I will use mostly Google Cloud tools. My worries lie within the official limits stated by Google Cloud Datastore.

Maximum size for an entity: 1,048,572 bytes or roughly 1MB.

From my application's point of view, one entity itself would never reach that size anyway. However, I would use the Google Datastore's concept of Ancestor paths in which you may hierarchically nest entities as children of a parent entity.

Tee problem is, one kind of entities will have dozens of direct children (entities) within it. Therefore, that will most likely reach the 1MB size limit.

My question being... does the size of nested child entities count against the parent entity total size... or not?

答:
No, child entities are separate entities so they don't count towards the size of any parent entities.

Child entities are related to parents through their keys, and the datastore backend will do thing differently for child-parent relationships, but that doesn't change the fact that each is its own entity with its own size limit.


问:
Can you use object that is synchronized as a lock
Is this code ok, or should I create a separate lock object? I am asking if there is a potential for some deadlock or something like that. I assume it should work since the language allows it, but just to be sure I would rather understand how it works and why it is ok or why it is not ok.</p>

```<code>var foo = false

fun bar()
{
    synchronized(foo) {
        foo = !foo
    }
}
</code>```

答:
Your code is broken in a way you might not expect. Here's the simplified version of the bytecode it generates, decompiled to Java:

```<code>private static boolean foo;

public static final void bar() {
    Boolean var0 = Boolean.valueOf(foo);
    synchronized(var0) {
        foo = !foo;
    }
}
</code>```

So you're essentially locking on whatever object the valueOf function returns, which in my JRE is either the TRUE or FALSE singletons inside the Boolean class (this is also simplified):

```<code>public class Boolean {
    public static final Boolean TRUE = new Boolean(true);
    public static final Boolean FALSE = new Boolean(false);

    public static Boolean valueOf(boolean var0) {
        return var0 ? TRUE : FALSE;
    }
}
</code>```

Your best bet for certain, runtime independent code is probably to create a separate instance of Any to synchronize on.

Body:
Is this code ok, or should I create a separate lock object? I am asking if there is a potential for some deadlock or something like that. I assume it should work since the language allows it, but just to be sure I would rather understand how it works and why it is ok or why it is not ok.

var foo = false

fun bar()
{
    synchronized(foo) {
        foo = !foo
    }
}

Body:
Your code is broken in a way you might not expect. Here's the simplified version of the bytecode it generates, decompiled to Java:

private static boolean foo;

public static final void bar() {
    Boolean var0 = Boolean.valueOf(foo);
    synchronized(var0) {
        foo = !foo;
    }
}

So you're essentially locking on whatever object the valueOf function returns, which in my JRE is either the TRUE or FALSE singletons inside the Boolean class (this is also simplified):

public class Boolean {
    public static final Boolean TRUE = new Boolean(true);
    public static final Boolean FALSE = new Boolean(false);

    public static Boolean valueOf(boolean var0) {
        return var0 ? TRUE : FALSE;
    }
}

Your best bet for certain, runtime independent code is probably to create a separate instance of Any to synchronize on.


问:
Boxing..do I get it right?

Possible Duplicate:

  What is boxing and unboxing and what are the trade offs?

Hi,
From my understanding:
When I assign data of value type to (reference) type object variable, it gets boxed and the result is not actual reference as it points to copy of the value stored on the heap. Is that right?
Thanks

答:
Well, not quite. (I misread your post to start with.)

The result is a real reference - but it doesn't refer to your original variable. It refers to an object on the heap which contains a copy of the value your variable held when boxing occurred. In particular, changing the value of your variable doesn't change the value in the box:

```<code>int i = 10;
object o = i;
i = 11;
Console.WriteLine(o); // Prints 10, not 11
</code>```

C# doesn't let you have direct access to the value inside the box - you can only get it by unboxing and taking a copy. C++/CLI, on the other hand, allows the value inside the box to be accessed separately, and even changed. (You can still potentially change the value in a box with C# - for example if the value type implements some interface and the interface methods mutate the value.)

Often the reference type which causes boxing is "object" but it could be some interface the value type implements, or just System.ValueType.

Title: Boxing..do I get it right?

Body:
Well, not quite. (I misread your post to start with.)

The result is a real reference - but it doesn't refer to your original variable. It refers to an object on the heap which contains a copy of the value your variable held when boxing occurred. In particular, changing the value of your variable doesn't change the value in the box:

int i = 10;
object o = i;
i = 11;
Console.WriteLine(o); // Prints 10, not 11

C# doesn't let you have direct access to the value inside the box - you can only get it by unboxing and taking a copy. C++/CLI, on the other hand, allows the value inside the box to be accessed separately, and even changed. (You can still potentially change the value in a box with C# - for example if the value type implements some interface and the interface methods mutate the value.)

Often the reference type which causes boxing is "object" but it could be some interface the value type implements, or just System.ValueType.


问:
Docker how to link two containers
I am a newbie to docker, I wanted to run a Java web application which takes  name and mail and stores it in a MySQL dB now my doubt is how can I use a MySQL image and where I need to mention dB config in my app and how to make these app and dB container to run on tomcat server .is it possible to upload this entire image in docker hub to run on another host  which pulls this image and can run this app successfully ..

I want to dockerize  my name and mail application so that I can convert into an image so that any host can pull that image and run the app without any dependencies

答:
you are talking about different things.

The image has its own instance on the port, which you mark it to be opened - it is like virtual machine - kinda like... And Java Spring app are just using this port to contact with your DB (look, how application.properties file should be written) and especially pom.xml, where you also should specify maven dependency for example for liquibase and pass the host url to it.

Have you ever connected to DB via IntelliJ? - retorical question

Here you have at least 4 questions... In my opinion you misunderstand the whole idea.

Read this at the beggining: https://stackoverflow.com/a/2840358/6237822

This is a small thing about docker: cant connect to postgres db - docker swarm - I think you should write one, very similiar for your Mysql

I also encourage you for lecturing: https://docs.docker.com/get-started/

Title: Docker how to link two containers


问:
Bounding boxes around text regions with different font size
I am trying to implement some kind of text detection algorithm and I want to separate the image into regions where each region contains different font size.

As in this image, for example:

Is there any easy way to implement it using python and/or opencv? if so, how?

I did tried googling it but could not find anything useful..
Thanks.

答:
This is an interesting question. There are a few steps that you need to take in order to achieve your goals. I hope you are sufficiently informed of basic computer vision algorithm (knowledge in openCV function helps) to understand the steps i am suggesting.

Group all the words together using morphological dilation process.

Use openCV findcountour function to label all the blobs. This will give you the width and height information of each blob as well.

Here is the tricky part, now that you have data on each blob, try to run a clustering algorithm on the data with the location(x,y) and geometry(width,height) as your features.

Once you cluster them correctly, its a matter of finding the leftmost, rightmost, topmost and bottom data to draw the bounding rect.

I hope this will provide you enough information to start you work. Its is not detailed but i think its enough to guide you.


问:
How to use real quotation (citation) in PHP with variable?
I couldn't find an answer to my question. What I am trying to do, is as follows: I am trying to make sure that "the.. " and "them... " are quoted within this quote. It seems like using a variable makes it hard, though.

```<code>    $title = get_the_title();
    echo '<meta name="description" content= "Blabla some '.title.' here as well, but is it "the '.$title.'" or "them '.$title.'"?" />';
</code>```

But what I see when I check my page source, is that it doesn't take the quotes as intended. I want to make sure that "the [titlevariablehere]" and "them [titlevariablehere]", are actually outputted as citation marks and seen as part of the content.

How can I make this possible? I tried using backslashes, but still the same problem.

答:
The problem here is not PHP, it's becuase the quotes are also the same as your surrounding quotes to define content.

So change the surrounding quotes like this:

```<code> echo '<meta name="description" content=\'Blabla, but is it "the '.$title.'" or "them '.$title.'"?\' />';
</code>```

Body:
I couldn't find an answer to my question. What I am trying to do, is as follows: I am trying to make sure that "the.. " and "them... " are quoted within this quote. It seems like using a variable makes it hard, though.

    $title = get_the_title();
    echo '<meta name="description" content= "Blabla some '.title.' here as well, but is it "the '.$title.'" or "them '.$title.'"?" />';

But what I see when I check my page source, is that it doesn't take the quotes as intended. I want to make sure that "the [titlevariablehere]" and "them [titlevariablehere]", are actually outputted as citation marks and seen as part of the content.

How can I make this possible? I tried using backslashes, but still the same problem.

Body:
The problem here is not PHP, it's becuase the quotes are also the same as your surrounding quotes to define content.

So change the surrounding quotes like this:

 echo '<meta name="description" content=\'Blabla, but is it "the '.$title.'" or "them '.$title.'"?\' />';


问:
Mocking methods from package
I am looking to mock some methods for unit testing. Unfortunately, the code is not structured really well.

```<code>var config = struct { abc *abc }

func Init(arg1) {
    // config.abc = newAbc(arg2, arg3)
}

func UnitTestThis() {
    //some code here

    config.abc.Search(arg4,arg5)

    //code here
}
</code>```

How do I unit test the UnitTestThis function, mocking the results of Search method? I have been trying to create an interface and mock the methods, but been unable to do so.

答:
If the config.abc field is a concrete type (or a pointer to a concrete type), you can't really mock it.

You need some refactoring.

With interface

Best would be to change the type of config.abc to interface type, so in tests you can create your own implementation and assign to it, and you can do whatever you want to in it.

Using a method value on config.abc.Search

Another option is to create a variable of function type to hold the method value of config.abc.Search(), and in tests assign a new value to it, a function literal for example.

This is how it could look like:

```<code>var config = struct{ abc *abc }{}

var searchFunc func(arg4Type, arg5Type)

func Init(arg1) {
    config.abc = newAbc(arg2,arg3)
    searchFunc = config.abc.Search
}

func UnitTestThis() {
    //some code here
    searchFunc(arg4, arg5)
    //code here
}
</code>```

Read more about this here: Is it possible to mock a function imported from a package in golang?

Title: Mocking methods from package

Body:
I am looking to mock some methods for unit testing. Unfortunately, the code is not structured really well.

var config = struct { abc *abc }

func Init(arg1) {
    // config.abc = newAbc(arg2, arg3)
}

func UnitTestThis() {
    //some code here

    config.abc.Search(arg4,arg5)

    //code here
}

How do I unit test the UnitTestThis function, mocking the results of Search method? I have been trying to create an interface and mock the methods, but been unable to do so.

Body:
If the config.abc field is a concrete type (or a pointer to a concrete type), you can't really mock it.

You need some refactoring.

With interface

Best would be to change the type of config.abc to interface type, so in tests you can create your own implementation and assign to it, and you can do whatever you want to in it.

Using a method value on config.abc.Search

Another option is to create a variable of function type to hold the method value of config.abc.Search(), and in tests assign a new value to it, a function literal for example.

This is how it could look like:

var config = struct{ abc *abc }{}

var searchFunc func(arg4Type, arg5Type)

func Init(arg1) {
    config.abc = newAbc(arg2,arg3)
    searchFunc = config.abc.Search
}

func UnitTestThis() {
    //some code here
    searchFunc(arg4, arg5)
    //code here
}

Read more about this here: Is it possible to mock a function imported from a package in golang?


问:
How to get password field value in react's material-ui
I am not able to access the value of <TextField />, if i don't write <input type='password'/> then it works fine, but for this i am getting a TypeError,  'this.refs[this._getRef(...)].getInputNode is not a function'.

```<code> dialogAction(tag,e){

  console.log(this.refs.password);
  console.log(this.refs.password.getValue());
  this.refs.dialog.dismiss();
}

render(){
let self = this;

let row = this.row,col = this.column;
let standardActions = [
  { text: 'Cancel',onTouchTap: this.dialogAction.bind(this,ProductConstants.CANCEL)},
  { text: 'Submit',onTouchTap: this.dialogAction.bind(this,ProductConstants.SUBMIT)}
];

return (
  <div className="ProductRepository">

    <Dialog ref = 'dialog'
      title="Dialog With Standard Actions"
      actions={standardActions}
      actionFocus="submit"
      modal={true}>
      <TextField ref='password'
        hintText="Password"
        floatingLabelText="Password">
        <input type="password" />
      </TextField>
    </Dialog>
    </div>
    );}

   }
</code>```

image below is the console output of the above code.

答:
For material 1.0 and react 16.1.1

Use inputRef

```<code>  <TextField autoFocus={true} inputRef={el => this.fv = el}
        placeholder="Required" size="30"></TextField >
</code>```

To read the value use below line

```<code>console.log(this.fv.value);
</code>```

Body:
I am not able to access the value of <TextField />, if i don't write <input type='password'/> then it works fine, but for this i am getting a TypeError,  'this.refs[this._getRef(...)].getInputNode is not a function'.

 dialogAction(tag,e){

  console.log(this.refs.password);
  console.log(this.refs.password.getValue());
  this.refs.dialog.dismiss();
}

render(){
let self = this;

let row = this.row,col = this.column;
let standardActions = [
  { text: 'Cancel',onTouchTap: this.dialogAction.bind(this,ProductConstants.CANCEL)},
  { text: 'Submit',onTouchTap: this.dialogAction.bind(this,ProductConstants.SUBMIT)}
];

return (

    <Dialog ref = 'dialog'
      title="Dialog With Standard Actions"
      actions={standardActions}
      actionFocus="submit"
      modal={true}>
      <TextField ref='password'
        hintText="Password"
        floatingLabelText="Password">
        <input type="password" />
      </TextField>
    </Dialog>

    );}

   }

image below is the console output of the above code.

Body:
For material 1.0 and react 16.1.1

Use inputRef

  <TextField autoFocus={true} inputRef={el => this.fv = el}
        placeholder="Required" size="30"></TextField >

To read the value use below line

console.log(this.fv.value);


问:
passing a json data object from php to javascript
I has probaly been asked a lot of times and i ve read lots of threads about it, but simple cant figure it out how to pass a json data pbject from php to java script.

I am am trying to get two single-dimensional json data object arrays(?) from a database and merge them into a two dinemsional json data object array which is then further passed to sigma.js for further processing.

Reading data from database:

```<code><?php
require 'sort.php';
$db = db();
$rawNodes = $db->select('node', '*', ['ico_id' => $_GET['id']]);
$rawEdges = $db->select("edge", "*", ["ico_id" => $_GET['id']]);

$nodes = array();
foreach ($rawNodes as $node) {
    array_push($out, $node);
}

$edges = array();
foreach ($rawEdges as $edge) {
    array_push($out, $edge);
}

?>
</code>```

now accessing it in Java Script:

```<code><html>
    <head>
        <title>Dynamic Transaction Visualization </title>
        <style type="text/css">
            #container {
                max-width: 1200px;
                height: 800px;
                margin: auto;
            }
        </style>
        <script src="build/sigma.min.js"></script>
        <script src="build/plugins/sigma.parsers.json.min.js"></script>
        <script src="build/plugins/sigma.renderers.edgeLabels.min.js"></script>
        <script>
            function getData () {
                var gnodes = <?php echo json_encode($nodes); ?>;
                var gedges = <?php echo json_encode($edges); ?>;

                alert(gnodes[0].adress); // <- Doesnt work? Why?

                var g = {nodes: [], edges: []}; //<- i think this sytanx is correct for making a 2 dim json data object array- is it?

                var xpos = 0;
                var ypos = 0;

                var n;

                for (n in gnodes) { //<- Does not iterate through any Elements
                    alert(gnodes[n].adress); //<- Doest work either? Why?
                    xpos = n.block_number - 4400000;
                    ypos += 100;
                    if (ypos > 10000) { ypos = 0 }
                    g.nodes.push(
                        {
                            "id": n.adress,
                            "label": n.adress,
                            "size": 1, //n.size,
                            "x": xpos,
                            "y": ypos
                        }
                    )
                }

                var edgecount = 0;

                var m;
                for (m in gedges) {
                    g.edges.push(
                        {
                            "id": edgecount++,
                            "source": m.node_adress1,
                            "target": m.node_adress2,
                            "label": m.erc20_value,
                            "type": "arrow"
                        }
                    )
                }
                //g.nodes = gnodes; geht nicht benennung nicht passend
                //g.edges = gedges;

                return g;
            }

            function dispGraph() {
                var gdata = getData();
                s = new sigma({
                    graph: gdata,
                    container: 'container',
                    renderer: {
                        container: "container",
                        type: "canvas"
                    },
                    settings: {
                        edgeLabelSize: 'fixed',
                        nodeLabelSize: 'fixed',
                        defaultNodeColor: '#ec5148',
                        maxNodeSize: 15,
                        minNodeSize: 5,
                        minEdgeSize: 4,
                        maxEdgeSize: 4,
                        minArrowSize: 4,
                        //edgeLabelSize: 'fixed',
                        // {string} The opposite power ratio between the font size of the label and
                        // the edge size:
                        // Math.pow(size, -1 / edgeLabelSizePowRatio) * size * defaultEdgeLabelSize
                        edgeLabelSizePowRatio: 1,
                        // {number} The minimum size an edge must have to see its label displayed.
                        edgeLabelThreshold: 1,
                    }
                });
            }
        </script>
    </head>
    <body>
    <div id="container"></div>
        <script>dispGraph()</script>
    </body>
</html>
</code>```

答:
Paul Zheng helped me with the following correcttion

1)

```<code>Change $out to $nodes in array_push()
</code>```

2)

```<code><?php $json = json_encode($data); ?>
<script>
    var json = <?=$json?>;
</script>
</code>```

Thanks everybody for your contribution!

ContentLicense: CC BY-SA 4.0

Body:
I has probaly been asked a lot of times and i ve read lots of threads about it, but simple cant figure it out how to pass a json data pbject from php to java script.

I am am trying to get two single-dimensional json data object arrays(?) from a database and merge them into a two dinemsional json data object array which is then further passed to sigma.js for further processing.

Reading data from database:

<?php
require 'sort.php';
$db = db();
$rawNodes = $db->select('node', '*', ['ico_id' => $_GET['id']]);
$rawEdges = $db->select("edge", "*", ["ico_id" => $_GET['id']]);

$nodes = array();
foreach ($rawNodes as $node) {
    array_push($out, $node);
}

$edges = array();
foreach ($rawEdges as $edge) {
    array_push($out, $edge);
}

?>

now accessing it in Java Script:

        <title>Dynamic Transaction Visualization </title>

Body:
Paul Zheng helped me with the following correcttion

1)

Change $out to $nodes in array_push()

2)

<?php $json = json_encode($data); ?>
<script>
    var json = <?=$json?>;
</script>

Thanks everybody for your contribution!


问:
Get Max value from range (multiple sheets) grouped by Name
I have 3 sheets that have the exact same format

Sheet1

```<code>A           B          C         D
George      10         2         8
Nick        15         89        0
Mike        13         1         50
Lucas       9         -5         12
</code>```

Sheet2

```<code>A           B          C         D
Nick        1          9         5
Mike        1          10        6
George      11         22        5
Lucas       10         5         2
Panos       55         0         1
</code>```

Sheet3

```<code>A            B         C         D
Panos        0         9         1
George       1         2         5
Nick         7         2         1
Lucas        1         5         1
</code>```

I want to query the range {'Sheet1'!A1:D5; 'Sheet2'!A1:D5; 'Sheet3'!A1:D5}

And get something like MAX(Col2:Col4) Group By Col1

Which would return something like:

```<code>George 22
Nick   89
Mike   50
Lucas  12
Panos  55
</code>```

I tried:

```<code>=sort(query({'Sheet1'!A1:D5; 'Sheet2'!A1:D5;'Sheet3'!A1:D5}, "select Col1, MAX(Col2:Col4) Group by Col1 Label MAX(Col2:Col4) '' " ),2, FALSE)
</code>```

and

```<code>=sort(query({'Sheet1'!A1:D5; 'Sheet2'!A1:D5;'Sheet3'!A1:D5}, "select Col1, MAX(MAX(Col2),MAX(Col3), MAX(Col4)) Group by Col1 " ),2, FALSE)
</code>```

Both didn't work. Any ideas?

答:
Please try:</p>

```<code>=query(sort(transpose(query({Sheet1!A1:D5;Sheet2!A1:D5;Sheet3!A1:D5},"select max(Col2), max(Col3), max(Col4) pivot Col1"))),"select Col1, max(Col2) group by Col1 label(Col1) ''")
</code>```

Body:
I have 3 sheets that have the exact same format

Sheet1

A           B          C         D
George      10         2         8
Nick        15         89        0
Mike        13         1         50
Lucas       9         -5         12

Sheet2

A           B          C         D
Nick        1          9         5
Mike        1          10        6
George      11         22        5
Lucas       10         5         2
Panos       55         0         1

Sheet3

A            B         C         D
Panos        0         9         1
George       1         2         5
Nick         7         2         1
Lucas        1         5         1

I want to query the range {'Sheet1'!A1:D5; 'Sheet2'!A1:D5; 'Sheet3'!A1:D5}

And get something like MAX(Col2:Col4) Group By Col1

Which would return something like:

George 22
Nick   89
Mike   50
Lucas  12
Panos  55

I tried:

=sort(query({'Sheet1'!A1:D5; 'Sheet2'!A1:D5;'Sheet3'!A1:D5}, "select Col1, MAX(Col2:Col4) Group by Col1 Label MAX(Col2:Col4) '' " ),2, FALSE)

and

=sort(query({'Sheet1'!A1:D5; 'Sheet2'!A1:D5;'Sheet3'!A1:D5}, "select Col1, MAX(MAX(Col2),MAX(Col3), MAX(Col4)) Group by Col1 " ),2, FALSE)

Both didn't work. Any ideas?

Body:
Please try:

=query(sort(transpose(query({Sheet1!A1:D5;Sheet2!A1:D5;Sheet3!A1:D5},"select max(Col2), max(Col3), max(Col4) pivot Col1"))),"select Col1, max(Col2) group by Col1 label(Col1) ''")