问:
Unchecked cast behavior with and without assignment to variable
Why first line in main does not throw ClassCastException while the second does? </p>

```<code>import java.util.function.Function;

class Scratch {

    static <T> T getSomething(Function<Integer, T> fun) {
        return (T) fun;
    }

    public static void main(String[] args) {
        Scratch.<String>getSomething(x -> "hello");
        String something = Scratch.<String>getSomething(x -> "hello");
    }
}
</code>```

答:
Generics are only a compile time check (read about type erasure). So at runtime your getSomething() method looks similar to this:

```<code>static Object getSomething(Function fun) {
    return fun;
}
</code>```

Now you see clearly that the first line will never throw an exception

```<code>Scratch.getSomething(x -> "hello");
</code>```

because Function is an Object and thus can be returned without a problem.

The second line however will throw one, because it will look similar to this:

```<code>String something = (String) Scratch.getSomething(x -> "hello");
</code>```

A Function is still an Object so it can be returned from the method, but it is not a String and thus, you get your ClassCastException.

The code compiles fine, because you instruct the compiler that you know what you're doing. You will get an Unchecked cast warning though on this line:

```<code> return (T) fun;
</code>```

This warning should be an indicator from the compiler to you, the programmer, that it (the compiler) can't be sure that that cast will succeed.

Body:
Why first line in main does not throw ClassCastException while the second does?

import java.util.function.Function;

class Scratch {

    static <T> T getSomething(Function<Integer, T> fun) {
        return (T) fun;
    }

    public static void main(String[] args) {
        Scratch.<String>getSomething(x -> "hello");
        String something = Scratch.<String>getSomething(x -> "hello");
    }
}

Body:
Generics are only a compile time check (read about type erasure). So at runtime your getSomething() method looks similar to this:

static Object getSomething(Function fun) {
    return fun;
}

Now you see clearly that the first line will never throw an exception

Scratch.getSomething(x -> "hello");

because Function is an Object and thus can be returned without a problem.

The second line however will throw one, because it will look similar to this:

String something = (String) Scratch.getSomething(x -> "hello");

A Function is still an Object so it can be returned from the method, but it is not a String and thus, you get your ClassCastException.

The code compiles fine, because you instruct the compiler that you know what you're doing. You will get an Unchecked cast warning though on this line:

 return (T) fun;

This warning should be an indicator from the compiler to you, the programmer, that it (the compiler) can't be sure that that cast will succeed.


来源: stackexchange

问:
How to add icon to downloadButton in Shiny?
May I add my own icon to downloadButton or downloadBttn (shinyWidgets) in my shiny app?

```<code>downloadBttn("downloadDataxlsx",
      label = "Download .xlsx",
      style = "stretch",
      color = "primary",
      size = "md"))
</code>```

答:
```<code>> downloadButton
function (outputId, label = "Download", class = NULL, ...)
{
    aTag <- tags$a(id = outputId, class = paste("btn btn-default shiny-download-link",
        class), href = "", target = "_blank", download = NA,
        icon("download"), label, ...)
}
<bytecode: 0x000000001a919c58>
<environment: namespace:shiny>
</code>```

This is the function behind the Download button. Just use your own function.

```<code>customDownloadbutton <- function(outputId, label = "Download"){
    tags$a(id = outputId, class = "btn btn-default shiny-download-link", href = "",
           target = "_blank", download = NA, icon("accessible-icon"), label)
}
</code>```

Just insert in icon the icon you want and use the function like the normal downloadbutton

How to use it:

#

```<code># This is a Shiny web application. You can run the application by clicking
# the 'Run App' button above.
#
# Find out more about building applications with Shiny here:
#
#    http://shiny.rstudio.com/
#

customDownloadbutton <- function(outputId, label = "Download"){
    tags$a(id = outputId, class = "btn btn-default shiny-download-link", href = "",
           target = "_blank", download = NA, icon("accessible-icon"), label)
}

library(shiny)

# Define UI for application that draws a histogram
ui <- fluidPage(

   # Application title
   titlePanel("Old Faithful Geyser Data"),

   # Sidebar with a slider input for number of bins
   sidebarLayout(
      sidebarPanel(
         sliderInput("bins",
                     "Number of bins:",
                     min = 1,
                     max = 50,
                     value = 30),
         customDownloadbutton("myDownloadButton")
      ),

      # Show a plot of the generated distribution
      mainPanel(
         plotOutput("distPlot")
      )
   )
)

# Define server logic required to draw a histogram
server <- function(input, output) {

   output$distPlot <- renderPlot({
      # generate bins based on input$bins from ui.R
      x    <- faithful[, 2]
      bins <- seq(min(x), max(x), length.out = input$bins + 1)

      # draw the histogram with the specified number of bins
      hist(x, breaks = bins, col = 'darkgray', border = 'white')
   })
}

# Run the application
shinyApp(ui = ui, server = server)
</code>```

ContentLicense: CC BY-SA 4.0

Body:
May I add my own icon to downloadButton or downloadBttn (shinyWidgets) in my shiny app?

downloadBttn("downloadDataxlsx",
      label = "Download .xlsx",
      style = "stretch",
      color = "primary",
      size = "md"))

Body:
> downloadButton
function (outputId, label = "Download", class = NULL, ...)
{
    aTag <- tags$a(id = outputId, class = paste("btn btn-default shiny-download-link",
        class), href = "", target = "_blank", download = NA,
        icon("download"), label, ...)
}
<bytecode: 0x000000001a919c58>
<environment: namespace:shiny>

This is the function behind the Download button. Just use your own function.

customDownloadbutton <- function(outputId, label = "Download"){
    tags$a(id = outputId, class = "btn btn-default shiny-download-link", href = "",
           target = "_blank", download = NA, icon("accessible-icon"), label)
}

Just insert in icon the icon you want and use the function like the normal downloadbutton

How to use it:

#

# This is a Shiny web application. You can run the application by clicking
# the 'Run App' button above.
#
# Find out more about building applications with Shiny here:
#
#    http://shiny.rstudio.com/
#

customDownloadbutton <- function(outputId, label = "Download"){
    tags$a(id = outputId, class = "btn btn-default shiny-download-link", href = "",
           target = "_blank", download = NA, icon("accessible-icon"), label)
}

library(shiny)

# Define UI for application that draws a histogram
ui <- fluidPage(

   # Application title
   titlePanel("Old Faithful Geyser Data"),

   # Sidebar with a slider input for number of bins
   sidebarLayout(
      sidebarPanel(
         sliderInput("bins",
                     "Number of bins:",
                     min = 1,
                     max = 50,
                     value = 30),
         customDownloadbutton("myDownloadButton")
      ),

      # Show a plot of the generated distribution
      mainPanel(
         plotOutput("distPlot")
      )
   )
)

# Define server logic required to draw a histogram
server <- function(input, output) {

   output$distPlot <- renderPlot({
      # generate bins based on input$bins from ui.R
      x    <- faithful[, 2]
      bins <- seq(min(x), max(x), length.out = input$bins + 1)

      # draw the histogram with the specified number of bins
      hist(x, breaks = bins, col = 'darkgray', border = 'white')
   })
}

# Run the application
shinyApp(ui = ui, server = server)


问:
Implementing search in mvc 4
I have a textbox where user types the string he needs to search. If the user enters only a single word string, then I am able to retrieve the correct data from database but if the user enters a multi-word string then my code fails.

I am using EntityFramework to get the data.

Here is my code to get the data using a single word string.

```<code>public ActionResult SearchResult(string search)
{
   var j = objCon.Mobiles.Where(oh => oh.MobileName.Contains(search) || oh.Description.Contains(search));
   List<Mobiles> prod = new List<Mobiles>();
   foreach (var p in j)
   {
       Mobiles Mob = new Mobiles();
       Mob.Description = p.Description;
       Mob.ImgUrl = p.Url;
       Mob.MobileName = p.MobileName;
       Mob.Price = Convert.ToString(p.Price);
       Mob.SlNo = p.SlNo;
       prod.Add(Mob);
   }

   return View("~/Views/Product/Index.cshtml", prod);
}
</code>```

I tried breaking the string into single word using split but could not get the correct data.

```<code>string str = null;
string[] strArr = null;
int count = 0;
str = //UserInput;
char[] splitchar = { ' ' };
strArr = str.Split(splitchar);
</code>```

答:
I Wrote an Example to Solve your Problem. Hope That You will Be Benefited From The Code.

First Create Mobile Class:

```<code> public class Mobile
        {
            public int Id { get; set; }

            public string Name { get; set; }

            public string Description { get; set; }
        }
</code>```

Next Create Extension method To Check If there is Value:

```<code>public static bool ContainsAny(this string haystack, params string[] needles)
        {
            foreach (var needle in needles)
            {
                if (haystack.Contains(needle))
                    return true;
            }

            return false;
        }
</code>```

Finally Create Main Body Along with Test Data:

```<code>using System;
using System.Collections.Generic;
using System.Linq;

namespace StackOverFlow
{
    static class Program
    {
        static void Main()
        {
            List<Mobile> mobiles = new List<Mobile>
            {
                new Mobile{Id = 1,Name = "samsung galaxy s3",Description = "model"},
                new Mobile{Id = 2,Name = "nokia N67",Description = "nokia n96 time"},
                new Mobile{Id = 3,Name = "iphone 5s",Description = "test"},
                new Mobile{Id = 4,Name = "samsung galaxy packet",Description = "this time"},
                new Mobile{Id = 5,Name = "iphone ipad",Description = "now"},
                new Mobile{Id = 6,Name = "glx c5",Description = "time"},
            };

            string[] search = "galaxy time 5s".Split(' ');

            var result = mobiles.Where(c => c.Name.ContainsAny(search) ||
                                            c.Description.ContainsAny(search)).ToList();

            foreach (var item in result)
            {
                Console.WriteLine(item.Id + "-" + item.Name + "-" + item.Description);
            }

            Console.ReadKey();
        }
</code>```

Title: Implementing search in mvc 4

Body:
I have a textbox where user types the string he needs to search. If the user enters only a single word string, then I am able to retrieve the correct data from database but if the user enters a multi-word string then my code fails.

I am using EntityFramework to get the data.

Here is my code to get the data using a single word string.

public ActionResult SearchResult(string search)
{
   var j = objCon.Mobiles.Where(oh => oh.MobileName.Contains(search) || oh.Description.Contains(search));
   List<Mobiles> prod = new List<Mobiles>();
   foreach (var p in j)
   {
       Mobiles Mob = new Mobiles();
       Mob.Description = p.Description;
       Mob.ImgUrl = p.Url;
       Mob.MobileName = p.MobileName;
       Mob.Price = Convert.ToString(p.Price);
       Mob.SlNo = p.SlNo;
       prod.Add(Mob);
   }

   return View("~/Views/Product/Index.cshtml", prod);
}

I tried breaking the string into single word using split but could not get the correct data.

string str = null;
string[] strArr = null;
int count = 0;
str = //UserInput;
char[] splitchar = { ' ' };
strArr = str.Split(splitchar);

Body:
I Wrote an Example to Solve your Problem. Hope That You will Be Benefited From The Code.

First Create Mobile Class:

 public class Mobile
        {
            public int Id { get; set; }

            public string Name { get; set; }

            public string Description { get; set; }
        }

Next Create Extension method To Check If there is Value:

public static bool ContainsAny(this string haystack, params string[] needles)
        {
            foreach (var needle in needles)
            {
                if (haystack.Contains(needle))
                    return true;
            }

            return false;
        }

Finally Create Main Body Along with Test Data:

using System;
using System.Collections.Generic;
using System.Linq;

namespace StackOverFlow
{
    static class Program
    {
        static void Main()
        {
            List<Mobile> mobiles = new List<Mobile>
            {
                new Mobile{Id = 1,Name = "samsung galaxy s3",Description = "model"},
                new Mobile{Id = 2,Name = "nokia N67",Description = "nokia n96 time"},
                new Mobile{Id = 3,Name = "iphone 5s",Description = "test"},
                new Mobile{Id = 4,Name = "samsung galaxy packet",Description = "this time"},
                new Mobile{Id = 5,Name = "iphone ipad",Description = "now"},
                new Mobile{Id = 6,Name = "glx c5",Description = "time"},
            };

            string[] search = "galaxy time 5s".Split(' ');

            var result = mobiles.Where(c => c.Name.ContainsAny(search) ||
                                            c.Description.ContainsAny(search)).ToList();

            foreach (var item in result)
            {
                Console.WriteLine(item.Id + "-" + item.Name + "-" + item.Description);
            }

            Console.ReadKey();
        }


答:
If the rows from which you want the time difference are not guaranteed to be consecutive, you can do it with this (which works in any case):

```<code>select
  t.id,
  t.date,
  datediff(
    s,
    t.date,
    (select min(date) from tablename where id = t.id and date > t.date)
  ) restdateinseconds
from tablename t
</code>```

See the demo.

Results:

```<code>> id | date                | restdateinseconds
> -: | :------------------ | ----------------:
>  1 | 28/03/2019 10:05:27 |                11
>  1 | 28/03/2019 10:05:38 |               516
>  1 | 28/03/2019 10:14:14 |                 2
>  1 | 28/03/2019 10:14:16 |                30
>  1 | 28/03/2019 10:14:46 |                44
>  1 | 28/03/2019 10:15:30 |
</code>```

Title: rest every two rows of the same column in sql and and convert the result in seconds

Body:
I need help with my Hive table .

   DATE               |    ID     |    RESTDATESINSECONDS

 2019-03-28 10:05:27        1             (2019-03-28 10:05:38)-(2019-03-28 10:05:27)
 2019-03-28 10:05:38        1             (2019-03-28 10:14:14)- (2019-03-28 10:05:38)
 2019-03-28 10:14:14        1             (2019-03-28 10:14:16) -(2019-03-28 10:14:14)
 2019-03-28 10:14:16        1             (2019-03-28 10:14:46) -(2019-03-28 10:14:16)
 2019-03-28 10:14:46        1             (2019-03-28 10:15:30) -(2019-03-28 10:15:30)
 2019-03-28 10:15:30        1                """
     """                    """              """

I have two hundred thousand rows. I would like to return this as:

   DATE               |    ID     |    RESTDATESINSECONDS

 2019-03-28 10:05:27        1             11
 2019-03-28 10:05:38        1             516
 2019-03-28 10:14:14        1             2
 2019-03-28 10:14:16        1             30
 2019-03-28 10:14:46        1             44
 2019-03-28 10:15:30        1             ""

I have tried all kinds of parenthesis and counts, but everything I try throws different errors for syntax! Does anyone have any ideas?

Many Thanks :)

Body:
If the rows from which you want the time difference are not guaranteed to be consecutive, you can do it with this (which works in any case):

select
  t.id,
  t.date,
  datediff(
    s,
    t.date,
    (select min(date) from tablename where id = t.id and date > t.date)
  ) restdateinseconds
from tablename t

See the demo.

Results:

> id | date                | restdateinseconds
> -: | :------------------ | ----------------:
>  1 | 28/03/2019 10:05:27 |                11
>  1 | 28/03/2019 10:05:38 |               516
>  1 | 28/03/2019 10:14:14 |                 2
>  1 | 28/03/2019 10:14:16 |                30
>  1 | 28/03/2019 10:14:46 |                44
>  1 | 28/03/2019 10:15:30 |


问:
Silverlight 3 class library
I'm currently working on a UI development project and it was decided to implement it in Silvelight. I understand that Microsoft developers try to minimize size of the distribution and that's why some (lots) of the classes in regular .NET Framework are not included. Is it expected to see more classes included in Silverlight 3 library?</p>

答: Silverlight 3 no doubt will grow in size.  Silverlight 3 will target mobile devices, so there is no doubt that the package will remain small.  However, people are asking for a lot of new features: printing, WCF additions, 3D, microphone support, hardware acceleration...no doubt its going to grow.  Its just a matter if they integrate this into the core or will your XAP file have to get a lot bigger.</p>

OwnerDisplayName: buckethead

ContentLicense: CC BY-SA 2.5

Title: Silverlight 3 class library

LastEditorDisplayName: buckethead

OwnerDisplayName: Bart Czernicki


问:
Registering a custom element from a chrome extension
I have tried registering custom elements through native javascript and polymer from within a chrome extension with no luck. The issue is documented here.

```<code>Uncaught NotSupportedError: Failed to execute 'registerElement' on 'Document': Registration failed for type 'polymer-element'. Elements cannot be registered from extensions.
</code>```

I am looking for suggestions for using web components in an extension - has anyone done so? My instinct is to register these elements using standard html tags (div, etc...) on the shadow DOM for the time being until it is supported. Any suggestions?

答:
For me it was sufficient to use @webcomponents/custom-elements package in my content.js.
Also webcomponents-sd-ce.js weights about 80kb and custom-elements.min.js only 16kb.

Just install it as a dependancy and import it in the content.js and let your package manager do the rest.

npm i @webcomponents/custom-elements
```
<pre class="lang-js prettyprint-override"><code>import '@webcomponents/custom-elements'
</code>```

This is how I use it in my project popup-tab-switcher.

I believe that all these workarounds are not necessary and custom elements must work for extensions without polyfills. Please vote for the issue in Chrome bugs to draw attention to this flaw.

ContentLicense: CC BY-SA 3.0

Body:
I have tried registering custom elements through native javascript and polymer from within a chrome extension with no luck. The issue is documented here.

Uncaught NotSupportedError: Failed to execute 'registerElement' on 'Document': Registration failed for type 'polymer-element'. Elements cannot be registered from extensions.

I am looking for suggestions for using web components in an extension - has anyone done so? My instinct is to register these elements using standard html tags (div, etc...) on the shadow DOM for the time being until it is supported. Any suggestions?

Body:
For me it was sufficient to use @webcomponents/custom-elements package in my content.js.
Also webcomponents-sd-ce.js weights about 80kb and custom-elements.min.js only 16kb.

Just install it as a dependancy and import it in the content.js and let your package manager do the rest.

npm i @webcomponents/custom-elements

import '@webcomponents/custom-elements'

This is how I use it in my project popup-tab-switcher.

I believe that all these workarounds are not necessary and custom elements must work for extensions without polyfills. Please vote for the issue in Chrome bugs to draw attention to this flaw.


问:
Hive optimization on count distinct multiple columns with groupby
I am working on a hive(1.4-cdh) code optimization on MapReduce, in my project we have used lot of count distinct operation with groupby clause, an example hql is shown below.

```<code>DROP TABLE IF EXISTS testdb.NewTable PURGE;
CREATE TABLE testdb.NewTable AS
SELECT a.* FROM (
SELECT col1,
COUNT(DISTINCT col2) AS col2,
COUNT(DISTINCT col3) AS col3,
COUNT(DISTINCT col4) AS col4,
COUNT(DISTINCT col5) AS col5
FROM BaseTable
GROUP BY col1) a
WHERE  a.col3 > 1 OR a.col4 > 1 OR a.col2 > 1 OR a.col5 > 1;
</code>```

Can you please help me with a better approach on this, to minimize the processing time of query.

Adding explain path for CountDistinct and CollectSet:

CountDistinct Explain Plan:

```<code>OK
STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: BaseTable
            Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
            Select Operator
              expressions: col1 (type: string), col2 (type: decimal(3,0)), col3 (type: string), col4 (type: string), col5 (type: string)
              outputColumnNames: col1, col2, col3, col4, col5
              Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
              Group By Operator
                aggregations: count(DISTINCT col5), count(DISTINCT col2), count(DISTINCT col4), count(DISTINCT col3)
                keys: col1 (type: string), col5 (type: string), col2 (type: decimal(3,0)), col4 (type: string), col3 (type: string)
                mode: hash
                outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
                Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                Reduce Output Operator
                  key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: decimal(3,0)), _col3 (type: string), _col4 (type: string)
                  sort order: +++++
                  Map-reduce partition columns: _col0 (type: string)
                  Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
      Reduce Operator Tree:
        Group By Operator
          aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), count(DISTINCT KEY._col1:2._col0), count(DISTINCT KEY._col1:3._col0)
          keys: KEY._col0 (type: string)
          mode: mergepartial
          outputColumnNames: _col0, _col1, _col2, _col3, _col4
          Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
          Filter Operator
            predicate: ((((_col2 > 1) or (_col3 > 1)) or (_col1 > 1)) or (_col4 > 1)) (type: boolean)
            Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
            File Output Operator
              compressed: false
              Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
              table:
                  input format: org.apache.hadoop.mapred.TextInputFormat
                  output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                  serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink
</code>```

CollectSet Explain Plan:

```<code>OK
STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: BaseTable
            Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
            Select Operator
              expressions: col1 (type: string), col2 (type: decimal(3,0)), col3 (type: string), col4 (type: string), col5 (type: string)
              outputColumnNames: col1, col2, col3, col4, col5
              Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
              Group By Operator
                aggregations: collect_set(col5), collect_set(col2), collect_set(col4), collect_set(col3)
                keys: col1 (type: string)
                mode: hash
                outputColumnNames: _col0, _col1, _col2, _col3, _col4
                Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                Reduce Output Operator
                  key expressions: _col0 (type: string)
                  sort order: +
                  Map-reduce partition columns: _col0 (type: string)
                  Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                  value expressions: _col1 (type: array<string>), _col2 (type: array<decimal(3,0)>), _col3 (type: array<string>), _col4 (type: array<string>)
      Reduce Operator Tree:
        Group By Operator
          aggregations: collect_set(VALUE._col0), collect_set(VALUE._col1), collect_set(VALUE._col2), collect_set(VALUE._col3)
          keys: KEY._col0 (type: string)
          mode: mergepartial
          outputColumnNames: _col0, _col1, _col2, _col3, _col4
          Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
          Select Operator
            expressions: _col0 (type: string), size(_col1) (type: int), size(_col2) (type: int), size(_col3) (type: int), size(_col4) (type: int)
            outputColumnNames: _col0, _col1, _col2, _col3, _col4
            Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
            Filter Operator
              predicate: ((((_col2 > 1) or (_col3 > 1)) or (_col1 > 1)) or (_col4 > 1)) (type: boolean)
              Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
              File Output Operator
                compressed: false
                Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
                table:
                    input format: org.apache.hadoop.mapred.TextInputFormat
                    output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                    serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink
</code>```

答:
Try using collect_set, it will collect distinct values excluding nulls.</p>

```<code>CREATE TABLE testdb.NewTable AS
SELECT a.* FROM (
SELECT col1,
size(collect_set(col2)) AS col2,
size(collect_set(col3)) AS col3,
size(collect_set(col4)) AS col4,
size(collect_set(col5)) AS col5
FROM BaseTable
GROUP BY col1) a
WHERE  a.col3 > 1 OR a.col4 > 1 OR a.col2 > 1 OR a.col5 > 1;
</code>```

Body:
I am working on a hive(1.4-cdh) code optimization on MapReduce, in my project we have used lot of count distinct operation with groupby clause, an example hql is shown below.

DROP TABLE IF EXISTS testdb.NewTable PURGE;
CREATE TABLE testdb.NewTable AS
SELECT a.* FROM (
SELECT col1,
COUNT(DISTINCT col2) AS col2,
COUNT(DISTINCT col3) AS col3,
COUNT(DISTINCT col4) AS col4,
COUNT(DISTINCT col5) AS col5
FROM BaseTable
GROUP BY col1) a
WHERE  a.col3 > 1 OR a.col4 > 1 OR a.col2 > 1 OR a.col5 > 1;

Can you please help me with a better approach on this, to minimize the processing time of query.

Adding explain path for CountDistinct and CollectSet:

CountDistinct Explain Plan:

OK
STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: BaseTable
            Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
            Select Operator
              expressions: col1 (type: string), col2 (type: decimal(3,0)), col3 (type: string), col4 (type: string), col5 (type: string)
              outputColumnNames: col1, col2, col3, col4, col5
              Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
              Group By Operator
                aggregations: count(DISTINCT col5), count(DISTINCT col2), count(DISTINCT col4), count(DISTINCT col3)
                keys: col1 (type: string), col5 (type: string), col2 (type: decimal(3,0)), col4 (type: string), col3 (type: string)
                mode: hash
                outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
                Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                Reduce Output Operator
                  key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: decimal(3,0)), _col3 (type: string), _col4 (type: string)
                  sort order: +++++
                  Map-reduce partition columns: _col0 (type: string)
                  Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
      Reduce Operator Tree:
        Group By Operator
          aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), count(DISTINCT KEY._col1:2._col0), count(DISTINCT KEY._col1:3._col0)
          keys: KEY._col0 (type: string)
          mode: mergepartial
          outputColumnNames: _col0, _col1, _col2, _col3, _col4
          Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
          Filter Operator
            predicate: ((((_col2 > 1) or (_col3 > 1)) or (_col1 > 1)) or (_col4 > 1)) (type: boolean)
            Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
            File Output Operator
              compressed: false
              Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
              table:
                  input format: org.apache.hadoop.mapred.TextInputFormat
                  output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                  serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink

CollectSet Explain Plan:

OK
STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: BaseTable
            Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
            Select Operator
              expressions: col1 (type: string), col2 (type: decimal(3,0)), col3 (type: string), col4 (type: string), col5 (type: string)
              outputColumnNames: col1, col2, col3, col4, col5
              Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
              Group By Operator
                aggregations: collect_set(col5), collect_set(col2), collect_set(col4), collect_set(col3)
                keys: col1 (type: string)
                mode: hash
                outputColumnNames: _col0, _col1, _col2, _col3, _col4
                Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                Reduce Output Operator
                  key expressions: _col0 (type: string)
                  sort order: +
                  Map-reduce partition columns: _col0 (type: string)
                  Statistics: Num rows: 16863109255 Data size: 2613966713222 Basic stats: COMPLETE Column stats: NONE
                  value expressions: _col1 (type: array<string>), _col2 (type: array<decimal(3,0)>), _col3 (type: array<string>), _col4 (type: array<string>)
      Reduce Operator Tree:
        Group By Operator
          aggregations: collect_set(VALUE._col0), collect_set(VALUE._col1), collect_set(VALUE._col2), collect_set(VALUE._col3)
          keys: KEY._col0 (type: string)
          mode: mergepartial
          outputColumnNames: _col0, _col1, _col2, _col3, _col4
          Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
          Select Operator
            expressions: _col0 (type: string), size(_col1) (type: int), size(_col2) (type: int), size(_col3) (type: int), size(_col4) (type: int)
            outputColumnNames: _col0, _col1, _col2, _col3, _col4
            Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
            Filter Operator
              predicate: ((((_col2 > 1) or (_col3 > 1)) or (_col1 > 1)) or (_col4 > 1)) (type: boolean)
              Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
              File Output Operator
                compressed: false
                Statistics: Num rows: 8431554627 Data size: 1306983356533 Basic stats: COMPLETE Column stats: NONE
                table:
                    input format: org.apache.hadoop.mapred.TextInputFormat
                    output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                    serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink

Body:
Try using collect_set, it will collect distinct values excluding nulls.

CREATE TABLE testdb.NewTable AS
SELECT a.* FROM (
SELECT col1,
size(collect_set(col2)) AS col2,
size(collect_set(col3)) AS col3,
size(collect_set(col4)) AS col4,
size(collect_set(col5)) AS col5
FROM BaseTable
GROUP BY col1) a
WHERE  a.col3 > 1 OR a.col4 > 1 OR a.col2 > 1 OR a.col5 > 1;


问:
How to restore administrative templates under the user configuration group policy?
I removed administrative templates under the user configurations group policies while I'm working on. How can I restore it?

答:
I found the souliton. Open the Command Prompt as Administrator and write below line by line;</p>

```<code>RD /S /Q "%WinDir%\System32\GroupPolicyUsers"
RD /S /Q "%WinDir%\System32\GroupPolicy"
gpupdate /force
</code>```

Body:
I found the souliton. Open the Command Prompt as Administrator and write below line by line;

RD /S /Q "%WinDir%\System32\GroupPolicyUsers"
RD /S /Q "%WinDir%\System32\GroupPolicy"
gpupdate /force


问:
Flutter set Container background with Text child
in flutter i want to have simple Text inside Container with background color on container, for example:

unfortunately i can't set or make this screen shot with flutter widgets

```<code>Expanded(
  child: Container(
    margin: EdgeInsets.only(top: 10.0),
    child: ListView(
      children: <Widget>[
        Container(
          padding: EdgeInsets.only(right: 1.0, left: 1.0),
          child: Center(
            child: bottomSheetDashBoardItems(
                widget.dashboardItems),
          ),
        ), //container
      ], //list view children
    ), //list view
  ), //container
), //Expanded

Widget bottomSheetDashBoardItems(List<DashboardItems> dashboardItems) {
  return Wrap(
    children: dashboardItems
        .map((item) => Container(
              width: 75.0,
              height: 75.0,
              margin: EdgeInsets.all(5.0),
              decoration: BoxDecoration(
                  color: Colors.white,
                  borderRadius: BorderRadius.all(Radius.circular(5.0)),
                  boxShadow: [
                    BoxShadow(
                      blurRadius: 0.5,
                      color: Colors.black,
                      offset: Offset(0.0, 0.0),
                    )
                  ]),
              child: Center(
                  child: Stack(
                children: <Widget>[
                  Padding(
                    padding: const EdgeInsets.only(bottom: 12.0),
                    child: Align(
                      alignment: Alignment.center,
                      child: Image.asset(
                        item.icon,
                        width: 40.0,
                        height: 40.0,
                      ),
                    ),
                  ),
                  Align(
                    alignment: Alignment.bottomCenter,
                    child: Container(
                      decoration: BoxDecoration(
                        color: Colors.indigo[400],
                        borderRadius: BorderRadius.only(
                          bottomLeft: Radius.circular(5.0),
                          bottomRight: Radius.circular(5.0),
                        ),
                      ),
                      child: Padding(
                        padding: const EdgeInsets.only(top: 3.0),
                        child: Text(
                          item.title,
                          textAlign: TextAlign.center,
                          style: TextStyle(
                              color: Colors.white,
                              fontFamily: 'IranSansLight',
                              fontSize: 9.0),
                        ),
                      ),
                    ),
                  ),
                ],
              )),
            ))
        .toList());
</code>```

}

result:

答:
```<code>         Align(
                alignment: Alignment.bottomCenter,
                child: Container(
            //      width: 75.0,   <--- remove this width and try
                  height: 19.0,
                  decoration: BoxDecoration(
                    color: Colors.indigo[400],
                    borderRadius: BorderRadius.only(
                      bottomLeft: Radius.circular(5.0),
                      bottomRight: Radius.circular(5.0),
                    ),
                  ),
</code>```

hope work

Body:
in flutter i want to have simple Text inside Container with background color on container, for example:

unfortunately i can't set or make this screen shot with flutter widgets

Expanded(
  child: Container(
    margin: EdgeInsets.only(top: 10.0),
    child: ListView(
      children: <Widget>[
        Container(
          padding: EdgeInsets.only(right: 1.0, left: 1.0),
          child: Center(
            child: bottomSheetDashBoardItems(
                widget.dashboardItems),
          ),
        ), //container
      ], //list view children
    ), //list view
  ), //container
), //Expanded

Widget bottomSheetDashBoardItems(List<DashboardItems> dashboardItems) {
  return Wrap(
    children: dashboardItems
        .map((item) => Container(
              width: 75.0,
              height: 75.0,
              margin: EdgeInsets.all(5.0),
              decoration: BoxDecoration(
                  color: Colors.white,
                  borderRadius: BorderRadius.all(Radius.circular(5.0)),
                  boxShadow: [
                    BoxShadow(
                      blurRadius: 0.5,
                      color: Colors.black,
                      offset: Offset(0.0, 0.0),
                    )
                  ]),
              child: Center(
                  child: Stack(
                children: <Widget>[
                  Padding(
                    padding: const EdgeInsets.only(bottom: 12.0),
                    child: Align(
                      alignment: Alignment.center,
                      child: Image.asset(
                        item.icon,
                        width: 40.0,
                        height: 40.0,
                      ),
                    ),
                  ),
                  Align(
                    alignment: Alignment.bottomCenter,
                    child: Container(
                      decoration: BoxDecoration(
                        color: Colors.indigo[400],
                        borderRadius: BorderRadius.only(
                          bottomLeft: Radius.circular(5.0),
                          bottomRight: Radius.circular(5.0),
                        ),
                      ),
                      child: Padding(
                        padding: const EdgeInsets.only(top: 3.0),
                        child: Text(
                          item.title,
                          textAlign: TextAlign.center,
                          style: TextStyle(
                              color: Colors.white,
                              fontFamily: 'IranSansLight',
                              fontSize: 9.0),
                        ),
                      ),
                    ),
                  ),
                ],
              )),
            ))
        .toList());

}

result:

Body:
         Align(
                alignment: Alignment.bottomCenter,
                child: Container(
            //      width: 75.0,   <--- remove this width and try
                  height: 19.0,
                  decoration: BoxDecoration(
                    color: Colors.indigo[400],
                    borderRadius: BorderRadius.only(
                      bottomLeft: Radius.circular(5.0),
                      bottomRight: Radius.circular(5.0),
                    ),
                  ),

hope work


问:
AttributeError: 'list' object has no attribute 'h3' ( Beautifulsoup )
I am a beginner in Web-scraping and I am following this tutorial ( https://www.dataquest.io/blog/web-scraping-beautifulsoup/ ) to extract movie data, I think that I have badly defined " first_movie " !

this is the code

```<code>  from requests import get
  from bs4 import BeautifulSoup

  first_movie =[]

  url = 'http://www.imdb.com/search/title?
  release_date=2017&sort=num_votes,desc&page=1'
  response = get(url)
  html_soup = BeautifulSoup(response.text, 'html.parser')
  type(html_soup)

  movie_containers = html_soup.find_all('div', class_ = 'lister-item mode-advanced')

  first_name = first_movie.h3.a.text
</code>```

I get this error :

```<code>Traceback (most recent call last):
File "mov1.py", line 13, in <module>
first_name = first_movie.h3.a.text
AttributeError: 'list' object has no attribute 'h3'
</code>```

答:
Try the following code.

```<code>import requests
from bs4 import BeautifulSoup
url = 'https://www.imdb.com/search/title?release_date=2017&sort=num_votes,desc&page=1'
r = requests.get(url, headers = {'User-Agent' : 'Mozilla/5.0'})
soup = BeautifulSoup(r.content, 'html.parser')
items=soup.find_all('h3',class_='lister-item-header')
for item in items:
    print(item.find('a').text)
</code>```

Output:

```<code>Logan
Wonder Woman
Guardians of the Galaxy: Vol. 2
Thor: Ragnarok
Dunkirk
Star Wars: Episode VIII - The Last Jedi
Spider-Man: Homecoming
Get Out
Blade Runner 2049
Baby Driver
It
Three Billboards Outside Ebbing, Missouri
Justice League
The Shape of Water
John Wick: Chapter 2
Coco
Jumanji: Welcome to the Jungle
Beauty and the Beast
Kong: Skull Island
Kingsman: The Golden Circle
Pirates of the Caribbean: Salazar's Revenge
Alien: Covenant
13 Reasons Why
War for the Planet of the Apes
The Greatest Showman
Life
Fast & Furious 8
Murder on the Orient Express
Lady Bird
Ghost in the Shell
King Arthur: Legend of the Sword
Wind River
The Hitman's Bodyguard
Mother!
The Mummy
Call Me by Your Name
Atomic Blonde
The Punisher
Bright
I, Tonya
Valerian and the City of a Thousand Planets
Baywatch
Darkest Hour
American Made
La Casa de Papel
Mindhunter
Transformers: The Last Knight
The Handmaid's Tale
The Lego Batman Movie
The Disaster Artist
</code>```

Body:
I am a beginner in Web-scraping and I am following this tutorial ( https://www.dataquest.io/blog/web-scraping-beautifulsoup/ ) to extract movie data, I think that I have badly defined " first_movie " !

this is the code

  from requests import get
  from bs4 import BeautifulSoup

  first_movie =[]

  url = 'http://www.imdb.com/search/title?
  release_date=2017&sort=num_votes,desc&page=1'
  response = get(url)
  html_soup = BeautifulSoup(response.text, 'html.parser')
  type(html_soup)

  movie_containers = html_soup.find_all('div', class_ = 'lister-item mode-advanced')

  first_name = first_movie.h3.a.text

I get this error :

Traceback (most recent call last):
File "mov1.py", line 13, in <module>
first_name = first_movie.h3.a.text
AttributeError: 'list' object has no attribute 'h3'

Body:
Try the following code.

import requests
from bs4 import BeautifulSoup
url = 'https://www.imdb.com/search/title?release_date=2017&sort=num_votes,desc&page=1'
r = requests.get(url, headers = {'User-Agent' : 'Mozilla/5.0'})
soup = BeautifulSoup(r.content, 'html.parser')
items=soup.find_all('h3',class_='lister-item-header')
for item in items:
    print(item.find('a').text)

Output:

Logan
Wonder Woman
Guardians of the Galaxy: Vol. 2
Thor: Ragnarok
Dunkirk
Star Wars: Episode VIII - The Last Jedi
Spider-Man: Homecoming
Get Out
Blade Runner 2049
Baby Driver
It
Three Billboards Outside Ebbing, Missouri
Justice League
The Shape of Water
John Wick: Chapter 2
Coco
Jumanji: Welcome to the Jungle
Beauty and the Beast
Kong: Skull Island
Kingsman: The Golden Circle
Pirates of the Caribbean: Salazar's Revenge
Alien: Covenant
13 Reasons Why
War for the Planet of the Apes
The Greatest Showman
Life
Fast & Furious 8
Murder on the Orient Express
Lady Bird
Ghost in the Shell
King Arthur: Legend of the Sword
Wind River
The Hitman's Bodyguard
Mother!
The Mummy
Call Me by Your Name
Atomic Blonde
The Punisher
Bright
I, Tonya
Valerian and the City of a Thousand Planets
Baywatch
Darkest Hour
American Made
La Casa de Papel
Mindhunter
Transformers: The Last Knight
The Handmaid's Tale
The Lego Batman Movie
The Disaster Artist


问:
String that ends wih a single underscore character
I am trying to find the correct regular expression that detects string that end with a single underscore, e.g. hello_, this_. So far I have re.match("[aA-zZ](_)", string) but detects strings like, day__, ball__, i.e, with two underscores which i do not want. Any ideas how to get string with a single underscore in the end.

Thanks

答:
You may use

```<code>re.search(r"(?<!_)_$", s)
</code>```

See the regex demo. See the regex graph:

Details

(?<!_) - a negative lookbehind that makes sure there is no _ immediately the left of the current position

_ - an underscore

$ - end of string.

Note you need re.search that looks for a match anywhere inside a string, not re.match that matches only at the start of a string.

Body:
You may use

re.search(r"(?<!_)_$", s)

See the regex demo. See the regex graph:

Details

(?<!_) - a negative lookbehind that makes sure there is no _ immediately the left of the current position

_ - an underscore

$ - end of string.

Note you need re.search that looks for a match anywhere inside a string, not re.match that matches only at the start of a string.


问:
What packages do I need for a working CPAN tool chain under cygwin64?
There are a couple of perl modules I often use on my *nix systems (FreeBSD, Linux, Mac). I always install these from cpan. Earlier this year I needed to use some of these on a 64-bit Windows system, and in about 2 hours I got the then-most-recent Strawberry Perl package installed, and from cpan I installed the modules of my desire the same way I used to do under *nix. Everything worked as expected.

On another 64-bit Windows system I am now attempting to set up a cygwin64 environment instead of installing Strawberry Perl. There are a handful of tools we need on that system, and those are already working fine under cygwin64, so it seems like a good idea to add the new perl functionality to the already existing cygwin64 environment. To test this possibility without messing with the production environment, I am performing trial/error setups on a separate, clean Windows system.

I downloaded the now current cygwin64 installer, and can get all the tools we required in the past installed fine. I can also get perl5 installed. But installing modules via "cpan install" does not work.
Unfortunatelly, I am not really aware of the *nix packages that cpan depends on, because all on my unices (and apparently with Strawberry Perl under Windows too) every dependency is installed automatically, and I never had to care about these.
My cygwin64 environment obviously misses some packages required by the build process run by cpan. But cpan in cygwin64 does a very bad job of communicating what is missing or what is wrong.

I tried installing multiple combinations of clang, gcc, mingw and other developer tools using the cygwin installer, but still could not get "cpan install desired::module" working. Depending on the package mixture I used, I get different errors (or signs of errors).

As a fairly reliable starting point, could you please give me a list of cygwin packages I should most certainly install, in order to have an environment where "cpan install ..." is supposed to succeed.

Addition#1:

If I install only the most basic cygwin packages (zsh, perl) and those that come anyway, I end up with this ...

```<code>keve@cygtest ~
$ perl -v

This is perl 5, version 26, subversion 3 (v5.26.3) built for     x86_64-cygwin-threads-multi
(with 7 registered patches, see perl -V for more detail)

Copyright 1987-2018, Larry Wall

Perl may be copied only under the terms of either the Artistic License or     the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

keve@cygtest ~
$ curl --version
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: [unreleased]
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps     telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL
</code>```

Any of make, gmake, cmake, nmake, gcc, clang results in a "command not found" response.

Starting cpan does this ...

```<code>$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

 ALERT: 'make' is an essential tool for building perl Modules.
Please make sure you have 'make' (or some equivalent) working.

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
</code>```

and that "Press SPACE and ENTER" line keeps filling my screen until I press Ctrl+C.

So, I start the Cygwin installer again, and install the devel/make package. That way I have this ...

```<code>$ make -v
GNU Make 4.2.1
Built for x86_64-unknown-cygwin
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
</code>```

This time there is no warning about "make" missing, but the same "Press SPACE and ENTER" line fills the screen. There is still that "uninitialised $what", and the warning about not having proper permissions to folders. Which I have seen in questions by others, usually fixed by reinstalling the perl package with the most recent one (I have the most recent one).

So, what package do I miss?
Or did I make a wrong choice above by accepting the default?
Please note: I have installed additional devel packages with my earlier attempts. But to keep things simple for now, I only have "perl" and "make".

Addition#2

Following the answers by Håkon and mob, I started the cygwin installer again, and installed the recommended packages. So now I have devel/binutils, devel/clang, devel/cmake, devel/gcc-core, devel/gcc-g++, gccmakedep, and make installed. Also available is curl, which was already present so I did not need to install (perhaps came as a dependency).

Unfortunately, I still get this when starting "cpan":

```<code>$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
</code>```

Regarding the $what warning:
Lines 660 to 670 of Cpan.pm go like this ...

```<code>BEGIN {
my $scalar = '';

sub _hook_into_CPANpm_report
    {
    no warnings 'redefine';

    *CPAN::Shell::myprint = sub {
        my($self,$what) = @_;
        $scalar .= $what;
</code>```

I can get rid of the warning by modifying line 669 to this ...

```<code>$scalar .= ($what // '');
</code>```

... but this does not solve the problem of why $what is undefined anyway. However, I still end up seeing that "Press SPACE and ENTER to disable curl" filling up the screen. And the same happens if I type "manual" and hit Enter instead of accepting the default local::lib option.

答:
Based on your cpan output, you can try use the Cygwin installer to install the following additional packages before you rerun cpan:

ALERT: 'make' is an essential tool for building perl Modules. -->  install make package.

Press SPACE and ENTER to disable curl, install curl package. Also, I think packages gcc and gcc-g++ is something that it is required for many CPAN modules to install. Then libcrypt-devel can be required for SSH related things, and libreadline-devel is nice to have for history recall in the Cygwin terminal window.

Optionally, the GNU MP Bignum library could be installed: libgmp-devel. I'll add more as I get feedback from you :)

Body:
There are a couple of perl modules I often use on my *nix systems (FreeBSD, Linux, Mac). I always install these from cpan. Earlier this year I needed to use some of these on a 64-bit Windows system, and in about 2 hours I got the then-most-recent Strawberry Perl package installed, and from cpan I installed the modules of my desire the same way I used to do under *nix. Everything worked as expected.

On another 64-bit Windows system I am now attempting to set up a cygwin64 environment instead of installing Strawberry Perl. There are a handful of tools we need on that system, and those are already working fine under cygwin64, so it seems like a good idea to add the new perl functionality to the already existing cygwin64 environment. To test this possibility without messing with the production environment, I am performing trial/error setups on a separate, clean Windows system.

I downloaded the now current cygwin64 installer, and can get all the tools we required in the past installed fine. I can also get perl5 installed. But installing modules via "cpan install" does not work.
Unfortunatelly, I am not really aware of the *nix packages that cpan depends on, because all on my unices (and apparently with Strawberry Perl under Windows too) every dependency is installed automatically, and I never had to care about these.
My cygwin64 environment obviously misses some packages required by the build process run by cpan. But cpan in cygwin64 does a very bad job of communicating what is missing or what is wrong.

I tried installing multiple combinations of clang, gcc, mingw and other developer tools using the cygwin installer, but still could not get "cpan install desired::module" working. Depending on the package mixture I used, I get different errors (or signs of errors).

As a fairly reliable starting point, could you please give me a list of cygwin packages I should most certainly install, in order to have an environment where "cpan install ..." is supposed to succeed.

Addition#1:

If I install only the most basic cygwin packages (zsh, perl) and those that come anyway, I end up with this ...

keve@cygtest ~
$ perl -v

This is perl 5, version 26, subversion 3 (v5.26.3) built for     x86_64-cygwin-threads-multi
(with 7 registered patches, see perl -V for more detail)

Copyright 1987-2018, Larry Wall

Perl may be copied only under the terms of either the Artistic License or     the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

keve@cygtest ~
$ curl --version
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: [unreleased]
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps     telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL

Any of make, gmake, cmake, nmake, gcc, clang results in a "command not found" response.

Starting cpan does this ...

$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

 ALERT: 'make' is an essential tool for building perl Modules.
Please make sure you have 'make' (or some equivalent) working.

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl

and that "Press SPACE and ENTER" line keeps filling my screen until I press Ctrl+C.

So, I start the Cygwin installer again, and install the devel/make package. That way I have this ...

$ make -v
GNU Make 4.2.1
Built for x86_64-unknown-cygwin
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl

This time there is no warning about "make" missing, but the same "Press SPACE and ENTER" line fills the screen. There is still that "uninitialised $what", and the warning about not having proper permissions to folders. Which I have seen in questions by others, usually fixed by reinstalling the perl package with the most recent one (I have the most recent one).

So, what package do I miss?
Or did I make a wrong choice above by accepting the default?
Please note: I have installed additional devel packages with my earlier attempts. But to keep things simple for now, I only have "perl" and "make".

Addition#2

Following the answers by Håkon and mob, I started the cygwin installer again, and installed the recommended packages. So now I have devel/binutils, devel/clang, devel/cmake, devel/gcc-core, devel/gcc-g++, gccmakedep, and make installed. Also available is curl, which was already present so I did not need to install (perhaps came as a dependency).

Unfortunately, I still get this when starting "cpan":

$ cpan
Loading internal null logger. Install Log::Log4perl for logging messages
Sorry, we have to rerun the configuration dialog for CPAN.pm due to
some missing parameters. Configuration will be written to
 <</home/Valaki/.cpan/CPAN/MyConfig.pm>>

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes]

Use of uninitialized value $what in concatenation (.) or string at /usr/share/perl5/5.26/App/Cpan.pm line 669, <STDIN> line 1.
 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib]

Press SPACE and ENTER to disable curl
Press SPACE and ENTER to disable curl

Regarding the $what warning:
Lines 660 to 670 of Cpan.pm go like this ...

BEGIN {
my $scalar = '';

sub _hook_into_CPANpm_report
    {
    no warnings 'redefine';

    *CPAN::Shell::myprint = sub {
        my($self,$what) = @_;
        $scalar .= $what;

I can get rid of the warning by modifying line 669 to this ...

$scalar .= ($what // '');

... but this does not solve the problem of why $what is undefined anyway. However, I still end up seeing that "Press SPACE and ENTER to disable curl" filling up the screen. And the same happens if I type "manual" and hit Enter instead of accepting the default local::lib option.


问:
zsh auto suggestion with vscode integrated terminal
Problem:

The zsh auto suggestion is very hard to see.

I tried to

```<code>"terminal.background" : "#000000",
"terminal.foreground" : "#00FD61",
</code>```

change the background and foreground.

How to override the default color of zsh auto-suggestion in vs-code integrated terminal.?

答:
Using </p>

```<code>"terminal.ansiBrightBlack": "#ff0000"
</code>```

Body:
Problem:

The zsh auto suggestion is very hard to see.

I tried to

"terminal.background" : "#000000",
"terminal.foreground" : "#00FD61",

change the background and foreground.

How to override the default color of zsh auto-suggestion in vs-code integrated terminal.?

Body:
Using

"terminal.ansiBrightBlack": "#ff0000"


问:
How to initialise form group with validator but error free initially
I create a new form group with required validator but when a user touched the field, the form control is already marked as invalid at the beginning.

```<code>const group: any = {};
group['subject'] = new FormControl(null, Validators.required);
const formGroup = new FormGroup(group);

<form [formGroup]="formGroup">
    <input matInput [formControlName]="subject" id="subject">
</form>
</code>```

I want to make it invalid only on blur or dirty. How can I prevent this behaviour? Thanks.

答:
You can use reactive forms like this:

```<code>import { Component, OnInit } from '@angular/core';
import { FormBuilder, FormGroup, Validators } from '@angular/forms';

// import custom validator to validate that password and confirm password fields match
import { MustMatch } from './_helpers/must-match.validator';

@Component({
    selector: 'app',
    templateUrl: 'app.component.html'
})

export class AppComponent implements OnInit {
    registerForm: FormGroup;
    submitted = false;

    constructor(private formBuilder: FormBuilder) { }

    ngOnInit() {
        this.registerForm = this.formBuilder.group({
            firstName: ['', Validators.required],
            lastName: ['', Validators.required],
            email: ['', [Validators.required, Validators.email]],
            password: ['', [Validators.required, Validators.minLength(6)]],
            confirmPassword: ['', Validators.required]
        }, {
            validator: MustMatch('password', 'confirmPassword')
        });
    }

    // convenience getter for easy access to form fields
    get f() { return this.registerForm.controls; }

    onSubmit() {
        this.submitted = true;

        // stop here if form is invalid
        if (this.registerForm.invalid) {
            return;
        }

        alert('SUCCESS!! :-)\n\n' + JSON.stringify(this.registerForm.value))
    }
}
</code>```

and html file:

```<code><!-- main app container -->
<div class="jumbotron">
    <div class="container">
        <div class="row">
            <div class="col-md-6 offset-md-3">
                <h3>Angular 7 Reactive Form Validation</h3>
                <form [formGroup]="registerForm" (ngSubmit)="onSubmit()">
                    <div class="form-group">
                        <label>First Name</label>
                        <input type="text" formControlName="firstName" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.firstName.errors }" />
                        <div *ngIf="submitted && f.firstName.errors" class="invalid-feedback">
                            <div *ngIf="f.firstName.errors.required">First Name is required</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Last Name</label>
                        <input type="text" formControlName="lastName" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.lastName.errors }" />
                        <div *ngIf="submitted && f.lastName.errors" class="invalid-feedback">
                            <div *ngIf="f.lastName.errors.required">Last Name is required</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Email</label>
                        <input type="text" formControlName="email" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.email.errors }" />
                        <div *ngIf="submitted && f.email.errors" class="invalid-feedback">
                            <div *ngIf="f.email.errors.required">Email is required</div>
                            <div *ngIf="f.email.errors.email">Email must be a valid email address</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Password</label>
                        <input type="password" formControlName="password" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.password.errors }" />
                        <div *ngIf="submitted && f.password.errors" class="invalid-feedback">
                            <div *ngIf="f.password.errors.required">Password is required</div>
                            <div *ngIf="f.password.errors.minlength">Password must be at least 6 characters</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Confirm Password</label>
                        <input type="password" formControlName="confirmPassword" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.confirmPassword.errors }" />
                        <div *ngIf="submitted && f.confirmPassword.errors" class="invalid-feedback">
                            <div *ngIf="f.confirmPassword.errors.required">Confirm Password is required</div>
                            <div *ngIf="f.confirmPassword.errors.mustMatch">Passwords must match</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <button class="btn btn-primary">Register</button>
                    </div>
                </form>
            </div>
        </div>
    </div>
</div>
</code>```

custom validator:

```<code>import { FormGroup } from '@angular/forms';

// custom validator to check that two fields match
export function MustMatch(controlName: string, matchingControlName: string) {
    return (formGroup: FormGroup) => {
        const control = formGroup.controls[controlName];
        const matchingControl = formGroup.controls[matchingControlName];

        if (matchingControl.errors && !matchingControl.errors.mustMatch) {
            // return if another validator has already found an error on the matchingControl
            return;
        }

        // set error on matchingControl if validation fails
        if (control.value !== matchingControl.value) {
            matchingControl.setErrors({ mustMatch: true });
        } else {
            matchingControl.setErrors(null);
        }
    }
}
</code>```

and here is the full example: link

Body:
I create a new form group with required validator but when a user touched the field, the form control is already marked as invalid at the beginning.

const group: any = {};
group['subject'] = new FormControl(null, Validators.required);
const formGroup = new FormGroup(group);

<form [formGroup]="formGroup">
    <input matInput [formControlName]="subject" id="subject">
</form>

I want to make it invalid only on blur or dirty. How can I prevent this behaviour? Thanks.

Body:
You can use reactive forms like this:

import { Component, OnInit } from '@angular/core';
import { FormBuilder, FormGroup, Validators } from '@angular/forms';

// import custom validator to validate that password and confirm password fields match
import { MustMatch } from './_helpers/must-match.validator';

@Component({
    selector: 'app',
    templateUrl: 'app.component.html'
})

export class AppComponent implements OnInit {
    registerForm: FormGroup;
    submitted = false;

    constructor(private formBuilder: FormBuilder) { }

    ngOnInit() {
        this.registerForm = this.formBuilder.group({
            firstName: ['', Validators.required],
            lastName: ['', Validators.required],
            email: ['', [Validators.required, Validators.email]],
            password: ['', [Validators.required, Validators.minLength(6)]],
            confirmPassword: ['', Validators.required]
        }, {
            validator: MustMatch('password', 'confirmPassword')
        });
    }

    // convenience getter for easy access to form fields
    get f() { return this.registerForm.controls; }

    onSubmit() {
        this.submitted = true;

        // stop here if form is invalid
        if (this.registerForm.invalid) {
            return;
        }

        alert('SUCCESS!! :-)\n\n' + JSON.stringify(this.registerForm.value))
    }
}

and html file:

<!-- main app container -->
<div class="jumbotron">
    <div class="container">
        <div class="row">
            <div class="col-md-6 offset-md-3">
                <h3>Angular 7 Reactive Form Validation</h3>
                <form [formGroup]="registerForm" (ngSubmit)="onSubmit()">
                    <div class="form-group">
                        <label>First Name</label>
                        <input type="text" formControlName="firstName" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.firstName.errors }" />
                        <div *ngIf="submitted && f.firstName.errors" class="invalid-feedback">
                            <div *ngIf="f.firstName.errors.required">First Name is required</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Last Name</label>
                        <input type="text" formControlName="lastName" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.lastName.errors }" />
                        <div *ngIf="submitted && f.lastName.errors" class="invalid-feedback">
                            <div *ngIf="f.lastName.errors.required">Last Name is required</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Email</label>
                        <input type="text" formControlName="email" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.email.errors }" />
                        <div *ngIf="submitted && f.email.errors" class="invalid-feedback">
                            <div *ngIf="f.email.errors.required">Email is required</div>
                            <div *ngIf="f.email.errors.email">Email must be a valid email address</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Password</label>
                        <input type="password" formControlName="password" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.password.errors }" />
                        <div *ngIf="submitted && f.password.errors" class="invalid-feedback">
                            <div *ngIf="f.password.errors.required">Password is required</div>
                            <div *ngIf="f.password.errors.minlength">Password must be at least 6 characters</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <label>Confirm Password</label>
                        <input type="password" formControlName="confirmPassword" class="form-control" [ngClass]="{ 'is-invalid': submitted && f.confirmPassword.errors }" />
                        <div *ngIf="submitted && f.confirmPassword.errors" class="invalid-feedback">
                            <div *ngIf="f.confirmPassword.errors.required">Confirm Password is required</div>
                            <div *ngIf="f.confirmPassword.errors.mustMatch">Passwords must match</div>
                        </div>
                    </div>
                    <div class="form-group">
                        <button class="btn btn-primary">Register</button>
                    </div>
                </form>
            </div>
        </div>
    </div>
</div>

custom validator:

import { FormGroup } from '@angular/forms';

// custom validator to check that two fields match
export function MustMatch(controlName: string, matchingControlName: string) {
    return (formGroup: FormGroup) => {
        const control = formGroup.controls[controlName];
        const matchingControl = formGroup.controls[matchingControlName];

        if (matchingControl.errors && !matchingControl.errors.mustMatch) {
            // return if another validator has already found an error on the matchingControl
            return;
        }

        // set error on matchingControl if validation fails
        if (control.value !== matchingControl.value) {
            matchingControl.setErrors({ mustMatch: true });
        } else {
            matchingControl.setErrors(null);
        }
    }
}

and here is the full example: link


问:
Java based SIP endpoint
I am looking to implement a sip endpoint using java.

I came across jain-sip. But, it's not clear whether it's used to implement a sip client or sip endpoint itself.

答: You should be able to implement any sip endpoint with jain-sip. It is used in user agents, servers, b2buas, proxies, sbcs and registrars. But if you need something beyond the SIP such as media/RTP then you need additional RTP or relay stack.</p>

Title: Java based SIP endpoint


答:
You may use this regex:</p>

```<code>REGEX = re.compile(r'[^_]_$')
REGEX.search(string)`
</code>```

Title: String that ends wih a single underscore character

Body:
I am trying to find the correct regular expression that detects string that end with a single underscore, e.g. hello_, this_. So far I have re.match("[aA-zZ](_)", string) but detects strings like, day__, ball__, i.e, with two underscores which i do not want. Any ideas how to get string with a single underscore in the end.

Thanks

Body:
You may use this regex:

REGEX = re.compile(r'[^_]_$')
REGEX.search(string)`


问:
SQLAlchemy unusual not null failure when appending to a relationship
I'm having a very odd error with SQLAlchemy. I've removed as much of my code as possible to narrow down the issue, and removing any more will cause the error to disappear. I was able to reproduce the issue on another PC with a fresh pip install of SQLAlchemy (on Python 2.7).

If I do any variant of appending something like this:

```<code>python = Application(name='Python')
python.versions.append(ApplicationVersion(version=27))
session.add(python)
session.commit()

#or

python = Application(name='Python')
session.add(python)
session.commit()
python.versions.append(ApplicationVersion(version=27))
session.commit()
</code>```

I get this error (if I don't remove anything else from the code):

```<code>sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: ApplicationVersion.application_id
[SQL: INSERT INTO "ApplicationVersion" (application_id, version_int) VALUES (?, ?)]
[parameters: (None, 27)]
</code>```

However, like I mentioned, it works perfectly if I remove anything. For example, by removing the docstring of this below function, it will assign application_id correctly and work as expected.

```<code>@contextmanager
def Session():
    """Setup session to allow for usage with a context manager."""
    session = _Session()
    yield session
    session.close()
</code>```

I have literally no clue what's going on. As a disclaimer, the other PC I tested is on the same work network, but since I'm testing with sqlite, I can't imagine it being something network based.

Here is the code to reproduce the error (it's a single file joined from multiple ones):

```<code>######### CONNECT.PY #######
import os
from contextlib import contextmanager

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy.orm import sessionmaker

class BaseTable(object):
    """General things to apply to each table.

    Help: https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/mixins.html
    """
    @declared_attr
    def __tablename__(cls):
        """Set the table name to that of the model."""
        return cls.__name__

if 'DATABASE_URL' not in os.environ:
    os.environ['DATABASE_URL'] = 'sqlite://'

Engine = create_engine(os.environ['DATABASE_URL'])

Base = declarative_base(bind=Engine, cls=BaseTable)

_Session = sessionmaker(bind=Base.metadata.bind)

@contextmanager
def Session():
    """Setup session to allow for usage with a context manager."""
    session = _Session()
    yield session
    session.close()

########## MODELS.PY ###########
import time
import os
from sqlalchemy import Column, Integer, SmallInteger, String, Text
from sqlalchemy import ForeignKey, UniqueConstraint, Table, event
from sqlalchemy.ext.hybrid import hybrid_property
from sqlalchemy.orm import backref, relationship, validates

class Category(Base):
    row_id = Column(Integer, primary_key=True)
    name = Column(String(64), nullable=False)
    parent_id = Column(Integer, ForeignKey('Category.row_id'), nullable=True)

    parent = relationship('Category', foreign_keys=parent_id, remote_side=row_id)
    children = relationship('Category')

    __table_args__ = (
        UniqueConstraint('name', 'parent_id', name='unique_name_parent'),
    )

    @hybrid_property
    def fullname(self):
        parent = self.parent
        visited = set()
        chain = [self.name]
        while parent:
            if parent in visited:
                break
            visited.add(parent)
            chain.append(parent.name)
            parent = parent.parent

        return '.'.join(chain[::-1])

    def __init__(self, name, collection, parent=None, creator=None, **kwargs):
        super(Category, self).__init__(name=name, collection=collection, parent=parent, creator=creator, **kwargs)

    def __repr__(self):
        return '<{cls} "{fullname}">'.format(
            cls=self.__class__.__name__,
            fullname=self.fullname,
        )

class Application(Base):
    row_id = Column(Integer, primary_key=True)
    name = Column(String(16), nullable=False)
    versions = relationship('ApplicationVersion', order_by='ApplicationVersion.version_int')

class ApplicationVersion(Base):
    row_id = Column(Integer, primary_key=True)
    application_id = Column(Integer, ForeignKey('Application.row_id'), nullable=False)
    version_int = Column(Integer, nullable=False)

    application = relationship('Application', foreign_keys=application_id)

    __table_args__ = (
        UniqueConstraint('application_id', 'version_int', name='unique_application_version'),
    )

    def __init__(self, version, application=None, **kwargs):
        super(ApplicationVersion, self).__init__(application=application, version_int=version, **kwargs)

    def __repr__(self):
        return '<{cls} "{application} {version}">'.format(
            cls=self.__class__.__name__,
            application=self.application.name,
            version=self.version_int,
        )

    def __eq__(self, num):
        return self.version_int == num

    def __neq__(self, num):
        return self.version_int != num

######## TEST.PY ########
Base.metadata.create_all()

if __name__ == '__main__':
    with Session() as session:

        # Setup programs and versions
        python = Application(name='Python')
        python.versions.append(ApplicationVersion(version=27))
        session.add(python)
        session.commit()

        print python.versions
</code>```

These are various actions that will stop the error:

Remove docstring from BaseTable or Session

Remove if 'DATABASE_URL' not in os.environ:

Replace create_engine(os.environ['DATABASE_URL']) with
create_engine('sqlite://')

Remove Category

Remove relationships from Category

Remove fullname, __init__ or __repr__ from Category

Remove __init__, __repr__, __eq__ or __neq__ from
ApplicationVersion

Any help would be appreciated as it's driving me a little crazy. I can get around the issue by using session.add(ApplicationVersion(python, 27)), but I would like to know what is actually going on here, as I've never seen Python behave like this before.

答:
I found the problem to be with the custom constructor that you've defined on ApplicationVersion:

def __init__(self, version, application=None, **kwargs):
    super(ApplicationVersion, self).__init__(application=application, version_int=version, **kwargs)
```

<p>Specifically, you allow for a default value of <code>None</code> for <code>ApplicationVersion.application</code>. I'm not sure what value this serves to you as the default constructor doesn't require that you pass an explicit value for any of the model's fields, and so if it's not provided it will be <code>None</code> when accessed anyway.</p>

<p>Then in this line in your test:</p>

<pre class="lang-py prettyprint-override"><code>python.versions.append(ApplicationVersion(version=27))
</code>```

... you explicitly create an ApplicationVersion object with application=None due to the constructor, but at the same time, append it to the python.versions collection. The manner in which these relationships resolve foreign key values appears to be inconsistent, so sometimes it's trying to flush with application_id=1 which is the pk of the new Application object, and other times it's trying to flush with application_id=None as the constructor dictates. But application_id is not nullable:

application_id = Column(Integer, ForeignKey('Application.row_id'), nullable=False)
```

<p>... and that is when you get the <code>IntegrityError</code>.</p>

<p>SQLAlchemy must make some distinction between explicitly setting a relationship attribute to <code>None</code> and it never having been set at all, because if you stop setting <code>application</code> to <code>None</code> in your constructor, the problem stops:</p>

<pre class="lang-py prettyprint-override"><code>def __init__(self, version, **kwargs):
    super(ApplicationVersion, self).__init__(version_int=version, **kwargs)
</code>```

I was able to whittle your example down to this generic example (Python 3 I'm sorry, so you'll need to adjust the print calls):

from sqlalchemy import create_engine, Column, Integer
from sqlalchemy import ForeignKey
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker

engine = create_engine('sqlite://')

Base = declarative_base()

Session = sessionmaker(bind=engine)

class Parent(Base):
    __tablename__ = 'parent'
    id = Column(Integer, primary_key=True)
    children = relationship('Child')

class Child(Base):
    __tablename__ = 'child'
    id = Column(Integer, primary_key=True)
    num = Column(Integer)
    parent_id = Column(Integer, ForeignKey('parent.id'), nullable=False)
    parent = relationship('Parent')

    def __init__(self, parent=None, **kwargs):
        super(Child, self).__init__(parent=parent, **kwargs)

if __name__ == '__main__':
    Base.metadata.create_all(engine)
    error_cnt = 0
    success_cnt = 0
    for _ in range(20):
        s = Session()
        try:
            parent = Parent()
            parent.children.append(Child())
            s.add(parent)
            s.commit()
        except IntegrityError:
            error_cnt += 1
        else:
            success_cnt += 1
        finally:
            s.close()
    print('errors', error_cnt)
    print('successes', success_cnt)
```

When you run that, you should get random counts of successes and errors. Then delete Child.__init__() method and it works all the time.

Body:
I'm having a very odd error with SQLAlchemy. I've removed as much of my code as possible to narrow down the issue, and removing any more will cause the error to disappear. I was able to reproduce the issue on another PC with a fresh pip install of SQLAlchemy (on Python 2.7).

If I do any variant of appending something like this:

python = Application(name='Python')
python.versions.append(ApplicationVersion(version=27))
session.add(python)
session.commit()

#or

python = Application(name='Python')
session.add(python)
session.commit()
python.versions.append(ApplicationVersion(version=27))
session.commit()

I get this error (if I don't remove anything else from the code):

sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: ApplicationVersion.application_id
[SQL: INSERT INTO "ApplicationVersion" (application_id, version_int) VALUES (?, ?)]
[parameters: (None, 27)]

However, like I mentioned, it works perfectly if I remove anything. For example, by removing the docstring of this below function, it will assign application_id correctly and work as expected.

@contextmanager
def Session():
    """Setup session to allow for usage with a context manager."""
    session = _Session()
    yield session
    session.close()

I have literally no clue what's going on. As a disclaimer, the other PC I tested is on the same work network, but since I'm testing with sqlite, I can't imagine it being something network based.

Here is the code to reproduce the error (it's a single file joined from multiple ones):

######### CONNECT.PY #######
import os
from contextlib import contextmanager

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy.orm import sessionmaker

class BaseTable(object):
    """General things to apply to each table.

    Help: https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/mixins.html
    """
    @declared_attr
    def __tablename__(cls):
        """Set the table name to that of the model."""
        return cls.__name__

if 'DATABASE_URL' not in os.environ:
    os.environ['DATABASE_URL'] = 'sqlite://'

Engine = create_engine(os.environ['DATABASE_URL'])

Base = declarative_base(bind=Engine, cls=BaseTable)

_Session = sessionmaker(bind=Base.metadata.bind)

@contextmanager
def Session():
    """Setup session to allow for usage with a context manager."""
    session = _Session()
    yield session
    session.close()

########## MODELS.PY ###########
import time
import os
from sqlalchemy import Column, Integer, SmallInteger, String, Text
from sqlalchemy import ForeignKey, UniqueConstraint, Table, event
from sqlalchemy.ext.hybrid import hybrid_property
from sqlalchemy.orm import backref, relationship, validates

class Category(Base):
    row_id = Column(Integer, primary_key=True)
    name = Column(String(64), nullable=False)
    parent_id = Column(Integer, ForeignKey('Category.row_id'), nullable=True)

    parent = relationship('Category', foreign_keys=parent_id, remote_side=row_id)
    children = relationship('Category')

    __table_args__ = (
        UniqueConstraint('name', 'parent_id', name='unique_name_parent'),
    )

    @hybrid_property
    def fullname(self):
        parent = self.parent
        visited = set()
        chain = [self.name]
        while parent:
            if parent in visited:
                break
            visited.add(parent)
            chain.append(parent.name)
            parent = parent.parent

        return '.'.join(chain[::-1])

    def __init__(self, name, collection, parent=None, creator=None, **kwargs):
        super(Category, self).__init__(name=name, collection=collection, parent=parent, creator=creator, **kwargs)

    def __repr__(self):
        return '<{cls} "{fullname}">'.format(
            cls=self.__class__.__name__,
            fullname=self.fullname,
        )

class Application(Base):
    row_id = Column(Integer, primary_key=True)
    name = Column(String(16), nullable=False)
    versions = relationship('ApplicationVersion', order_by='ApplicationVersion.version_int')

class ApplicationVersion(Base):
    row_id = Column(Integer, primary_key=True)
    application_id = Column(Integer, ForeignKey('Application.row_id'), nullable=False)
    version_int = Column(Integer, nullable=False)

    application = relationship('Application', foreign_keys=application_id)

    __table_args__ = (
        UniqueConstraint('application_id', 'version_int', name='unique_application_version'),
    )

    def __init__(self, version, application=None, **kwargs):
        super(ApplicationVersion, self).__init__(application=application, version_int=version, **kwargs)

    def __repr__(self):
        return '<{cls} "{application} {version}">'.format(
            cls=self.__class__.__name__,
            application=self.application.name,
            version=self.version_int,
        )

    def __eq__(self, num):
        return self.version_int == num

    def __neq__(self, num):
        return self.version_int != num

######## TEST.PY ########
Base.metadata.create_all()

if __name__ == '__main__':
    with Session() as session:

        # Setup programs and versions
        python = Application(name='Python')
        python.versions.append(ApplicationVersion(version=27))
        session.add(python)
        session.commit()

        print python.versions

These are various actions that will stop the error:

Remove docstring from BaseTable or Session

Remove if 'DATABASE_URL' not in os.environ:

Replace create_engine(os.environ['DATABASE_URL']) with
create_engine('sqlite://')

Remove Category

Remove relationships from Category

Remove fullname, __init__ or __repr__ from Category

Remove __init__, __repr__, __eq__ or __neq__ from
ApplicationVersion

Any help would be appreciated as it's driving me a little crazy. I can get around the issue by using session.add(ApplicationVersion(python, 27)), but I would like to know what is actually going on here, as I've never seen Python behave like this before.

Body:
I found the problem to be with the custom constructor that you've defined on ApplicationVersion:

def __init__(self, version, application=None, **kwargs):
    super(ApplicationVersion, self).__init__(application=application, version_int=version, **kwargs)

Specifically, you allow for a default value of None for ApplicationVersion.application. I'm not sure what value this serves to you as the default constructor doesn't require that you pass an explicit value for any of the model's fields, and so if it's not provided it will be None when accessed anyway.

Then in this line in your test:

python.versions.append(ApplicationVersion(version=27))

... you explicitly create an ApplicationVersion object with application=None due to the constructor, but at the same time, append it to the python.versions collection. The manner in which these relationships resolve foreign key values appears to be inconsistent, so sometimes it's trying to flush with application_id=1 which is the pk of the new Application object, and other times it's trying to flush with application_id=None as the constructor dictates. But application_id is not nullable:

application_id = Column(Integer, ForeignKey('Application.row_id'), nullable=False)

... and that is when you get the IntegrityError.

SQLAlchemy must make some distinction between explicitly setting a relationship attribute to None and it never having been set at all, because if you stop setting application to None in your constructor, the problem stops:

def __init__(self, version, **kwargs):
    super(ApplicationVersion, self).__init__(version_int=version, **kwargs)

I was able to whittle your example down to this generic example (Python 3 I'm sorry, so you'll need to adjust the print calls):

from sqlalchemy import create_engine, Column, Integer
from sqlalchemy import ForeignKey
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker

engine = create_engine('sqlite://')

Base = declarative_base()

Session = sessionmaker(bind=engine)

class Parent(Base):
    __tablename__ = 'parent'
    id = Column(Integer, primary_key=True)
    children = relationship('Child')

class Child(Base):
    __tablename__ = 'child'
    id = Column(Integer, primary_key=True)
    num = Column(Integer)
    parent_id = Column(Integer, ForeignKey('parent.id'), nullable=False)
    parent = relationship('Parent')

    def __init__(self, parent=None, **kwargs):
        super(Child, self).__init__(parent=parent, **kwargs)

if __name__ == '__main__':
    Base.metadata.create_all(engine)
    error_cnt = 0
    success_cnt = 0
    for _ in range(20):
        s = Session()
        try:
            parent = Parent()
            parent.children.append(Child())
            s.add(parent)
            s.commit()
        except IntegrityError:
            error_cnt += 1
        else:
            success_cnt += 1
        finally:
            s.close()
    print('errors', error_cnt)
    print('successes', success_cnt)

When you run that, you should get random counts of successes and errors. Then delete Child.__init__() method and it works all the time.