问:
When is the (default-variant) PTX instruction `prmt` useful?
PTX has a prmt instruction with many variants. This question regards the default one, which, if formatted as a C/C++ function, would look like this:

```<code>uint32_t prmt(uint32_t a, uint32_t b, uint32_t byte_selectors);
</code>```

and this is what it does (adapted from the official docs):

In the generic form (no mode specified), byte_selectors consists of four 4-bit selection values. The bytes in the two source parameters a and b are numbered from 0 to 7: {b, a} = {{b7, b6, b5, b4}, {b3, b2, b1, b0}}. For each byte in the function's output, a 4-bit selection value is defined.

The 3 lsbs of the selection value specify which of the 8 source bytes should be moved into the target position. The msb defines if the byte value should be copied, or if the sign (msb of the byte) should be replicated over all 8 bits of the target position (sign extend of the byte value); msb=0 means copy the literal value; msb=1 means replicate the sign.

My question: When is this kind of operation useful? What kind of computation can make use of it?

答:
The PTX instruction prmt exposes the functionality of the machine instruction PRMT. The default mode of the prmt instruction is used when none of the special modes .f4e, .b4e, .rc8, .ecl, .ecr, .rc16 is specified.

The default mode has two per-byte sub-modes, controlled by the most significant bit of the 4-bit selector field for each of eight source bytes. The commonly used sub-mode is to have the msb of the selector field zero, which means the destination byte is copied verbatim from the specified source byte. This sub-mode is exposed via a device function intrinsic __byte_perm(), and generally used to extract, insert, and permute bytes or perform bit shifts by multiples of 8. Example usage can be seen in this answer.

The other sub-mode is special, in that instead of copying the entire source byte, it replicates the most significant bit of the specified source byte across the  destination byte. For this, the msb of the selector field needs to be set to one. Programmers must use PTX inline assembly to access this functionality.

I did not design the GPU hardware, so cannot speak to why that sub-mode was implemented. It is generally useful when the msb of each byte serves as a boolean value that one needs to convert into a mask for the whole byte. This in turn is generally useful for byte-wise processing within a 32-bit register. Note that CUDA includes a good many device function intrinsics for such processing, and disassembly will confirm that the msb replication sub-mode of the prmt default mode is used for many of those.

A fully worked example, emulation of the paddsb operation (byte-wise addition with signed saturation), is shown below. Note the use of prmt with msb replication inside masked_sign_to_byte_mask().

```<code>#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

#if (__CUDACC__)
#define __HOST__ __host__
#define __DEVICE__ __device__
#else // __CUDACC__
#define __HOST__
#define __DEVICE__
#endif // __CUDACC__

#define MSB_MASK (0x80808080U)  // mask for msb of each byte

// r = (a ^ b) & ~c
__HOST__ __DEVICE__ uint32_t lop3_14 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0x14;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a ^ b) & ~c;
#endif // __CUDA_ARCH__
    return r;
}

// r = (a ^ b) & c
__HOST__ __DEVICE__ uint32_t lop3_28 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0x28;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a ^ b) & c;
#endif // __CUDA_ARCH__
    return r;
}

// r = a ^ (~b & c)
__HOST__ __DEVICE__ uint32_t lop3_d2 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0xd2;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = a ^ (~b & c);
#endif // __CUDA_ARCH__
    return r;
}

// r = (a & c) | (b & ~c)
__HOST__ __DEVICE__ uint32_t lop3_f4 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0xf4;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a & c) | (b & ~c);
#endif // __CUDA_ARCH__
    return r;
}

__HOST__ __DEVICE__ uint32_t masked_sign_to_byte_mask (uint32_t a)
{
#if (__CUDA_ARCH__ >= 200)
    asm ("prmt.b32 %0,%0,0,0xba98;" : "+r"(a)); // convert MSBs to masks
#else
    a = a & MSB_MASK;
    a = a + a - (a >> 7); // extend MSBs to full byte to create mask
#endif
    return a;
}

__HOST__ __DEVICE__ uint32_t masked_select (uint32_t a, uint32_t b, uint32_t m)
{
#if (__CUDA_ARCH__ >= 500)
    return lop3_f4 (a, b, m);
#elif 0
    return (((a)&(m))|((b)&(~(m))));
#else
    return((((a)^(b))&(m))^(b));
#endif
}

/*
   my_paddsb() performs byte-wise addition with signed saturation. In the
   case of overflow, positive results are clamped at 127, while negative
   results are clamped at -128.
*/
__HOST__ __DEVICE__ uint32_t my_paddsb (uint32_t a, uint32_t b)
{
    uint32_t sum, res, ofl, sga, msk;
    res = (a & ~MSB_MASK) + (b & ~MSB_MASK);
    sum = a ^ b;
    ofl = lop3_14 (res, a, sum); // ofl = (res ^ a) & ~sum
    sga = masked_sign_to_byte_mask (a);  // sign(a)-mask
    msk = masked_sign_to_byte_mask (ofl);// overflow-mask
    res = lop3_d2 (res, ~MSB_MASK, sum); // res = res ^ (MSB_MASK & sum)
    sga = lop3_28 (sga, ~MSB_MASK, msk); // sga = (sga ^ ~MSB_MASK) & msk
    res = masked_select (sga, res, msk); // res = (sga & msk) | (res & ~msk)
    return res;
}

__global__ void kernel (uint32_t a, uint32_t b)
{
    printf ("GPU: %08x\n", my_paddsb (a, b));
}

int main (void)
{
    uint32_t a = 0x12ef70a0;
    uint32_t b = 0x34cd6090;
    kernel<<<1,1>>>(a, b);
    cudaDeviceSynchronize();
    printf ("CPU: %08x\n", my_paddsb (a, b));
    return EXIT_SUCCESS;
}
</code>```

Body:
PTX has a prmt instruction with many variants. This question regards the default one, which, if formatted as a C/C++ function, would look like this:

uint32_t prmt(uint32_t a, uint32_t b, uint32_t byte_selectors);

and this is what it does (adapted from the official docs):

In the generic form (no mode specified), byte_selectors consists of four 4-bit selection values. The bytes in the two source parameters a and b are numbered from 0 to 7: {b, a} = {{b7, b6, b5, b4}, {b3, b2, b1, b0}}. For each byte in the function's output, a 4-bit selection value is defined.

The 3 lsbs of the selection value specify which of the 8 source bytes should be moved into the target position. The msb defines if the byte value should be copied, or if the sign (msb of the byte) should be replicated over all 8 bits of the target position (sign extend of the byte value); msb=0 means copy the literal value; msb=1 means replicate the sign.

My question: When is this kind of operation useful? What kind of computation can make use of it?

Body:
The PTX instruction prmt exposes the functionality of the machine instruction PRMT. The default mode of the prmt instruction is used when none of the special modes .f4e, .b4e, .rc8, .ecl, .ecr, .rc16 is specified.

The default mode has two per-byte sub-modes, controlled by the most significant bit of the 4-bit selector field for each of eight source bytes. The commonly used sub-mode is to have the msb of the selector field zero, which means the destination byte is copied verbatim from the specified source byte. This sub-mode is exposed via a device function intrinsic __byte_perm(), and generally used to extract, insert, and permute bytes or perform bit shifts by multiples of 8. Example usage can be seen in this answer.

The other sub-mode is special, in that instead of copying the entire source byte, it replicates the most significant bit of the specified source byte across the  destination byte. For this, the msb of the selector field needs to be set to one. Programmers must use PTX inline assembly to access this functionality.

I did not design the GPU hardware, so cannot speak to why that sub-mode was implemented. It is generally useful when the msb of each byte serves as a boolean value that one needs to convert into a mask for the whole byte. This in turn is generally useful for byte-wise processing within a 32-bit register. Note that CUDA includes a good many device function intrinsics for such processing, and disassembly will confirm that the msb replication sub-mode of the prmt default mode is used for many of those.

A fully worked example, emulation of the paddsb operation (byte-wise addition with signed saturation), is shown below. Note the use of prmt with msb replication inside masked_sign_to_byte_mask().

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

#if (__CUDACC__)
#define __HOST__ __host__
#define __DEVICE__ __device__
#else // __CUDACC__
#define __HOST__
#define __DEVICE__
#endif // __CUDACC__

#define MSB_MASK (0x80808080U)  // mask for msb of each byte

// r = (a ^ b) & ~c
__HOST__ __DEVICE__ uint32_t lop3_14 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0x14;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a ^ b) & ~c;
#endif // __CUDA_ARCH__
    return r;
}

// r = (a ^ b) & c
__HOST__ __DEVICE__ uint32_t lop3_28 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0x28;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a ^ b) & c;
#endif // __CUDA_ARCH__
    return r;
}

// r = a ^ (~b & c)
__HOST__ __DEVICE__ uint32_t lop3_d2 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0xd2;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = a ^ (~b & c);
#endif // __CUDA_ARCH__
    return r;
}

// r = (a & c) | (b & ~c)
__HOST__ __DEVICE__ uint32_t lop3_f4 (uint32_t a, uint32_t b, uint32_t c)
{
    uint32_t r;
#if (__CUDA_ARCH__ >= 500)
    asm ("lop3.b32 %0,%1,%2,%3,0xf4;\n\t" : "=r"(r) : "r"(a), "r"(b), "r"(c));
#else // __CUDA_ARCH__
    r = (a & c) | (b & ~c);
#endif // __CUDA_ARCH__
    return r;
}

__HOST__ __DEVICE__ uint32_t masked_sign_to_byte_mask (uint32_t a)
{
#if (__CUDA_ARCH__ >= 200)
    asm ("prmt.b32 %0,%0,0,0xba98;" : "+r"(a)); // convert MSBs to masks
#else
    a = a & MSB_MASK;
    a = a + a - (a >> 7); // extend MSBs to full byte to create mask
#endif
    return a;
}

__HOST__ __DEVICE__ uint32_t masked_select (uint32_t a, uint32_t b, uint32_t m)
{
#if (__CUDA_ARCH__ >= 500)
    return lop3_f4 (a, b, m);
#elif 0
    return (((a)&(m))|((b)&(~(m))));
#else
    return((((a)^(b))&(m))^(b));
#endif
}

/*
   my_paddsb() performs byte-wise addition with signed saturation. In the
   case of overflow, positive results are clamped at 127, while negative
   results are clamped at -128.
*/
__HOST__ __DEVICE__ uint32_t my_paddsb (uint32_t a, uint32_t b)
{
    uint32_t sum, res, ofl, sga, msk;
    res = (a & ~MSB_MASK) + (b & ~MSB_MASK);
    sum = a ^ b;
    ofl = lop3_14 (res, a, sum); // ofl = (res ^ a) & ~sum
    sga = masked_sign_to_byte_mask (a);  // sign(a)-mask
    msk = masked_sign_to_byte_mask (ofl);// overflow-mask
    res = lop3_d2 (res, ~MSB_MASK, sum); // res = res ^ (MSB_MASK & sum)
    sga = lop3_28 (sga, ~MSB_MASK, msk); // sga = (sga ^ ~MSB_MASK) & msk
    res = masked_select (sga, res, msk); // res = (sga & msk) | (res & ~msk)
    return res;
}

__global__ void kernel (uint32_t a, uint32_t b)
{
    printf ("GPU: %08x\n", my_paddsb (a, b));
}

int main (void)
{
    uint32_t a = 0x12ef70a0;
    uint32_t b = 0x34cd6090;
    kernel<<<1,1>>>(a, b);
    cudaDeviceSynchronize();
    printf ("CPU: %08x\n", my_paddsb (a, b));
    return EXIT_SUCCESS;
}


来源: stackexchange

问:
What language (or framework) can i use to build auto do some specific task in a website
I want to write a program to do some specific task in a website (ie: Auto order, auto login and post some comments). But i don't know what language or framework can help me do that.
If you know plase help me.</p>

答:
It sounds like you want to automate some user actions on a website, for that Selenium/Webdriver is the best library/framework if you want to do it on a Desktop or Appium if you want to do it on mobile.

Seeing as you are a beginner I would also recommend to use Python as its not only easy to get started with but its also one of the better languages in current times for anything related to automation.

I actually have multiple playlists that teach:

beginner to senior concepts in Selenium

beginner Appium concepts

how to build an advanced framework in Appium

ContentLicense: CC BY-SA 4.0


问:
Threaded console app throwing error, but execution carries on as normal and nothing is logged
I've got a small console application which has a fairly simple task of collecting realtime data from a data server and throwing it in a database. The app makes a TCP connection to a server which then holds the connection open and throws data down the pipe. The app collects it, reformats it and then chucks it in an MSSQL database.

The application is multi-threaded - on startup, I spin up a 'Collector' thread that's in charge of managing the TCP connection to the data source and throwing data messages onto an internal queue. There's a separate 'processor' thread that looks for messages in the queue, pulls them off individually and creates a database insert transaction. The initial startup thread does pretty much nothing at all once the other threads are up and running.

Everything's fairly extensively logged using NLog, both to file and to console.

The issue I'm having is that on occasion, Windows (Server 2008) throws up an error dialog similar (but obviously not the same!) to the one below. When this happens, the console logging pauses so i can see the approximate time the error is thrown. If I click 'Close the program', the app continues running as normal, but the size of the message queue rapidly goes up, as if the Collector thread has been running all along but the processor thread has been blocked. There's nothing recorded in the logs about any exceptions being thrown.

http://img833.imageshack.us/img833/7533/pleaseclosethisprogram.png

Given that I think I'm catching as many exceptions as possible and logging as far as possible, I'm stumped as to what this might be. I originally thought it might be the logging mechanism throwing an exception, but after ripping out log4net and replacing it with Nlog, I'm seeing the same behavior. This is extremely difficult to reproduce given it only happens irregularly, about twice per week. The app is written in C# .NET 4.

If anyone's got any ideas about steps I can take to troubleshoot this issue, that would be hugely appreciated.

edit

Ok, I've got some details from the event log, I'd completely forgotten that .NET exceptions get logged in there as well. here's the data:

```<code>Faulting application name: DataArchiver.exe, version: 1.0.0.0, time stamp: 0x4dd027df
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0x00000000
Fault offset: 0x0039b8f0
Faulting process id: 0x59c
Faulting application start time: 0x01cc13358473b6da
Faulting application path: D:\DataArchiver\DataArchiver.exe
Faulting module path: unknown
Report Id: f352ef11-7ffc-11e0-9cd0-005056bc0006
</code>```

答:
For start, you could make sure that you catch any unhandled exceptions in your app by wrapping the entire Main method in a try/catch block. Also, add an event handler for the AppDomain.CurrentDomain.UnhandledException event. This will catch event thrown by your background threads. Both handlers should log the error and then allow the application to close.

If you have a thread running in background, and its IsBackground property is not set to true, it will keep your application alive. Your main program logic should also take care to notify all threads that they are supposed to finish in case of an exception, user close request or windows shutdown.

ContentLicense: CC BY-SA 3.0

Body:
I've got a small console application which has a fairly simple task of collecting realtime data from a data server and throwing it in a database. The app makes a TCP connection to a server which then holds the connection open and throws data down the pipe. The app collects it, reformats it and then chucks it in an MSSQL database.

The application is multi-threaded - on startup, I spin up a 'Collector' thread that's in charge of managing the TCP connection to the data source and throwing data messages onto an internal queue. There's a separate 'processor' thread that looks for messages in the queue, pulls them off individually and creates a database insert transaction. The initial startup thread does pretty much nothing at all once the other threads are up and running.

Everything's fairly extensively logged using NLog, both to file and to console.

The issue I'm having is that on occasion, Windows (Server 2008) throws up an error dialog similar (but obviously not the same!) to the one below. When this happens, the console logging pauses so i can see the approximate time the error is thrown. If I click 'Close the program', the app continues running as normal, but the size of the message queue rapidly goes up, as if the Collector thread has been running all along but the processor thread has been blocked. There's nothing recorded in the logs about any exceptions being thrown.

http://img833.imageshack.us/img833/7533/pleaseclosethisprogram.png

Given that I think I'm catching as many exceptions as possible and logging as far as possible, I'm stumped as to what this might be. I originally thought it might be the logging mechanism throwing an exception, but after ripping out log4net and replacing it with Nlog, I'm seeing the same behavior. This is extremely difficult to reproduce given it only happens irregularly, about twice per week. The app is written in C# .NET 4.

If anyone's got any ideas about steps I can take to troubleshoot this issue, that would be hugely appreciated.

edit

Ok, I've got some details from the event log, I'd completely forgotten that .NET exceptions get logged in there as well. here's the data:

Faulting application name: DataArchiver.exe, version: 1.0.0.0, time stamp: 0x4dd027df
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0x00000000
Fault offset: 0x0039b8f0
Faulting process id: 0x59c
Faulting application start time: 0x01cc13358473b6da
Faulting application path: D:\DataArchiver\DataArchiver.exe
Faulting module path: unknown
Report Id: f352ef11-7ffc-11e0-9cd0-005056bc0006


问:
LDAP Change base number for uID and gID
So, I want to use LDAP for some servers access and group sudo.

Problem: when creating users in LDAP the uids (and gids) start at 1000. How can i change that base number to 10000?

So when i add an user it would start at 10000 and going up with the next when creating new users.

答: In my case i solved it by modifying phpldamadmin code for both uid and gid.</p>

Title: LDAP Change base number for uID and gID


问:
Suppresing a newline breaks JSX. Why?
```<code>const addForm = <Formik>
  {(props) => (
    <Form>
        (ommited...)
    </Form>
  )}
  </Formik>;
</code>```

That works as expected. To my surprise, this (first newline suppresed) doesn't

```<code>const addForm = <Formik> {(props) => (
    <Form>
        (ommited...)
    </Form>
  )}
  </Formik>;
</code>```

I get Error: React.Children.only expected to receive a single React element child.

Is this expected behaviour? How can I understand it?

答:
You can see the issue a bit more clearly if you look at the transpiled code. The first code produces this

```<code>"use strict";

var addForm = React.createElement(Formik, null, function (props) {
  return React.createElement(Form, null, "(ommited...)");
});
</code>```

And the second code produces this

```<code>"use strict";

var addForm = React.createElement(Formik, null, " ", function (props) {
  return React.createElement(Form, null, "(ommited...)");
});
</code>```

The difference is that in the second case there's an extra " " as a child. That means the Formik component has two children, which isn't allowed.

So where's that extra space in the original code? It's the space between <Formik> and {(props) => (. If you delete that, you'll get the same behavior between both versions. The first example of code only has whitespace at the beginning of the lines, which gets ignored when transpiling, but whitespace in the middle of a line is significant.

Title: Suppresing a newline breaks JSX. Why?

Body:
const addForm = <Formik>
  {(props) => (
    <Form>
        (ommited...)
    </Form>
  )}
  </Formik>;

That works as expected. To my surprise, this (first newline suppresed) doesn't

const addForm = <Formik> {(props) => (
    <Form>
        (ommited...)
    </Form>
  )}
  </Formik>;

I get Error: React.Children.only expected to receive a single React element child.

Is this expected behaviour? How can I understand it?

Body:
You can see the issue a bit more clearly if you look at the transpiled code. The <a href="https://babeljs.io/repl/#?browsers=&build=&builtIns=false&spec=false&loose=false&code_lz=MYewdgzgLgBAhgEwQMRAJwLYwLwwDyqYCWA1gHwwBQMMA3gBQAOaIjEAlDhfdTfoRjK8-feiAwYiUAKYIAdAvbD8AegFCa7AL688a9JPIBuIA&debug=false&forceAllTransforms=false&shippedProposals=false&circleciRepo=&evaluate=false&fileSize=false&timeTravel=false&sourceType=module&lineWrap=true&presets=es2015%2Creact%2Cstage-2&prettier=false&targets=&version=7.8.4&externalPlugins=" rel="nofollow noreferrer">first code produces this

"use strict";

var addForm = React.createElement(Formik, null, function (props) {
  return React.createElement(Form, null, "(ommited...)");
});

And <a href="https://babeljs.io/repl/#?browsers=&build=&builtIns=false&spec=false&loose=false&code_lz=MYewdgzgLgBAhgEwQMRAJwLYwLwwDyqYCWA1gHwwDeAFAA5oi0QCUOF1AUDN_oRmVx5DqIDBiJQApggB0c5oO54A9HwHdmAX0Eq-pMgG4gA&debug=false&forceAllTransforms=false&shippedProposals=false&circleciRepo=&evaluate=false&fileSize=false&timeTravel=false&sourceType=module&lineWrap=true&presets=es2015%2Creact%2Cstage-2&prettier=false&targets=&version=7.8.4&externalPlugins=" rel="nofollow noreferrer">the second code produces this

"use strict";

var addForm = React.createElement(Formik, null, " ", function (props) {
  return React.createElement(Form, null, "(ommited...)");
});

The difference is that in the second case there's an extra " " as a child. That means the Formik component has two children, which isn't allowed.

So where's that extra space in the original code? It's the space between <Formik> and {(props) => (. If you delete that, you'll get the same behavior between both versions. The first example of code only has whitespace at the beginning of the lines, which gets ignored when transpiling, but whitespace in the middle of a line is significant.


问:
How to deal with changes to ios 13 fonts?
I'm developing my first pet-project app. It's almost ready for release, but the recent ios 13 release has changed all the fonts. I can see there was an issue filed on Github (https://github.com/flutter/flutter/issues/43998).

My code is pretty standard:

```<code>Widget build(BuildContext context) {
  return Text(
    'Find out which bins to put out and when your collection day is',
    style: TextStyle(
      fontWeight: FontWeight.bold,
      fontSize: 16.0,
    ),
    textAlign: TextAlign.center,
  );
}
</code>```

But the results are like this: https://imgur.com/a/u2Esgym

How are people dealing with this?

答:
So, it looks like there is another Github issue to address this
https://github.com/flutter/flutter/issues/49492</p>

Body:
I'm developing my first pet-project app. It's almost ready for release, but the recent ios 13 release has changed all the fonts. I can see there was an issue filed on Github (https://github.com/flutter/flutter/issues/43998).

My code is pretty standard:

Widget build(BuildContext context) {
  return Text(
    'Find out which bins to put out and when your collection day is',
    style: TextStyle(
      fontWeight: FontWeight.bold,
      fontSize: 16.0,
    ),
    textAlign: TextAlign.center,
  );
}

But the results are like this: https://imgur.com/a/u2Esgym

How are people dealing with this?


问:
How to improve cubejs preagregations creation process? (its taking too long to build preaggs even with partitionGranularity)
We are having trouble with the preaggregations creation performance. We currently have specific filters for the data for each one of our clients, and we generate different cubes for each one of them by extending a base cube (called Metrics) and defining a segment that represents those filters.

To summarize, we have a Metrics base cube, and we generate dynamic cubes MetricsA, MetricsB, MetricsC for clients A, B, C. Each one of these cubes has a segment that we call z, which contains a specific SQL query for each of our clients. The data to build the segment is retrieved from our API using asyncModule, and then we extend the Metrics cube to generate all the clients specific cubes by overriding the z segment with the client's filter.
By doing this, when a client queries the cube service, the data retrieved will come from their specific cube, with the data already filtered (by the enforced z segment).

This Metrics cube is built by joining large tables, so we also added a partitionGranularity (monthly) to reduce the size of the preaggregations, but they are still taking too long to build (> 10 minutes).

We need to edit the specific query that the cube service submits to create the preaggregation tables, so we only keep the rows with the z segment = 1 (because that is the relevant data), or at least we want to be able to rearrange/modify the query to improve performance. Which is the best place to do such changes? or what is the recommended practice to intervene this process?

答:
There're two approaches you can use to leverage pre-aggregations in multi-tenant environments.

Override sql for each customer cube such as OrdersC1, OrdersC2, etc. In this case all pre-aggregations defined in base Orders cube will be inherited. Each customer cube will have it's own set of pre-aggregations. It means if there're N customers and M pre-aggregations then N * M pre-aggregation tables should be built which can be costly in some scenarios.

cube(`Orders`, {
  sql: `SELECT * FROM orders`,

  preAggregations: {
    date: {
      type: `rollup`,
      measureReferences: [someMeasure],
      dimensionReferences: [someDimension],
      timeDimensionReference: date,
      granularity: `month`
    },
    // ...
  }
});

cube(`OrdersC1`, {
  extends: Orders,
  sql: `SELECT * FROM orders WHERE customer_id = 'C1'`,
});
```

<ol start="2">
<li>Use tenant field as a dimension of rollup. Every segment can be converted to dimension which provides an opportunity to use single rollup table for all customers. To route requests to right tenant data queryTransformer can be used.</li>
</ol>

<pre class="lang-js prettyprint-override"><code>cube(`Orders`, {
  sql: `SELECT * FROM orders`,

  // ...

  dimensions: {
    // ...

    customerId: {
      sql: `customer_id`,
      type: `string`
    }
  },

  preAggregations: {
    date: {
      type: `rollup`,
      measureReferences: [someMeasure],
      dimensionReferences: [customerId, someDimension],
      timeDimensionReference: date,
      granularity: `month`
    },

    // ...
  }
});
</code>```

Body:
There're two approaches you can use to leverage pre-aggregations in multi-tenant environments.

Override sql for each customer cube such as OrdersC1, OrdersC2, etc. In this case all pre-aggregations defined in base Orders cube will be inherited. Each customer cube will have it's own set of pre-aggregations. It means if there're N customers and M pre-aggregations then N * M pre-aggregation tables should be built which can be costly in some scenarios.

cube(`Orders`, {
  sql: `SELECT * FROM orders`,

  preAggregations: {
    date: {
      type: `rollup`,
      measureReferences: [someMeasure],
      dimensionReferences: [someDimension],
      timeDimensionReference: date,
      granularity: `month`
    },
    // ...
  }
});

cube(`OrdersC1`, {
  extends: Orders,
  sql: `SELECT * FROM orders WHERE customer_id = 'C1'`,
});

Use tenant field as a dimension of rollup. Every segment can be converted to dimension which provides an opportunity to use single rollup table for all customers. To route requests to right tenant data queryTransformer can be used.

cube(`Orders`, {
  sql: `SELECT * FROM orders`,

  // ...

  dimensions: {
    // ...

    customerId: {
      sql: `customer_id`,
      type: `string`
    }
  },

  preAggregations: {
    date: {
      type: `rollup`,
      measureReferences: [someMeasure],
      dimensionReferences: [customerId, someDimension],
      timeDimensionReference: date,
      granularity: `month`
    },

    // ...
  }
});


问:
How to choose HugePage size
From what I understand we need an entry in a page table in order to access physical memory in RAM. So if I have 60TB of data in an Oracle database and 125GB RAM, what is the optimal size and number of HugePages?

Oracle docs [1] recommend allowing memlock to lock "at least 90 percent of current RAM".

If I allocate 1GB (1048576KB) page entry size, does it mean that I need to statically allocate 90% of 125GB as follows,

112 000 000 KB / 1 048 576 KB ~= 106 pages

Or for 2MB page entry,

112 000 000 KB / (2 MB * 1024 KB / 1MB) ~= 54687 pages

My biggest confusion is that if page table entries take up 90% of RAM where does an actual data go?

答: The size of the database is irrelevant.  You want to configure enough hugepages to hold your SGA (or "all" of your SGAs if running multiple instance of Oracle on the same server).  Probably the best calculator script (and the one I use) is My Oracle Support Doc ID 401749.1.  Of course you have to have a paid support contract to get there.  Lacking that, there is a script and description at https://oracle-base.com/articles/linux/configuring-huge-pages-for-oracle-on-linux-64. On review, I see that your cited doc also contains a script for recommendation of vm.nr_hugepages=value in /etc/sysctl.conf.  All in all, I'd just with your cited doc, since you already have it.  As far as your question of "where does the actual data go?"  It goes on the disk.  Until such time as it is read, in which case it goes into the data buffers in memory. Sounds like you need to review 'Instance Architecture' in the Concepts Manual. </p>

Title: How to choose HugePage size


问:
Mahout runs out of heap space
I am running NaiveBayes on a set of tweets using Mahout. Two files, one 100 MB and one 300 MB. I changed JAVA_HEAP_MAX to JAVA_HEAP_MAX=-Xmx2000m ( earlier it was 1000). But even then,  mahout ran for a few hours ( 2 to be precise) before it complained of heap space error.   What should i do to  resolve ?

Some more info if it helps : I am running on a single node, my laptop infact and it has 3GB of RAM (only) .

Thanks.

EDIT: I ran it the third time with <1/2 of the data that i used the first time ( first time i used 5.5 million tweets, second i used 2million ) and i still got a heap space problem. I am posting the complete error for completion purposes :

```<code>17 May, 2011 2:16:22 PM
 org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO:  map 50% reduce 0%

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:62)
    at java.lang.StringBuilder.<init>(StringBuilder.java:85)
    at org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1283)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1251)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureDriver.runJob(BayesFeatureDriver.java:63)
    at org.apache.mahout.classifier.bayes.mapreduce.bayes.BayesDriver.runJob(BayesDriver.java:44)
    at org.apache.mahout.classifier.bayes.TrainClassifier.trainNaiveBayes(TrainClassifier.java:54)
    at org.apache.mahout.classifier.bayes.TrainClassifier.main(TrainClassifier.java:162)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:184)
17 May, 2011 7:14:53 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
WARNING: job_local_0001
java.lang.OutOfMemoryError: Java heap space
    at java.lang.String.substring(String.java:1951)
    at java.lang.String.subSequence(String.java:1984)
    at java.util.regex.Pattern.split(Pattern.java:1019)
    at java.util.regex.Pattern.split(Pattern.java:1076)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureMapper.map(BayesFeatureMapper.java:78)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureMapper.map(BayesFeatureMapper.java:46)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
</code>```

And i am posting the part of the bin/mahout script that i changed :
 Original :

```<code>JAVA=$JAVA_HOME/bin/java
            JAVA_HEAP_MAX=-Xmx1000m

if [ "$MAHOUT_HEAPSIZE" != "" ]; then
  #echo "run with heapsize $MAHOUT_HEAPSIZE"
  JAVA_HEAP_MAX="-Xmx""$MAHOUT_HEAPSIZE""m"
  #echo $JAVA_HEAP_MAX
fi
</code>```

Modified :

```<code>JAVA=$JAVA_HOME/bin/java
 JAVA_HEAP_MAX=-Xmx2000m

if [ "$MAHOUT_HEAPSIZE" != "" ]; then
  #echo "run with heapsize $MAHOUT_HEAPSIZE"
  JAVA_HEAP_MAX="-Xmx""$MAHOUT_HEAPSIZE""m"
  #echo $JAVA_HEAP_MAX
fi
</code>```

答:
Did you modify the heap size for the hadoop environment or the mahout one? See if this query on mahout list helps. From personal experience, I can suggest that you reduce the data size that you are trying to process. Whenever I tried to execute the Bayes classifier on my laptop, after running for a few hours, the heap space would get exhausted.

I'd suggest that you run this off EC2. I think the basic S3/EC2 option is free for usage.

Title: Mahout runs out of heap space

Body:
I am running NaiveBayes on a set of tweets using Mahout. Two files, one 100 MB and one 300 MB. I changed JAVA_HEAP_MAX to JAVA_HEAP_MAX=-Xmx2000m ( earlier it was 1000). But even then,  mahout ran for a few hours ( 2 to be precise) before it complained of heap space error.   What should i do to  resolve ?

Some more info if it helps : I am running on a single node, my laptop infact and it has 3GB of RAM (only) .

Thanks.

EDIT: I ran it the third time with <1/2 of the data that i used the first time ( first time i used 5.5 million tweets, second i used 2million ) and i still got a heap space problem. I am posting the complete error for completion purposes :

17 May, 2011 2:16:22 PM
 org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO:  map 50% reduce 0%

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:62)
    at java.lang.StringBuilder.<init>(StringBuilder.java:85)
    at org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1283)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1251)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureDriver.runJob(BayesFeatureDriver.java:63)
    at org.apache.mahout.classifier.bayes.mapreduce.bayes.BayesDriver.runJob(BayesDriver.java:44)
    at org.apache.mahout.classifier.bayes.TrainClassifier.trainNaiveBayes(TrainClassifier.java:54)
    at org.apache.mahout.classifier.bayes.TrainClassifier.main(TrainClassifier.java:162)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:184)
17 May, 2011 7:14:53 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
WARNING: job_local_0001
java.lang.OutOfMemoryError: Java heap space
    at java.lang.String.substring(String.java:1951)
    at java.lang.String.subSequence(String.java:1984)
    at java.util.regex.Pattern.split(Pattern.java:1019)
    at java.util.regex.Pattern.split(Pattern.java:1076)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureMapper.map(BayesFeatureMapper.java:78)
    at org.apache.mahout.classifier.bayes.mapreduce.common.BayesFeatureMapper.map(BayesFeatureMapper.java:46)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

And i am posting the part of the bin/mahout script that i changed :
 Original :

JAVA=$JAVA_HOME/bin/java
            JAVA_HEAP_MAX=-Xmx1000m

if [ "$MAHOUT_HEAPSIZE" != "" ]; then
  #echo "run with heapsize $MAHOUT_HEAPSIZE"
  JAVA_HEAP_MAX="-Xmx""$MAHOUT_HEAPSIZE""m"
  #echo $JAVA_HEAP_MAX
fi

Modified :

JAVA=$JAVA_HOME/bin/java
 JAVA_HEAP_MAX=-Xmx2000m

if [ "$MAHOUT_HEAPSIZE" != "" ]; then
  #echo "run with heapsize $MAHOUT_HEAPSIZE"
  JAVA_HEAP_MAX="-Xmx""$MAHOUT_HEAPSIZE""m"
  #echo $JAVA_HEAP_MAX
fi


问:
How can I install openMP on my new MacBook Pro (with Mac OS Catalina)?
I installed Xcode (and also the command line tools) but terminal says (when I'm compiling):

```<code>gcc -o task -fopenmp task.c
clang: error: unsupported option '-fopenmp'
</code>```

I tried to install openmp via brew but people say that it's not available anymore on homebrew, they suggest to try

```<code>brew instal llvm
</code>```

But I get the same error.
I tried also in the boneyard

```<code>brew install homebrew/boneyard/clang-omp
</code>```

but the repository doesn't exist anymore.

Could you help me?
I just need to learn openMP, I don't think that is so difficult to install...

Thank you!

Kind regards,

Nico

答:
To build with OpenMP support, you need to make sure you're not invoking Apple's clang from Xcode. Even if you install llvm or gcc via brew, you should try gcc -v and clang -v in your terminal session. Both likely invoke Apple's version.

You can use GNU gcc or LLVM; both are available via brew. If you use LLVM, you'll also need to install libomp.

GNU gcc (currently at version 9)

```<code>brew install gcc
gcc-9 -o task -fopenmp task.c
</code>```

Note that you invoke this version of gcc explicitly by suffixing gcc with a - and the major version number, e.g.: gcc-9

LLVM

Brew installs LLVM as keg-only so that it doesn't conflict with Apple's version. You'll therefore need to make sure you invoke the right clang. You'll also need to specify where the libomp library is located.

```<code>brew install llvm libomp
`brew --prefix llvm`/bin/clang -L`brew --prefix`/lib -o task -fopenmp task.c
</code>```

Body:
I installed Xcode (and also the command line tools) but terminal says (when I'm compiling):

gcc -o task -fopenmp task.c
clang: error: unsupported option '-fopenmp'

I tried to install openmp via brew but people say that it's not available anymore on homebrew, they suggest to try

brew instal llvm

But I get the same error.
I tried also in the boneyard

brew install homebrew/boneyard/clang-omp

but the repository doesn't exist anymore.

Could you help me?
I just need to learn openMP, I don't think that is so difficult to install...

Thank you!

Kind regards,

Nico

Body:
To build with OpenMP support, you need to make sure you're not invoking Apple's clang from Xcode. Even if you install llvm or gcc via brew, you should try gcc -v and clang -v in your terminal session. Both likely invoke Apple's version.

You can use GNU gcc or LLVM; both are available via brew. If you use LLVM, you'll also need to install libomp.

GNU gcc (currently at version 9)

brew install gcc
gcc-9 -o task -fopenmp task.c

Note that you invoke this version of gcc explicitly by suffixing gcc with a - and the major version number, e.g.: gcc-9

LLVM

Brew installs LLVM as keg-only so that it doesn't conflict with Apple's version. You'll therefore need to make sure you invoke the right clang. You'll also need to specify where the libomp library is located.

brew install llvm libomp
`brew --prefix llvm`/bin/clang -L`brew --prefix`/lib -o task -fopenmp task.c


问:
I have a vector that has pointers to elements from 2 subtypes accessed via an interface. Can't seem to re-arrange the vector
I'm working on a really over engineered matrix calculator for school and hit a brick wall with the very last function I need. The full version of the transpose() is supposed to re-arrange my matrix and to switch columns and rows around.

I posted this question earlier with a snippet but was asked to write a minimum reproducable sample. This isn't very "minimum" but I tried.

So. I have a std::vector<std::unique_ptr<Interface>> that can hold objects of types Sub1 and Sub2.

In the full version I have a class that bypasses Interface and works fine since it only holds Sub2 objects.

However I can't seem to get it to work when there are multiple types in the vector. Nothing I've tried in the past few hours has done anything.

```<code>#include <iostream>
#include "main.h"
#include <memory>
#include <utility>
#include <vector>
#include <sstream>

//this is the one that won't work
Super Super::transpose() const {
    Super newMatrix = *this;
    int i = 1;
    for(auto& elem : elements){
        *elem = *newMatrix.elements[i];
        i--;
    }
return *this;
}

int main() {
    std::vector<std::unique_ptr<Interface>> newMatrix;
    newMatrix.push_back(std::unique_ptr<Sub1>(new Sub1('x')));
    newMatrix.push_back(std::unique_ptr<Sub2>(new Sub2(1)));
    Super mtx(std::move(newMatrix),2 );
    std::cout << mtx << std::endl;
    mtx.transpose();
    std::cout << mtx;
    return 0;
}

//busywork from here on
Super::Super(std::vector<std::unique_ptr<Interface>> matrix, int pN){
    elements = std::move(matrix);
    n = pN;
}

std::unique_ptr<Interface> Sub1::clone() const {
    char newVal = val;
    return std::unique_ptr<Sub1>(new Sub1(newVal));
}

std::unique_ptr<Interface> Sub2::clone() const {
    int newVal = i;
    return std::unique_ptr<Sub2>(new Sub2(newVal));
}

Sub1::Sub1(char pVal) {
    val = pVal;
}

Sub2::Sub2(int pI) {
    i = pI;
}

std::string Sub1::toString() const {
    std::stringstream ss;
    ss << val;
    return ss.str();
}

std::string Sub2::toString() const {
    std::stringstream ss;
    ss << i;
    return ss.str();
}

std::ostream &operator<<(std::ostream &os, const Super &matrix) {
    os << "[";
    int i = 0;
    for (auto &elem : matrix.elements) {
        os << elem->toString();
        if (i < matrix.n - 1) {
            os << ",";
        }
        i++;
    }
    os << "]";
    return os;
}

Super::Super(const Super &matrix) {
    n = matrix.n;
    for (auto &elem : matrix.elements) {
        std::unique_ptr<Interface> newElem = elem->clone();
        elements.push_back(std::move(newElem));
    }
}
</code>```

And heres the header

```<code>#ifndef EXAMPLE_MAIN_H
#define EXAMPLE_MAIN_H

#include <memory>
#include <vector>

class Interface{
public:
    virtual ~Interface() = default;
    virtual std::unique_ptr<Interface> clone() const = 0;
    virtual std::string toString() const = 0;
};

class Super{
private:
    std::vector<std::unique_ptr<Interface>> elements;
    int n = 2;
public:
    Super(std::vector<std::unique_ptr<Interface>> elements, int n);
    Super(const Super &matrix);
    Super transpose() const;
    friend std::ostream &operator<<(std::ostream &os, const Super &matrix);
};

class Sub1: public Interface{
private:
    char val;
public:
    Sub1(char pVal);
    std::string toString() const override;
    std::unique_ptr<Interface> clone() const override;
};

class Sub2: public Interface{
private:
    int i;
public:
    Sub2(int pI);
    std::string toString() const override;
    std::unique_ptr<Interface> clone() const override;
};

std::ostream &operator<<(std::ostream &os, const Super &matrix);

#endif //EXAMPLE_MAIN_H
</code>```

答:
Super Super::transpose() const;

This looks like a function that shouldn't change *this when called, but should return a transposed version of *this.

Example:

```<code>#include <algorithm>

Super Super::transpose() const {
    Super newMatrix = *this;
    std::reverse(newMatrix.elements.begin(), newMatrix.elements.end());
    return newMatrix;
}
</code>```

If you'd like to transpose *this in-place, change it to:

Super& Super::transpose();

```<code>Super& Super::transpose() {
    std::reverse(elements.begin(), elements.end());
    return *this;
}
</code>```

If you must create a temporary object first, you can, but you can't assign a dereferenced Interface* to another dereferenced Interface*. Moving the unique_ptr works though:

```<code>Super& Super::transpose() {
    Super newMatrix = *this;
    size_t s = elements.size();
    for(size_t i = 0; i < s; ++i) {
        elements[i] = std::move(newMatrix.elements[s - i - 1]);
    }
    return *this;
}
</code>```

Demo

You can also encapsulate the std::unique_ptr<Interface> in a wrapper class to add a copy constructor and assignment operator (which is not supported by std::unique_ptr). By doing that, you can greatly simplify your other classes. Your Super class wouldn't need to know anything about cloning. Copying/moving elements would work out-of-the-box.

Example wrapper:

```<code>class Cell {
public:
    Cell() noexcept = default;                              // empty Cell ctor
    explicit Cell(std::unique_ptr<Interface>&& d) noexcept; // converting ctor

    // rule of five
    Cell(const Cell& rhs);                           // must be implemented
    Cell(Cell&& rhs) noexcept = default;             // handled by unique_ptr
    Cell& operator=(const Cell& rhs);                // must be implemented
    Cell& operator=(Cell&& rhs) noexcept = default;  // handled by unique_ptr
    ~Cell() = default;                               // handled by unique_ptr

    explicit operator bool() const noexcept; // proxy for unique_ptr operator bool
    void reset() noexcept;                   // empty the Cell

    // dereferencing
    Interface& operator*();
    const Interface& operator*() const;
    Interface* operator->();
    const Interface* operator->() const;

    std::ostream& print(std::ostream& os) const;     // calls: os << data->toString()

    // A helper factory to make a Cell of a certain type using the converting ctor
    template<typename T, class... Args>
    static Cell make(Args&&... args) {
        return Cell(std::make_unique<T>(std::forward<Args>(args)...));
    }

private:
    std::unique_ptr<Interface> data{};
};

std::ostream& operator<<(std::ostream& os, const Cell& c); // calls c.print(os)
</code>```

The copy constructor and copy assignment operator can be implemented like this:

```<code>Cell::Cell(const Cell& rhs) : data(rhs ? rhs.data->clone() : nullptr) {}

Cell& Cell::operator=(const Cell& rhs) {
    if(this != &rhs) data = rhs ? rhs.data->clone() : nullptr;
    return *this;
}
</code>```

Here's a demo with a 2D matrix using a std::vector<std::vector<Cell>>. Such 2D vectors are not very effective but it serves the purpose of the demo.

Body:
I'm working on a really over engineered matrix calculator for school and hit a brick wall with the very last function I need. The full version of the transpose() is supposed to re-arrange my matrix and to switch columns and rows around.

I posted this question earlier with a snippet but was asked to write a minimum reproducable sample. This isn't very "minimum" but I tried.

So. I have a std::vector<std::unique_ptr<Interface>> that can hold objects of types Sub1 and Sub2.

In the full version I have a class that bypasses Interface and works fine since it only holds Sub2 objects.

However I can't seem to get it to work when there are multiple types in the vector. Nothing I've tried in the past few hours has done anything.

#include <iostream>
#include "main.h"
#include <memory>
#include <utility>
#include <vector>
#include <sstream>

//this is the one that won't work
Super Super::transpose() const {
    Super newMatrix = *this;
    int i = 1;
    for(auto& elem : elements){
        *elem = *newMatrix.elements[i];
        i--;
    }
return *this;
}

int main() {
    std::vector<std::unique_ptr<Interface>> newMatrix;
    newMatrix.push_back(std::unique_ptr<Sub1>(new Sub1('x')));
    newMatrix.push_back(std::unique_ptr<Sub2>(new Sub2(1)));
    Super mtx(std::move(newMatrix),2 );
    std::cout << mtx << std::endl;
    mtx.transpose();
    std::cout << mtx;
    return 0;
}

//busywork from here on
Super::Super(std::vector<std::unique_ptr<Interface>> matrix, int pN){
    elements = std::move(matrix);
    n = pN;
}

std::unique_ptr<Interface> Sub1::clone() const {
    char newVal = val;
    return std::unique_ptr<Sub1>(new Sub1(newVal));
}

std::unique_ptr<Interface> Sub2::clone() const {
    int newVal = i;
    return std::unique_ptr<Sub2>(new Sub2(newVal));
}

Sub1::Sub1(char pVal) {
    val = pVal;
}

Sub2::Sub2(int pI) {
    i = pI;
}

std::string Sub1::toString() const {
    std::stringstream ss;
    ss << val;
    return ss.str();
}

std::string Sub2::toString() const {
    std::stringstream ss;
    ss << i;
    return ss.str();
}

std::ostream &operator<<(std::ostream &os, const Super &matrix) {
    os << "[";
    int i = 0;
    for (auto &elem : matrix.elements) {
        os << elem->toString();
        if (i < matrix.n - 1) {
            os << ",";
        }
        i++;
    }
    os << "]";
    return os;
}

Super::Super(const Super &matrix) {
    n = matrix.n;
    for (auto &elem : matrix.elements) {
        std::unique_ptr<Interface> newElem = elem->clone();
        elements.push_back(std::move(newElem));
    }
}

And heres the header

#ifndef EXAMPLE_MAIN_H
#define EXAMPLE_MAIN_H

#include <memory>
#include <vector>

class Interface{
public:
    virtual ~Interface() = default;
    virtual std::unique_ptr<Interface> clone() const = 0;
    virtual std::string toString() const = 0;
};

class Super{
private:
    std::vector<std::unique_ptr<Interface>> elements;
    int n = 2;
public:
    Super(std::vector<std::unique_ptr<Interface>> elements, int n);
    Super(const Super &matrix);
    Super transpose() const;
    friend std::ostream &operator<<(std::ostream &os, const Super &matrix);
};

class Sub1: public Interface{
private:
    char val;
public:
    Sub1(char pVal);
    std::string toString() const override;
    std::unique_ptr<Interface> clone() const override;
};

class Sub2: public Interface{
private:
    int i;
public:
    Sub2(int pI);
    std::string toString() const override;
    std::unique_ptr<Interface> clone() const override;
};

std::ostream &operator<<(std::ostream &os, const Super &matrix);

#endif //EXAMPLE_MAIN_H

Body:
Super Super::transpose() const;

This looks like a function that shouldn't change *this when called, but should return a transposed version of *this.

Example:

#include <algorithm>

Super Super::transpose() const {
    Super newMatrix = *this;
    std::reverse(newMatrix.elements.begin(), newMatrix.elements.end());
    return newMatrix;
}

If you'd like to transpose *this in-place, change it to:

Super& Super::transpose();

Super& Super::transpose() {
    std::reverse(elements.begin(), elements.end());
    return *this;
}

If you must create a temporary object first, you can, but you can't assign a dereferenced Interface* to another dereferenced Interface*. Moving the unique_ptr works though:

Super& Super::transpose() {
    Super newMatrix = *this;
    size_t s = elements.size();
    for(size_t i = 0; i < s; ++i) {
        elements[i] = std::move(newMatrix.elements[s - i - 1]);
    }
    return *this;
}

Demo

You can also encapsulate the std::unique_ptr<Interface> in a wrapper class to add a copy constructor and assignment operator (which is not supported by std::unique_ptr). By doing that, you can greatly simplify your other classes. Your Super class wouldn't need to know anything about cloning. Copying/moving elements would work out-of-the-box.

Example wrapper:

class Cell {
public:
    Cell() noexcept = default;                              // empty Cell ctor
    explicit Cell(std::unique_ptr<Interface>&& d) noexcept; // converting ctor

    // rule of five
    Cell(const Cell& rhs);                           // must be implemented
    Cell(Cell&& rhs) noexcept = default;             // handled by unique_ptr
    Cell& operator=(const Cell& rhs);                // must be implemented
    Cell& operator=(Cell&& rhs) noexcept = default;  // handled by unique_ptr
    ~Cell() = default;                               // handled by unique_ptr

    explicit operator bool() const noexcept; // proxy for unique_ptr operator bool
    void reset() noexcept;                   // empty the Cell

    // dereferencing
    Interface& operator*();
    const Interface& operator*() const;
    Interface* operator->();
    const Interface* operator->() const;

    std::ostream& print(std::ostream& os) const;     // calls: os << data->toString()

    // A helper factory to make a Cell of a certain type using the converting ctor
    template<typename T, class... Args>
    static Cell make(Args&&... args) {
        return Cell(std::make_unique<T>(std::forward<Args>(args)...));
    }

private:
    std::unique_ptr<Interface> data{};
};

std::ostream& operator<<(std::ostream& os, const Cell& c); // calls c.print(os)

The copy constructor and copy assignment operator can be implemented like this:

Cell::Cell(const Cell& rhs) : data(rhs ? rhs.data->clone() : nullptr) {}

Cell& Cell::operator=(const Cell& rhs) {
    if(this != &rhs) data = rhs ? rhs.data->clone() : nullptr;
    return *this;
}

Here's a demo with a 2D matrix using a std::vector<std::vector<Cell>>. Such 2D vectors are not very effective but it serves the purpose of the demo.


问:
Laravel Slug Duplicate
when i trying to create post with the same slug it create post normally also in update the same problem
in this case i'v duplicate slugs for post

i search a lot about this problem but i didn't get any answers

any idea to solve this problem ?

Model

```<code>protected $fillable = [
    'title', 'slug', 'body',
];
</code>```

Controller

```<code>    public function slug($string, $separator = '-') {
    if (is_null($string)) {
        return "";
    }

    $string = trim($string);

    $string = mb_strtolower($string, "UTF-8");;

    $string = preg_replace("/[^a-z0-9_\sءاأإآؤئبتثجحخدذرزسشصضطظعغفقكلمنهويةى]#u/", "", $string);

    $string = preg_replace("/[\s-]+/", " ", $string);

    $string = preg_replace("/[\s_]/", $separator, $string);

    return $string;
}
public function store(Request $request)
{
    $this->validate($request, array(
        'title'         => 'required|max:255',
        'slug'          => 'required|min:3|max:255|unique:posts',
        'body'          => 'required',
    ));
    $post = new Post;
    $post->title = $request->input('title');
    $post->slug = $this->slug($request->slug);
    $post->body = $request->input('body');
    $post->save();

    return redirect('admin/posts')->with('success', 'post is successfully saved');
}
    public function update(Request $request, $id)
{
    if ($request->isMethod('get'))
        return view('content.admin.post.index', ['url' => Post::find($id)]);
    else {
        $rules = [
            'title'         => 'required|max:255',
            'slug'          => 'required|min:3|max:255|unique:posts,slug,{$post->slug},slug',
            'body'          => 'required',
            ];
        $this->validate($request, $rules);
        $post = Post::find($id);

        $post->title = $request->title;
        $post->slug = $this->slug($request->slug);
        $post->body = $request->body;

        $post->save();

        return redirect('/admin/posts');
    }
}
</code>```

答:
Why not just set slug column as unique in database migration?
Like this:

$table->string('slug')->unique();

Title: Laravel Slug Duplicate

Body:
when i trying to create post with the same slug it create post normally also in update the same problem
in this case i'v duplicate slugs for post

i search a lot about this problem but i didn't get any answers

any idea to solve this problem ?

Model

protected $fillable = [
    'title', 'slug', 'body',
];

Controller

    public function slug($string, $separator = '-') {
    if (is_null($string)) {
        return "";
    }

    $string = trim($string);

    $string = mb_strtolower($string, "UTF-8");;

    $string = preg_replace("/[^a-z0-9_\sءاأإآؤئبتثجحخدذرزسشصضطظعغفقكلمنهويةى]#u/", "", $string);

    $string = preg_replace("/[\s-]+/", " ", $string);

    $string = preg_replace("/[\s_]/", $separator, $string);

    return $string;
}
public function store(Request $request)
{
    $this->validate($request, array(
        'title'         => 'required|max:255',
        'slug'          => 'required|min:3|max:255|unique:posts',
        'body'          => 'required',
    ));
    $post = new Post;
    $post->title = $request->input('title');
    $post->slug = $this->slug($request->slug);
    $post->body = $request->input('body');
    $post->save();

    return redirect('admin/posts')->with('success', 'post is successfully saved');
}
    public function update(Request $request, $id)
{
    if ($request->isMethod('get'))
        return view('content.admin.post.index', ['url' => Post::find($id)]);
    else {
        $rules = [
            'title'         => 'required|max:255',
            'slug'          => 'required|min:3|max:255|unique:posts,slug,{$post->slug},slug',
            'body'          => 'required',
            ];
        $this->validate($request, $rules);
        $post = Post::find($id);

        $post->title = $request->title;
        $post->slug = $this->slug($request->slug);
        $post->body = $request->body;

        $post->save();

        return redirect('/admin/posts');
    }
}


问:
How can I "rotate" a relationship in laravel most efficiently?
Im building an ecommerce website in Laravel. I have an attributes table filled with the attribute information, like "Color" and "Size". The **attribute_values table holds all the attribute values like "red", "blue" and "XXL".

Since I want to re-use the attributes I don't create a new attribute set per product, but I link all the attribute values to the products with the pivot table attribute_value_product.

The model relationship structure is:

```<code>Attribute -> hasMany -> AttributeValue
Product -> belongsToMany -> AttributeValue
</code>```

The tables are as follows:

```<code>products
--------------------------------
| id | name            | price |
--------------------------------
| 1  | Example product | 200   |
--------------------------------

attributes
--------------
| id | name  |
--------------
| 1  | Color |
--------------

attribute_values
-----------------------------
| id | attribute_id | value |
-----------------------------
| 1  | 1            | Red   |
-----------------------------
| 2  | 1            | Blue  |
-----------------------------

attribute_value_product
----------------------------------------
| id | attribute_value_id | product_id |
----------------------------------------
| 1  | 1                  | 1          |
----------------------------------------
| 2  | 2                  | 1          |
----------------------------------------

</code>```

The controller code (a bit simplified) looks like this:

```<code>public function show($id) {

  $product = Product::where('id', $id)
    ->with('attributeValues.attribute')
    ->first();

  return view('product')->withProduct($product);

}
</code>```

the JSON result when I return the $product variable from the controller looks like this:

```<code>{
    id: 1,
    name: "Example product",
    price: 200,
    attribute_values: [
        {
            id: 1,
            attribute_id: 1,
            value: "Red",
            attribute: {
                id: 1,
                name: "Color",
            }
        },
        {
            id: 2,
            attribute_id: 1,
            value: "Blue",
            attribute: {
                id: 1,
                name: "Color",
            }
        }
    ]
}
</code>```

As you can see, this is kinda 'nasty'. When I retrieve lots of attributes for lots of products the attribute get's loaded for every value. When I only need it once actually. Also the way the data is structured is "wrong" in my opinion.

The preferred output I would like to see is:

```<code>{
    id: 1,
    name: "Example product",
    price: 200,
    attributes: [
        {
            id: 1,
            name: "Color",
            values: [
                {
                    id: 1,
                    attribute_id: 1,
                    value: "Blue",
                },
                {
                    id: 2,
                    attribute_id: 1,
                    value: "Red",
                }
            ]
        },
    ]
}
</code>```

So the values would be loaded per attribute per product. Is there a (Laravel) way to structure the relationships in a way that I can retrieve the data in the latter format while still keeping the database schema I have?

I'd rather not loop over all the items every time I need to retrieve the attributes. Especially when querying for lot's of products with some other relations.

答:
You probably want to look up Eloquent Resources. It's basically a way to format your models when transforming them into an array.

```<code><?php

namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\JsonResource;

class Product extends JsonResource
{
    /**
     * Transform the resource into an array.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function toArray($request)
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'price' => $this->price,
            'attributes' => $this->attribute_values->mapToGroups(fn($item, $key) => [
                [
                    'id' => $item->pivot->id,     // color id
                    'name' => $item->pivot->name, // color name
                    'values' => [
                        'id' => $item->id,
                        'attribute_id' => $item->attribute_id,
                        'value' => $item->value,
                    ]
                ]
            ])
            ->flatten(1),
        ];
    }
}
</code>```

```<code>use App\Http\Resources\Product as ProductResource;
...
$product = Product::with('attributeValues')->first();

return new ProductResource($product);
</code>```

Since I don't have your exact structure, I can only do guesswork. Just play around with the collection grouping methods until the data takes the format you want.

Eloquent Resources

Collections - Available Methods

Body:
Im building an ecommerce website in Laravel. I have an attributes table filled with the attribute information, like "Color" and "Size". The **attribute_values table holds all the attribute values like "red", "blue" and "XXL".

Since I want to re-use the attributes I don't create a new attribute set per product, but I link all the attribute values to the products with the pivot table attribute_value_product.

The model relationship structure is:

Attribute -> hasMany -> AttributeValue
Product -> belongsToMany -> AttributeValue

The tables are as follows:

products
--------------------------------
| id | name            | price |
--------------------------------
| 1  | Example product | 200   |
--------------------------------

attributes
--------------
| id | name  |
--------------
| 1  | Color |
--------------

attribute_values
-----------------------------
| id | attribute_id | value |
-----------------------------
| 1  | 1            | Red   |
-----------------------------
| 2  | 1            | Blue  |
-----------------------------

attribute_value_product
----------------------------------------
| id | attribute_value_id | product_id |
----------------------------------------
| 1  | 1                  | 1          |
----------------------------------------
| 2  | 2                  | 1          |
----------------------------------------

The controller code (a bit simplified) looks like this:

public function show($id) {

  $product = Product::where('id', $id)
    ->with('attributeValues.attribute')
    ->first();

  return view('product')->withProduct($product);

}

the JSON result when I return the $product variable from the controller looks like this:

{
    id: 1,
    name: "Example product",
    price: 200,
    attribute_values: [
        {
            id: 1,
            attribute_id: 1,
            value: "Red",
            attribute: {
                id: 1,
                name: "Color",
            }
        },
        {
            id: 2,
            attribute_id: 1,
            value: "Blue",
            attribute: {
                id: 1,
                name: "Color",
            }
        }
    ]
}

As you can see, this is kinda 'nasty'. When I retrieve lots of attributes for lots of products the attribute get's loaded for every value. When I only need it once actually. Also the way the data is structured is "wrong" in my opinion.

The preferred output I would like to see is:

{
    id: 1,
    name: "Example product",
    price: 200,
    attributes: [
        {
            id: 1,
            name: "Color",
            values: [
                {
                    id: 1,
                    attribute_id: 1,
                    value: "Blue",
                },
                {
                    id: 2,
                    attribute_id: 1,
                    value: "Red",
                }
            ]
        },
    ]
}

So the values would be loaded per attribute per product. Is there a (Laravel) way to structure the relationships in a way that I can retrieve the data in the latter format while still keeping the database schema I have?

I'd rather not loop over all the items every time I need to retrieve the attributes. Especially when querying for lot's of products with some other relations.

Body:
You probably want to look up Eloquent Resources. It's basically a way to format your models when transforming them into an array.

<?php

namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\JsonResource;

class Product extends JsonResource
{
    /**
     * Transform the resource into an array.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function toArray($request)
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'price' => $this->price,
            'attributes' => $this->attribute_values->mapToGroups(fn($item, $key) => [
                [
                    'id' => $item->pivot->id,     // color id
                    'name' => $item->pivot->name, // color name
                    'values' => [
                        'id' => $item->id,
                        'attribute_id' => $item->attribute_id,
                        'value' => $item->value,
                    ]
                ]
            ])
            ->flatten(1),
        ];
    }
}

use App\Http\Resources\Product as ProductResource;
...
$product = Product::with('attributeValues')->first();

return new ProductResource($product);

Since I don't have your exact structure, I can only do guesswork. Just play around with the collection grouping methods until the data takes the format you want.

Eloquent Resources

Collections - Available Methods


问:
jQuery - 2 different functions for 2 different submit buttons in one form
I have had a search around at other suggestions for what i want to do, but none seems to have what i need.

Basically i have a form with 2 submit buttons, i want one to save the data to the database, which i have done and works perfect ... the second submit button ( preview ) i need to open in a new window, take all the form data and display it on a form while leaving the previous page as it is ... if that makes sense.

I figure i cant do this with PHP as to get the post data it needs to basically reload a page ... i thought there might be a way with jQuery.

Any help?

答:
It sound like you need to use script for the 2nd button.

Would look something like this (using jQuery):

```<code><button type="button" id="preview_button">preview</button>
<script type="text/javascript">
    $(document).ready(function(){
      $('#preview_button').click(function(){
        var pUrl = 'http://server/preview.php?' + $('#form_id').serialize();
        window.open(pUrl);
      });
    });
</script>
</code>```

Body:
It sound like you need to use script for the 2nd button.

Would look something like this (using jQuery):

<button type="button" id="preview_button">preview</button>
<script type="text/javascript">
    $(document).ready(function(){
      $('#preview_button').click(function(){
        var pUrl = 'http://server/preview.php?' + $('#form_id').serialize();
        window.open(pUrl);
      });
    });
</script>


问:
The code will count the number of times a name appears in a column and will analyze the date when the data was stored in different Sheets
I updated the code with the suggestions and worl, thank you all

But, I have another problem. I need to do this for every month of the year and for all the names on the worksheet (about 20 names).

This way the code will be very long, do you have any suggestions to make it more automated?

'''

```<code>Sub CountHSEcards()

Sub CountHSEcards()

Dim Sh As Worksheet
Dim Br As Worksheet
Set Sh = Sheets("Planilha1")
Set Br = Sheets("Brasil")

NultimaCelula = Sh.Cells(Sh.Rows.Count, "A").End(xlUp).Row

For i = 1 To NultimaCelula

    If InStr(Sh.Range(CStr("AD" & i + 1)).Value, "Lucas") > 0 Then

        If Month(Sh.Range("AC" & i + 1).Value) = 1 Then
            Br.Range("C4") = Br.Range("C4").Value + 1
        End If

    End If

Next i

End Sub
</code>```

'''

答:
I could not test if the program is doing porecisely what you need because I do not know precisely your data in the columns and what you expect, but this code could work properly in my opinion (every variables seem to be correctly read) :

```<code>Sub CountHSEcards()

Dim Sh, Br As Worksheet
Set Sh = Sheets("Planilha1")
Set Br = Sheets("Brazil")

NultimaCelula = Sh.Cells(Sh.Rows.Count, "A").End(xlUp).Row

For i = 1 To NultimaCelula

    Sh.Select
    If InStr(Range(CStr("AD" & i + 1)).Value, "") > 0 Then

        If Month(Sh.Range(CStr("C" & i + 1)).Value) = i Then
            Br.Range(CStr("C" & i + 3)) = Br.Range(CStr("C" & i + 3)).Value + 1
        End If

    End If
Next i

End Sub
</code>```

You chose a notation for the range of a string type. So "AD" for example is OK. However, after you had directly an integer (i or i + x) to this string. It is problematic because it is a mix between different type. You can do like that, but after you convert the overall operation as a string (with CStr). Thus, the range has a string as argument and the address is ok.

Cheers

Body:
I updated the code with the suggestions and worl, thank you all

But, I have another problem. I need to do this for every month of the year and for all the names on the worksheet (about 20 names).

This way the code will be very long, do you have any suggestions to make it more automated?

'''

Sub CountHSEcards()

Sub CountHSEcards()

Dim Sh As Worksheet
Dim Br As Worksheet
Set Sh = Sheets("Planilha1")
Set Br = Sheets("Brasil")

NultimaCelula = Sh.Cells(Sh.Rows.Count, "A").End(xlUp).Row

For i = 1 To NultimaCelula

    If InStr(Sh.Range(CStr("AD" & i + 1)).Value, "Lucas") > 0 Then

        If Month(Sh.Range("AC" & i + 1).Value) = 1 Then
            Br.Range("C4") = Br.Range("C4").Value + 1
        End If

    End If

Next i

End Sub

'''

Body:
I could not test if the program is doing porecisely what you need because I do not know precisely your data in the columns and what you expect, but this code could work properly in my opinion (every variables seem to be correctly read) :

Sub CountHSEcards()

Dim Sh, Br As Worksheet
Set Sh = Sheets("Planilha1")
Set Br = Sheets("Brazil")

NultimaCelula = Sh.Cells(Sh.Rows.Count, "A").End(xlUp).Row

For i = 1 To NultimaCelula

    Sh.Select
    If InStr(Range(CStr("AD" & i + 1)).Value, "") > 0 Then

        If Month(Sh.Range(CStr("C" & i + 1)).Value) = i Then
            Br.Range(CStr("C" & i + 3)) = Br.Range(CStr("C" & i + 3)).Value + 1
        End If

    End If
Next i

End Sub

You chose a notation for the range of a string type. So "AD" for example is OK. However, after you had directly an integer (i or i + x) to this string. It is problematic because it is a mix between different type. You can do like that, but after you convert the overall operation as a string (with CStr). Thus, the range has a string as argument and the address is ok.

Cheers


问:
CoreImage: CIImage write JPG is shifting colors [macOS]
Using CoreImage to filter photos, I have found that saving to JPG file will result in an image that has a subtle but visible blue hue. In this example using a B&W image, the histogram reveals how the colors have been shifted in the saved file.

---Input

Output [] Histogram shows the color layers are offset

-- Issue demonstrated with MacOS 'Preview' App

I can show a similar result using only the Preview App.

test image here: https://i.stack.imgur.com/Y3f03.jpg

Open the JPG image using Preview.

Export to JPEG at any 'Quality' other than the default (85%?)

Open the exported file and look at the Histogram, and the same color shifting can be seen as I experience within my app.

-- Issue demonstrated in custom MacOS App

The code here is as bare bones as possible, creating a CIImage from the photo and immediately saving it without performing any filters. In this example I chose 0.61 for compression as it resulted in a similar file size as the original. The distortion seems to be broader if using a higher compression ratio, but I could not find any value that would eliminate it.

```<code>if let img = CIImage(contentsOf: url) {
   let dest = procFolder.url(named: "InOut.jpg")
   img.jpgWrite(url: dest)
}

extension CIImage {
    func jpgWrite(url: URL) {

        let prop: [NSBitmapImageRep.PropertyKey: Any] = [
            .compressionFactor: 0.61
        ]

        let bitmap = NSBitmapImageRep(ciImage: self)
        let data = bitmap.representation(using: NSBitmapImageRep.FileType.jpeg, properties: prop)

        do {
            try data?.write(to: url, options: .atomic)
        } catch {
            log.error(error)
        }
    }
}
</code>```

Update 1: Using @Frank Schlegel's answer for saving JPG file

The JPG now carries a Color Sync Profile, and I can (unscientifically) track a ~10% performance boost for portrait images (less for landscape), which are nice improvements. But, unfortunately the resulting file is still skewing the colors in the same way demonstrated in the histograms above.

```<code>extension CIImage {
   static let writeContext = CIContext(mtlDevice: MTLCreateSystemDefaultDevice()!, options: [
        // using an extended working color space allows you to retain wide gamut information, e.g., if the input is in DisplayP3
        .workingColorSpace: CGColorSpace(name: CGColorSpace.extendedSRGB)!,
        .workingFormat: CIFormat.RGBAh // 16 bit color depth, needed in extended space
    ])

    func jpgWrite(url: URL) {
       // write the output in the same color space as the input; fallback to sRGB if it can't be determined
        let outputColorSpace = colorSpace ?? CGColorSpace(name: CGColorSpace.sRGB)!
        do {
            try CIImage.writeContext.writeJPEGRepresentation(of: self, to: url, colorSpace: outputColorSpace, options: [:])
        } catch {
        }
    }
}
</code>```

Question:

How can I open a B&W JPG as a CIImage, and re-save a JPG file avoiding any color shifting?

答:
I never found the underlying cause of this issue and therefore no 'true' solution as I was seeking. In discussion with @Frank Schlegel, it led to a belief that it is an artifact of Apple's jpeg converter. And the issue was certainly more apparent when using test files that appear monochrome but actually had small amount of color info in them.

The simplest fix for my app was to ensure there was no color in the source image, so I drop the saturation to 0 prior to saving the file.

```<code>    let params = [
        "inputBrightness": brightness,  // -1...1, This filter calculates brightness by adding a bias value: color.rgb + vec3(brightness)
        "inputContrast": contrast,      // 0...2, this filter uses the following formula: (color.rgb - vec3(0.5)) * contrast + vec3(0.5)
        "inputSaturation": saturation   // 0...2
    ]
    image.applyingFilter("CIColorControls", parameters: params)
</code>```

Body:
Using CoreImage to filter photos, I have found that saving to JPG file will result in an image that has a subtle but visible blue hue. In this example using a B&W image, the histogram reveals how the colors have been shifted in the saved file.

---Input

Output [] Histogram shows the color layers are offset

-- Issue demonstrated with MacOS 'Preview' App

I can show a similar result using only the Preview App.

test image here: https://i.stack.imgur.com/Y3f03.jpg

Open the JPG image using Preview.

Export to JPEG at any 'Quality' other than the default (85%?)

Open the exported file and look at the Histogram, and the same color shifting can be seen as I experience within my app.

-- Issue demonstrated in custom MacOS App

The code here is as bare bones as possible, creating a CIImage from the photo and immediately saving it without performing any filters. In this example I chose 0.61 for compression as it resulted in a similar file size as the original. The distortion seems to be broader if using a higher compression ratio, but I could not find any value that would eliminate it.

if let img = CIImage(contentsOf: url) {
   let dest = procFolder.url(named: "InOut.jpg")
   img.jpgWrite(url: dest)
}

extension CIImage {
    func jpgWrite(url: URL) {

        let prop: [NSBitmapImageRep.PropertyKey: Any] = [
            .compressionFactor: 0.61
        ]

        let bitmap = NSBitmapImageRep(ciImage: self)
        let data = bitmap.representation(using: NSBitmapImageRep.FileType.jpeg, properties: prop)

        do {
            try data?.write(to: url, options: .atomic)
        } catch {
            log.error(error)
        }
    }
}

Update 1: Using @Frank Schlegel's answer for saving JPG file

The JPG now carries a Color Sync Profile, and I can (unscientifically) track a ~10% performance boost for portrait images (less for landscape), which are nice improvements. But, unfortunately the resulting file is still skewing the colors in the same way demonstrated in the histograms above.

extension CIImage {
   static let writeContext = CIContext(mtlDevice: MTLCreateSystemDefaultDevice()!, options: [
        // using an extended working color space allows you to retain wide gamut information, e.g., if the input is in DisplayP3
        .workingColorSpace: CGColorSpace(name: CGColorSpace.extendedSRGB)!,
        .workingFormat: CIFormat.RGBAh // 16 bit color depth, needed in extended space
    ])

    func jpgWrite(url: URL) {
       // write the output in the same color space as the input; fallback to sRGB if it can't be determined
        let outputColorSpace = colorSpace ?? CGColorSpace(name: CGColorSpace.sRGB)!
        do {
            try CIImage.writeContext.writeJPEGRepresentation(of: self, to: url, colorSpace: outputColorSpace, options: [:])
        } catch {
        }
    }
}

Question:

How can I open a B&W JPG as a CIImage, and re-save a JPG file avoiding any color shifting?

Body:
I never found the underlying cause of this issue and therefore no 'true' solution as I was seeking. In discussion with @Frank Schlegel, it led to a belief that it is an artifact of Apple's jpeg converter. And the issue was certainly more apparent when using test files that appear monochrome but actually had small amount of color info in them.

The simplest fix for my app was to ensure there was no color in the source image, so I drop the saturation to 0 prior to saving the file.

    let params = [
        "inputBrightness": brightness,  // -1...1, This filter calculates brightness by adding a bias value: color.rgb + vec3(brightness)
        "inputContrast": contrast,      // 0...2, this filter uses the following formula: (color.rgb - vec3(0.5)) * contrast + vec3(0.5)
        "inputSaturation": saturation   // 0...2
    ]
    image.applyingFilter("CIColorControls", parameters: params)


问:
How can I use onClick on stateless component in React?
I want to use onClick on a stateless compoenent but it's reject an error like : onClick listener to be a function, instead got a value of object type.

I need to show and hide component on click.
Example when I click on the <ResultCard/> component I want to hide him and show <ResultDetail/>

State React Component :

```<code>import React, { Component } from "react";
import ResultCard from "./ResultCard";
import "../../assets/css/Result.css";
import Spinner from "../Spinner";
import { getApiToken, getParisByPrice } from "../../services/api";
import Modal from "../Modal";
import "../../assets/css/BudgetEntry.css";
import modify from "../../assets/images/modify.png";
import ResultDetail from "./ResultDetail";

class Results extends Component {
  state = {
    priceValue: "",
    showResult: true
  };

  showResultDetail = () => {

    this.setState({ showResult: false });
  };

  closeResultDetails = () => {

    this.setState({ showResult: true });
  };

  render() {
    return (
      <div className="results-container">

        {this.state.loading ? (
          <Spinner />
        ) : (
          <div className={"row"}>
            {this.state.showResult ?
            (
              this.state.paris.map(details => (
                <ResultCard
                  key={details.id}
                  id={details.id}
                  showResultDetail={this.showResultDetail}
                  prefix={details.prefix}
                  costPerDay={details.average_cost_per_day}
                  logoSports={details.infrastructure.map(home =>
                    home.logo_path.map(path_image => (
                      <img
                        src={path_image}
                        alt="icon-sports"
                        style={{ width: 20 }}
                        key={path_image}
                      />
                    ))
                  )}
                />
              ))
            )
              :
            (
              <ResultDetail closeResultDetail={this.closeResultDetails}/>
            )
            }
          </div>
        )}
      </div>
    );
  }
}

export default Results;

</code>```

ResultCard (who is stateless component):

```<code>import React from 'react';
import '../../assets/css/ResultCard.css';

const ResultCard = ({prefix, costPerDay, logoSports, showResultDetail, id}) => {

  return (
    <div className="card" onClick={showResultDetail} id={id}>
      <p style={{margin:5}}>{prefix}</p>
      <p style={{margin:1}}>arrondissement</p>
      <p>{costPerDay} $</p>
      {logoSports}
    </div>
  )

};

export default ResultCard;
</code>```

ResultDetail (who is stateless component):

```<code>import React from 'react';

const ResultDetail = (closeResultDetail) => (

      <div onClick={closeResultDetail}>
        <p>Result detail</p>
      </div>
)

export default ResultDetail;
</code>```

thank for your help

答:
The issue is here

const ResultDetail = (closeResultDetail) => (
```

<p>You need to destructure it from the <code>props</code> object like this:</p>

<pre class="lang-js prettyprint-override"><code>const ResultDetail = ({closeResultDetail}) => (
</code>```

Or use it from props directly like this:

const ResultDetail = (props) => (

...
```

Body:
I want to use onClick on a stateless compoenent but it's reject an error like : onClick listener to be a function, instead got a value of object type.

I need to show and hide component on click.
Example when I click on the <ResultCard/> component I want to hide him and show <ResultDetail/>

State React Component :

import React, { Component } from "react";
import ResultCard from "./ResultCard";
import "../../assets/css/Result.css";
import Spinner from "../Spinner";
import { getApiToken, getParisByPrice } from "../../services/api";
import Modal from "../Modal";
import "../../assets/css/BudgetEntry.css";
import modify from "../../assets/images/modify.png";
import ResultDetail from "./ResultDetail";

class Results extends Component {
  state = {
    priceValue: "",
    showResult: true
  };

  showResultDetail = () => {

    this.setState({ showResult: false });
  };

  closeResultDetails = () => {

    this.setState({ showResult: true });
  };

  render() {
    return (

        {this.state.loading ? (
          <Spinner />
        ) : (

            {this.state.showResult ?
            (
              this.state.paris.map(details => (
                <ResultCard
                  key={details.id}
                  id={details.id}
                  showResultDetail={this.showResultDetail}
                  prefix={details.prefix}
                  costPerDay={details.average_cost_per_day}
                  logoSports={details.infrastructure.map(home =>
                    home.logo_path.map(path_image => (

                    ))
                  )}
                />
              ))
            )
              :
            (
              <ResultDetail closeResultDetail={this.closeResultDetails}/>
            )
            }

        )}

    );
  }
}

export default Results;

ResultCard (who is stateless component):

import React from 'react';
import '../../assets/css/ResultCard.css';

const ResultCard = ({prefix, costPerDay, logoSports, showResultDetail, id}) => {

  return (

{prefix}

arrondissement

{costPerDay} $

      {logoSports}

  )

};

export default ResultCard;

ResultDetail (who is stateless component):

import React from 'react';

const ResultDetail = (closeResultDetail) => (

Result detail

)

export default ResultDetail;

thank for your help

Body:
The issue is here

const ResultDetail = (closeResultDetail) => (

You need to destructure it from the props object like this:

const ResultDetail = ({closeResultDetail}) => (

Or use it from props directly like this:

const ResultDetail = (props) => (
 <div onClick={props.closeResultDetail}>
...


问:
How do I set the class path on Linux?
I need an answer for the question in the title.

Thanks.

答:
Create a small shell script which sets the classpath:</p>

```<code>#!/bin/bash
export JAVA_HOME=...

cp=$(find lib -name "*.jar" -exec printf :{} ';')
if [[ -n "$CLASSPATH" ]]; then
    cp="$cp;CLASSPATH"
fi

"$JAVA_HOME/bin/java" -classpath "$cp" ...
</code>```

ContentLicense: CC BY-SA 2.5

Title: How do I set the class path on Linux?

LastEditorDisplayName: Andrew Hare

Body:
Create a small shell script which sets the classpath:

#!/bin/bash
export JAVA_HOME=...

cp=$(find lib -name "*.jar" -exec printf :{} ';')
if [[ -n "$CLASSPATH" ]]; then
    cp="$cp;CLASSPATH"
fi

"$JAVA_HOME/bin/java" -classpath "$cp" ...

OwnerDisplayName: Aaron Digulla

LastEditorDisplayName: Aaron Digulla


问:
Dynamically Rendering a React component
In React JSX it does not appear to be possible to do something like this:

render: function() {
  return (
    <{this.props.component.slug} className='text'>
      {this.props.component.value}
    </{this.props.component.slug}>
  );
}```

I get a parse error: Unexpected token {.  Is this not something React
  can handle?

I'm designing this component so that under the hood, the values stored in this.props.component.slug will contain valid HTML elements (h1, p, etc.).  Is there any way to make this work?

答:
The solution for me was to assign the imported Component to a variable(with CapitalCase) and then render that variable.

Example:

```<code>import React, { Component } from 'react';
import FooComponent from './foo-component';
import BarComponent from './bar-component';

class MyComponent extends Component {
    components = {
        foo: FooComponent,
        bar: BarComponent
    };

        //this is the most important step
       const TagName = this.components.foo;

    render() {
       return <TagName />
    }
}
export default MyComponent;
</code>```

Title: Dynamically Rendering a React component

Body:
In React JSX it does not appear to be possible to do something like this:

render: function() {
  return (
    <{this.props.component.slug} className='text'>
      {this.props.component.value}
    </{this.props.component.slug}>
  );
}

I get a parse error: Unexpected token {.  Is this not something React
  can handle?

I'm designing this component so that under the hood, the values stored in this.props.component.slug will contain valid HTML elements (h1, p, etc.).  Is there any way to make this work?

Body:
The solution for me was to assign the imported Component to a variable(with CapitalCase) and then render that variable.

Example:

import React, { Component } from 'react';
import FooComponent from './foo-component';
import BarComponent from './bar-component';

class MyComponent extends Component {
    components = {
        foo: FooComponent,
        bar: BarComponent
    };

        //this is the most important step
       const TagName = this.components.foo;

    render() {
       return <TagName />
    }
}
export default MyComponent;


问:
how do I resolve IndexError: list index out of range?
I am trying to replicate this repository: https://github.com/sujiongming/UCF-101_video_classification. I get the following error when I run the 2_extract_files.py file.

```<code>Traceback (most recent call last):
  File "2_extract_files.py", line 99, in <module>
    main()
  File "2_extract_files.py", line 96, in main
    extract_files()
  File "2_extract_files.py", line 38, in extract_files
    video_parts = get_video_parts(video_path)
  File "2_extract_files.py", line 76, in get_video_parts
    filename = parts[3]
IndexError: list index out of range
</code>```

The code is as follows:

```<code>def extract_files():
    data_file = []
    folders = ['./train/', './test/']

    for folder in folders:
        class_folders = glob.glob(folder + '*')

        for vid_class in class_folders:
            class_files = glob.glob(vid_class + '/*.avi')

            for video_path in class_files:
                video_parts = get_video_parts(video_path)

                train_or_test, classname, filename_no_ext, filename = video_parts
                if not check_already_extracted(video_parts):

                    src = train_or_test + '/' + classname + '/' + \
                        filename
                    dest = train_or_test + '/' + classname + '/' + \
                        filename_no_ext + '-%04d.jpg'
                    call(["ffmpeg", "-i", src, dest])

                nb_frames = get_nb_frames_for_video(video_parts)

                data_file.append([train_or_test, classname, filename_no_ext, nb_frames])

                print("Generated %d frames for %s" % (nb_frames, filename_no_ext))

    with open('data_file.csv', 'w') as fout:
        writer = csv.writer(fout)
        writer.writerows(data_file)

    print("Extracted and wrote %d video files." % (len(data_file)))

def get_nb_frames_for_video(video_parts):
    train_or_test, classname, filename_no_ext, _ = video_parts
    generated_files = glob.glob(train_or_test + '/' + classname + '/' +
                                filename_no_ext + '*.jpg')
    return len(generated_files)

def get_video_parts(video_path):
    parts = video_path.split('/')
    filename = parts[3]
    filename_no_ext = filename.split('.')[0]
    classname = parts[2]
    train_or_test = parts[1]

    return train_or_test, classname, filename_no_ext, filename
</code>```

can anyone tell me what I'm doing wrong and guide me on how to get the list index right. Thanks in advance.

```<code>Window 10
Python 3.7.6
</code>```

答:
It is recommended to use just os.path.split(video_path) and os.path.splitext() and work your way through, it safer and also more portable:

```<code>def get_video_parts(video_path):
    head, filename = os.path.split(video_path)
    filename_no_ext, ext = os.path.splitext(filename)
    head, classname = os.path.split(head)
    head, train_or_test = os.path.split(head)

    return train_or_test, classname, filename_no_ext, filename
</code>```

https://docs.python.org/3/library/os.path.html#os.path.split

I'm a bit outdated - so chances that you would like to try out pathlib for more high level operations on paths objects - in this case it would probably be combination of path.stem() to get the name of the last part without extension and path.parent() to go up.

https://docs.python.org/3/library/pathlib.html#module-pathlib

Body:
I am trying to replicate this repository: https://github.com/sujiongming/UCF-101_video_classification. I get the following error when I run the 2_extract_files.py file.

Traceback (most recent call last):
  File "2_extract_files.py", line 99, in <module>
    main()
  File "2_extract_files.py", line 96, in main
    extract_files()
  File "2_extract_files.py", line 38, in extract_files
    video_parts = get_video_parts(video_path)
  File "2_extract_files.py", line 76, in get_video_parts
    filename = parts[3]
IndexError: list index out of range

The code is as follows:

def extract_files():
    data_file = []
    folders = ['./train/', './test/']

    for folder in folders:
        class_folders = glob.glob(folder + '*')

        for vid_class in class_folders:
            class_files = glob.glob(vid_class + '/*.avi')

            for video_path in class_files:
                video_parts = get_video_parts(video_path)

                train_or_test, classname, filename_no_ext, filename = video_parts
                if not check_already_extracted(video_parts):

                    src = train_or_test + '/' + classname + '/' + \
                        filename
                    dest = train_or_test + '/' + classname + '/' + \
                        filename_no_ext + '-%04d.jpg'
                    call(["ffmpeg", "-i", src, dest])

                nb_frames = get_nb_frames_for_video(video_parts)

                data_file.append([train_or_test, classname, filename_no_ext, nb_frames])

                print("Generated %d frames for %s" % (nb_frames, filename_no_ext))

    with open('data_file.csv', 'w') as fout:
        writer = csv.writer(fout)
        writer.writerows(data_file)

    print("Extracted and wrote %d video files." % (len(data_file)))

def get_nb_frames_for_video(video_parts):
    train_or_test, classname, filename_no_ext, _ = video_parts
    generated_files = glob.glob(train_or_test + '/' + classname + '/' +
                                filename_no_ext + '*.jpg')
    return len(generated_files)

def get_video_parts(video_path):
    parts = video_path.split('/')
    filename = parts[3]
    filename_no_ext = filename.split('.')[0]
    classname = parts[2]
    train_or_test = parts[1]

    return train_or_test, classname, filename_no_ext, filename

can anyone tell me what I'm doing wrong and guide me on how to get the list index right. Thanks in advance.

Window 10
Python 3.7.6

Body:
It is recommended to use just os.path.split(video_path) and os.path.splitext() and work your way through, it safer and also more portable:

def get_video_parts(video_path):
    head, filename = os.path.split(video_path)
    filename_no_ext, ext = os.path.splitext(filename)
    head, classname = os.path.split(head)
    head, train_or_test = os.path.split(head)

    return train_or_test, classname, filename_no_ext, filename

https://docs.python.org/3/library/os.path.html#os.path.split

I'm a bit outdated - so chances that you would like to try out pathlib for more high level operations on paths objects - in this case it would probably be combination of path.stem() to get the name of the last part without extension and path.parent() to go up.

https://docs.python.org/3/library/pathlib.html#module-pathlib