问:
Oracle How to Map Data Types to CODE/IDs - Dump and dbms_sql.define_column produce data type codes
When using DUMP() or when using dbms_sql.define_column both return the CODE of the Oracle Data type.

Is there a MAP that ties the Oracle Data Types to their Oracle CODEs that are returned by the DUMP() function or the dbms_sql.define_column feature?

I'm looking for this to aid in testing as well as to locate custom data types such as a NESTED TABLE.  A nested table exists in one of my tables and produces a Data Type Code of 109.  How do I lookup the type that is associated with Type Code of 109?  Examples below of dump and dbms_sql.define_column.

I must assume this is available within the database to query due to the fact that when I write a query I define the columns using the 'TEXT' name of the column type and NOT the Code.  So where is the text aka name of my data types  stored whether they are 'built-in' or 'custom'?  It's got to be somewhere.

DUMP

```<code>SELECT
    DUMP(to_date('15-JAN-18'),10,1,1) AS date_type
  , DUMP(123,10,1,1)   AS num_type
  , DUMP('abc',10,1,1) AS var_type
FROM dual
;
</code>```

Dump Output

```<code>/* OUTPUT:
 |
 | "DATE_TYPE"          "NUM_TYPE"          "VAR_TYPE"
 |--------------------|-------------------|------------------
 | "Typ=13 Len=8: 226"  "Typ=2 Len=3: 194"  "Typ=96 Len=3: 97"
*/
</code>```

DBMS_SQL.DEFINE_COLUMN

```<code>... some code above ...

-- Define columns:
FOR i IN 1..n_colcnt
LOOP
    dbms_output.put_line(v_desctab(i).col_name ||': ' || v_desctab(i).col_type);

    IF v_desctab(i).col_type = 2
    THEN dbms_sql.DEFINE_COLUMN(n_curid, i, v_num_var);
    ELSIF v_desctab(i).col_type = 12 THEN
        dbms_sql.define_column(n_curid, i, v_date_var);
    ELSE
        dbms_sql.define_column(n_curid, i, v_name_var, 50);
    END IF;
END LOOP;

... some code below ...
</code>```

DBMS_SQL.DEFINE_COLUMN Output - Outputs column names and their data type CODEs.  I want Text instead.

```<code>QUERY_ID: 2
OTHER_ID: 2
ACTIVE: 1
QUERY_NAME: 1
QUERY_DESC: 1
QUERY_DEF: 112
QUERY_TYPE: 2
CREATED: 12
UPDATED: 12
CREATED_BY: 2
UPDATED_BY: 2
</code>```

One thing I have regarding a mapping of Oracle Data Types to their corresponding IDs is an excel file I continue to reference.  I am hoping there is something I can query within Oracle to give me what I'm looking for rather than referencing an external document.

There is THIS link however this doesn't assist me when I'm looking for Custom Types and it missing documentation for CODE 13.

Here is another very nice Data Type Map somebody put together: Oracle Type Code Mappings

答:
Using the data obtained from this link (Oracle Type Code Mappings) I decided to create my own table and populate it with the Oracle Data Types mapping the Oracle Type CODEs back to their 'TEXT' of the datatype.

Maybe somebody will come along and show a way to just obtain the NAME of the TYPE based on the code without this custom work around.  For now this is going to have to be my accepted answer as it gets me as close as possible to what I needed in the first place.  The other answers certainly help me to get here.  Unfortunately it still isn't able to pull back the actual NAME of Custom Types that come back with a CODE of 109.

```<code>CREATE TABLE oracle_data_types
(
  Data_Type                 VARCHAR2(100 CHAR),
  Uniform_Type_Code         NUMBER(10),
  Oracle_Doc                NUMBER(10),
  DBMS_SQL                  NUMBER(10),
  DBMS_TYPES_TYPECODE_PERC  VARCHAR2(100 CHAR),
  JDBC_java_sql_Types       VARCHAR2(100 CHAR),
  OCI                       VARCHAR2(100 CHAR),
  DUMP                      NUMBER(10),
  V$_SQL_BIND_DATA_DATATYPE  NUMBER(10)
)
;

/

INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('VARCHAR2',1,1,1,'9 VARCHAR2 1 VARCHAR','12 VARCHAR','1 SQLT_CHR',1,1);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NVARCHAR2',1,1,1,'287 NVARCHAR2','12 VARCHAR','1 SQLT_CHR',1,1);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NUMBER',2,2,2,'NUMBER','2 NUMERIC','2 SQLT_NUM',2,2);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('FLOAT',2,2,2,'2 NUMBER','2 NUMERIC','2 SQLT_NUM',2,2);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('LONG',8,8,8,'','-1 LONGVARCHAR','8 SQLT_LNG',NULL,8);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('DATE',12,12,12,'12 DATE','93 TIMESTAMP1','12 SQLT_DAT',12,12);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BINARY_FLOAT',21,21,100,'100 BFLOAT','100','100 SQLT_IBFLOAT',100,100);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BINARY_DOUBLE',22,22,101,'101 BDOUBLE','101','101 SQLT_IBFLOAT',101,101);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP',180,180,180,'187 TIMESTAMP','93 TIMESTAMP1','187 SQLT_TIMESTAMP',180,180);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP WITH TIME ZONE',181,181,181,'188 TIMESTAMP_TZ','-101','188 SQLT_TIMESTAMP_TZ',181,181);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP WITH LOCAL TIME ZONE',231,231,231,'232 TIMESTAMP_LTZ','-102','232 SQLT_TIMESTAMP_LTZ',231,231);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('INTERVAL YEAR TO MONTH',182,182,182,'189 INTERVAL_YM','-103','189 SQLT_INTERVAL_YM',182,182);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('INTERVAL DAY TO SECOND',183,183,183,'190 INTERVAL_DS','-104','190 SQLT_INTERVAL_DS',183,183);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('RAW',23,23,23,'95 RAW','-3 VARBINARY','23 SQLT_BIN',23,23);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('LONG RAW',24,24,24,'','-4 LONGVARBINARY','24 SQLT_LBI',NULL,24);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ROWID',69,69,11,'','-8','104 SQLT_RDD',69,69);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('UROWID',208,208,208,'104 UROWID','-8','104 SQLT_RDD',208,208);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('CHAR',96,96,96,'96 CHAR','1 CHAR','96 SQLT_AFC',96,96);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NCHAR',96,96,96,'286 NCHAR','1 CHAR','96 SQLT_AFC',96,96);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('CLOB',112,112,112,'112 CLOB','2005 CLOB','112 SQLT_CLOB',NULL,112);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NCLOB',112,112,112,'288 NCLOB','2005 CLOB','112 SQLT_CLOB',NULL,112);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BLOB',113,113,113,'113 BLOB','2004 BLOB','113 SQLT_BLOB',NULL,113);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BFILE',114,114,114,'114 BFILE','-13','114 SQLT_BFILEE',NULL,114);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('XMLTYPE',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYDATA',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYDATASET',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYTYPE',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Object type',109,NULL,109,'108 OBJECT','2002 STRUCT','108 SQLT_NTY',121,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('VARRAY',109,NULL,109,'247 VARRAY','2003 ARRAY','108 SQLT_NTY',NULL,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Nested table',109,NULL,109,'248 TABLE','2003 ARRAY','108 SQLT_NTY',NULL,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('REF',111,NULL,111,'110 REF','2006 REF','110 SQLT_REF',111,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Strong REF CURSOR',102,NULL,102,'','-10','116 SQLT_RSET',102,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Weak REF CURSOR',102,NULL,102,'','-10','116 SQLT_RSET',102,NULL);
</code>```

Screenshot of Data for a visual reference

Example of using the sys.DBMS_TYPES package

To select the PL/SQL code directly from the package:

```<code>SELECT TEXT
FROM dba_source
WHERE name = 'DBMS_TYPES'
   AND regexp_like(text, 'TYPECODE')
;
</code>```

Using PL/SQL to pull the Constants out:

```<code>BEGIN
  dbms_output.put_line(dbms_types.TYPECODE_CHAR            ||' TYPECODE_DATE');
  dbms_output.put_line(dbms_types.TYPECODE_NUMBER          ||' TYPECODE_NUMBER');
  dbms_output.put_line(dbms_types.TYPECODE_VARCHAR2        ||' TYPECODE_RAW');
  dbms_output.put_line(dbms_types.TYPECODE_VARCHAR         ||' TYPECODE_CHAR');
  dbms_output.put_line(dbms_types.TYPECODE_RAW             ||' TYPECODE_VARCHAR2');
  dbms_output.put_line(dbms_types.TYPECODE_MLSLABEL        ||' TYPECODE_VARCHAR');
  dbms_output.put_line(dbms_types.TYPECODE_BLOB            ||' TYPECODE_MLSLABEL');
  dbms_output.put_line(dbms_types.TYPECODE_BFILE           ||' TYPECODE_BLOB');
  dbms_output.put_line(dbms_types.TYPECODE_CLOB            ||' TYPECODE_BFILE');
  dbms_output.put_line(dbms_types.TYPECODE_CFILE           ||' TYPECODE_CLOB');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP       ||' TYPECODE_CFILE');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP_TZ    ||' TYPECODE_TIMESTAMP');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP_LTZ   ||' TYPECODE_TIMESTAMP_TZ');
  dbms_output.put_line(dbms_types.TYPECODE_INTERVAL_YM     ||' TYPECODE_TIMESTAMP_LTZ');
  dbms_output.put_line(dbms_types.TYPECODE_INTERVAL_DS     ||' TYPECODE_INTERVAL_YM');
  dbms_output.put_line(dbms_types.TYPECODE_REF             ||' TYPECODE_INTERVAL_DS');
  dbms_output.put_line(dbms_types.TYPECODE_OBJECT          ||' TYPECODE_REF');
  dbms_output.put_line(dbms_types.TYPECODE_VARRAY          ||' TYPECODE_OBJECT');
  dbms_output.put_line(dbms_types.TYPECODE_TABLE           ||' TYPECODE_VARRAY');
  dbms_output.put_line(dbms_types.TYPECODE_NAMEDCOLLECTION ||' TYPECODE_TABLE');
  dbms_output.put_line(dbms_types.TYPECODE_DATE            ||' TYPECODE_NAMEDCOLLECTION');
  dbms_output.put_line(dbms_types.TYPECODE_OPAQUE          ||' TYPECODE_OPAQUE');
  dbms_output.put_line(dbms_types.TYPECODE_NCHAR           ||' TYPECODE_NCHAR');
  dbms_output.put_line(dbms_types.TYPECODE_NVARCHAR2       ||' TYPECODE_NVARCHAR2');
  dbms_output.put_line(dbms_types.TYPECODE_NCLOB           ||' TYPECODE_NCLOB');
  dbms_output.put_line(dbms_types.TYPECODE_BFLOAT          ||' TYPECODE_BFLOAT');
  dbms_output.put_line(dbms_types.TYPECODE_BDOUBLE         ||' TYPECODE_BDOUBLE');
  dbms_output.put_line(dbms_types.TYPECODE_UROWID          ||' TYPECODE_UROWID');
END;
</code>```

Output

```<code>96 TYPECODE_DATE
2 TYPECODE_NUMBER
9 TYPECODE_RAW
1 TYPECODE_CHAR
95 TYPECODE_VARCHAR2
105 TYPECODE_VARCHAR
113 TYPECODE_MLSLABEL
114 TYPECODE_BLOB
112 TYPECODE_BFILE
115 TYPECODE_CLOB
187 TYPECODE_CFILE
188 TYPECODE_TIMESTAMP
232 TYPECODE_TIMESTAMP_TZ
189 TYPECODE_TIMESTAMP_LTZ
190 TYPECODE_INTERVAL_YM
110 TYPECODE_INTERVAL_DS
108 TYPECODE_REF
247 TYPECODE_OBJECT
248 TYPECODE_VARRAY
122 TYPECODE_TABLE
12 TYPECODE_NAMEDCOLLECTION
58 TYPECODE_OPAQUE
286 TYPECODE_NCHAR
287 TYPECODE_NVARCHAR2
288 TYPECODE_NCLOB
100 TYPECODE_BFLOAT
101 TYPECODE_BDOUBLE
104 TYPECODE_UROWID
</code>```

Using the DECODE function code pulled directly out of the view for ALL_TAB_COLUMNS

```<code>SELECT DISTINCT c.type# type_code,
decode(c.type#, 1, decode(c.charsetform, 2, 'NVARCHAR2', 'VARCHAR2'),
                2, decode(c.scale, null, decode(c.precision#, null, 'NUMBER', 'FLOAT'), 'NUMBER'),
                8, 'LONG',
                9, decode(c.charsetform, 2, 'NCHAR VARYING', 'VARCHAR'),
                12, 'DATE',
                23, 'RAW', 24, 'LONG RAW',
                69, 'ROWID',
                96, decode(c.charsetform, 2, 'NCHAR', 'CHAR'),
                100, 'BINARY_FLOAT',
                101, 'BINARY_DOUBLE',
                105, 'MLSLABEL',
                106, 'MLSLABEL',
                111, 'REF',
                112, decode(c.charsetform, 2, 'NCLOB', 'CLOB'),
                113, 'BLOB', 114, 'BFILE', 115, 'CFILE',
                121, 'USER_TYPE',
                122, 'USER_TYPE',
                123, 'USER_TYPE',
                178, 'TIME(' ||c.scale|| ')',
                179, 'TIME(' ||c.scale|| ')' || ' WITH TIME ZONE',
                180, 'TIMESTAMP(' ||c.scale|| ')',
                181, 'TIMESTAMP(' ||c.scale|| ')'||' WITH TIME ZONE',
                231, 'TIMESTAMP(' ||c.scale|| ')'||' WITH LOCAL TIME ZONE',
                182, 'INTERVAL YEAR(' ||c.precision#||') TO MONTH',
                183, 'INTERVAL DAY(' ||c.precision#||') TO SECOND(' ||
                      c.scale || ')',
                208, 'UROWID',
                'UNDEFINED') type_name
FROM sys.col$ c
ORDER BY c.type#
;
</code>```

Body:
When using DUMP() or when using dbms_sql.define_column both return the CODE of the Oracle Data type.

Is there a MAP that ties the Oracle Data Types to their Oracle CODEs that are returned by the DUMP() function or the dbms_sql.define_column feature?

I'm looking for this to aid in testing as well as to locate custom data types such as a NESTED TABLE.  A nested table exists in one of my tables and produces a Data Type Code of 109.  How do I lookup the type that is associated with Type Code of 109?  Examples below of dump and dbms_sql.define_column.

I must assume this is available within the database to query due to the fact that when I write a query I define the columns using the 'TEXT' name of the column type and NOT the Code.  So where is the text aka name of my data types  stored whether they are 'built-in' or 'custom'?  It's got to be somewhere.

DUMP

SELECT
    DUMP(to_date('15-JAN-18'),10,1,1) AS date_type
  , DUMP(123,10,1,1)   AS num_type
  , DUMP('abc',10,1,1) AS var_type
FROM dual
;

Dump Output

/* OUTPUT:
 |
 | "DATE_TYPE"          "NUM_TYPE"          "VAR_TYPE"
 |--------------------|-------------------|------------------
 | "Typ=13 Len=8: 226"  "Typ=2 Len=3: 194"  "Typ=96 Len=3: 97"
*/

DBMS_SQL.DEFINE_COLUMN

... some code above ...

-- Define columns:
FOR i IN 1..n_colcnt
LOOP
    dbms_output.put_line(v_desctab(i).col_name ||': ' || v_desctab(i).col_type);

    IF v_desctab(i).col_type = 2
    THEN dbms_sql.DEFINE_COLUMN(n_curid, i, v_num_var);
    ELSIF v_desctab(i).col_type = 12 THEN
        dbms_sql.define_column(n_curid, i, v_date_var);
    ELSE
        dbms_sql.define_column(n_curid, i, v_name_var, 50);
    END IF;
END LOOP;

... some code below ...

DBMS_SQL.DEFINE_COLUMN Output - Outputs column names and their data type CODEs.  I want Text instead.

QUERY_ID: 2
OTHER_ID: 2
ACTIVE: 1
QUERY_NAME: 1
QUERY_DESC: 1
QUERY_DEF: 112
QUERY_TYPE: 2
CREATED: 12
UPDATED: 12
CREATED_BY: 2
UPDATED_BY: 2

One thing I have regarding a mapping of Oracle Data Types to their corresponding IDs is an excel file I continue to reference.  I am hoping there is something I can query within Oracle to give me what I'm looking for rather than referencing an external document.

There is THIS link however this doesn't assist me when I'm looking for Custom Types and it missing documentation for CODE 13.

Here is another very nice Data Type Map somebody put together: Oracle Type Code Mappings

Body:
Using the data obtained from this link (Oracle Type Code Mappings) I decided to create my own table and populate it with the Oracle Data Types mapping the Oracle Type CODEs back to their 'TEXT' of the datatype.

Maybe somebody will come along and show a way to just obtain the NAME of the TYPE based on the code without this custom work around.  For now this is going to have to be my accepted answer as it gets me as close as possible to what I needed in the first place.  The other answers certainly help me to get here.  Unfortunately it still isn't able to pull back the actual NAME of Custom Types that come back with a CODE of 109.

CREATE TABLE oracle_data_types
(
  Data_Type                 VARCHAR2(100 CHAR),
  Uniform_Type_Code         NUMBER(10),
  Oracle_Doc                NUMBER(10),
  DBMS_SQL                  NUMBER(10),
  DBMS_TYPES_TYPECODE_PERC  VARCHAR2(100 CHAR),
  JDBC_java_sql_Types       VARCHAR2(100 CHAR),
  OCI                       VARCHAR2(100 CHAR),
  DUMP                      NUMBER(10),
  V$_SQL_BIND_DATA_DATATYPE  NUMBER(10)
)
;

/

INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('VARCHAR2',1,1,1,'9 VARCHAR2 1 VARCHAR','12 VARCHAR','1 SQLT_CHR',1,1);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NVARCHAR2',1,1,1,'287 NVARCHAR2','12 VARCHAR','1 SQLT_CHR',1,1);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NUMBER',2,2,2,'NUMBER','2 NUMERIC','2 SQLT_NUM',2,2);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('FLOAT',2,2,2,'2 NUMBER','2 NUMERIC','2 SQLT_NUM',2,2);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('LONG',8,8,8,'','-1 LONGVARCHAR','8 SQLT_LNG',NULL,8);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('DATE',12,12,12,'12 DATE','93 TIMESTAMP1','12 SQLT_DAT',12,12);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BINARY_FLOAT',21,21,100,'100 BFLOAT','100','100 SQLT_IBFLOAT',100,100);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BINARY_DOUBLE',22,22,101,'101 BDOUBLE','101','101 SQLT_IBFLOAT',101,101);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP',180,180,180,'187 TIMESTAMP','93 TIMESTAMP1','187 SQLT_TIMESTAMP',180,180);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP WITH TIME ZONE',181,181,181,'188 TIMESTAMP_TZ','-101','188 SQLT_TIMESTAMP_TZ',181,181);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('TIMESTAMP WITH LOCAL TIME ZONE',231,231,231,'232 TIMESTAMP_LTZ','-102','232 SQLT_TIMESTAMP_LTZ',231,231);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('INTERVAL YEAR TO MONTH',182,182,182,'189 INTERVAL_YM','-103','189 SQLT_INTERVAL_YM',182,182);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('INTERVAL DAY TO SECOND',183,183,183,'190 INTERVAL_DS','-104','190 SQLT_INTERVAL_DS',183,183);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('RAW',23,23,23,'95 RAW','-3 VARBINARY','23 SQLT_BIN',23,23);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('LONG RAW',24,24,24,'','-4 LONGVARBINARY','24 SQLT_LBI',NULL,24);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ROWID',69,69,11,'','-8','104 SQLT_RDD',69,69);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('UROWID',208,208,208,'104 UROWID','-8','104 SQLT_RDD',208,208);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('CHAR',96,96,96,'96 CHAR','1 CHAR','96 SQLT_AFC',96,96);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NCHAR',96,96,96,'286 NCHAR','1 CHAR','96 SQLT_AFC',96,96);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('CLOB',112,112,112,'112 CLOB','2005 CLOB','112 SQLT_CLOB',NULL,112);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('NCLOB',112,112,112,'288 NCLOB','2005 CLOB','112 SQLT_CLOB',NULL,112);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BLOB',113,113,113,'113 BLOB','2004 BLOB','113 SQLT_BLOB',NULL,113);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('BFILE',114,114,114,'114 BFILE','-13','114 SQLT_BFILEE',NULL,114);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('XMLTYPE',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYDATA',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYDATASET',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('ANYTYPE',109,NULL,109,'58 OPAQUE2','2007','108 SQLT_NTY',58,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Object type',109,NULL,109,'108 OBJECT','2002 STRUCT','108 SQLT_NTY',121,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('VARRAY',109,NULL,109,'247 VARRAY','2003 ARRAY','108 SQLT_NTY',NULL,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Nested table',109,NULL,109,'248 TABLE','2003 ARRAY','108 SQLT_NTY',NULL,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('REF',111,NULL,111,'110 REF','2006 REF','110 SQLT_REF',111,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Strong REF CURSOR',102,NULL,102,'','-10','116 SQLT_RSET',102,NULL);
INSERT INTO oracle_data_types (Data_Type, Uniform_Type_Code, Oracle_Doc, DBMS_SQL, DBMS_TYPES_TYPECODE_PERC, JDBC_java_sql_Types, OCI, DUMP, V$_SQL_BIND_DATA_DATATYPE) VALUES ('Weak REF CURSOR',102,NULL,102,'','-10','116 SQLT_RSET',102,NULL);

Screenshot of Data for a visual reference

Example of using the sys.DBMS_TYPES package

To select the PL/SQL code directly from the package:

SELECT TEXT
FROM dba_source
WHERE name = 'DBMS_TYPES'
   AND regexp_like(text, 'TYPECODE')
;

Using PL/SQL to pull the Constants out:

BEGIN
  dbms_output.put_line(dbms_types.TYPECODE_CHAR            ||' TYPECODE_DATE');
  dbms_output.put_line(dbms_types.TYPECODE_NUMBER          ||' TYPECODE_NUMBER');
  dbms_output.put_line(dbms_types.TYPECODE_VARCHAR2        ||' TYPECODE_RAW');
  dbms_output.put_line(dbms_types.TYPECODE_VARCHAR         ||' TYPECODE_CHAR');
  dbms_output.put_line(dbms_types.TYPECODE_RAW             ||' TYPECODE_VARCHAR2');
  dbms_output.put_line(dbms_types.TYPECODE_MLSLABEL        ||' TYPECODE_VARCHAR');
  dbms_output.put_line(dbms_types.TYPECODE_BLOB            ||' TYPECODE_MLSLABEL');
  dbms_output.put_line(dbms_types.TYPECODE_BFILE           ||' TYPECODE_BLOB');
  dbms_output.put_line(dbms_types.TYPECODE_CLOB            ||' TYPECODE_BFILE');
  dbms_output.put_line(dbms_types.TYPECODE_CFILE           ||' TYPECODE_CLOB');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP       ||' TYPECODE_CFILE');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP_TZ    ||' TYPECODE_TIMESTAMP');
  dbms_output.put_line(dbms_types.TYPECODE_TIMESTAMP_LTZ   ||' TYPECODE_TIMESTAMP_TZ');
  dbms_output.put_line(dbms_types.TYPECODE_INTERVAL_YM     ||' TYPECODE_TIMESTAMP_LTZ');
  dbms_output.put_line(dbms_types.TYPECODE_INTERVAL_DS     ||' TYPECODE_INTERVAL_YM');
  dbms_output.put_line(dbms_types.TYPECODE_REF             ||' TYPECODE_INTERVAL_DS');
  dbms_output.put_line(dbms_types.TYPECODE_OBJECT          ||' TYPECODE_REF');
  dbms_output.put_line(dbms_types.TYPECODE_VARRAY          ||' TYPECODE_OBJECT');
  dbms_output.put_line(dbms_types.TYPECODE_TABLE           ||' TYPECODE_VARRAY');
  dbms_output.put_line(dbms_types.TYPECODE_NAMEDCOLLECTION ||' TYPECODE_TABLE');
  dbms_output.put_line(dbms_types.TYPECODE_DATE            ||' TYPECODE_NAMEDCOLLECTION');
  dbms_output.put_line(dbms_types.TYPECODE_OPAQUE          ||' TYPECODE_OPAQUE');
  dbms_output.put_line(dbms_types.TYPECODE_NCHAR           ||' TYPECODE_NCHAR');
  dbms_output.put_line(dbms_types.TYPECODE_NVARCHAR2       ||' TYPECODE_NVARCHAR2');
  dbms_output.put_line(dbms_types.TYPECODE_NCLOB           ||' TYPECODE_NCLOB');
  dbms_output.put_line(dbms_types.TYPECODE_BFLOAT          ||' TYPECODE_BFLOAT');
  dbms_output.put_line(dbms_types.TYPECODE_BDOUBLE         ||' TYPECODE_BDOUBLE');
  dbms_output.put_line(dbms_types.TYPECODE_UROWID          ||' TYPECODE_UROWID');
END;

Output

96 TYPECODE_DATE
2 TYPECODE_NUMBER
9 TYPECODE_RAW
1 TYPECODE_CHAR
95 TYPECODE_VARCHAR2
105 TYPECODE_VARCHAR
113 TYPECODE_MLSLABEL
114 TYPECODE_BLOB
112 TYPECODE_BFILE
115 TYPECODE_CLOB
187 TYPECODE_CFILE
188 TYPECODE_TIMESTAMP
232 TYPECODE_TIMESTAMP_TZ
189 TYPECODE_TIMESTAMP_LTZ
190 TYPECODE_INTERVAL_YM
110 TYPECODE_INTERVAL_DS
108 TYPECODE_REF
247 TYPECODE_OBJECT
248 TYPECODE_VARRAY
122 TYPECODE_TABLE
12 TYPECODE_NAMEDCOLLECTION
58 TYPECODE_OPAQUE
286 TYPECODE_NCHAR
287 TYPECODE_NVARCHAR2
288 TYPECODE_NCLOB
100 TYPECODE_BFLOAT
101 TYPECODE_BDOUBLE
104 TYPECODE_UROWID

Using the DECODE function code pulled directly out of the view for ALL_TAB_COLUMNS

SELECT DISTINCT c.type# type_code,
decode(c.type#, 1, decode(c.charsetform, 2, 'NVARCHAR2', 'VARCHAR2'),
                2, decode(c.scale, null, decode(c.precision#, null, 'NUMBER', 'FLOAT'), 'NUMBER'),
                8, 'LONG',
                9, decode(c.charsetform, 2, 'NCHAR VARYING', 'VARCHAR'),
                12, 'DATE',
                23, 'RAW', 24, 'LONG RAW',
                69, 'ROWID',
                96, decode(c.charsetform, 2, 'NCHAR', 'CHAR'),
                100, 'BINARY_FLOAT',
                101, 'BINARY_DOUBLE',
                105, 'MLSLABEL',
                106, 'MLSLABEL',
                111, 'REF',
                112, decode(c.charsetform, 2, 'NCLOB', 'CLOB'),
                113, 'BLOB', 114, 'BFILE', 115, 'CFILE',
                121, 'USER_TYPE',
                122, 'USER_TYPE',
                123, 'USER_TYPE',
                178, 'TIME(' ||c.scale|| ')',
                179, 'TIME(' ||c.scale|| ')' || ' WITH TIME ZONE',
                180, 'TIMESTAMP(' ||c.scale|| ')',
                181, 'TIMESTAMP(' ||c.scale|| ')'||' WITH TIME ZONE',
                231, 'TIMESTAMP(' ||c.scale|| ')'||' WITH LOCAL TIME ZONE',
                182, 'INTERVAL YEAR(' ||c.precision#||') TO MONTH',
                183, 'INTERVAL DAY(' ||c.precision#||') TO SECOND(' ||
                      c.scale || ')',
                208, 'UROWID',
                'UNDEFINED') type_name
FROM sys.col$ c
ORDER BY c.type#
;


来源: stackexchange

问:
How can I filter an Iterable based upon a Predicate?
I want to do a string list filter function using an Iterable<String> and a predicate to select the strings to keep, the other ones must be removed from the list, but I'm not understating how I do the remove.

```<code>static <T> Iterable<T> select(Iterable<T> it, Predicate<T> pred) {
    for (T s: it) {
        if (pred.test(s)==false) {
            // what to do here?
        }
    }
    return ...;
}
</code>```

For this input:

```<code>{"a","","b",""}
</code>```

I expect

```<code>{"a","b"}
</code>```

答:
The alternative solution to Holger's I meant in the comment looks like this:

```<code>static <T> Iterable<T> select(Iterable<T> toIterate, Predicate<T> pred) {
    return () -> new Iterator<T>() {
        Iterator<T> delegate = toIterate.iterator();
        T next = findNextValid();
        public boolean hasNext() {
            return next != null;
        }
        public T next() {
            if (next == null) throw new NoSuchElementException();
            T result = next;
            next = findNextValid();
            return result;
        }
        private T findNextValid() {
            T result = null;
            while (result == null && delegate.hasNext()) {
                T candidate = delegate.next();
                if (pred.test(candidate)) {
                    result = candidate;
                }
            }
            return result;
        }
    };
}
</code>```

The difference is that there's no need for an additional marker for the hasCurrent, and it advances the Iterator before the next element is actually requested. You might consider the latter to be undesirable though.

ContentLicense: CC BY-SA 4.0

Body:
I want to do a string list filter function using an Iterable<String> and a predicate to select the strings to keep, the other ones must be removed from the list, but I'm not understating how I do the remove.

static <T> Iterable<T> select(Iterable<T> it, Predicate<T> pred) {
    for (T s: it) {
        if (pred.test(s)==false) {
            // what to do here?
        }
    }
    return ...;
}

For this input:

{"a","","b",""}

I expect

{"a","b"}

Body:
The alternative solution to Holger's I meant in the comment looks like this:

static <T> Iterable<T> select(Iterable<T> toIterate, Predicate<T> pred) {
    return () -> new Iterator<T>() {
        Iterator<T> delegate = toIterate.iterator();
        T next = findNextValid();
        public boolean hasNext() {
            return next != null;
        }
        public T next() {
            if (next == null) throw new NoSuchElementException();
            T result = next;
            next = findNextValid();
            return result;
        }
        private T findNextValid() {
            T result = null;
            while (result == null && delegate.hasNext()) {
                T candidate = delegate.next();
                if (pred.test(candidate)) {
                    result = candidate;
                }
            }
            return result;
        }
    };
}

The difference is that there's no need for an additional marker for the hasCurrent, and it advances the Iterator before the next element is actually requested. You might consider the latter to be undesirable though.


问:
simple python for loop doesn't update the element
My first time trying anything in python and I can't even get the for loop to update the element. Please, what am I not seeing?

I have this data in a text file

Y|Contents/MacOS/Safari|1500

   N|Contents/MacOS/firefox|1200

  Y|Contents/MacOS/Evernote|1100

And I want to change N in the second line to Y. It prints out the form I want inside the loop but not outside? What is this?

```<code>def setTrackable(self,appName):
        lookfor = 'Contents/MacOS/'+appName
        with open (self.__dataFile,"r") as myfile:
            data = myfile.readlines()

        for line in data:
            temp = line.split("|")
            if (temp[1] == lookfor):
                line = 'Y|'+lookfor+"|"+temp[2]
                print("line: "+line)

        print (data[1])
        with open (self.__dataFile, "w") as myfile:
            myfile.writelines(data)
</code>```

答:
Here is a code sample to produce your desired results, please disregard my use of StringIO since I am reading your file from memory, otherwise you can use the csv module to easily parse this data.

```<code>import csv
from io import StringIO

s = """Y|Contents/MacOS/Safari|1500
N|Contents/MacOS/firefox|1200
Y|Contents/MacOS/Evernote|1100"""

fin = StringIO(s)

def parser(file_obj):
    reader = csv.reader(file_obj, delimiter='|')
    data = []
    for line in reader:
        if line[0] == 'N':
            line[0] = 'Y'
        data.append(line)
    return data

p = parser(fin)

print(p)

[['Y', 'Contents/MacOS/Safari', '1500'],
 ['Y', 'Contents/MacOS/firefox', '1200'],
 ['Y', 'Contents/MacOS/Evernote', '1100']]
</code>```

Body:
My first time trying anything in python and I can't even get the for loop to update the element. Please, what am I not seeing?

I have this data in a text file

Y|Contents/MacOS/Safari|1500

   N|Contents/MacOS/firefox|1200

  Y|Contents/MacOS/Evernote|1100

And I want to change N in the second line to Y. It prints out the form I want inside the loop but not outside? What is this?

def setTrackable(self,appName):
        lookfor = 'Contents/MacOS/'+appName
        with open (self.__dataFile,"r") as myfile:
            data = myfile.readlines()

        for line in data:
            temp = line.split("|")
            if (temp[1] == lookfor):
                line = 'Y|'+lookfor+"|"+temp[2]
                print("line: "+line)

        print (data[1])
        with open (self.__dataFile, "w") as myfile:
            myfile.writelines(data)

Body:
Here is a code sample to produce your desired results, please disregard my use of StringIO since I am reading your file from memory, otherwise you can use the csv module to easily parse this data.

import csv
from io import StringIO

s = """Y|Contents/MacOS/Safari|1500
N|Contents/MacOS/firefox|1200
Y|Contents/MacOS/Evernote|1100"""

fin = StringIO(s)

def parser(file_obj):
    reader = csv.reader(file_obj, delimiter='|')
    data = []
    for line in reader:
        if line[0] == 'N':
            line[0] = 'Y'
        data.append(line)
    return data

p = parser(fin)

print(p)

[['Y', 'Contents/MacOS/Safari', '1500'],
 ['Y', 'Contents/MacOS/firefox', '1200'],
 ['Y', 'Contents/MacOS/Evernote', '1100']]


问:
avrdude: ser_open(): can't open device "\\.\COM3": Access is denied
I get this error message when i try to upload code to my arduino. Does anyone know what is going on?

I have installed the proper drivers and it works fine. My port appears on the menu and it is the corresponent to the device.

bless

答:
If you are on Linux,
just set current user to belong to "dialout" group which is the one who members are allowed to access ports.

Run following:

```<code>sudo adduser yourUserName dialout
</code>```

then relog to take effect

(Ubuntu 18.04)

Body:
If you are on Linux,
just set current user to belong to "dialout" group which is the one who members are allowed to access ports.

Run following:

sudo adduser yourUserName dialout

then relog to take effect

(Ubuntu 18.04)


问:
Connect to SOCKS proxy that requires authentication
I'm using Fiddler Core in my project and have had success routing requests through a SOCKS proxy that does not require authentication.

```<code>oS["x-OverrideGateway"] = string.Format("socks={0}:{1}", ProxyHost, ProxyPort);
</code>```

Now I'm trying to connect to a SOCKS proxy that requires authentication. I've tried adding the below, per this non-SOCKS guide: Authenticate With CBT.

```<code>oS["X-AutoAuth"] = Auth;
</code>```

Where Auth is a string containing the credentials in the form username:password. But the connection always fails with a Bad Gateway exception.  I've been successful connecting to the same SOCKS proxy using curl and following it's guidelines for an authenticated SOCKS proxy.

After X-AutoAuth didn't work I reflected some of the code and found that this authentication mechanism is not used for SOCKS.

答:
We faced a similar issue and created a NuGet library which helps you "transform" SOCKS v4 traffic to SOCKS v5, and we added authentication support as well.

This small example shows how can you hook up FiddlerCore and a SOCKS5 proxy with username and password:

```<code>var localEndpoint = new IPEndPoint(IPAddress.Loopback, 4321);
var remoteEndpoint = new IPEndPoint(IPAddress.Parse("remote proxy IP"), 8080);
ISocksRelayServer relay = new SocksRelayServer.SocksRelayServer(localEndpoint, remoteEndpoint)
{
    Username = "...",
    Password = "..."
};

// Debug to console
relay.OnLogMessage += (sender, s) => Console.WriteLine($"OnLogMessage: {s}");
relay.OnLocalConnect += (sender, endpoint) => Console.WriteLine($"OnLocalConnect: {endpoint}");
relay.OnRemoteConnect += (sender, endpoint) => Console.WriteLine($"OnRemoteConnect: {endpoint}");

// Start relay server
relay.Start();

// Start FiddlerCore
FiddlerApplication.Startup(...);

// Set upstream gateway before requests
FiddlerApplication.BeforeRequest += session =>
{
    session["x-OverrideGateway"] = relay.LocalEndPoint.ToString();
}
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I'm using Fiddler Core in my project and have had success routing requests through a SOCKS proxy that does not require authentication.

oS["x-OverrideGateway"] = string.Format("socks={0}:{1}", ProxyHost, ProxyPort);

Now I'm trying to connect to a SOCKS proxy that requires authentication. I've tried adding the below, per this non-SOCKS guide: Authenticate With CBT.

oS["X-AutoAuth"] = Auth;

Where Auth is a string containing the credentials in the form username:password. But the connection always fails with a Bad Gateway exception.  I've been successful connecting to the same SOCKS proxy using curl and following it's guidelines for an authenticated SOCKS proxy.

After X-AutoAuth didn't work I reflected some of the code and found that this authentication mechanism is not used for SOCKS.

Body:
We faced a similar issue and created a NuGet library which helps you "transform" SOCKS v4 traffic to SOCKS v5, and we added authentication support as well.

This small example shows how can you hook up FiddlerCore and a SOCKS5 proxy with username and password:

var localEndpoint = new IPEndPoint(IPAddress.Loopback, 4321);
var remoteEndpoint = new IPEndPoint(IPAddress.Parse("remote proxy IP"), 8080);
ISocksRelayServer relay = new SocksRelayServer.SocksRelayServer(localEndpoint, remoteEndpoint)
{
    Username = "...",
    Password = "..."
};

// Debug to console
relay.OnLogMessage += (sender, s) => Console.WriteLine($"OnLogMessage: {s}");
relay.OnLocalConnect += (sender, endpoint) => Console.WriteLine($"OnLocalConnect: {endpoint}");
relay.OnRemoteConnect += (sender, endpoint) => Console.WriteLine($"OnRemoteConnect: {endpoint}");

// Start relay server
relay.Start();

// Start FiddlerCore
FiddlerApplication.Startup(...);

// Set upstream gateway before requests
FiddlerApplication.BeforeRequest += session =>
{
    session["x-OverrideGateway"] = relay.LocalEndPoint.ToString();
}


问:
Parsing Returned JSON
I am trying to print some of the values from the API call to the console, but I am receiving an error I am unfamiliar with:

System.Xml.XmlException: 'Data at the root level is invalid. Line 1, position 1.'

I think that I might be incorrectly accessing the values returned to by the API, but I am not 100% sure whats wrong

I've looked at a few other posts about this same issue and tried to implement the fixes suggested, but none of them were able to solve my problem. I think mine is unique as my JSON object is different from others I have seen.

```<code>namespace CarQueryAPI
{
    class CarResult
    {
        public string model_name { get; set; }
    }
}

namespace CarQueryAPI
{
    class Program
    {
        public static void Main(string[] args)
    {
        apiCall();
        Console.ReadKey();
    }

    public static void apiCall()
    {
        //client to invoke the car query api
        WebClient client = new WebClient();

        Task<String> carQueryTask = null;// Task<string> that queries carQuery

        // url for API (no key required)
        string url = "https://www.carqueryapi.com/api/0.3/?callback=?&cmd=getTrims&make=toyota&model=camry&year=2005";

        try
        {
           //request info from the api
            carQueryTask = client.DownloadStringTaskAsync(url);

            string result = carQueryTask.Result.Remove(0, 2);
            result = result.Remove(result.Length - 4, 4);

            dynamic json = JsonConvert.DeserializeObject(result);

            string model_name = json.model_name;

            Console.WriteLine(model_name);
        }

        catch (WebException)
        {
            // check whether Task failed
            if (carQueryTask.Status == TaskStatus.Faulted)
            {
                Console.WriteLine("Call Failed");
            }
        }
    }
}
</code>```

}

i need the result to e just the model_name of the car being returned from the call so that I can see how to capture the others as well.

The following is some of the JSON object being returned:

```<code>?({"Trims":[{"model_id":"38988","model_make_id":"toyota","model_name":"Camry","model_trim":"2.4 GLi Automatic","model_year":"2005","model_body":"Sedan","model_engine_position":"Front","model_engine_cc":"2360","model_engine_cyl":null,"model_engine_type":null,"model_engine_valves_per_cyl":"4","model_engine_power_ps":"152","model_engine_power_rpm":"5600","model_engine_torque_nm":"218","model_engine_torque_rpm":"4000","model_engine_bore_mm":"88.5","model_engine_stroke_mm":"96.0","model_engine_compression":"9.6:1","model_engine_fuel":null,"model_top_speed_kph":"197","model_0_to_100_kph":"10.6","model_drive":"Front","model_transmission_type":"Automatic","model_seats":"5","model_doors":"4","model_weight_kg":"920","model_length_mm":null,"model_width_mm":null,"model_height_mm":null,"model_wheelbase_mm":"2730","model_lkm_hwy":null,"model_lkm_mixed":null,"model_lkm_city":null,"model_fuel_cap_l":"70","model_sold_in_us":"1","model_co2":null,"model_make_display":"Toyota","make_display":"Toyota","make_country":"Japan"},{"model_id":"38999","model_make_id":"toyota","model_name":"Camry","model_trim":"2.4 WT-i","model_year":"2005","model_body":"Sedan","model_engine_position":"Front","model_engine_cc":"2362","model_engine_cyl":"4","model_engine_type":"in-line","model_engine_valves_per_cyl":"4","model_engine_power_ps":"153","model_engine_power_rpm":"5600","model_engine_torque_nm":"218","model_engine_torque_rpm":"3800","model_engine_bore_mm":"88.5","model_engine_stroke_mm":"96.0","model_engine_compression":"9.6:1","model_engine_fuel":"Gasoline - Premium","model_top_speed_kph":"210","model_0_to_100_kph":"9.4","model_drive":"Front","model_transmission_type":null,"model_seats":"5","model_doors":"4","model_weight_kg":"1465","model_length_mm":"4820","model_width_mm":"1800","model_height_mm":"1510","model_wheelbase_mm":"2680","model_lkm_hwy":null,"model_lkm_mixed":"8.7","model_lkm_city":null,"model_fuel_cap_l":"70","model_sold_in_us":"1","model_co2":null,"model_make_display":"Toyota","make_display":"Toyota","make_country":"Japan"}
</code>```

...etc

EDIT:

Changed some of the code as you all suggested, but now I am receiving this error:

Unexpected end when deserializing object. Path 'Trims[18]', line 1, position 19143

答: I followed the carqueryapi link that was in your provided code, and after removing "?(" from the beginning, and ");" from the end the JSON appears to format correctly. Is there any reason those extra characters are needed? You may need to remove them.</p>

Title: Parsing Returned JSON

Body:
I am trying to print some of the values from the API call to the console, but I am receiving an error I am unfamiliar with:

System.Xml.XmlException: 'Data at the root level is invalid. Line 1, position 1.'

I think that I might be incorrectly accessing the values returned to by the API, but I am not 100% sure whats wrong

I've looked at a few other posts about this same issue and tried to implement the fixes suggested, but none of them were able to solve my problem. I think mine is unique as my JSON object is different from others I have seen.

namespace CarQueryAPI
{
    class CarResult
    {
        public string model_name { get; set; }
    }
}

namespace CarQueryAPI
{
    class Program
    {
        public static void Main(string[] args)
    {
        apiCall();
        Console.ReadKey();
    }

    public static void apiCall()
    {
        //client to invoke the car query api
        WebClient client = new WebClient();

        Task<String> carQueryTask = null;// Task<string> that queries carQuery

        // url for API (no key required)
        string url = "https://www.carqueryapi.com/api/0.3/?callback=?&cmd=getTrims&make=toyota&model=camry&year=2005";

        try
        {
           //request info from the api
            carQueryTask = client.DownloadStringTaskAsync(url);

            string result = carQueryTask.Result.Remove(0, 2);
            result = result.Remove(result.Length - 4, 4);

            dynamic json = JsonConvert.DeserializeObject(result);

            string model_name = json.model_name;

            Console.WriteLine(model_name);
        }

        catch (WebException)
        {
            // check whether Task failed
            if (carQueryTask.Status == TaskStatus.Faulted)
            {
                Console.WriteLine("Call Failed");
            }
        }
    }
}

}

i need the result to e just the model_name of the car being returned from the call so that I can see how to capture the others as well.

The following is some of the JSON object being returned:

?({"Trims":[{"model_id":"38988","model_make_id":"toyota","model_name":"Camry","model_trim":"2.4 GLi Automatic","model_year":"2005","model_body":"Sedan","model_engine_position":"Front","model_engine_cc":"2360","model_engine_cyl":null,"model_engine_type":null,"model_engine_valves_per_cyl":"4","model_engine_power_ps":"152","model_engine_power_rpm":"5600","model_engine_torque_nm":"218","model_engine_torque_rpm":"4000","model_engine_bore_mm":"88.5","model_engine_stroke_mm":"96.0","model_engine_compression":"9.6:1","model_engine_fuel":null,"model_top_speed_kph":"197","model_0_to_100_kph":"10.6","model_drive":"Front","model_transmission_type":"Automatic","model_seats":"5","model_doors":"4","model_weight_kg":"920","model_length_mm":null,"model_width_mm":null,"model_height_mm":null,"model_wheelbase_mm":"2730","model_lkm_hwy":null,"model_lkm_mixed":null,"model_lkm_city":null,"model_fuel_cap_l":"70","model_sold_in_us":"1","model_co2":null,"model_make_display":"Toyota","make_display":"Toyota","make_country":"Japan"},{"model_id":"38999","model_make_id":"toyota","model_name":"Camry","model_trim":"2.4 WT-i","model_year":"2005","model_body":"Sedan","model_engine_position":"Front","model_engine_cc":"2362","model_engine_cyl":"4","model_engine_type":"in-line","model_engine_valves_per_cyl":"4","model_engine_power_ps":"153","model_engine_power_rpm":"5600","model_engine_torque_nm":"218","model_engine_torque_rpm":"3800","model_engine_bore_mm":"88.5","model_engine_stroke_mm":"96.0","model_engine_compression":"9.6:1","model_engine_fuel":"Gasoline - Premium","model_top_speed_kph":"210","model_0_to_100_kph":"9.4","model_drive":"Front","model_transmission_type":null,"model_seats":"5","model_doors":"4","model_weight_kg":"1465","model_length_mm":"4820","model_width_mm":"1800","model_height_mm":"1510","model_wheelbase_mm":"2680","model_lkm_hwy":null,"model_lkm_mixed":"8.7","model_lkm_city":null,"model_fuel_cap_l":"70","model_sold_in_us":"1","model_co2":null,"model_make_display":"Toyota","make_display":"Toyota","make_country":"Japan"}

...etc

EDIT:

Changed some of the code as you all suggested, but now I am receiving this error:

Unexpected end when deserializing object. Path 'Trims[18]', line 1, position 19143


问:
Azure guest user access issue
I am running a webapp on Azure. This app is a botservice whats connecting to Cortana. When I logged in with original account which I creted the Azure portal its works but if I try use the app with an other ms account (ofc I added the account in azure ad as a guest user and gave it GA).

Tried to remove the account and add it back again - desnt help.
Tried to add it as a co - admin - doesnt helped at all.
Tried to add to Access control (IAM) as an owner - doesnt helped.
Tried to add ti Access control (IAM) in the app services - doesnt helped.

答:
Cortana Skills can be shared with other users either by "Publish to group " or "Publish to world". You could create a group and then invite users to join the group to be able to access the skill.
Please go through the documentation : https://learn.microsoft.com/en-us/cortana/skills/mva71-share-skill

If not shared , the skill will be accessible only to the developer who created the skill.

Title: Azure guest user access issue


答:
Use enumeration and modify the data accordingly:

```<code>for i, line in enumerate(data):
    temp = line.split("|")
    if (temp[1] == lookfor):
        line = '|'.join(['Y', lookfor, temp[2]])
        data[i] = line
        print("line: "+line)
</code>```

Title: simple python for loop doesn't update the element

Body:
Use enumeration and modify the data accordingly:

for i, line in enumerate(data):
    temp = line.split("|")
    if (temp[1] == lookfor):
        line = '|'.join(['Y', lookfor, temp[2]])
        data[i] = line
        print("line: "+line)


答:
You never do anything to update the values in data; all you do is define a new variable called line, print it, then discard it. You should be appending that new value to a list.

A more Pythonic implementation would be:

```<code>with open (self.__dataFile,"r") as myfile:
    data = []

    for line in myfile:
        temp = line.split("|")
        if temp[1] == lookfor:
            line = 'Y|'+lookfor+"|"+temp[2]
            print("line: "+line)
        data.append(line)
</code>```

Body:
You never do anything to update the values in data; all you do is define a new variable called line, print it, then discard it. You should be appending that new value to a list.

A more Pythonic implementation would be:

with open (self.__dataFile,"r") as myfile:
    data = []

    for line in myfile:
        temp = line.split("|")
        if temp[1] == lookfor:
            line = 'Y|'+lookfor+"|"+temp[2]
            print("line: "+line)
        data.append(line)


问:
how to create crontab for auto run command "service php-fpm restart" in centos 6?
I want to run command "service php-fpm restart" each 1 minute and I tried to using contab.

I tried this code in contab file:

```<code>0/1 * * * * service php-fpm restart
</code>```

In the file crontab from /var/spool/con/root, I put these code:

```<code>0 2 * * * chown -R nginx:nginx /home/
0 3 * * * echo 1 > /proc/sys/vm/drop_caches
0 4 * * * reboot
0/1 * * * * service php-fpm restart
</code>```

All the command almost are running , but the command "service php-fpm restart" not.

答:
Besides the big question of why you need to restart every minute (feels like fixing a flat by making sure you have a airpump handy instead of stopping the leak) I wonder how you got a minute of "0/1"? To run something every minute you give it five stars</p>

```<code>* * * * * /sbin/service php-fpm restart
</code>```

Body:
I want to run command "service php-fpm restart" each 1 minute and I tried to using contab.

I tried this code in contab file:

0/1 * * * * service php-fpm restart

In the file crontab from /var/spool/con/root, I put these code:

0 2 * * * chown -R nginx:nginx /home/
0 3 * * * echo 1 > /proc/sys/vm/drop_caches
0 4 * * * reboot
0/1 * * * * service php-fpm restart

All the command almost are running , but the command "service php-fpm restart" not.

Body:
Besides the big question of why you need to restart every minute (feels like fixing a flat by making sure you have a airpump handy instead of stopping the leak) I wonder how you got a minute of "0/1"? To run something every minute you give it five stars

* * * * * /sbin/service php-fpm restart


问:
Could the STAN number be repeteable and random?
I'm developing a Connector with some bank, and we're using the ISO8583 protocol, right now, i'm setting the STAN(field 11) with some random number generated with a random generator but sometimes I have some number collisions, the question is, could I safely use this generator or do I need to make the STAN a sequential number?

Thanks in advance.

答:
The System Trace Audit Number (STAN) ISO-8583 number has different values and is maintained basically between relationships within the transaction. That is it can stay the same or the same transaction will have many STANs over its transaction path but it SHOULD be the same between two end point and it is usually controlled in settings whos STAN to use.

For Example:

Terminal -> Terminal Driver -> Switch 1->Switch 2->Issuer

The STAN is say assign by the terminal driver and then remains constant at minimum for the following relationships... though may change for each relationship.

Terminal Driver - Switch 1

Switch 1 -> Switch 2

Switch 2 -> Issuer

Note that internally within each system to the STAN may be unique as well but it needs to keep a unique STAN for each relationship.. and it shouldn't change between the request and response as it is needed for multi-part transactions (Single PA, Multiple Completions & Multi-PA, Single Completion) as well as for reversals and such in Data Element 90.


问:
How is my simple latch diagram wrong and why is flip-flop latches correct?
I'm having a hard time understanding the way flip-flops actually flip states and wondering why is it such a design commonly used, when simpler design could suffice, from my current opinion.

I'm hoping that after showing you my version of a latch diagram, someone could point out the flaws and that may help me understand why a flip-flop latch is better.

I was reading a book and bumped into some "general" form of latch:
https://i.imgur.com/nkldf4u.png (sorry, I don't have the reputation insert images)

I've been on it for about 2 hours trying to truly grasp the mechanism. Seeing that I can't do it, I've draw my version of a latch:

https://i.imgur.com/fFgpNzR.png

The blue diagram, the one from the book, is harder to follow because some gates will switch 2 times when the inputs switches once, because as the output is tunneled back as input to the same gate, the output may change base on its previous value.

My version of the diagram, the one in black, uses a more programmable approach. I take the current state C and decide if it differs from the input state and output it into A. I use A value in an AND gate with the enable wire to decide if both criteria is met and put it in B. Finally, I'm using a XOR to change the state and output as C.

I'm hoping someone can tell me why is this bad, what I haven't taken into consideration or why a more complex mechanism is needed.

Thank you in anticipation.

答:
As far as I can tell, your latch implementation should work.

However, there is more to low-level digital design than just gate count. In actual circuits, not all gates are created equal as the actual implementation of these gates can make some more "costly" than others (usually measured in area/transistor count and complexity in routing). For typical CMOS implementations, NAND gates are really cheap (only 4 transistors for two input NAND) so alot of primitives use NAND (or NOR) as a building block for more complex designs. XOR is generally a more complicated gate to implement, most CMOS implementations Ive seen use 8 transistors. Without going through and optimizing your design, it might take at least 20 or more transistors to implement while the latch design from the textbook only takes 16 (A 20%+ savings in area per bit stored, which is quite significant). There is alot more at play here than just transistor count as well; things like transistor sizing, routing and trace sizing, power considerations and glitch protection when actually going through and implementing designs, so even this simple analysis is incomplete and might be missing reasons for the textbook implementation vs yours (or vice versa).

Asynchronous sequential logic (which is what latch/flipflop implementations are) can be difficult to understand which is why most circuits use higher-level constructs and treat these details as black boxes (and it also creates a nice abstraction where the actual implementation doesnt matter so long as the properties of that element are preserved).


问:
How to programmatically transfer a lot of data between tables?
i have two tables where in the first one i have 14 millions and in the second one i have 1.5 million of data.

So i wonder how could i transfer this data to another table to be normalized ?
And how do i convert some type to another, for example: i have a field called 'year' but its type is varchar, but i want it an integer instead, how do i do that ?

I thought about do this using JDBC in a loop while from java, but i think this is not effeciently.

```<code>// 1.5 million of data
CREATE TABLE dbo.directorsmovies
    (
    movieid    INT NULL,
    directorid INT NULL,
    dname      VARCHAR (500) NULL,
    addition   VARCHAR (1000) NULL
    )

//14 million of data
CREATE TABLE dbo.movies
    (
    movieid      VARCHAR (20) NULL,
    title        VARCHAR (400) NULL,
    mvyear       VARCHAR (100) NULL,
    actorid      VARCHAR (20) NULL,
    actorname    VARCHAR (250) NULL,
    sex          CHAR (1) NULL,
    as_character VARCHAR (1500) NULL,
    languages    VARCHAR (1500) NULL,
    genres       VARCHAR (100) NULL
    )
</code>```

And this is my new tables:

```<code>DROP TABLE actor
CREATE TABLE actor (
    id INT PRIMARY KEY IDENTITY,
    name VARCHAR(200) NOT NULL,
    sex VARCHAR(1) NOT NULL
)

DROP TABLE actor_character
CREATE TABLE actor_character(
    id INT PRIMARY KEY IDENTITY,
    character VARCHAR(100)
)

DROP TABLE director
CREATE TABLE director(
    id INT PRIMARY KEY IDENTITY,
    name VARCHAR(200) NOT NULL,
    addition VARCHAR(150)
)

DROP TABLE movie
CREATE TABLE movie(
    id INT PRIMARY KEY IDENTITY,
    title VARCHAR(200) NOT NULL,
    year INT
)

DROP TABLE language
CREATE TABLE language(
    id INT PRIMARY KEY IDENTITY,
    language VARCHAR (100) NOT NULL
)

DROP TABLE genre
CREATE TABLE genre(
    id INT PRIMARY KEY IDENTITY,
    genre VARCHAR(100) NOT NULL
)

DROP TABLE director_movie
CREATE TABLE director_movie(
    idDirector INT,
    idMovie INT,
    CONSTRAINT fk_director_movie_1 FOREIGN KEY (idDirector) REFERENCES director(id),
    CONSTRAINT fk_director_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT pk_director_movie PRIMARY KEY(idDirector,idMovie)
)

DROP TABLE genre_movie
CREATE TABLE genre_movie(
    idGenre INT,
    idMovie INT,
    CONSTRAINT fk_genre_movie_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT fk_genre_movie_2 FOREIGN KEY (idGenre) REFERENCES genre(id),
    CONSTRAINT pk_genre_movie PRIMARY KEY (idMovie, idGenre)
)

DROP TABLE language_movie
CREATE TABLE language_movie(
    idLanguage INT,
    idMovie INT,
    CONSTRAINT fk_language_movie_1 FOREIGN KEY (idLanguage) REFERENCES language(id),
    CONSTRAINT fk_language_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT pk_language_movie PRIMARY KEY (idLanguage, idMovie)
)

DROP TABLE movie_actor
CREATE TABLE movie_actor(
    idMovie INT,
    idActor INT,
    CONSTRAINT fk_movie_actor_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT fk_movie_actor_2 FOREIGN KEY (idActor) REFERENCES actor(id),
    CONSTRAINT pk_movie_actor PRIMARY KEY (idMovie,idActor)
)
</code>```

UPDATE:
I'm using SQL Server 2008.
Sorry guys i forgot to mention that are different databases :

The not normalized is call disciplinedb and the my normalized call imdb.

Best regards,
Valter Henrique.

答:
Though my tables were dwarfs compared to yours, I got over this kind of problem once with stored procedures. For MySQL, below is a simplified (and untested) essence of my script, but something similar should work with all major SQL bases.

First you should just add a new integer year column (int_year in example) and then iterate over all rows using the procedure below:

```<code>DROP PROCEDURE IF EXISTS move_data;
CREATE PROCEDURE move_data()
BEGIN
  DECLARE done INT DEFAULT 0;
  DECLARE orig_id INT DEFAULT 0;
  DECLARE orig_year VARCHAR DEFAULT "";
  DECLARE cur1 CURSOR FOR SELECT id, year FROM table1;
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;

  OPEN cur1;

  PREPARE stmt FROM "UPDATE table1 SET int_year = ? WHERE id = ?";

  read_loop: LOOP
    FETCH cur1 INTO orig_id, orig_year;
    IF done THEN
      LEAVE read_loop;
    END IF;

    SET @year= orig_year;
    SET @id = orig_id;

    EXECUTE stmt USING @orig_year, @id;
  END LOOP;
  CLOSE cur1;
END;
</code>```

And to start the procedure, just CALL move_data().

The above SQL has two major ideas to speed it up:

Use CURSORS to iterate over a large table

Use PREPARED statement to quickly execute pre-known commands

PS. for my case this speeded things up from ages to seconds, though in your case it can still take a considerable amount of time. So it would be probably best to execute from command line, not some web interface (e.g. PhpMyAdmin).

Body:
i have two tables where in the first one i have 14 millions and in the second one i have 1.5 million of data.

So i wonder how could i transfer this data to another table to be normalized ?
And how do i convert some type to another, for example: i have a field called 'year' but its type is varchar, but i want it an integer instead, how do i do that ?

I thought about do this using JDBC in a loop while from java, but i think this is not effeciently.

// 1.5 million of data
CREATE TABLE dbo.directorsmovies
    (
    movieid    INT NULL,
    directorid INT NULL,
    dname      VARCHAR (500) NULL,
    addition   VARCHAR (1000) NULL
    )

//14 million of data
CREATE TABLE dbo.movies
    (
    movieid      VARCHAR (20) NULL,
    title        VARCHAR (400) NULL,
    mvyear       VARCHAR (100) NULL,
    actorid      VARCHAR (20) NULL,
    actorname    VARCHAR (250) NULL,
    sex          CHAR (1) NULL,
    as_character VARCHAR (1500) NULL,
    languages    VARCHAR (1500) NULL,
    genres       VARCHAR (100) NULL
    )

And this is my new tables:

DROP TABLE actor
CREATE TABLE actor (
    id INT PRIMARY KEY IDENTITY,
    name VARCHAR(200) NOT NULL,
    sex VARCHAR(1) NOT NULL
)

DROP TABLE actor_character
CREATE TABLE actor_character(
    id INT PRIMARY KEY IDENTITY,
    character VARCHAR(100)
)

DROP TABLE director
CREATE TABLE director(
    id INT PRIMARY KEY IDENTITY,
    name VARCHAR(200) NOT NULL,
    addition VARCHAR(150)
)

DROP TABLE movie
CREATE TABLE movie(
    id INT PRIMARY KEY IDENTITY,
    title VARCHAR(200) NOT NULL,
    year INT
)

DROP TABLE language
CREATE TABLE language(
    id INT PRIMARY KEY IDENTITY,
    language VARCHAR (100) NOT NULL
)

DROP TABLE genre
CREATE TABLE genre(
    id INT PRIMARY KEY IDENTITY,
    genre VARCHAR(100) NOT NULL
)

DROP TABLE director_movie
CREATE TABLE director_movie(
    idDirector INT,
    idMovie INT,
    CONSTRAINT fk_director_movie_1 FOREIGN KEY (idDirector) REFERENCES director(id),
    CONSTRAINT fk_director_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT pk_director_movie PRIMARY KEY(idDirector,idMovie)
)

DROP TABLE genre_movie
CREATE TABLE genre_movie(
    idGenre INT,
    idMovie INT,
    CONSTRAINT fk_genre_movie_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT fk_genre_movie_2 FOREIGN KEY (idGenre) REFERENCES genre(id),
    CONSTRAINT pk_genre_movie PRIMARY KEY (idMovie, idGenre)
)

DROP TABLE language_movie
CREATE TABLE language_movie(
    idLanguage INT,
    idMovie INT,
    CONSTRAINT fk_language_movie_1 FOREIGN KEY (idLanguage) REFERENCES language(id),
    CONSTRAINT fk_language_movie_2 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT pk_language_movie PRIMARY KEY (idLanguage, idMovie)
)

DROP TABLE movie_actor
CREATE TABLE movie_actor(
    idMovie INT,
    idActor INT,
    CONSTRAINT fk_movie_actor_1 FOREIGN KEY (idMovie) REFERENCES movie(id),
    CONSTRAINT fk_movie_actor_2 FOREIGN KEY (idActor) REFERENCES actor(id),
    CONSTRAINT pk_movie_actor PRIMARY KEY (idMovie,idActor)
)

UPDATE:
I'm using SQL Server 2008.
Sorry guys i forgot to mention that are different databases :

The not normalized is call disciplinedb and the my normalized call imdb.

Best regards,
Valter Henrique.

Body:
Though my tables were dwarfs compared to yours, I got over this kind of problem once with stored procedures. For MySQL, below is a simplified (and untested) essence of my script, but something similar should work with all major SQL bases.

First you should just add a new integer year column (int_year in example) and then iterate over all rows using the procedure below:

DROP PROCEDURE IF EXISTS move_data;
CREATE PROCEDURE move_data()
BEGIN
  DECLARE done INT DEFAULT 0;
  DECLARE orig_id INT DEFAULT 0;
  DECLARE orig_year VARCHAR DEFAULT "";
  DECLARE cur1 CURSOR FOR SELECT id, year FROM table1;
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;

  OPEN cur1;

  PREPARE stmt FROM "UPDATE table1 SET int_year = ? WHERE id = ?";

  read_loop: LOOP
    FETCH cur1 INTO orig_id, orig_year;
    IF done THEN
      LEAVE read_loop;
    END IF;

    SET @year= orig_year;
    SET @id = orig_id;

    EXECUTE stmt USING @orig_year, @id;
  END LOOP;
  CLOSE cur1;
END;

And to start the procedure, just CALL move_data().

The above SQL has two major ideas to speed it up:

Use CURSORS to iterate over a large table

Use PREPARED statement to quickly execute pre-known commands

PS. for my case this speeded things up from ages to seconds, though in your case it can still take a considerable amount of time. So it would be probably best to execute from command line, not some web interface (e.g. PhpMyAdmin).


问:
How to select (slice) in Pandas multiple rows and multiple non continuous columns?
I have a dataFrame and I want a range of specific rows and at the same time a range that is composed of a continuous range of columns plus an additional one.  Included is a piece of code that produces such a dataFrame

```<code>import pandas as pd
import numpy as np
np.random.seed(5)
dF = pd.DataFrame(np.random.randint(100, size=(100, 6)),
              columns=list('ABCDEF'),
              index=['R{}'.format(i) for i in range(100)])
dF.head()
</code>```

This works fine:

```<code>dF.loc[:, 'C':'E']
</code>```

But I need something like this, that produces an error:

```<code>dF.loc['R95':, ['A':'C', 'F']]
</code>```

The expected result has to include from row 'R95' onwards and columns 'A', 'C' and 'F'

答:
pandas indexing allows the following ways to indexing a dataframe (quoting from the docs):

A single label, e.g. 5 or 'a' (Note that 5 is interpreted as a label of the index. This use is not an integer position along the index.).

A list or array of labels ['a', 'b', 'c'].

A slice object with labels 'a':'f' (Note that contrary to usual python slices, both the start and the stop are included, when present in the index! See Slicing with labels.).

A boolean array

A callable function with one argument (the calling Series, DataFrame or Panel) and that returns valid output for indexing (one of the above).

So you need to use something a bit more complex.

For example, you could use pandas.concat to select the columns separately and then join together the dataframes:

```<code>pd.concat([dF.loc['R95':, 'A':'C'], dF.loc['R95':,'F']], axis=1)
</code>```

This gives:

```<code>      A   B   C   F
R95  19   7  76   0
R96  57  31  86  64
R97  51  12  59  33
R98  24   7  68  69
R99  81  20  86  70
</code>```

Body:
I have a dataFrame and I want a range of specific rows and at the same time a range that is composed of a continuous range of columns plus an additional one.  Included is a piece of code that produces such a dataFrame

import pandas as pd
import numpy as np
np.random.seed(5)
dF = pd.DataFrame(np.random.randint(100, size=(100, 6)),
              columns=list('ABCDEF'),
              index=['R{}'.format(i) for i in range(100)])
dF.head()

This works fine:

dF.loc[:, 'C':'E']

But I need something like this, that produces an error:

dF.loc['R95':, ['A':'C', 'F']]

The expected result has to include from row 'R95' onwards and columns 'A', 'C' and 'F'

Body:
pandas indexing allows the following ways to indexing a dataframe (quoting from the docs):

A single label, e.g. 5 or 'a' (Note that 5 is interpreted as a label of the index. This use is not an integer position along the index.).

A list or array of labels ['a', 'b', 'c'].

A slice object with labels 'a':'f' (Note that contrary to usual python slices, both the start and the stop are included, when present in the index! See Slicing with labels.).

A boolean array

A callable function with one argument (the calling Series, DataFrame or Panel) and that returns valid output for indexing (one of the above).

So you need to use something a bit more complex.

For example, you could use pandas.concat to select the columns separately and then join together the dataframes:

pd.concat([dF.loc['R95':, 'A':'C'], dF.loc['R95':,'F']], axis=1)

This gives:

      A   B   C   F
R95  19   7  76   0
R96  57  31  86  64
R97  51  12  59  33
R98  24   7  68  69
R99  81  20  86  70


问:
Generate random string/characters in JavaScript
I want a 5 character string composed of characters picked randomly from the set [a-zA-Z0-9].

What's the best way to do this with JavaScript?

答:
const c = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
const s = [...Array(5)].map(_ => c[~~(Math.random()*c.length)]).join('')
```


问:
python ldap search with flask-kerberos credentials
Is there any way to use the authenticated user or the server keytab in a python3/flask-kerberos application to query ldap?

I am using flask-kerberos connecting to active directory and is authenticating correctly.  I need to add authorization now based on the AD group and understand ldap is typically the best practice.
I am able to query successfully via ldap3, but I do not want to hardcode ldap bind credentials in my code.

The sasl/kerberos bind option for ldap3 looks like it needs an OS level credential-cache, but I would like to do this completely within the application.

Looking for any suggestions/alternatives to do this.

答:
sounds like you are mixing up authentication and authorization, but with no useful information provided regarding your code or setup its hard to help.

Take a look at something like this, and use ldap as impersonation to authenticate your users using their own credentials.


问:
Link error while using MinGW compiler (can't find __main)
I'm trying to compile a very simple program with MinGW on Windows, but I still got link errors. The program to be compiled is just C++ hello world.

```<code>Z:\dev>type test.cpp
#include <iostream>

int main() {
  std::cout << "Hello World!\n";
  return 0;
}
</code>```

Of course, just using MinGW's g++ is okay.

```<code>Z:\dev>g++ test.cpp -o test.exe
Z:\dev>test.exe
Hello World!
</code>```

However, I tried to separate compilation and linking, but failed.

```<code>Z:\dev>g++ -c test.cpp -o test.o

Z:\dev>ld test.o -o test.exe
test.o:test.cpp:(.text+0xa): undefined reference to `__main'
test.o:test.cpp:(.text+0x19): undefined reference to `std::cout'
test.o:test.cpp:(.text+0x1e): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& s
 <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)'
test.o:test.cpp:(.text+0x37): undefined reference to `std::ios_base::Init::~Init()'
test.o:test.cpp:(.text+0x5a): undefined reference to `std::ios_base::Init::Init()'
test.o:test.cpp:(.text+0x66): undefined reference to `atexit'
</code>```

It is obvious that I've missed some libraries. So, I tried to link with several MinGW's libraries, but still no good such as -lmsvcrt. I also did lstdc++, but still __main cannot be found and tons of warning messages.

Could you help me out which libraries should be linked together?

答: Link with -lgcc if you need to use ld. Otherwise, use g++ to link.

ContentLicense: CC BY-SA 2.5

Body:
I'm trying to compile a very simple program with MinGW on Windows, but I still got link errors. The program to be compiled is just C++ hello world.

Z:\dev>type test.cpp
#include <iostream>

int main() {
  std::cout << "Hello World!\n";
  return 0;
}

Of course, just using MinGW's g++ is okay.

Z:\dev>g++ test.cpp -o test.exe
Z:\dev>test.exe
Hello World!

However, I tried to separate compilation and linking, but failed.

Z:\dev>g++ -c test.cpp -o test.o

Z:\dev>ld test.o -o test.exe
test.o:test.cpp:(.text+0xa): undefined reference to `__main'
test.o:test.cpp:(.text+0x19): undefined reference to `std::cout'
test.o:test.cpp:(.text+0x1e): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& s
 <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)'
test.o:test.cpp:(.text+0x37): undefined reference to `std::ios_base::Init::~Init()'
test.o:test.cpp:(.text+0x5a): undefined reference to `std::ios_base::Init::Init()'
test.o:test.cpp:(.text+0x66): undefined reference to `atexit'

It is obvious that I've missed some libraries. So, I tried to link with several MinGW's libraries, but still no good such as -lmsvcrt. I also did lstdc++, but still __main cannot be found and tons of warning messages.

Could you help me out which libraries should be linked together?


问:
why I can't get all documents stored in firestore sub collection?
I have users collection that has sub collection called attendedEvents like the picture below. as you can see there are 3 documents in the attendedEvents sub collection

I try to get all documents available on that sub collection by using the code below, I just want to get it all, without order, limit or anything using getDocuments

```<code>    func getAttendedEventsFromBeginning(completion: @escaping (_ eventID: [String]?,QueryDocumentSnapshot?)->Void) {

        FirestoreDocumentReference.users(uidUser: uid).reference().collection("attendedEvents")
            .getDocuments { (snapshot, error) in

                let lastDocument = snapshot?.documents.last

                if let error = error {
                    completion(nil,lastDocument)
                    print("Error when fetching attended events documents in user subcollection: \(error.localizedDescription)")
                } else {
                    print("Successfully fetching attended events documents in user subcollection from Firestore ")

                    guard let documentsSnapshot = snapshot else {
                        completion(nil,lastDocument)
                        return
                    }

                    let eventDocuments = documentsSnapshot.documents
                    print("xxxxx")
                    print(eventDocuments)

                    var attendedeventIDs = [String]()

                    for document in eventDocuments {
                        let eventDictionary = document.data()
                        let theEvent = eventDictionary["eventID"] as! String
                        attendedeventIDs.append(theEvent)
                    }

                    print(attendedeventIDs)

                    completion(attendedeventIDs,lastDocument)
                }
            }
    }
</code>```

but as a result, I just got 2 document snapshot, it should be 3 documents

but if I delete the app from simulator and install it again, I got all the three data. is is cached or what?

答:
I have been in the same situation. In my case, this happens because in my document it only contains subcollection. And that will cause the document itself to not be shown in queries or snapshots.
My walkaround method is to add some random info in the document to make it exist.</p>

Body:
I have users collection that has sub collection called attendedEvents like the picture below. as you can see there are 3 documents in the attendedEvents sub collection

I try to get all documents available on that sub collection by using the code below, I just want to get it all, without order, limit or anything using getDocuments

    func getAttendedEventsFromBeginning(completion: @escaping (_ eventID: [String]?,QueryDocumentSnapshot?)->Void) {

        FirestoreDocumentReference.users(uidUser: uid).reference().collection("attendedEvents")
            .getDocuments { (snapshot, error) in

                let lastDocument = snapshot?.documents.last

                if let error = error {
                    completion(nil,lastDocument)
                    print("Error when fetching attended events documents in user subcollection: \(error.localizedDescription)")
                } else {
                    print("Successfully fetching attended events documents in user subcollection from Firestore ")

                    guard let documentsSnapshot = snapshot else {
                        completion(nil,lastDocument)
                        return
                    }

                    let eventDocuments = documentsSnapshot.documents
                    print("xxxxx")
                    print(eventDocuments)

                    var attendedeventIDs = [String]()

                    for document in eventDocuments {
                        let eventDictionary = document.data()
                        let theEvent = eventDictionary["eventID"] as! String
                        attendedeventIDs.append(theEvent)
                    }

                    print(attendedeventIDs)

                    completion(attendedeventIDs,lastDocument)
                }
            }
    }

but as a result, I just got 2 document snapshot, it should be 3 documents

but if I delete the app from simulator and install it again, I got all the three data. is is cached or what?


问:
Regex with multiple email address seperated by a semi colon
I am trying to build a regex what would allow multiple emails separated with a semicolon for only one domain.

This is what I have now:

@"^(?>\b[\w+-.%]+@gmail\.com(?:[,;]\s*)?)+\b$"

This allow 1@gmail.com%2@gmail.com or 1@gmail.com&&2@gmail.com. I just want to have it to be allowed with semicolmn seperator as this. 1@gmail.com;2@gmail.com.

答:
You could split your string then use regex for validating the email on each one:

public string addresses = "1@gmail.com;2@gmail.com";
public List<string> addressList = addresses.Split(";",StringSplitOptions.RemoveEmptyEntries);
public Regex rx = new Regex(@"^\w+([-+.']\w+)*@gmail.com$");

foreach(var email in addressList)
{
     var m = rx.Match(email);

     if(m.Success)
     {
        //Do something with match
     }
}
```

Regex adapted from .Net General Match

Body:
You could split your string then use regex for validating the email on each one:

public string addresses = "1@gmail.com;2@gmail.com";
public List<string> addressList = addresses.Split(";",StringSplitOptions.RemoveEmptyEntries);
public Regex rx = new Regex(@"^\w+([-+.']\w+)*@gmail.com$");

foreach(var email in addressList)
{
     var m = rx.Match(email);

     if(m.Success)
     {
        //Do something with match
     }
}

Regex adapted from .Net General Match


问:
Django UnicodeEncodeError when logging sql with unicode from BinaryField
Django 1.11, Python 3.5, Windows OS

I have a Django model with a BinaryField. When I save an instance of the model to the database Django prints out an error message like this:

```<code>UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 216: character maps to <undefined>
</code>```

The last line of the traceback indicates that the error is occurring in django.db.backends.utils.py -- I've added a comment to the offending line:

```<code>class CursorDebugWrapper(CursorWrapper):

    # XXX callproc isn't instrumented at this time.

    def execute(self, sql, params=None):
        start = time()
        try:
            return super(CursorDebugWrapper, self).execute(sql, params)
        finally:
            stop = time()
            duration = stop - start
            sql = self.db.ops.last_executed_query(self.cursor, sql, params)
            self.db.queries_log.append({
                'sql': sql,
                'time': "%.3f" % duration,
            })
            ##### Error is reported from the logger.debug statement
            logger.debug(
                '(%.3f) %s; args=%s', duration, sql, params,
                extra={'duration': duration, 'sql': sql, 'params': params}
            )
</code>```

So I think what's happening is that when Django tries to print the SQL insert statement it hits a non-printable Unicode character and throws an error. I don't want to disable Django debug logging while I'm developing (it is disabled in production, of course).  Any way to work around this issue?

答:
Setting the encoding of the log handler to utf-8 seems to work. If there's a better way please advise.

```<code>LOGGING = {
    # other config omitted
    'handlers': {
        'django-debug': {
            'level': 'DEBUG',
            'class':'logging.FileHandler',
            'filename': os.path.join(LOCAL_LOGGING_PATH,'django-debug.log'),
            'formatter': 'standard',
            'encoding':'utf8', # this fixes UnicodeEncodeError
        }
    }
}
</code>```

Things I tried that didn't work. These all still produce UnicodeEncodeError:

Add from __future__ import unicode_literals at the top of settings.py

Add the Byte Order Mark (BOM) character \ufeff to the formatter as suggested by the python 3 logging cookbook

Use %r instead of %s in the log formatter

Use the python unicode string on the formatter 'format': u'%(asctime)-s %(levelname)s [%(name)s]: %(message)s', and then logging._defaultFormatter = logging.Formatter(u"%(message)s") as suggested in this thread

Body:
Django 1.11, Python 3.5, Windows OS

I have a Django model with a BinaryField. When I save an instance of the model to the database Django prints out an error message like this:

UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 216: character maps to <undefined>

The last line of the traceback indicates that the error is occurring in django.db.backends.utils.py -- I've added a comment to the offending line:

class CursorDebugWrapper(CursorWrapper):

    # XXX callproc isn't instrumented at this time.

    def execute(self, sql, params=None):
        start = time()
        try:
            return super(CursorDebugWrapper, self).execute(sql, params)
        finally:
            stop = time()
            duration = stop - start
            sql = self.db.ops.last_executed_query(self.cursor, sql, params)
            self.db.queries_log.append({
                'sql': sql,
                'time': "%.3f" % duration,
            })
            ##### Error is reported from the logger.debug statement
            logger.debug(
                '(%.3f) %s; args=%s', duration, sql, params,
                extra={'duration': duration, 'sql': sql, 'params': params}
            )

So I think what's happening is that when Django tries to print the SQL insert statement it hits a non-printable Unicode character and throws an error. I don't want to disable Django debug logging while I'm developing (it is disabled in production, of course).  Any way to work around this issue?

Body:
Setting the encoding of the log handler to utf-8 seems to work. If there's a better way please advise.

LOGGING = {
    # other config omitted
    'handlers': {
        'django-debug': {
            'level': 'DEBUG',
            'class':'logging.FileHandler',
            'filename': os.path.join(LOCAL_LOGGING_PATH,'django-debug.log'),
            'formatter': 'standard',
            'encoding':'utf8', # this fixes UnicodeEncodeError
        }
    }
}

Things I tried that didn't work. These all still produce UnicodeEncodeError:

Add from __future__ import unicode_literals at the top of settings.py

Add the Byte Order Mark (BOM) character \ufeff to the formatter as suggested by the python 3 logging cookbook

Use %r instead of %s in the log formatter

Use the python unicode string on the formatter 'format': u'%(asctime)-s %(levelname)s [%(name)s]: %(message)s', and then logging._defaultFormatter = logging.Formatter(u"%(message)s") as suggested in this thread


问:
Linq groupby and count operations in mvc
So here is my table

```<code>MyCourse (MyCourseID, CourseNumber, CourseName, SemesterName, Year)
</code>```

Now if the table consists of the following data

```<code>ENG1200 'Basic English' Spring 2011
ENG1200 'Basic English' Spring 2011
</code>```

I am simply trying to extract the count along with the CourseNumber as follows

```<code>ENG1200 'Basic English' Spring 2011 2
</code>```

and then insert it into a new strongly typed class so I can use it in my View.
But the linq statement prints every class with a count of 1 on each.

```<code>ENG1200 'Basic English' Spring 2011 1
ENG1200 'Basic English' Spring 2011 1
</code>```

Any help will be greatly appreciated.

Here is my linq code that I am using:

```<code> public IQueryable<VotedCourses> GetVoted_Courses()
    {

        var votedCourses = courseDB.MyCourses.GroupBy
            (x => new
            {
                x.MyCourseID,
                x.CourseNumber,
                x.CourseName,
                x.SemesterName,
                x.Year
            })
            .Select(x =>
             new VotedCourses
             {
                 MyCourseID = x.Key.MyCourseID,
                 CourseNumber = x.Key.CourseNumber,
                 CourseName = x.Key.CourseName,
                 SemesterName = x.Key.SemesterName,
                 Year = x.Key.Year,
                 VoteCount = x.Count()
             });
        return votedCourses;
    }
</code>```

答: I notice you're including x.MyCourseID in your anonymous type. This column isn't present in your samples, and it looks like an identity. My guess is that this column is different between the two records. I'd remove it from the type and try again.

Body:
So here is my table

MyCourse (MyCourseID, CourseNumber, CourseName, SemesterName, Year)

Now if the table consists of the following data

ENG1200 'Basic English' Spring 2011
ENG1200 'Basic English' Spring 2011

I am simply trying to extract the count along with the CourseNumber as follows

ENG1200 'Basic English' Spring 2011 2

and then insert it into a new strongly typed class so I can use it in my View.
But the linq statement prints every class with a count of 1 on each.

ENG1200 'Basic English' Spring 2011 1
ENG1200 'Basic English' Spring 2011 1

Any help will be greatly appreciated.

Here is my linq code that I am using:

 public IQueryable<VotedCourses> GetVoted_Courses()
    {

        var votedCourses = courseDB.MyCourses.GroupBy
            (x => new
            {
                x.MyCourseID,
                x.CourseNumber,
                x.CourseName,
                x.SemesterName,
                x.Year
            })
            .Select(x =>
             new VotedCourses
             {
                 MyCourseID = x.Key.MyCourseID,
                 CourseNumber = x.Key.CourseNumber,
                 CourseName = x.Key.CourseName,
                 SemesterName = x.Key.SemesterName,
                 Year = x.Key.Year,
                 VoteCount = x.Count()
             });
        return votedCourses;
    }


问:
Ktor application with spring config server
wanted to try some new stuff and decided to build next micro-app in Ktor + Koin + Exposed. Everything looks really nice but I found one problem that is actually destroying the whole idea.

App needs DB access and the connection cannot be stored within repository but should be encrypted on config-server. Every other micro-app is suing spring boot and fetches configs with lib spring-cloud-config-client but I don't know if it's even possible to use that somehow from Ktor app. Anyone had the same problem and managed to fix it somehow?

答:
Cfg4j seems to be an alternative to Spring Cloud Config Server.

https://github.com/cfg4j/cfg4j

Other than that, there are a few articles and some questions on SO about integrating Spring Cloud Config Server with non Spring Boot projects.

Spring Cloud Config Client Without Spring Boot

Also, I am working on an external configuration parser for Kotlin and am considering to implement similar functionality.


问:
Easy way to validate the shape of an object in Javascript
I’m wondering if there’s a simple way of validating the shape of an object in Javascript.

Right now, I have a function that validates the shape of endpoint objects like this:

```<code>function validateEndpointShape(endpoint: any, hasId: boolean): boolean
{
return endpoint
&& (hasId ? typeof endpoint.id === 'string' : true)
&& typeof endpoint.name === 'string'
&& typeof endpoint.description === 'string'
&& typeof endpoint.url === 'string'
&& GenericApiEndpointMethods[endpoint.method] !== undefined
&& ApiEndpointTypes[endpoint.apiEndpointType] !== undefined
&& endpoint.group
&& typeof endpoint.group.groupPublicKey === 'string'
&& typeof endpoint.group.groupName === 'string'
&& typeof endpoint.reason === 'string'
&& typeof endpoint.isPublic === 'boolean'
&& typeof endpoint.isActive === 'boolean'
&& authTypes[endpoint.authType] !== undefined
&& Array.isArray(endpoint.parameters)
&& Array.isArray(endpoint.headers);
}
</code>```

This can get cumbersome and unwieldy. And I don’t want to have to do this for every object I create.

When an endpoint comes into our cloud firebase function, we have to do a bunch of validation on it so that we know when to reject bad data. The shape of the endpoint is one of those validations.

I tried doing this:

```<code>Delete req.body.reason;
req.body[‘extraField’] = ‘xxx’;
Const endpoint: GenericApiEndpoint = req.body;
console.log(‘endpoint =‘, endpoint);
</code>```

But Javascript doesn’t care. It will accept an endpoint without reason (a mandatory field) and with extraField (a non-existent field in the model) and assign it to an object typed as GenericApiEndpoint. Endpoint prints out without reason and with extraField.

I also tried:

```<code>Const endpoint = <GenericApiEndpoint>req.body;
</code>```

…but Javascript doesn’t care about that either.

Does anybody know of a simple way to validate the shape of objects in Javascript?

答:
There are plenty ways to validate data, and I'd say that any system where you want data to persist and match a specific model, you would want some kind of field validation. ORM's usually do this but you could also use a library such as:

https://validatejs.org (for pure field validation)

https://github.com/hapijs/joi (for creating schemas + validation)

https://json-schema.org/ (A standard for creating schemas. Can be used with validation libraries such as https://github.com/epoberezkin/ajv)

Basically the bottom line is that if you want to validate objects to make sure they fit a specific shape (model/schema), you will have to define that shape beforehand.

Body:
I’m wondering if there’s a simple way of validating the shape of an object in Javascript.

Right now, I have a function that validates the shape of endpoint objects like this:

function validateEndpointShape(endpoint: any, hasId: boolean): boolean
{
return endpoint
&& (hasId ? typeof endpoint.id === 'string' : true)
&& typeof endpoint.name === 'string'
&& typeof endpoint.description === 'string'
&& typeof endpoint.url === 'string'
&& GenericApiEndpointMethods[endpoint.method] !== undefined
&& ApiEndpointTypes[endpoint.apiEndpointType] !== undefined
&& endpoint.group
&& typeof endpoint.group.groupPublicKey === 'string'
&& typeof endpoint.group.groupName === 'string'
&& typeof endpoint.reason === 'string'
&& typeof endpoint.isPublic === 'boolean'
&& typeof endpoint.isActive === 'boolean'
&& authTypes[endpoint.authType] !== undefined
&& Array.isArray(endpoint.parameters)
&& Array.isArray(endpoint.headers);
}

This can get cumbersome and unwieldy. And I don’t want to have to do this for every object I create.

When an endpoint comes into our cloud firebase function, we have to do a bunch of validation on it so that we know when to reject bad data. The shape of the endpoint is one of those validations.

I tried doing this:

Delete req.body.reason;
req.body[‘extraField’] = ‘xxx’;
Const endpoint: GenericApiEndpoint = req.body;
console.log(‘endpoint =‘, endpoint);

But Javascript doesn’t care. It will accept an endpoint without reason (a mandatory field) and with extraField (a non-existent field in the model) and assign it to an object typed as GenericApiEndpoint. Endpoint prints out without reason and with extraField.

I also tried:

Const endpoint = <GenericApiEndpoint>req.body;

…but Javascript doesn’t care about that either.

Does anybody know of a simple way to validate the shape of objects in Javascript?