问:
Wordpress Database Optimisation for large Sites
I have a large Wordpress site with 170.000 users and a lot of daily page views.
I just tuned all MySQL indexes based on several comments but actually in my slow logs the SELECT distinct wp_usermeta.meta_key FROM wp_usermeta; takes around 3 seconds.
Server Hardware is: Dedicated Server with AMD Epyc 64 Cores, 128Gb DDR4, 2x480 NVMe SSD.

DB Server is MariaDB newest Version and config is (only innoDB tables):

```<code>innodb_buffer_pool_size = 64G
innodb_log_file_size = 16G
innodb_buffer_pool_instances = 16
innodb_io_capacity = 5000
max_binlog_size = 200M
max_connections = 250
wait_timeout = 28700
interactive_timeout = 28700
join_buffer_size = 128M
expire_logs_days = 3
skip-host-cache
skip-name-resolve
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 1
sql-mode = "STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION"

tmp_table_size = 256M
max_heap_table_size = 256M
table_definition_cache = 500
sort_buffer_size = 24M
key_buffer_size = 32M
performance_schema = on
</code>```

Maybe someone has some suggestions

答:
```<code>Analysis of GLOBAL STATUS and VARIABLES:

</code>```

Observations:

Version: 10.6.5-MariaDB-1:10.6.5+maria~bullseye-log

128 GB of RAM

Uptime = 1d 02:48:55

384 QPS

The More Important Issues:

I do not see any items that seem critical to help with the problem you are having.

Details and other observations:

( innodb_lru_scan_depth ) = 1,536
-- "InnoDB: page_cleaner: 1000ms intended loop took ..." may be fixed by lowering lru_scan_depth

( innodb_io_capacity_max ) = 10,000 -- When urgently flushing, use this many IOPs.
-- Reads could be slugghish or spiky.

( Innodb_buffer_pool_pages_free / Innodb_buffer_pool_pages_total ) = 2,787,201 / 4145152 = 67.2% -- Pct of buffer_pool currently not in use
-- innodb_buffer_pool_size (now 68719476736) is bigger than necessary?

( Innodb_buffer_pool_bytes_data / innodb_buffer_pool_size ) = 22,248,669,184 / 65536M = 32.4% -- Percent of buffer pool taken up by data
-- A small percent may indicate that the buffer_pool is unnecessarily big.

( Innodb_log_writes ) = 5,298,275 / 96535 = 55 /sec

( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 96,535 / 60 * 16384M / 6560327680 = 4,213 -- Minutes between InnoDB log rotations Beginning with 5.6.8, innodb_log_file_size can be changed dynamically; I don't know about MariaDB.  Be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size (now 17179869184). (Cannot change in AWS.)

( Innodb_row_lock_waits ) = 83,931 / 96535 = 0.87 /sec -- How often there is a delay in getting a row lock.
-- May be caused by complex queries that could be optimized.

( Innodb_row_lock_waits/Innodb_rows_inserted ) = 83,931/1560067 = 5.4% -- Frequency of having to wait for a row.

( innodb_flush_neighbors ) = 1 -- A minor optimization when writing blocks to disk.
-- Use 0 for SSD drives; 1 for HDD.

( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.

( join_buffer_size * Max_used_connections ) = (128M * 127) / 131072M = 12.4% -- (A metric for pondering the size of join_buffer_size.)
-- join_buffer_size (now 134217728) should probably be shrunk to avoid running out of RAM.

( (Com_show_create_table + Com_show_fields) / Questions ) = (66 + 1370563) / 37103211 = 3.7% -- Naughty framework -- spending a lot of effort rediscovering the schema.
-- Complain to the 3rd party vendor.

( local_infile ) = local_infile = ON
-- local_infile (now ON) = ON is a potential security issue

( Created_tmp_tables ) = 2,088,713 / 96535 = 22 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.

( Created_tmp_disk_tables ) = 1,751,146 / 96535 = 18 /sec -- Frequency of creating disk "temp" tables as part of complex SELECTs
-- increase tmp_table_size (now 268435456) and max_heap_table_size (now 268435456).
Check the rules for temp tables on when MEMORY is used instead of MyISAM. Perhaps minor schema or query changes can avoid MyISAM.
Better indexes and reformulation of queries are more likely to help.

( Created_tmp_disk_tables / Questions ) = 1,751,146 / 37103211 = 4.7% -- Pct of queries that needed on-disk tmp table.
-- Better indexes / No blobs / etc.

( Created_tmp_disk_tables / Created_tmp_tables ) = 1,751,146 / 2088713 = 83.8% -- Percent of temp tables that spilled to disk
-- Maybe increase tmp_table_size (now 268435456) and max_heap_table_size (now 268435456); improve indexes; avoid blobs, etc.

( Handler_read_rnd_next ) = 104,164,660,719 / 96535 = 1079035 /sec -- High if lots of table scans
-- possibly inadequate keys

( (Com_insert + Com_update + Com_delete + Com_replace) / Com_commit ) = (1561842 + 4652536 + 13886 + 42) / 352 = 17,694 -- Statements per Commit (assuming all InnoDB)
-- High: long transactions strain various things.

( Com_insert + Com_delete + Com_delete_multi + Com_replace + Com_update + Com_update_multi ) = (1561842 + 13886 + 0 + 42 + 4652536 + 794) / 96535 = 65 /sec -- writes/sec
-- 50 writes/sec + log flushes will probably max out I/O write capacity of HDD drives

( ( Com_stmt_prepare - Com_stmt_close ) / ( Com_stmt_prepare + Com_stmt_close ) ) = ( 2208251 - 1415 ) / ( 2208251 + 1415 ) = 99.9% -- Are you closing your prepared statements?
-- Add Closes.

( Com_stmt_prepare - Com_stmt_close ) = 2,208,251 - 1415 = 2.21e+6 -- How many prepared statements have not been closed.
-- CLOSE prepared statements

( Com_stmt_close / Com_stmt_prepare ) = 1,415 / 2208251 = 0.06% -- Prepared statements should be Closed.
-- Check whether all Prepared statements are "Closed".

( binlog_format ) = binlog_format = MIXED -- STATEMENT/ROW/MIXED.
-- ROW is preferred by 5.7 (10.3)

( Syncs ) = 5,727,396 / 96535 = 59 /sec -- Sync to disk for binlog.

( Com_change_db ) = 1,168,504 / 96535 = 12 /sec -- Probably comes from USE statements.
-- Consider connecting with DB, using db.tbl syntax, eliminating spurious USE statements, etc.

( Connections ) = 3,377,949 / 96535 = 35 /sec -- Connections
-- Increase wait_timeout (now 28700); use pooling?

( thread_cache_size / Max_used_connections ) = 250 / 127 = 196.9%
-- There is no advantage in having the thread cache bigger than your likely number of connections. Wasting space is the disadvantage.

( thread_pool_size ) = 64 -- Number of 'thread groups'. Limits how many treads can be executing at once. Probably should not be much bigger than the number of CPUs.
-- Don't set much higher than the number of CPU cores.

You have the Query Cache half-off. You should set both query_cache_type = OFF and query_cache_size = 0 . There is (according to a rumor) a 'bug' in the QC code that leaves some code on unless you turn off both of those settings.

VSClasses.inc.256 Error with eval('((1048576 - 1031304) / 0) / 4096') expr=[[((query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache) / query_cache_min_res_unit]]
VSClasses.inc.256 Error with eval('(1048576 - 1031304) / 0 / 16384') expr=[[(query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache / query_alloc_block_size]]
VSClasses.inc.256 Error with eval('0/0') expr=[[Innodb_dblwr_pages_written/Innodb_pages_written]]
VSClasses.inc.256 Error with eval('0 / (0 + 0 + 0)') expr=[[Qcache_hits / (Qcache_hits + Qcache_inserts + Qcache_not_cached)]]
VSClasses.inc.256 Error with eval('0/0') expr=[[Qcache_lowmem_prunes/Qcache_inserts]]

Abnormally small:

```<code>Innodb_adaptive_hash_non_hash_searches = 0
Innodb_buffer_pool_pages_flushed / max(Questions, Queries) = 0
Innodb_buffer_pool_pages_misc = 0
Innodb_buffer_pool_pages_misc * 16384 / innodb_buffer_pool_size = 0
Innodb_data_writes - Innodb_log_writes - Innodb_dblwr_writes = 6 /HR
Innodb_data_written = 0
Innodb_dblwr_pages_written = 0
Innodb_master_thread_active_loops = 13
Innodb_mem_adaptive_hash = 0
Innodb_pages_written = 0
Memory_used = 0.04%
Memory_used_initial = 15.7MB
</code>```

Abnormally large:

```<code>Aria_pagecache_reads = 18 /sec
Aria_pagecache_write_requests = 1180 /sec
Com_show_fields = 14 /sec
Com_stmt_prepare = 23 /sec
Handler_discover = 3 /HR
Handler_read_next = 1805396 /sec
Handler_read_next / Handler_read_key = 121
Innodb_buffer_pool_pages_dirty = 77,929
Innodb_buffer_pool_pages_free = 2.79e+6
Innodb_buffer_pool_pages_total = 4.15e+6
Innodb_checkpoint_age = 2.3e+9
Innodb_log_writes / Innodb_log_write_requests = 6636.2%
Innodb_os_log_fsyncs = 55 /sec
Innodb_rows_read = 2894484 /sec
Open_streams = 4
Opened_views = 0.058 /sec
Performance_schema_file_instances_lost = 6
Rows_read = 2887256 /sec
Select_full_range_join = 0.4 /sec
Select_full_range_join / Com_select = 0.18%
Slaves_connected = 0.037 /HR
Threads_cached = 113
performance_schema_max_statement_classes = 222
</code>```

Abnormal strings:

```<code>Slave_heartbeat_period = 0
Slave_received_heartbeats = 0
aria_recover_options = BACKUP,QUICK
binlog_row_metadata = NO_LOG
character_set_system = utf8mb3
disconnect_on_expired_password = OFF
innodb_fast_shutdown = 1
log_slow_admin_statements = ON
myisam_stats_method = NULLS_UNEQUAL
old_alter_table = DEFAULT
old_mode = UTF8_IS_UTF8MB3
optimizer_trace = enabled=off
slave_parallel_mode = optimistic
sql_slave_skip_counter = 0
</code>```

Body:
I have a large Wordpress site with 170.000 users and a lot of daily page views.
I just tuned all MySQL indexes based on several comments but actually in my slow logs the SELECT distinct wp_usermeta.meta_key FROM wp_usermeta; takes around 3 seconds.
Server Hardware is: Dedicated Server with AMD Epyc 64 Cores, 128Gb DDR4, 2x480 NVMe SSD.

DB Server is MariaDB newest Version and config is (only innoDB tables):

innodb_buffer_pool_size = 64G
innodb_log_file_size = 16G
innodb_buffer_pool_instances = 16
innodb_io_capacity = 5000
max_binlog_size = 200M
max_connections = 250
wait_timeout = 28700
interactive_timeout = 28700
join_buffer_size = 128M
expire_logs_days = 3
skip-host-cache
skip-name-resolve
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 1
sql-mode = "STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION"

tmp_table_size = 256M
max_heap_table_size = 256M
table_definition_cache = 500
sort_buffer_size = 24M
key_buffer_size = 32M
performance_schema = on

Maybe someone has some suggestions

Body:
Analysis of GLOBAL STATUS and VARIABLES:

Observations:

Version: 10.6.5-MariaDB-1:10.6.5+maria~bullseye-log

128 GB of RAM

Uptime = 1d 02:48:55

384 QPS

The More Important Issues:

I do not see any items that seem critical to help with the problem you are having.

Details and other observations:

( innodb_lru_scan_depth ) = 1,536
-- "InnoDB: page_cleaner: 1000ms intended loop took ..." may be fixed by lowering lru_scan_depth

( innodb_io_capacity_max ) = 10,000 -- When urgently flushing, use this many IOPs.
-- Reads could be slugghish or spiky.

( Innodb_buffer_pool_pages_free / Innodb_buffer_pool_pages_total ) = 2,787,201 / 4145152 = 67.2% -- Pct of buffer_pool currently not in use
-- innodb_buffer_pool_size (now 68719476736) is bigger than necessary?

( Innodb_buffer_pool_bytes_data / innodb_buffer_pool_size ) = 22,248,669,184 / 65536M = 32.4% -- Percent of buffer pool taken up by data
-- A small percent may indicate that the buffer_pool is unnecessarily big.

( Innodb_log_writes ) = 5,298,275 / 96535 = 55 /sec

( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 96,535 / 60 * 16384M / 6560327680 = 4,213 -- Minutes between InnoDB log rotations Beginning with 5.6.8, innodb_log_file_size can be changed dynamically; I don't know about MariaDB.  Be sure to also change my.cnf.
-- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size (now 17179869184). (Cannot change in AWS.)

( Innodb_row_lock_waits ) = 83,931 / 96535 = 0.87 /sec -- How often there is a delay in getting a row lock.
-- May be caused by complex queries that could be optimized.

( Innodb_row_lock_waits/Innodb_rows_inserted ) = 83,931/1560067 = 5.4% -- Frequency of having to wait for a row.

( innodb_flush_neighbors ) = 1 -- A minor optimization when writing blocks to disk.
-- Use 0 for SSD drives; 1 for HDD.

( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks.
-- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.

( join_buffer_size * Max_used_connections ) = (128M * 127) / 131072M = 12.4% -- (A metric for pondering the size of join_buffer_size.)
-- join_buffer_size (now 134217728) should probably be shrunk to avoid running out of RAM.

( (Com_show_create_table + Com_show_fields) / Questions ) = (66 + 1370563) / 37103211 = 3.7% -- Naughty framework -- spending a lot of effort rediscovering the schema.
-- Complain to the 3rd party vendor.

( local_infile ) = local_infile = ON
-- local_infile (now ON) = ON is a potential security issue

( Created_tmp_tables ) = 2,088,713 / 96535 = 22 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.

( Created_tmp_disk_tables ) = 1,751,146 / 96535 = 18 /sec -- Frequency of creating disk "temp" tables as part of complex SELECTs
-- increase tmp_table_size (now 268435456) and max_heap_table_size (now 268435456).
Check the rules for temp tables on when MEMORY is used instead of MyISAM. Perhaps minor schema or query changes can avoid MyISAM.
Better indexes and reformulation of queries are more likely to help.

( Created_tmp_disk_tables / Questions ) = 1,751,146 / 37103211 = 4.7% -- Pct of queries that needed on-disk tmp table.
-- Better indexes / No blobs / etc.

( Created_tmp_disk_tables / Created_tmp_tables ) = 1,751,146 / 2088713 = 83.8% -- Percent of temp tables that spilled to disk
-- Maybe increase tmp_table_size (now 268435456) and max_heap_table_size (now 268435456); improve indexes; avoid blobs, etc.

( Handler_read_rnd_next ) = 104,164,660,719 / 96535 = 1079035 /sec -- High if lots of table scans
-- possibly inadequate keys

( (Com_insert + Com_update + Com_delete + Com_replace) / Com_commit ) = (1561842 + 4652536 + 13886 + 42) / 352 = 17,694 -- Statements per Commit (assuming all InnoDB)
-- High: long transactions strain various things.

( Com_insert + Com_delete + Com_delete_multi + Com_replace + Com_update + Com_update_multi ) = (1561842 + 13886 + 0 + 42 + 4652536 + 794) / 96535 = 65 /sec -- writes/sec
-- 50 writes/sec + log flushes will probably max out I/O write capacity of HDD drives

( ( Com_stmt_prepare - Com_stmt_close ) / ( Com_stmt_prepare + Com_stmt_close ) ) = ( 2208251 - 1415 ) / ( 2208251 + 1415 ) = 99.9% -- Are you closing your prepared statements?
-- Add Closes.

( Com_stmt_prepare - Com_stmt_close ) = 2,208,251 - 1415 = 2.21e+6 -- How many prepared statements have not been closed.
-- CLOSE prepared statements

( Com_stmt_close / Com_stmt_prepare ) = 1,415 / 2208251 = 0.06% -- Prepared statements should be Closed.
-- Check whether all Prepared statements are "Closed".

( binlog_format ) = binlog_format = MIXED -- STATEMENT/ROW/MIXED.
-- ROW is preferred by 5.7 (10.3)

( Syncs ) = 5,727,396 / 96535 = 59 /sec -- Sync to disk for binlog.

( Com_change_db ) = 1,168,504 / 96535 = 12 /sec -- Probably comes from USE statements.
-- Consider connecting with DB, using db.tbl syntax, eliminating spurious USE statements, etc.

( Connections ) = 3,377,949 / 96535 = 35 /sec -- Connections
-- Increase wait_timeout (now 28700); use pooling?

( thread_cache_size / Max_used_connections ) = 250 / 127 = 196.9%
-- There is no advantage in having the thread cache bigger than your likely number of connections. Wasting space is the disadvantage.

( thread_pool_size ) = 64 -- Number of 'thread groups'. Limits how many treads can be executing at once. Probably should not be much bigger than the number of CPUs.
-- Don't set much higher than the number of CPU cores.

You have the Query Cache half-off. You should set both query_cache_type = OFF and query_cache_size = 0 . There is (according to a rumor) a 'bug' in the QC code that leaves some code on unless you turn off both of those settings.

VSClasses.inc.256 Error with eval('((1048576 - 1031304) / 0) / 4096') expr=[[((query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache) / query_cache_min_res_unit]]
VSClasses.inc.256 Error with eval('(1048576 - 1031304) / 0 / 16384') expr=[[(query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache / query_alloc_block_size]]
VSClasses.inc.256 Error with eval('0/0') expr=[[Innodb_dblwr_pages_written/Innodb_pages_written]]
VSClasses.inc.256 Error with eval('0 / (0 + 0 + 0)') expr=[[Qcache_hits / (Qcache_hits + Qcache_inserts + Qcache_not_cached)]]
VSClasses.inc.256 Error with eval('0/0') expr=[[Qcache_lowmem_prunes/Qcache_inserts]]

Abnormally small:

Innodb_adaptive_hash_non_hash_searches = 0
Innodb_buffer_pool_pages_flushed / max(Questions, Queries) = 0
Innodb_buffer_pool_pages_misc = 0
Innodb_buffer_pool_pages_misc * 16384 / innodb_buffer_pool_size = 0
Innodb_data_writes - Innodb_log_writes - Innodb_dblwr_writes = 6 /HR
Innodb_data_written = 0
Innodb_dblwr_pages_written = 0
Innodb_master_thread_active_loops = 13
Innodb_mem_adaptive_hash = 0
Innodb_pages_written = 0
Memory_used = 0.04%
Memory_used_initial = 15.7MB

Abnormally large:

Aria_pagecache_reads = 18 /sec
Aria_pagecache_write_requests = 1180 /sec
Com_show_fields = 14 /sec
Com_stmt_prepare = 23 /sec
Handler_discover = 3 /HR
Handler_read_next = 1805396 /sec
Handler_read_next / Handler_read_key = 121
Innodb_buffer_pool_pages_dirty = 77,929
Innodb_buffer_pool_pages_free = 2.79e+6
Innodb_buffer_pool_pages_total = 4.15e+6
Innodb_checkpoint_age = 2.3e+9
Innodb_log_writes / Innodb_log_write_requests = 6636.2%
Innodb_os_log_fsyncs = 55 /sec
Innodb_rows_read = 2894484 /sec
Open_streams = 4
Opened_views = 0.058 /sec
Performance_schema_file_instances_lost = 6
Rows_read = 2887256 /sec
Select_full_range_join = 0.4 /sec
Select_full_range_join / Com_select = 0.18%
Slaves_connected = 0.037 /HR
Threads_cached = 113
performance_schema_max_statement_classes = 222

Abnormal strings:

Slave_heartbeat_period = 0
Slave_received_heartbeats = 0
aria_recover_options = BACKUP,QUICK
binlog_row_metadata = NO_LOG
character_set_system = utf8mb3
disconnect_on_expired_password = OFF
innodb_fast_shutdown = 1
log_slow_admin_statements = ON
myisam_stats_method = NULLS_UNEQUAL
old_alter_table = DEFAULT
old_mode = UTF8_IS_UTF8MB3
optimizer_trace = enabled=off
slave_parallel_mode = optimistic
sql_slave_skip_counter = 0


来源: stackexchange

问:
Selenium find element error, but same code with js can work
use the python execute can find the element bottom and click, it works:

```<code>driver.execute_script("document.getElementsByClassName(\"g-c-R  webstore-test-button-label\")[0].click()")
</code>```

but with the similar code, the python code can not work:

```<code>element_install_bottom=driver.find_element(by=By.ID, value=r'g-c-R  webstore-test-button-label')
</code>```

and throw the exception:

```<code>selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="g-c-R  webstore-test-button-label"]"}
</code>```

and have try by.ID, by.ClassName; but always throw the exception.

the whole code as below:
from selenium import webdriver
from selenium.webdriver.common.by import By
import selenium.webdriver.support.ui as ui

```<code>import time

option = webdriver.ChromeOptions()

USER_DATA_PATH = r"D:\Chrome\User Data 3"
option.add_argument(f'--user-data-dir={USER_DATA_PATH}')
option.add_experimental_option('excludeSwitches', ['enable-automation'])

print(option.arguments)

driver = webdriver.Chrome(options=option)

extension_url = "https://chrome.google.com/webstore/detail/dark-reader/eimadpbcbfnmbkopoojfekhnkhdbieeh?hl=zh-CN"
driver.get(extension_url)
time.sleep(60)

element_install_bottom=driver.find_element(by=By.ID, value=r'g-c-R  webstore-test-button-label')
print(element_install_bottom)
</code>```

答:
You're seeing that exception as there is no element with that ID, that value is it's class.

Python:

```<code>driver.find_element_by_class_name("g-c-R  webstore-test-button-label")
</code>```

or:

```<code>driver.find_element(By.CLASS_NAME, "g-c-R  webstore-test-button-label")
</code>```

ContentLicense: CC BY-SA 4.0

Body:
use the python execute can find the element bottom and click, it works:

driver.execute_script("document.getElementsByClassName(\"g-c-R  webstore-test-button-label\")[0].click()")

but with the similar code, the python code can not work:

element_install_bottom=driver.find_element(by=By.ID, value=r'g-c-R  webstore-test-button-label')

and throw the exception:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="g-c-R  webstore-test-button-label"]"}

and have try by.ID, by.ClassName; but always throw the exception.

the whole code as below:
from selenium import webdriver
from selenium.webdriver.common.by import By
import selenium.webdriver.support.ui as ui

import time

option = webdriver.ChromeOptions()

USER_DATA_PATH = r"D:\Chrome\User Data 3"
option.add_argument(f'--user-data-dir={USER_DATA_PATH}')
option.add_experimental_option('excludeSwitches', ['enable-automation'])

print(option.arguments)

driver = webdriver.Chrome(options=option)

extension_url = "https://chrome.google.com/webstore/detail/dark-reader/eimadpbcbfnmbkopoojfekhnkhdbieeh?hl=zh-CN"
driver.get(extension_url)
time.sleep(60)

element_install_bottom=driver.find_element(by=By.ID, value=r'g-c-R  webstore-test-button-label')
print(element_install_bottom)

Body:
You're seeing that exception as there is no element with that ID, that value is it's class.

Python:

driver.find_element_by_class_name("g-c-R  webstore-test-button-label")

or:

driver.find_element(By.CLASS_NAME, "g-c-R  webstore-test-button-label")


问:
How to create nested JSON using first entry from from flat JSON as parent in javascript
I've tried various answers to this (same/similar) question here, and none have given the desired result.
I have a flat Json file that I need to convert to nested.
The first item in each chunk is the item that the data needs grouping on when converted to nested data.
The example data just shows two "types" but there are several more in the data, and a few more other elements along with colour and size as well - I kept it short for posting here.

Example data input:

```<code>[
  {
    "type": "Background",
    "colour": "Blue",
    "size": "5"
  },
  {
    "type": "Background",
    "colour": "Grey",
    "size": "3"
  },
  {
    "type": "Foreground",
    "colour": "Red",
    "size": "5"
  },
  {
    "type": "Foreground",
    "colour": "White",
    "size": "10"
  }
]
</code>```

Desired Output:

```<code>[
  {
    "type": "Background",
    "elements": [
      {
        "colour": "Blue",
        "size": "5"
      },
      {
        "colour": "Grey",
        "size": "3"
      }
    ]
  }
  {
    "type": "Foreground",
    "elements": [
      {
        "colour": "Red",
        "size": "5"
      },
      {
        "colour": "White",
        "size": "10"
      }
    ]
  }
]
</code>```

The closest I came to an answer for this was here:
Create nested json from flattened JSON in javascript

But the output I received from that (in VSCode) was this:

```<code>{
  type: { elements: [ [Object], [Object] ] }
}
</code>```

While I am trying to follow the code, it is beyond my javascript knowledge currently, so I don't know what's going wrong, or why I get [object] and not the actual elements?

Can anyone help me get this sorted out so my output looks like my above "Desired output"?

Cheers!

答:
It seems that there are two steps here. First group all entries with the same type (which would be easier under the type name as a property, then build the nesting.

const input =
    [
      {
        "type": "Background",
        "colour": "Blue",
        "size": "5"
      },
      {
        "type": "Background",
        "colour": "Grey",
        "size": "3"
      },
      {
        "type": "Foreground",
        "colour": "Red",
        "size": "5"
      },
      {
        "type": "Foreground",
        "colour": "White",
        "size": "10"
      }
    ];

var assist = {},
    output = [];

// Group original objects based on type
// The type is used as a property value
for (let i = 0; i < input.length; i++) {
  let type = input[i].type;

  if (assist.hasOwnProperty(type)) {
    assist[type].push(input[i]);
  } else {
    assist[type] = [input[i]];
  }
  delete input[i].type;
}

// Turn the grouped entries into new array entries
let keys = Object.keys(assist);
for (let i = 0; i < keys.length; i++) {
  output.push({
    type: keys[i],
    elements: assist[keys[i]]
  });
};

// Show results
console.log(output);```

Note that while the above code does not modify the original array, it does modify it's members (deleting their type property) and using them in the output array. You will have to clone it if you need the original too.

Body:
I've tried various answers to this (same/similar) question here, and none have given the desired result.
I have a flat Json file that I need to convert to nested.
The first item in each chunk is the item that the data needs grouping on when converted to nested data.
The example data just shows two "types" but there are several more in the data, and a few more other elements along with colour and size as well - I kept it short for posting here.

Example data input:

[
  {
    "type": "Background",
    "colour": "Blue",
    "size": "5"
  },
  {
    "type": "Background",
    "colour": "Grey",
    "size": "3"
  },
  {
    "type": "Foreground",
    "colour": "Red",
    "size": "5"
  },
  {
    "type": "Foreground",
    "colour": "White",
    "size": "10"
  }
]

Desired Output:

[
  {
    "type": "Background",
    "elements": [
      {
        "colour": "Blue",
        "size": "5"
      },
      {
        "colour": "Grey",
        "size": "3"
      }
    ]
  }
  {
    "type": "Foreground",
    "elements": [
      {
        "colour": "Red",
        "size": "5"
      },
      {
        "colour": "White",
        "size": "10"
      }
    ]
  }
]

The closest I came to an answer for this was here:
Create nested json from flattened JSON in javascript

But the output I received from that (in VSCode) was this:

{
  type: { elements: [ [Object], [Object] ] }
}

While I am trying to follow the code, it is beyond my javascript knowledge currently, so I don't know what's going wrong, or why I get [object] and not the actual elements?

Can anyone help me get this sorted out so my output looks like my above "Desired output"?

Cheers!

Body:
It seems that there are two steps here. First group all entries with the same type (which would be easier under the type name as a property, then build the nesting.

const input =
    [
      {
        "type": "Background",
        "colour": "Blue",
        "size": "5"
      },
      {
        "type": "Background",
        "colour": "Grey",
        "size": "3"
      },
      {
        "type": "Foreground",
        "colour": "Red",
        "size": "5"
      },
      {
        "type": "Foreground",
        "colour": "White",
        "size": "10"
      }
    ];

var assist = {},
    output = [];

// Group original objects based on type
// The type is used as a property value
for (let i = 0; i < input.length; i++) {
  let type = input[i].type;

  if (assist.hasOwnProperty(type)) {
    assist[type].push(input[i]);
  } else {
    assist[type] = [input[i]];
  }
  delete input[i].type;
}

// Turn the grouped entries into new array entries
let keys = Object.keys(assist);
for (let i = 0; i < keys.length; i++) {
  output.push({
    type: keys[i],
    elements: assist[keys[i]]
  });
};

// Show results
console.log(output);

Note that while the above code does not modify the original array, it does modify it's members (deleting their type property) and using them in the output array. You will have to clone it if you need the original too.


问:
Uncaught Error: Too many re-renders. React limits the number of renders to prevent an infinite loop - ProtectedRoutes Component
I am attempting to create a ProtectedRoutes component, however, I seem to have created an infinite loop somewhere that I can't seem to figure out. I'm a beginner.

It should check if there is a cookie stored, and if so, go to the component. If not, it should navigate back to the main page.

ProtectedRoutes.js

```<code>import React, { Component, useState } from "react";
import { Route, Navigate } from "react-router-dom";
import Cookies from "universal-cookie";
const cookies = new Cookies();

export default function ProtectedRoutes({component: Component, ...rest}) {
    const [auth, setAuth] = useState(false);
    //get cookie from browser if logged in
    const token = cookies.get("TOKEN");
    if (token) {
        setAuth(true);
    };

    return auth ? <Component /> : <Navigate to="/" />
}
</code>```

App.js

```<code>import { Container, Col, Row } from "react-bootstrap";
import "./App.css";
import Register from "./Register";
import Login from "./Login";
import { Routes, Route } from "react-router-dom";
import Account from "./Account";
import FreeComponent from "./FreeComponent";
import AuthComponent from "./AuthComponent";
import Private from "./ProtectedRoutes";
import ProtectedRoutes from "./ProtectedRoutes";

function App() {
  return (
    <Container>
      <Row>
        <Col className="text-center">
          <h1 className="header">React Authentication Tutorial</h1>

          <section id="navigation">
            <a href="/">Home</a>
            <a href="/free">Free Component</a>
            <a href="/auth">Auth Component</a>
          </section>
        </Col>
      </Row>

      {/* Routes */ }
      <Routes>
        <Route exact path="/" element={ <Account /> } />
        <Route exact path="/free" element={ <FreeComponent /> } />
        <Route path="/auth" element={<ProtectedRoutes component={AuthComponent} />} />
      </Routes>
    </Container>
  );
}

export default App;
</code>```

AuthComponent.js

```<code>import React from 'react';

export default function AuthComponent() {
    return (
        <div>
            <h1 className="text-center">Auth Component</h1>
        </div>
    );
}
</code>```

答:
In ProtectedRoutes component, you're setting a state (setAuth in this case) directly inside the component, this is what happens when you do that:

React re-renders a component every time a state change is detected thus when you wrote

```<code>export default function ProtectedRoutes({component: Component, ...rest}) {
    ...
    if (token) {
        setAuth(true);
    };
    ...
}
</code>```

you're running setAuth(sets a new state true) every time the component renders(re-renders) and in turn, re-rendering the component every time because of the state change which is why have an infinite loop there.
, this works like this:

It runs everytime a dependency changes(passed as an array), and when you pass an empty array it runs just twice - when the component mounts(renders the first time) and when it unmounts.

What you need to do is to pass an empty array as a dependency as below

```<code>import { useEffect } from 'react'

export default function ProtectedRoutes({component: Component, ...rest}) {
    ...
    useEffect(() => {
        if (token) {
            setAuth(true);
        };
    }, [])
    ...
}
</code>```

this setAuth just once when the component mounts

Body:
I am attempting to create a ProtectedRoutes component, however, I seem to have created an infinite loop somewhere that I can't seem to figure out. I'm a beginner.

It should check if there is a cookie stored, and if so, go to the component. If not, it should navigate back to the main page.

ProtectedRoutes.js

import React, { Component, useState } from "react";
import { Route, Navigate } from "react-router-dom";
import Cookies from "universal-cookie";
const cookies = new Cookies();

export default function ProtectedRoutes({component: Component, ...rest}) {
    const [auth, setAuth] = useState(false);
    //get cookie from browser if logged in
    const token = cookies.get("TOKEN");
    if (token) {
        setAuth(true);
    };

    return auth ? <Component /> : <Navigate to="/" />
}

App.js

import { Container, Col, Row } from "react-bootstrap";
import "./App.css";
import Register from "./Register";
import Login from "./Login";
import { Routes, Route } from "react-router-dom";
import Account from "./Account";
import FreeComponent from "./FreeComponent";
import AuthComponent from "./AuthComponent";
import Private from "./ProtectedRoutes";
import ProtectedRoutes from "./ProtectedRoutes";

function App() {
  return (
    <Container>
      <Row>
        <Col className="text-center">

React Authentication Tutorial

            Home
            Free Component
            Auth Component

        </Col>
      </Row>

      {/* Routes */ }
      <Routes>
        <Route exact path="/" element={ <Account /> } />
        <Route exact path="/free" element={ <FreeComponent /> } />
        <Route path="/auth" element={<ProtectedRoutes component={AuthComponent} />} />
      </Routes>
    </Container>
  );
}

export default App;

AuthComponent.js

import React from 'react';

export default function AuthComponent() {
    return (

Auth Component

    );
}

Body:
In ProtectedRoutes component, you're setting a state (setAuth in this case) directly inside the component, this is what happens when you do that:

React re-renders a component every time a state change is detected thus when you wrote

export default function ProtectedRoutes({component: Component, ...rest}) {
    ...
    if (token) {
        setAuth(true);
    };
    ...
}

you're running setAuth(sets a new state true) every time the component renders(re-renders) and in turn, re-rendering the component every time because of the state change which is why have an infinite loop there.
, this works like this:

It runs everytime a dependency changes(passed as an array), and when you pass an empty array it runs just twice - when the component mounts(renders the first time) and when it unmounts.

What you need to do is to pass an empty array as a dependency as below

import { useEffect } from 'react'

export default function ProtectedRoutes({component: Component, ...rest}) {
    ...
    useEffect(() => {
        if (token) {
            setAuth(true);
        };
    }, [])
    ...
}

this setAuth just once when the component mounts


问:
pip/python: normal site-packages is not writeable
I have a new Macbook - a user installed it, and then I installed a new user (mine), granted admin privileges and deleted the old one. I am on OS Catalina.

Since the installation I've been having several permission problems.
VSCode can't find Jupyter Notebook, pip installs packages at ~/Library/Python/3.7/site-packages.

When I do which python3 I get usr/bin/python3.
When I do pip3 install <package> I get: Defaulting to user installation because normal site-packages is not writeable And then it says it has already been installed, even though I can't access it when I do import <package>.

It's seems clear that this is a permission problem, pip can't install to the "base" python, and them python can't find what I've installed into ~/Library/Python/3.7/site-packages.

I've tried reinstalling the OS, but since I haven't done a clean install, it didn't change anything.
What am I missing?
How exactly can I fix permissions? Where do I want packages to be installed (venv sure, but some packages I want global (like jupyter).

答:
I met exactly the same issue.

I just type sudo python3.8 -m pip install .... and the error disappeared.

If I remove the sudo, issue remains.


问:
Why do I need to check if a file can be read twice (once with built in File method once with Scanner)
So I'm reviewing my professor's code for a project (my goal is to just understand everything) and she verifies that a file can be opened/read three times, and I'm not sure why all three are necessary.

1: if (!File.exists()) { }  Checks if the file exists using the built in exists method on a File object

2: if (!File.canRead()){ checks if the file can be read using the built in canRead method on the File object

3 (the part I don't understand):

```<code>Scanner blank = null;
try {
            blank = new Scanner(File) ;
        } catch (FileNotFoundException e) {
           do whatever
    }

</code>```

I don't understand why if we already checked if the file can be opened using the canRead method why we need to check AGAIN if it can be opened using a Scanner.

I would appreciate any insight. Thanks.

答:
You understand correctly the idea behind the try/catch first you have to have an understanding of what the Exceptions are.

Basically, Exception is a special kind of object that contains information related to the abnormal behavior of the application.

Exceptions are divided into two categories:

checked;

unchecked (runtime exceptions).

Checked exceptions represent situations that are more or less expected (something could go wrong not because of some king of in your code, but because file system works this way or because database works in such way, because network connection could fail, etc). It's mandatory for your code to have a scenario for these cases. Checked exceptions are called this way because the compiler checks if they are treated or not. You have to provide a try with catch that matches the expected exception or its parent. Or as an option declare using the throws clause that method may throw an exception and then handle it downstream (as a rule of thumb remember that checked exceptions must be necessarily treated somewhere).

With regards to unchecked exceptions - these are exceptions caused by the drawbacks of code. And are represented by RuntimeException and its subclasses.

When your code tries to access a file a checked exception which is called FileNotFoundException has to be treated.

There are many things that could cause FileNotFoundException:

file-path is wrong;

file exist, but can not be accessed (due to security reasons, another process uses this file, etc);

target path exists, but it's a directory and not a file.

And even if this file was successfully accessed previously there could be some wrinkles next time. Every piece of code that deals files must threat FileNotFoundException or one of its parents like `IOException' just because a filesystem behaves this way.

Hope it answers your question.

Body:
So I'm reviewing my professor's code for a project (my goal is to just understand everything) and she verifies that a file can be opened/read three times, and I'm not sure why all three are necessary.

1: if (!File.exists()) { }  Checks if the file exists using the built in exists method on a File object

2: if (!File.canRead()){ checks if the file can be read using the built in canRead method on the File object

3 (the part I don't understand):

Scanner blank = null;
try {
            blank = new Scanner(File) ;
        } catch (FileNotFoundException e) {
           do whatever
    }

I don't understand why if we already checked if the file can be opened using the canRead method why we need to check AGAIN if it can be opened using a Scanner.

I would appreciate any insight. Thanks.


问:
What will happen if you insert 5 billion rows as independent transactions? What will the vaccum do to the xmin column?
I understand that xmin is limited to 2^32-1, about 4 billion entries. Vacuum cleans up dead tuples, postgres maintains a transaction for every update to a row and those are cleaned up as a process of vacuum. Lets say, i am just inserting 5 billion records all independent transactions.

What is the dead tuple, in the above scenario?

When does the vacuum run automatically, around the 2 billion entries, what does it clean at this point?

what happens to the xmin for the 1st 2 billion records as the vacuum process starts ?

Let's say, i stored the xmin of the 100,000 transactions, as in keep pushing the insert transactions. After i inserted the 5 billion transactions, i want to query for all of my transaction after the 100,000 xmin and xmin < 500,000. Will this work?

答:
In old versions, freezing a tuple involved changing the xmin to a magic value of 2. see src/include/access/transam.h:

```<code>#define FrozenTransactionId            ((TransactionId) 2)
</code>```

This might still be seen on disk for upgraded servers, and is still passed around in memory as an indicator the tuple is frozen.

But since 9.4 freezing a tuple involves leaving xmin alone and setting two specific bits in infomask to be on.  See src/include/access/htup_details.h

```<code>#define HEAP_XMIN_COMMITTED     0x0100  /* t_xmin committed */
#define HEAP_XMIN_INVALID       0x0200  /* t_xmin invalid/aborted */
#define HEAP_XMIN_FROZEN        (HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID)
</code>```

(Since it doesn't make sense for xmin to be committed and invalid at the same time, that combination was free to get a new meaning.)

Body:
In old versions, freezing a tuple involved changing the xmin to a magic value of 2. see src/include/access/transam.h:

#define FrozenTransactionId            ((TransactionId) 2)

This might still be seen on disk for upgraded servers, and is still passed around in memory as an indicator the tuple is frozen.

But since 9.4 freezing a tuple involves leaving xmin alone and setting two specific bits in infomask to be on.  See src/include/access/htup_details.h

#define HEAP_XMIN_COMMITTED     0x0100  /* t_xmin committed */
#define HEAP_XMIN_INVALID       0x0200  /* t_xmin invalid/aborted */
#define HEAP_XMIN_FROZEN        (HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID)

(Since it doesn't make sense for xmin to be committed and invalid at the same time, that combination was free to get a new meaning.)


问:
How can decrypt Cakephp3 encrypted data right from MySQL?
I have a very specific requirement where some columns need to be encrypted using aes_encrypt / aes_decrypt. We need to encrypt the information at SQL level using a eas so it can be read using another app or directly from MySQL using a query and aes_encrypt / aes_decrypt.

Our app was developed using CakePHP 3 and database is MySQL 5.6.25.

I found and carefully follow the instruction on this selected answer: Encyption/Decryption of Form Fields in CakePHP 3

Now the data is being saved encrypted on the database... the problem is that we still need to be able to use aes_decrypt on MySQL to decrypt the information and it's returning NULL.

On CakePHP 3, config/app.php:

```<code>'Security' => ['salt' => '1234567890']
</code>```

Then encrypted using:

```<code>Security::encrypt($value, Security::salt());
</code>```

Data is saved on MySQL but aes_decrypt() returns NULL

```<code>SELECT AES_DECRIPT(address_enc, '1234567890') FROM address;
</code>```

How can I setup CakePHP 3 to correctly encrypt information so I can later decrypt it on MySQL using aes_decrypt() ?

[EDIT]

My MYSQL table:

```<code>CREATE TABLE IF NOT EXISTS `address` (
`id` int(11) NOT NULL,
  `address` varchar(255) DEFAULT NULL,
  `address_enc` blob,
  `comment` varchar(255) DEFAULT NULL,
  `comment_enc` blob
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
</code>```

Note: address and comment are just for testings.

Then, on CakePHP, I created a custom database type:

src/Database/Type/CryptedType.php

```<code><?php
namespace App\Database\Type;

use Cake\Database\Driver;
use Cake\Database\Type;
use Cake\Utility\Security;

class CryptedType extends Type
{
    public function toDatabase($value, Driver $driver)
    {
        return Security::encrypt($value, Security::salt());
    }

    public function toPHP($value, Driver $driver)
    {
        if ($value === null) {
            return null;
        }
        return Security::decrypt($value, Security::salt());
    }
}
</code>```

src/config/bootstrap.php

Register the custom type.

```<code>use Cake\Database\Type;
Type::map('crypted', 'App\Database\Type\CryptedType');
</code>```

src/Model/Table/AddressTable.php

Finally map the cryptable columns to the registered type, and that's it, from now on everything's being handled automatically.

```<code>use Cake\Database\Schema\Table as Schema;

class AddressTable extends Table
{
    // ...

    protected function _initializeSchema(Schema $table)
    {
        $table->columnType('address_enc', 'crypted');
        $table->columnType('comment_enc', 'crypted');
        return $table;
    }

    // ...
}
</code>```

答:
FOR ANYONE STRUGGLING TO DECRYPT WITH MYSQL: This generally applies to anyone using symmetric AES encryption/decryption - specifically when trying to decrypt with AES_DECRYPT.

For instance, if you are using aes-128-ecb, and your encrypted data is 16 bytes long with no padding, you need to add padding bytes to your encrypted data before trying to decrypt (because mySQL is expecting PKCS7 padding). Because MySQL uses PKCS7, you need to add 16 more bytes, in this case those pad bytes are 0x10101010101010101010101010101010. We take the left 16 bytes because when we encrypt the 0x10101010101010101010101010101010, we get 32 bytes, and we only need the first 16.

```<code>aes_decrypt(concat(<ENCRYPTED_BYTES>, left(aes_encrypt(<PAD BYTES>, <KEY>), 16)), <KEY>)
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I have a very specific requirement where some columns need to be encrypted using aes_encrypt / aes_decrypt. We need to encrypt the information at SQL level using a eas so it can be read using another app or directly from MySQL using a query and aes_encrypt / aes_decrypt.

Our app was developed using CakePHP 3 and database is MySQL 5.6.25.

I found and carefully follow the instruction on this selected answer: Encyption/Decryption of Form Fields in CakePHP 3

Now the data is being saved encrypted on the database... the problem is that we still need to be able to use aes_decrypt on MySQL to decrypt the information and it's returning NULL.

On CakePHP 3, config/app.php:

'Security' => ['salt' => '1234567890']

Then encrypted using:

Security::encrypt($value, Security::salt());

Data is saved on MySQL but aes_decrypt() returns NULL

SELECT AES_DECRIPT(address_enc, '1234567890') FROM address;

How can I setup CakePHP 3 to correctly encrypt information so I can later decrypt it on MySQL using aes_decrypt() ?

[EDIT]

My MYSQL table:

CREATE TABLE IF NOT EXISTS `address` (
`id` int(11) NOT NULL,
  `address` varchar(255) DEFAULT NULL,
  `address_enc` blob,
  `comment` varchar(255) DEFAULT NULL,
  `comment_enc` blob
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;

Note: address and comment are just for testings.

Then, on CakePHP, I created a custom database type:

src/Database/Type/CryptedType.php

<?php
namespace App\Database\Type;

use Cake\Database\Driver;
use Cake\Database\Type;
use Cake\Utility\Security;

class CryptedType extends Type
{
    public function toDatabase($value, Driver $driver)
    {
        return Security::encrypt($value, Security::salt());
    }

    public function toPHP($value, Driver $driver)
    {
        if ($value === null) {
            return null;
        }
        return Security::decrypt($value, Security::salt());
    }
}

src/config/bootstrap.php

Register the custom type.

use Cake\Database\Type;
Type::map('crypted', 'App\Database\Type\CryptedType');

src/Model/Table/AddressTable.php

Finally map the cryptable columns to the registered type, and that's it, from now on everything's being handled automatically.

use Cake\Database\Schema\Table as Schema;

class AddressTable extends Table
{
    // ...

    protected function _initializeSchema(Schema $table)
    {
        $table->columnType('address_enc', 'crypted');
        $table->columnType('comment_enc', 'crypted');
        return $table;
    }

    // ...
}

Body:
FOR ANYONE STRUGGLING TO DECRYPT WITH MYSQL: This generally applies to anyone using symmetric AES encryption/decryption - specifically when trying to decrypt with AES_DECRYPT.

For instance, if you are using aes-128-ecb, and your encrypted data is 16 bytes long with no padding, you need to add padding bytes to your encrypted data before trying to decrypt (because mySQL is expecting PKCS7 padding). Because MySQL uses PKCS7, you need to add 16 more bytes, in this case those pad bytes are 0x10101010101010101010101010101010. We take the left 16 bytes because when we encrypt the 0x10101010101010101010101010101010, we get 32 bytes, and we only need the first 16.

aes_decrypt(concat(<ENCRYPTED_BYTES>, left(aes_encrypt(<PAD BYTES>, <KEY>), 16)), <KEY>)


问:
How to display the summation of two negative number in HEX while the input is decimal in x8086 assembly language?
; Hello, I'm new to assembly language x8086, the problem is how can I add two negative numbers (decimal) and display them in HEX. I'm stuck at the point the code does not detect the negative numbers and I don't know how to display it in HEX. can someone help me? here my code</p>
```<code>TITLE PROB12
PAGE 60,132
.MODEL SMALL
.STACK 64H
.DATA
NUM1 DB 8,?,8 DUP (?)
NUM2 DB 8,?,8 DUP (?)
PROMPT1 DB CR,LF,'Enter the first number','$'
PROMPT2 DB CR,LF,'Enter the second number','$'
PROMPT3 DB CR,LF,'The total sum is '
SUM DB 7 DUP (?),'$'
CR EQU 0DH
LF EQU 0AH
.CODE
MAIN: MOV AX,@DATA
MOV DS,AX
CALL CLEAR ;clear screen
MOV AH,09
MOV DX,OFFSET PROMPT1
INT 21H ;display first prompt
MOV AH,0AH
MOV DX,OFFSET NUM1
INT 21H ;get first number
MOV AH,09
MOV DX,OFFSET PROMPT2
INT 21H ;display second prompt
MOV AH,0AH
MOV DX,OFFSET NUM2
INT 21H ;get second number
MOV SI,OFFSET NUM1 + 8 ;point to LSD of number 1
MOV DI,OFFSET NUM2 + 8 ;point to LSD of number 2
MOV BX,OFFSET SUM + 6 ;point to LSD of sum
MOV CX,7 ;add 7 bytes
CLC ;clear carry
ADD_LP: MOV AL,[SI] ;get byte from number 1
ADC AL,[DI] ;add byte from number 2
PUSHF ;save any carry
AAA ;ASCII adjust
OR AL,30H ;make it ASCII
POPF ;restore flags
MOV [BX],AL ;store sum (in BCD)
DEC SI ;decrement pointers
DEC DI ;to point to next byte
DEC BX
LOOP ADD_LP ;loop through 7 bytes
MOV AH,09
MOV DX,OFFSET PROMPT3
INT 21H ;display result
MOV AH,4CH
INT 21H ;go back to DOS
;--------------------------------------------------
CLEAR PROC
MOV AH,06 ;clear screen function
MOV AL,00 ;page 0
MOV BH,07 ;normal attribute
MOV CX,0 ;entire screen
MOV DX,184FH
INT 10H
RET
CLEAR ENDP
END MAIN
</code>```

答:
I'm stuck at the point the code does not detect the negative numbers and I don't know how to display it in HEX

It makes no sense asking about the above if the code that you have written has more fundamental issues.

Your code uses the DOS.BufferedInput function 0Ah, and hopes to find the least significant digit of the inputted numerical characters at offset 8 of the input structure. This approach can only work if the user at the keyboard always provides precisely 7 digits. That's not very user-friendly.

The loop that later adds two such numbers is not propagating the carry like it should! Currently you are preserving the carry from the adc operation, but that's pointless since it will always be clear. You need to propagate throughout the loop the carry that you obtain from the aaa instruction.

```<code>  MOV  CX,7    ;add 7 bytes
  CLC
ADD_LP:
  MOV  AL,[SI] ;get byte from number 1
  ADC  AL,[DI] ;add byte from number 2
  AAA          ;ASCII adjust

  PUSHF        ;save any carry
  OR   AL,30h  ;make it ASCII
  POPF         ;restore flags

  MOV  [BX],AL ;store sum (in BCD)
  DEC  SI      ;decrement pointers
  DEC  DI      ;to point to next byte
  DEC  BX
  LOOP ADD_LP  ;loop through 7 bytes
</code>```

If you were going to detect a negative number then you would have to inspect the very first inputted characters cmp byte ptr [NUM1 + 2], '-' and cmp byte ptr [NUM2 + 2], '-'. If both are indeed present then the current addition is fine as long as you iterate over the remaining 6 digits (7-1) and that you write a "-" character at SUM.

Body:
; Hello, I'm new to assembly language x8086, the problem is how can I add two negative numbers (decimal) and display them in HEX. I'm stuck at the point the code does not detect the negative numbers and I don't know how to display it in HEX. can someone help me? here my code

TITLE PROB12
PAGE 60,132
.MODEL SMALL
.STACK 64H
.DATA
NUM1 DB 8,?,8 DUP (?)
NUM2 DB 8,?,8 DUP (?)
PROMPT1 DB CR,LF,'Enter the first number','$'
PROMPT2 DB CR,LF,'Enter the second number','$'
PROMPT3 DB CR,LF,'The total sum is '
SUM DB 7 DUP (?),'$'
CR EQU 0DH
LF EQU 0AH
.CODE
MAIN: MOV AX,@DATA
MOV DS,AX
CALL CLEAR ;clear screen
MOV AH,09
MOV DX,OFFSET PROMPT1
INT 21H ;display first prompt
MOV AH,0AH
MOV DX,OFFSET NUM1
INT 21H ;get first number
MOV AH,09
MOV DX,OFFSET PROMPT2
INT 21H ;display second prompt
MOV AH,0AH
MOV DX,OFFSET NUM2
INT 21H ;get second number
MOV SI,OFFSET NUM1 + 8 ;point to LSD of number 1
MOV DI,OFFSET NUM2 + 8 ;point to LSD of number 2
MOV BX,OFFSET SUM + 6 ;point to LSD of sum
MOV CX,7 ;add 7 bytes
CLC ;clear carry
ADD_LP: MOV AL,[SI] ;get byte from number 1
ADC AL,[DI] ;add byte from number 2
PUSHF ;save any carry
AAA ;ASCII adjust
OR AL,30H ;make it ASCII
POPF ;restore flags
MOV [BX],AL ;store sum (in BCD)
DEC SI ;decrement pointers
DEC DI ;to point to next byte
DEC BX
LOOP ADD_LP ;loop through 7 bytes
MOV AH,09
MOV DX,OFFSET PROMPT3
INT 21H ;display result
MOV AH,4CH
INT 21H ;go back to DOS
;--------------------------------------------------
CLEAR PROC
MOV AH,06 ;clear screen function
MOV AL,00 ;page 0
MOV BH,07 ;normal attribute
MOV CX,0 ;entire screen
MOV DX,184FH
INT 10H
RET
CLEAR ENDP
END MAIN

Body:
I'm stuck at the point the code does not detect the negative numbers and I don't know how to display it in HEX

It makes no sense asking about the above if the code that you have written has more fundamental issues.

Your code uses the DOS.BufferedInput function 0Ah, and hopes to find the least significant digit of the inputted numerical characters at offset 8 of the input structure. This approach can only work if the user at the keyboard always provides precisely 7 digits. That's not very user-friendly.

The loop that later adds two such numbers is not propagating the carry like it should! Currently you are preserving the carry from the adc operation, but that's pointless since it will always be clear. You need to propagate throughout the loop the carry that you obtain from the aaa instruction.

  MOV  CX,7    ;add 7 bytes
  CLC
ADD_LP:
  MOV  AL,[SI] ;get byte from number 1
  ADC  AL,[DI] ;add byte from number 2
  AAA          ;ASCII adjust

  PUSHF        ;save any carry
  OR   AL,30h  ;make it ASCII
  POPF         ;restore flags

  MOV  [BX],AL ;store sum (in BCD)
  DEC  SI      ;decrement pointers
  DEC  DI      ;to point to next byte
  DEC  BX
  LOOP ADD_LP  ;loop through 7 bytes

If you were going to detect a negative number then you would have to inspect the very first inputted characters cmp byte ptr [NUM1 + 2], '-' and cmp byte ptr [NUM2 + 2], '-'. If both are indeed present then the current addition is fine as long as you iterate over the remaining 6 digits (7-1) and that you write a "-" character at SUM.


问:
What is the difference and relationship between execution context and lexical environment?
In JavaScript: Understanding the Weird Parts lexical environment is explained as the scope of your code while execution context is a collection of lexical environments, and that it includes stuff beyond your written code.

The descriptions of these terms still sound overlapping in functionality and it's unclear as to what execution context does or how it does it.

答:
Execution Context
Execution Context is created every time, the JS engine calls a function. They stays in the call stack.

Lexical Environment
The lexical environment includes the scopes that are present at a certain point in time. Every execution context has its own lexical environment.

Check the image and see the scope and the call stack section on the right side of the image:

The call Stack section contains two execution context (test1 and anonymous).

The scope section contains the lexical environment of the test1 execution context.

test1 lexical environment has three objects Global, Local, and Block when the JS engine is executing line no 10.


问:
In for loops, is there a way to execute a module function with attributes assigned by the for loop?
Python 3.10.1

Totally new to programming, please be patient.

I want to print a list of all foreground colors available in the colorama module in the color named, like so:

PowerShell Colors

My attempt:

```<code>from colorama import init, Fore, Style
init()

# available foreground colors acquired via dir(Fore)
colors = [
    'BLACK',
    'BLUE',
    'CYAN',
    'GREEN',
    'LIGHTBLACK_EX',
    'LIGHTBLUE_EX',
    'LIGHTCYAN_EX',
    'LIGHTGREEN_EX',
    'LIGHTMAGENTA_EX',
    'LIGHTRED_EX',
    'LIGHTWHITE_EX',
    'LIGHTYELLOW_EX',
    'MAGENTA',
    'RED',
    'WHITE',
    'YELLOW'
]

for col in colors:
    fcol = "Fore." + col
    print(f"  {exec(fcol)}[{col}]{Style.RESET_ALL}")
</code>```

My output (no color changes):

```<code>  None[BLACK]
  None[BLUE]
  None[CYAN]
  None[GREEN]
  None[LIGHTBLACK_EX]
  None[LIGHTBLUE_EX]
  None[LIGHTCYAN_EX]
  None[LIGHTGREEN_EX]
  None[LIGHTMAGENTA_EX]
  None[LIGHTRED_EX]
  None[LIGHTWHITE_EX]
  None[LIGHTYELLOW_EX]
  None[MAGENTA]
  None[RED]
  None[WHITE]
  None[YELLOW]
</code>```

答:
Apologies, I searched around some more and found an answer by user Kasper I couldn't before, which I implemented to my code like so:</p>
```<code>from colorama import init, Fore, Style
init()

colors = dict(Fore.__dict__.items())

for color in colors.keys():
    print(colors[color] + f"  [{color}]")
</code>```

Body:
Python 3.10.1

Totally new to programming, please be patient.

I want to print a list of all foreground colors available in the colorama module in the color named, like so:

PowerShell Colors

My attempt:

from colorama import init, Fore, Style
init()

# available foreground colors acquired via dir(Fore)
colors = [
    'BLACK',
    'BLUE',
    'CYAN',
    'GREEN',
    'LIGHTBLACK_EX',
    'LIGHTBLUE_EX',
    'LIGHTCYAN_EX',
    'LIGHTGREEN_EX',
    'LIGHTMAGENTA_EX',
    'LIGHTRED_EX',
    'LIGHTWHITE_EX',
    'LIGHTYELLOW_EX',
    'MAGENTA',
    'RED',
    'WHITE',
    'YELLOW'
]

for col in colors:
    fcol = "Fore." + col
    print(f"  {exec(fcol)}[{col}]{Style.RESET_ALL}")

My output (no color changes):

  None[BLACK]
  None[BLUE]
  None[CYAN]
  None[GREEN]
  None[LIGHTBLACK_EX]
  None[LIGHTBLUE_EX]
  None[LIGHTCYAN_EX]
  None[LIGHTGREEN_EX]
  None[LIGHTMAGENTA_EX]
  None[LIGHTRED_EX]
  None[LIGHTWHITE_EX]
  None[LIGHTYELLOW_EX]
  None[MAGENTA]
  None[RED]
  None[WHITE]
  None[YELLOW]

Body:
Apologies, I searched around some more and found an answer by user Kasper I couldn't before, which I implemented to my code like so:

from colorama import init, Fore, Style
init()

colors = dict(Fore.__dict__.items())

for color in colors.keys():
    print(colors[color] + f"  [{color}]")


问:
How do I change the whole page background color with each route using React?
Im building a react app and I want every route to have a different background color.
But the only way to change the background color of the whole page is by using the * {} tag. Is there some kind of theme library or anything I can do to pass a variable from my routes to the * {} tag to change its background color?

Other solutions would include putting a min-height on my root component but that just feels wrong and I will not do it.

答:
I think you can use this as well

    function App() {

      const [color, changeColor] = useState("#282c34");

      document.body.style.backgroundColor = color;

      return (

          <Router>

                    <Link to="/" onClick={() => changeColor("#282c34")}>
                      Home
                    </Link>

                    <Link to="/about/" onClick={() => changeColor("#9c27b0")}>
                      About
                    </Link>

                    <Link to="/users/" onClick={() => changeColor("#007bff")}>
                      Users
                    </Link>

              <Route path="/" exact component={Index} />
              <Route path="/about/" component={About} />
              <Route path="/users/" component={Users} />

          </Router>

      );
    }
```

Body:
I think you can use this as well

    function App() {

      const [color, changeColor] = useState("#282c34");

      document.body.style.backgroundColor = color;

      return (
        <div>
          <Router>
            <div>
              <nav>
                <ul>
                  <li>
                    <Link to="/" onClick={() => changeColor("#282c34")}>
                      Home
                    </Link>
                  </li>
                  <li>
                    <Link to="/about/" onClick={() => changeColor("#9c27b0")}>
                      About
                    </Link>
                  </li>
                  <li>
                    <Link to="/users/" onClick={() => changeColor("#007bff")}>
                      Users
                    </Link>
                  </li>
                </ul>
              </nav>
              <Route path="/" exact component={Index} />
              <Route path="/about/" component={About} />
              <Route path="/users/" component={Users} />
            </div>
          </Router>
        </div>
      );
    }


问:
Implementing pacman in c, ghost movement
I am creating a pacman in c and currently I am usisng a single thread for each ghost and each ghost represents a '#' but when I run it all the screen gets full of ghosts and not all the ghosts move just one or two.

Im using this logic

create a struct of 5 ghost, each ghost contains the x,y position.
create an array of 5 threads and each thread implements one ghost
each ghost moves randomly on the screen, for each space that it moves I print
a space in the old position and then I print a '#' in the new position.

Could you provide me please an example of how to implement the movement of the ghost,
or the implementation Im doing is the correct way?

Thank you

答:
One thread per agent is not a very common approach to building games. It quickly becomes unworkable for large scenes. The conventional solution is to define a state machine representing a ghost, with some kind of "advance" method that gives it a chance to adjust its internal state to the next time quantum. Create multiple instances of this state machine, and call all their "advance" methods on each iteration of the game loop. All of this can happen in a single thread.

There's quite a bit more to it than this, but it'll get you started.


问:
Calling an overridden function from a base constructor
It makes sense that I should not call an overridden function from a base constructor, since the derived class is not yet constructed.

But I want to use this design pattern, where each derived class provides methods for calculating the properties of the base class since the properties should be immutable and assigned at the constructor.

Shape.cs

```<code>public abstract class Shape
{
    protected Shape()
    {
        Area = 0f;
        Center = Vector2.Zero;
        const int n = 36;
        float du = 1/(float)n, dv = 1/(float)n;
        for (int i = 0; i < n; i++)
        {
            float u = (i+0.5f)*du;
            for (int j = 0; j < n; j++)
            {
                float v = (i+0.5f)*dv;
                float f = GetAreaElement(u, v);
                // Warning: Remove this call from a constructor to the overridable 'GetAreaElement' method.
                Area += f*du*dv;
                Center += GetPosition(u, v)*f*du*dv;
                // Warning: Remove this call from a constructor to the overridable 'GetPosition' method.
            }
        }
        Center /= Area;
    }

    public abstract Vector2 GetPosition(float u, float v);
    protected abstract float GetAreaElement(float u, float v);

    public float Area { get;  }
    public Vector2 Center { get; }
}

public class Circle : Shape
{
    public Circle(float radius)
    {
        Radius=radius;
    }

    public float Radius { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        float r = u*Radius, θ = (float)(2*Math.PI)*v;
        return new Vector2(
            r*(float)Math.Cos(θ),
            r*(float)Math.Sin(θ));
    }

    protected override float GetAreaElement(float u, float v)
    {
        return u*Radius;
    }
}

public class Rectangle : Shape
{
    public Rectangle(float side1, float side2)
    {
        Side1=side1;
        Side2=side2;
    }

    public float Side1 { get; }
    public float Side2 { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        return new Vector2((u-0.5f)*Side1, (v-0.5f)*Side2);
    }

    protected override float GetAreaElement(float u, float v)
    {
        return Side1*Side2;
    }
}
</code>```

So what is the solution here? I want to use the base constructor to define the properties, and the calculation depends on the implementation of the derived class.

Workaround 1 - Future calculator

A workaround would be to provide a protected function that calculates the properties, each to be called from each constructor of the derived class, but there is no enforcement here. If one class forgets to call the calculator function the whole thing falls apart. And the properties are now private set which is not immutable really.

```<code>public abstract class Shape
{
    protected void Calculate()
    {
        ...
        float f = GetAreaElement(u, v);
        ...
        Center += GetPosition(u, v)*f*du*dv;
        ...
    }

    public abstract Vector2 GetPosition(float u, float v);
    protected abstract float GetAreaElement(float u, float v);

    public float Area { get; private set; }
    public Vector2 Center { get; private set; }
}

public class Circle : Shape
{
    public Circle(float radius)
    {
        Radius=radius;

        base.Calculate();
    }

    public float Radius { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        ...
    }

    protected override float GetAreaElement(float u, float v)
    {
        ...
    }
}
</code>```

Workaround 2 - Function delegates

Another workaround would be to supply the delegates to the required function implementations as arguments to the base class constructor.

```<code>public delegate float AreaFactor(float u, float v);
public delegate Vector2 PositionFunc(float u, float v);
public abstract class Shape
{
    protected Shape(AreaFactor a, PositionFunc p)
    {
        this.GetAreaElement = a;
        this.GetPosition = p;
        ...
        float f = a(u, v);
        this.Center += p(u, v)*f*du*dv;
        ...
    }

    public float Area { get; }
    public Vector2 Center { get;  }

    public AreaFactor GetAreaElement { get; }
    public PositionFunc GetPosition { get; }
}

public class Circle : Shape
{
    public Circle(float radius)
        : base(
              (u, v) => u*radius,
              (u,v)=>
                {
                    float r = u*radius, θ = (float)(2*Math.PI)*v;
                    return new Vector2(
                        r*(float)Math.Cos(θ),
                        r*(float)Math.Sin(θ));
                })
    {
        Radius=radius;
    }

    public float Radius { get; }
}
</code>```

This seems a bit clunky to me, and I am not sure I like the function delegate properties, instead of overridden methods.

Question/Challege

Can [SO] provide some other ways of achieving the above-stated goals

Base properties are immutable

Base properties are calculated at the constructor based on the implementation details of the derived classes.

Each derived class holds its own immutable properties used to describe the derived class.

答:
One option is to, instead of calculating the Area and Center in the constructor, lazily calculate them in the property getter. It will require a backing field to know if the property has been calculated or not, but it will get rid of the warning.

It also seems odd to me that you don't pass the center to the Shape constructor, but I don't fully understand the design you're using.

Body:
It makes sense that I should not call an overridden function from a base constructor, since the derived class is not yet constructed.

But I want to use this design pattern, where each derived class provides methods for calculating the properties of the base class since the properties should be immutable and assigned at the constructor.

Shape.cs

public abstract class Shape
{
    protected Shape()
    {
        Area = 0f;
        Center = Vector2.Zero;
        const int n = 36;
        float du = 1/(float)n, dv = 1/(float)n;
        for (int i = 0; i < n; i++)
        {
            float u = (i+0.5f)*du;
            for (int j = 0; j < n; j++)
            {
                float v = (i+0.5f)*dv;
                float f = GetAreaElement(u, v);
                // Warning: Remove this call from a constructor to the overridable 'GetAreaElement' method.
                Area += f*du*dv;
                Center += GetPosition(u, v)*f*du*dv;
                // Warning: Remove this call from a constructor to the overridable 'GetPosition' method.
            }
        }
        Center /= Area;
    }

    public abstract Vector2 GetPosition(float u, float v);
    protected abstract float GetAreaElement(float u, float v);

    public float Area { get;  }
    public Vector2 Center { get; }
}

public class Circle : Shape
{
    public Circle(float radius)
    {
        Radius=radius;
    }

    public float Radius { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        float r = u*Radius, θ = (float)(2*Math.PI)*v;
        return new Vector2(
            r*(float)Math.Cos(θ),
            r*(float)Math.Sin(θ));
    }

    protected override float GetAreaElement(float u, float v)
    {
        return u*Radius;
    }
}

public class Rectangle : Shape
{
    public Rectangle(float side1, float side2)
    {
        Side1=side1;
        Side2=side2;
    }

    public float Side1 { get; }
    public float Side2 { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        return new Vector2((u-0.5f)*Side1, (v-0.5f)*Side2);
    }

    protected override float GetAreaElement(float u, float v)
    {
        return Side1*Side2;
    }
}

So what is the solution here? I want to use the base constructor to define the properties, and the calculation depends on the implementation of the derived class.

Workaround 1 - Future calculator

A workaround would be to provide a protected function that calculates the properties, each to be called from each constructor of the derived class, but there is no enforcement here. If one class forgets to call the calculator function the whole thing falls apart. And the properties are now private set which is not immutable really.

public abstract class Shape
{
    protected void Calculate()
    {
        ...
        float f = GetAreaElement(u, v);
        ...
        Center += GetPosition(u, v)*f*du*dv;
        ...
    }

    public abstract Vector2 GetPosition(float u, float v);
    protected abstract float GetAreaElement(float u, float v);

    public float Area { get; private set; }
    public Vector2 Center { get; private set; }
}

public class Circle : Shape
{
    public Circle(float radius)
    {
        Radius=radius;

        base.Calculate();
    }

    public float Radius { get; }

    public override Vector2 GetPosition(float u, float v)
    {
        ...
    }

    protected override float GetAreaElement(float u, float v)
    {
        ...
    }
}

Workaround 2 - Function delegates

Another workaround would be to supply the delegates to the required function implementations as arguments to the base class constructor.

public delegate float AreaFactor(float u, float v);
public delegate Vector2 PositionFunc(float u, float v);
public abstract class Shape
{
    protected Shape(AreaFactor a, PositionFunc p)
    {
        this.GetAreaElement = a;
        this.GetPosition = p;
        ...
        float f = a(u, v);
        this.Center += p(u, v)*f*du*dv;
        ...
    }

    public float Area { get; }
    public Vector2 Center { get;  }

    public AreaFactor GetAreaElement { get; }
    public PositionFunc GetPosition { get; }
}

public class Circle : Shape
{
    public Circle(float radius)
        : base(
              (u, v) => u*radius,
              (u,v)=>
                {
                    float r = u*radius, θ = (float)(2*Math.PI)*v;
                    return new Vector2(
                        r*(float)Math.Cos(θ),
                        r*(float)Math.Sin(θ));
                })
    {
        Radius=radius;
    }

    public float Radius { get; }
}

This seems a bit clunky to me, and I am not sure I like the function delegate properties, instead of overridden methods.

Question/Challege

Can [SO] provide some other ways of achieving the above-stated goals

Base properties are immutable

Base properties are calculated at the constructor based on the implementation details of the derived classes.

Each derived class holds its own immutable properties used to describe the derived class.


问:
What is the fastest way to calculate the logical_and (&&) between elements of two __m256i variables, looking for any pair of non-zero elements
As far as I know, integers in C++ can be treated like booleans, and we can have a code like this:

```<code>int a = 6, b = 10;
if (a && b) do something ---> true as both a and b are non-zero
</code>```

Now, assume that we have:

```<code>__m256i a, b;
</code>```

I need to apply logical_and (&&) for all 4 long variables in __m256i, and return true if one pair is non-zero. I mean something like:

```<code>(a[0] && b[0]) || (a[1] && b[1]) || ...
</code>```

Do we have a fast code in AVX or AVX2 for this purpose?

I could not find any direct instruction for this purpose, and definitely, using the bitwise and (&) also is not the same.

答:
You can cleverly combine a vpcmpeqq with a vptest:

```<code>__m256i mask = _mm256_cmpeq_epi64(a, _mm256_set1_epi64x(0));
bool result = ! _mm256_testc_si256(mask, b);
</code>```

The result is true if and only if (~mask & b) != 0 or

```<code>((a[i]==0 ? 0 : -1) & b[i]) != 0 // for some i
// equivalent to
((a[i]==0 ? 0 : b[i])) != 0      // for some i
// equivalent to
a[i]!=0 && b[i]!=0               // for some i
</code>```

which is equivalent to what you want.

Godbolt-link (play around with a and b): https://godbolt.org/z/aTjx7vMKd

If result is a loop condition, the compiler should of course directly do a jb/jnb instruction instead of setnb.

Body:
As far as I know, integers in C++ can be treated like booleans, and we can have a code like this:

int a = 6, b = 10;
if (a && b) do something ---> true as both a and b are non-zero

Now, assume that we have:

__m256i a, b;

I need to apply logical_and (&&) for all 4 long variables in __m256i, and return true if one pair is non-zero. I mean something like:

(a[0] && b[0]) || (a[1] && b[1]) || ...

Do we have a fast code in AVX or AVX2 for this purpose?

I could not find any direct instruction for this purpose, and definitely, using the bitwise and (&) also is not the same.

Body:
You can cleverly combine a vpcmpeqq with a vptest:

__m256i mask = _mm256_cmpeq_epi64(a, _mm256_set1_epi64x(0));
bool result = ! _mm256_testc_si256(mask, b);

The result is true if and only if (~mask & b) != 0 or

((a[i]==0 ? 0 : -1) & b[i]) != 0 // for some i
// equivalent to
((a[i]==0 ? 0 : b[i])) != 0      // for some i
// equivalent to
a[i]!=0 && b[i]!=0               // for some i

which is equivalent to what you want.

Godbolt-link (play around with a and b): https://godbolt.org/z/aTjx7vMKd

If result is a loop condition, the compiler should of course directly do a jb/jnb instruction instead of setnb.


问:
OpenCV - Fastest method to check if two images are 100% same or not
There are many questions over here which checks if two images are "nearly" similar or not.

My task is simple. With OpenCV, I want to find out if two images are 100% identical or not.

They will be of same size but can be saved with different filenames.

答:
I have done this task.

Compare file sizes.
Compare exif data.
Compare first 'n' byte, where 'n' is 128 to 1024 or so.
Compare last 'n' bytes.
Compare middle 'n' bytes.
Compare checksum


问:
Input 0 of layer X is incompatible with the layer: expected shape= X, found shape= Y
I'm trying to view every channel for all of the intermediate activations for my model for an image, following this tutorial.

I wrote this:

```<code>import glob
import matplotlib
from matplotlib import pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import imageio as im
from keras import models
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.layers import Dropout
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint

images = []
for img_path in glob.glob('/content/pic1*.JPEG'):
    image1 = mpimg.imread(img_path)
    open_file = image1 / 255
    resize = cv2.resize(open_file,(150,150))
    images.append(open_file)

plt.figure(figsize=(20,10))
columns = 5
for i, image in enumerate(images):
    plt.subplot(len(images) / columns + 1, columns, i + 1)
    plt.imshow(image)

print(model.summary())

layer_outputs = [layer.output for layer in model.layers[:]]
activation_model = models.Model(inputs=model.input, outputs=layer_outputs)
activations = activation_model.predict(images[0])

  [1]: https://towardsdatascience.com/visualizing-intermediate-activation-in-convolutional-neural-networks-with-keras-260b36d60d0
</code>```

I get the error:

```<code>   ValueError: Input 0 of layer "model_12" is incompatible with the layer: expected shape=(None, 150, 150, 3), found shape=(32, 448, 3)
</code>```

I understand that it's saying that I have the wrong shape input, but is the line cv2.resize(open_file, (150,150)) not meant to be changing that?

答:
You are appending the image before resizing.

```<code>for img_path in glob.glob('/content/pic1*.JPEG'):
    image1 = mpimg.imread(img_path)
    open_file = image1 / 255
    resize = cv2.resize(open_file,(150,150))
    # images.append(open_file)  # WRONG
    images.append(resize)
</code>```

Also, I think the input shape for your model is 4-dimensional.

# activations = activation_model.predict(images[0])  # not sure this works
activations = activation_model.predict(np.expand_dims(images[0]), axis=0)
```

For details, check out section 'Predicting the class of unseen images' of the tutorial you mentioned.

Body:
I'm trying to view every channel for all of the intermediate activations for my model for an image, following this tutorial.

I wrote this:

import glob
import matplotlib
from matplotlib import pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import imageio as im
from keras import models
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.layers import Dropout
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint

images = []
for img_path in glob.glob('/content/pic1*.JPEG'):
    image1 = mpimg.imread(img_path)
    open_file = image1 / 255
    resize = cv2.resize(open_file,(150,150))
    images.append(open_file)

plt.figure(figsize=(20,10))
columns = 5
for i, image in enumerate(images):
    plt.subplot(len(images) / columns + 1, columns, i + 1)
    plt.imshow(image)

print(model.summary())

layer_outputs = [layer.output for layer in model.layers[:]]
activation_model = models.Model(inputs=model.input, outputs=layer_outputs)
activations = activation_model.predict(images[0])

  [1]: https://towardsdatascience.com/visualizing-intermediate-activation-in-convolutional-neural-networks-with-keras-260b36d60d0

I get the error:

   ValueError: Input 0 of layer "model_12" is incompatible with the layer: expected shape=(None, 150, 150, 3), found shape=(32, 448, 3)

I understand that it's saying that I have the wrong shape input, but is the line cv2.resize(open_file, (150,150)) not meant to be changing that?

Body:
You are appending the image before resizing.

for img_path in glob.glob('/content/pic1*.JPEG'):
    image1 = mpimg.imread(img_path)
    open_file = image1 / 255
    resize = cv2.resize(open_file,(150,150))
    # images.append(open_file)  # WRONG
    images.append(resize)

Also, I think the input shape for your model is 4-dimensional.

# activations = activation_model.predict(images[0])  # not sure this works
activations = activation_model.predict(np.expand_dims(images[0]), axis=0)

For details, check out section 'Predicting the class of unseen images' of the tutorial you mentioned.


问:
Find the index number of where a variable fits between in pandas column
I have the following dataframe:

```<code>import pandas as pd
#Create DF
d = {
     'Category': ['A','B','C','D','E','F','G'],
    'Value':[10,20,30,40,50,60,70],
    }

df = pd.DataFrame(data=d)
df
</code>```

I then have a defined variable of a number that may change:
e.g val = 45

How do i search the dataframe column Value and return the index number of the row where val  would fit.

Expected value would be:

for val = 45 the index i would like returned is 4

for val = 22 the index i would like returned is 2

for val = 60 the index i would like returned is 6 (would go after if it has a match)

Any help would be greatly appreciated!

答:
You can use pd.Series.searchsorted

```<code>df.Value.searchsorted([45,22,60], side='right')
</code>```

Output

```<code>array([4, 2, 6])
</code>```

Body:
I have the following dataframe:

import pandas as pd
#Create DF
d = {
     'Category': ['A','B','C','D','E','F','G'],
    'Value':[10,20,30,40,50,60,70],
    }

df = pd.DataFrame(data=d)
df

I then have a defined variable of a number that may change:
e.g val = 45

How do i search the dataframe column Value and return the index number of the row where val  would fit.

Expected value would be:

for val = 45 the index i would like returned is 4

for val = 22 the index i would like returned is 2

for val = 60 the index i would like returned is 6 (would go after if it has a match)

Any help would be greatly appreciated!

Body:
You can use pd.Series.searchsorted

df.Value.searchsorted([45,22,60], side='right')

Output

array([4, 2, 6])


问:
How to use linux command "cat" from a script?
Input:A.fasta

```<code>>A
sldkfjslkdcskd
</code>```

Input:B.fasta

```<code>>B
pofnvkweu
</code>```

Expected output:A_B.fasta

```<code>>A
sldkfjslkdcskd
>B
pofnvkweu
</code>```

Code:

```<code>my_model_dir_path='../my_model'
word='.fasta'

for f in os.listdir(my_model_dir_path):
    if word in f:
        subprocess.call(["cat", f"{f}"])
</code>```

Error message:

```<code>No such file or directory
</code>```

I used subprocess to call linux command cat to concatenate fasta files in a fasta format like expected output. However, I got a message like No such file or directory even though the files and the directory exist in right path.

答:
I saw this post: https://www.delftstack.com/howto/python/python-cat/

```<code>import os

os.system("echo 'Hello! Python is the best programming language.' >> ~/file.txt")
os.system("cat ~/file.txt")
</code>```

Output:
Hello! Python is the best programming language.

Body:
Input:A.fasta

>A
sldkfjslkdcskd

Input:B.fasta

>B
pofnvkweu

Expected output:A_B.fasta

>A
sldkfjslkdcskd
>B
pofnvkweu

Code:

my_model_dir_path='../my_model'
word='.fasta'

for f in os.listdir(my_model_dir_path):
    if word in f:
        subprocess.call(["cat", f"{f}"])

Error message:

No such file or directory

I used subprocess to call linux command cat to concatenate fasta files in a fasta format like expected output. However, I got a message like No such file or directory even though the files and the directory exist in right path.

Body:
I saw this post: https://www.delftstack.com/howto/python/python-cat/

import os

os.system("echo 'Hello! Python is the best programming language.' >> ~/file.txt")
os.system("cat ~/file.txt")

Output:
Hello! Python is the best programming language.


答:
Try this:

from colorama import Fore, Style

# available foreground colors acquired via dir(Fore)
colors = {
    "BLACK": Fore.BLACK,
    "BLUE": Fore.BLUE,
    "CYAN": Fore.CYAN,
    "GREEN": Fore.GREEN,
    "LIGHTBLACK_EX": Fore.LIGHTBLACK_EX,
    "LIGHTBLUE_EX": Fore.LIGHTBLUE_EX,
    "LIGHTCYAN_EX": Fore.LIGHTCYAN_EX,
    "LIGHTGREEN_EX": Fore.LIGHTGREEN_EX,
    "LIGHTMAGENTA_EX": Fore.LIGHTMAGENTA_EX,
    "LIGHTRED_EX": Fore.LIGHTRED_EX,
    "LIGHTWHITE_EX": Fore.LIGHTWHITE_EX,
    "LIGHTYELLOW_EX": Fore.LIGHTYELLOW_EX,
    "MAGENTA": Fore.MAGENTA,
    "RED": Fore.RED,
    "WHITE": Fore.WHITE,
    "YELLOW": Fore.YELLOW
}

for col in colors:
    print(f"{colors[col]}[{col}]{Style.RESET_ALL}")
```

Title: In for loops, is there a way to execute a module function with attributes assigned by the for loop?


问:
App crash on firestick 2nd gen when getting presigned URL
Gradle

```<code>implementation platform('software.amazon.awssdk:bom:2.17.119')
implementation 'software.amazon.awssdk:s3'
</code>```

Code Used To get Presigned Url

```<code>public static URL getPresignedUrl(String bucketName, String keyName ) {
    PresignedGetObjectRequest presignedGetObjectRequest = null;
    try {
        AwsCredentialsProvider credentialsProvider = StaticCredentialsProvider.create
                (AwsBasicCredentials.create(Access Key, Secret Key));

        S3Presigner preSigner = S3Presigner.builder()
                .credentialsProvider(credentialsProvider)
                .region(Region.US_WEST_2).build();
        GetObjectRequest getObjectRequest =
                GetObjectRequest.builder()
                        .bucket(bucketName)
                        .key(keyName)
                        .build();

        GetObjectPresignRequest getObjectPresignRequest = null;
        if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
            getObjectPresignRequest = GetObjectPresignRequest.builder()
                    .signatureDuration(Duration.ofDays(1))
                    .getObjectRequest(getObjectRequest)
                    .build();
        }

        // Generate the presigned request
         presignedGetObjectRequest =
                 preSigner.presignGetObject(getObjectPresignRequest);
    } catch (Exception e) {
        e.getStackTrace();
    }
    return  presignedGetObjectRequest.url();
}
</code>```

App working fine if the fire OS version is greater than or equal 7.x.x
Crashes on fire OS version 5.x.x

Crashing on the line  S3Presigner preSigner = S3Presigner.builder()

Need help

答:
Without exact knowledge about the versions it's hard to spot the reason directly.

But you can try it by yourself by separating the concatenated statements:

Currently:

```<code>        S3Presigner preSigner = S3Presigner.builder()
                .credentialsProvider(credentialsProvider)
                .region(Region.US_WEST_2).build();
</code>```

New:

```<code>        S3Presigner preSigner = S3Presigner.builder();
        preSigner = preSigner.credentialsProvider(credentialsProvider);
        preSigner = preSigner.region(Region.US_WEST_2):
        preSigner = preSigner.build();
</code>```

I'm neither firm with Java nor with the Fire OS API, so it's quite possible that my code wouldn't work even on the never Fire OS versions and you had to change a bit.

Nevertheless I think you can get the idea to disentangle all statements to
see which part of causing the problem.

After knowing this you can look if the corresponding function is perhaps missing in Fire OS 5 or if you have to change the parameters perhaps.

Body:
Gradle

implementation platform('software.amazon.awssdk:bom:2.17.119')
implementation 'software.amazon.awssdk:s3'

Code Used To get Presigned Url

public static URL getPresignedUrl(String bucketName, String keyName ) {
    PresignedGetObjectRequest presignedGetObjectRequest = null;
    try {
        AwsCredentialsProvider credentialsProvider = StaticCredentialsProvider.create
                (AwsBasicCredentials.create(Access Key, Secret Key));

        S3Presigner preSigner = S3Presigner.builder()
                .credentialsProvider(credentialsProvider)
                .region(Region.US_WEST_2).build();
        GetObjectRequest getObjectRequest =
                GetObjectRequest.builder()
                        .bucket(bucketName)
                        .key(keyName)
                        .build();

        GetObjectPresignRequest getObjectPresignRequest = null;
        if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
            getObjectPresignRequest = GetObjectPresignRequest.builder()
                    .signatureDuration(Duration.ofDays(1))
                    .getObjectRequest(getObjectRequest)
                    .build();
        }

        // Generate the presigned request
         presignedGetObjectRequest =
                 preSigner.presignGetObject(getObjectPresignRequest);
    } catch (Exception e) {
        e.getStackTrace();
    }
    return  presignedGetObjectRequest.url();
}

App working fine if the fire OS version is greater than or equal 7.x.x
Crashes on fire OS version 5.x.x

Crashing on the line  S3Presigner preSigner = S3Presigner.builder()

Need help

Body:
Without exact knowledge about the versions it's hard to spot the reason directly.

But you can try it by yourself by separating the concatenated statements:

Currently:

        S3Presigner preSigner = S3Presigner.builder()
                .credentialsProvider(credentialsProvider)
                .region(Region.US_WEST_2).build();

New:

        S3Presigner preSigner = S3Presigner.builder();
        preSigner = preSigner.credentialsProvider(credentialsProvider);
        preSigner = preSigner.region(Region.US_WEST_2):
        preSigner = preSigner.build();

I'm neither firm with Java nor with the Fire OS API, so it's quite possible that my code wouldn't work even on the never Fire OS versions and you had to change a bit.

Nevertheless I think you can get the idea to disentangle all statements to
see which part of causing the problem.

After knowing this you can look if the corresponding function is perhaps missing in Fire OS 5 or if you have to change the parameters perhaps.


问:
Delete specific rows based in conditions on rows from a dataframe pandas
I want to delete specific rows based in conditions on rows from a Pandas dataframe.

For example, since I have several currency pairs at the same time, I intend to select only one of the currencies of the same time.

This is the priority: EUR, USD, GBP, CHF.

```<code>currency    timebuy buyprice
CNHUSD  2021-01-05 08:30:00 0,00005073
CNHGBP  2021-01-05 08:30:00 1,588
ZARGBP  2021-01-07 05:15:00 0,2727
ZARUSD  2021-01-07 05:15:00 300
ZAREUR  2021-01-07 13:00:00 0,1936
ZARCHF  2021-01-07 13:00:00 0,0000052
JPYCHF  2021-01-13 06:00:00 0,0002222
JPYUSD  2021-01-13 06:00:00 8
JPYGBP  2021-01-13 06:00:00 8
</code>```

to

```<code>currency    timebuy buyprice
CNHUSD  2021-01-05 08:30:00 0,00005073
ZAREUR  2021-01-07 13:00:00 0,1936
JPYUSD  2021-01-13 06:00:00 8
</code>```

答:
For a priority list like this, it's easiest to work with numbers. So, you can create a nice numeric mapping from your priority list, and use it to pick rows:

```<code>priority = ['EUR', 'USD', 'GBP', 'CHF']
mapping = {p: i for i, p in enumerate(priority)}
indexes = df['currency'].str[-3:].map(mapping).groupby(df['currency'].str[:3]).idxmin().sort_values()
selected = df.loc[indexes]
</code>```

Output:

```<code>>>> selected
  currency             timebuy    buyprice
0   CNHUSD 2021-01-05 08:30:00  0,00005073
4   ZAREUR 2021-01-07 13:00:00      0,1936
7   JPYUSD 2021-01-13 06:00:00           8
</code>```

One-liner:

```<code>priority = ['EUR', 'USD', 'GBP', 'CHF']
filtered = df.loc[df['currency'].str[-3:].map({p: i for i, p in enumerate(priority)}).groupby(df['currency'].str[:3]).idxmin().sort_values()]
</code>```

If you want to group by each timestamp instead of the first 3 letters of currency, group by df['timestamp'] instead of df['currency'].str[:3], i.e.:

```<code>indexes = df['currency'].str[-3:].map(mapping).groupby(df['timestamp']).idxmin().sort_values()
                                                     # ^^^^^^^^^^^^^^^
</code>```

Body:
I want to delete specific rows based in conditions on rows from a Pandas dataframe.

For example, since I have several currency pairs at the same time, I intend to select only one of the currencies of the same time.

This is the priority: EUR, USD, GBP, CHF.

currency    timebuy buyprice
CNHUSD  2021-01-05 08:30:00 0,00005073
CNHGBP  2021-01-05 08:30:00 1,588
ZARGBP  2021-01-07 05:15:00 0,2727
ZARUSD  2021-01-07 05:15:00 300
ZAREUR  2021-01-07 13:00:00 0,1936
ZARCHF  2021-01-07 13:00:00 0,0000052
JPYCHF  2021-01-13 06:00:00 0,0002222
JPYUSD  2021-01-13 06:00:00 8
JPYGBP  2021-01-13 06:00:00 8

to

currency    timebuy buyprice
CNHUSD  2021-01-05 08:30:00 0,00005073
ZAREUR  2021-01-07 13:00:00 0,1936
JPYUSD  2021-01-13 06:00:00 8

Body:
For a priority list like this, it's easiest to work with numbers. So, you can create a nice numeric mapping from your priority list, and use it to pick rows:

priority = ['EUR', 'USD', 'GBP', 'CHF']
mapping = {p: i for i, p in enumerate(priority)}
indexes = df['currency'].str[-3:].map(mapping).groupby(df['currency'].str[:3]).idxmin().sort_values()
selected = df.loc[indexes]

Output:

>>> selected
  currency             timebuy    buyprice
0   CNHUSD 2021-01-05 08:30:00  0,00005073
4   ZAREUR 2021-01-07 13:00:00      0,1936
7   JPYUSD 2021-01-13 06:00:00           8

One-liner:

priority = ['EUR', 'USD', 'GBP', 'CHF']
filtered = df.loc[df['currency'].str[-3:].map({p: i for i, p in enumerate(priority)}).groupby(df['currency'].str[:3]).idxmin().sort_values()]

If you want to group by each timestamp instead of the first 3 letters of currency, group by df['timestamp'] instead of df['currency'].str[:3], i.e.:

indexes = df['currency'].str[-3:].map(mapping).groupby(df['timestamp']).idxmin().sort_values()
                                                     # ^^^^^^^^^^^^^^^


答:
Using groupby and reindex:

```<code># Hard-code your priority for the second currency in each pair
pri = ['EUR', 'USD', 'GBP', 'CHF']

# Create a new column for the second currency of each pair
df['2ndcurr'] = df['currency'].str[-3:]

# Group by time and second currency,
# Sort inner level (1) of resulting MultiIndex to match priority,
# Group by the outer level (0),
# Get the first row of each group,
# Reset timebuy from index into its own column

(df.set_index(['timebuy', '2ndcurr'])
   .reindex(pri, level=1)
   .groupby(level=0)
   .first()
   .reset_index())

               timebuy currency    buyprice
0  2021-01-05 08:30:00   CNHUSD  0,00005073
1  2021-01-07 05:15:00   ZARUSD         300
2  2021-01-07 13:00:00   ZAREUR      0,1936
3  2021-01-13 06:00:00   JPYUSD           8
</code>```

Title: Delete specific rows based in conditions on rows from a dataframe pandas

Body:
Using groupby and reindex:

# Hard-code your priority for the second currency in each pair
pri = ['EUR', 'USD', 'GBP', 'CHF']

# Create a new column for the second currency of each pair
df['2ndcurr'] = df['currency'].str[-3:]

# Group by time and second currency,
# Sort inner level (1) of resulting MultiIndex to match priority,
# Group by the outer level (0),
# Get the first row of each group,
# Reset timebuy from index into its own column

(df.set_index(['timebuy', '2ndcurr'])
   .reindex(pri, level=1)
   .groupby(level=0)
   .first()
   .reset_index())

               timebuy currency    buyprice
0  2021-01-05 08:30:00   CNHUSD  0,00005073
1  2021-01-07 05:15:00   ZARUSD         300
2  2021-01-07 13:00:00   ZAREUR      0,1936
3  2021-01-13 06:00:00   JPYUSD           8