问:
Adding in max() to query with many joins kills performance - how to optimise?
Let me preface this by saying I'm working in SSMS2016, and that my SQL is not the greatest.

I have a query with quite a few joins, which runs in slow-but-bearable 20-odd seconds.

I'd like to add a column which is basically the max value of the row_number in the CCR subquery (the 2 lines commented out in the code), however it massively impacts query time (60+ seconds for 1000 records). Is there any way I can get the required column without the huge jump in run time, and any way I can clean up the query in general?

```<code>SELECT DISTINCT AWI.*
FROM
(SELECT
    4 'SORT'
    , CCR.URN
    , CCR.SPECIAL_COMMENT
    , SL.SCHEDULED_SHIP_DATE
    , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE NULL END AS 'ACTUAL_SHIP_DATE'
    , SL.LOAD_CLOSED
    , SH.SHIPPING_LOAD_NUM 'SHIPPING_LOAD'
--  , MAX(CCR.RNK) OVER(PARTITION BY CCR.SHIPPING_LOAD_NUM) 'CTNS_ON_LOAD'
    , SH.CARRIER
    , SH.CARRIER_SERVICE 'SERVICE'
    , IWI.PARENT_INSTR 'WORK_UNIT'
    , IWI.CONDITION 'WORK_STATUS'
    , IWI.WORK_TYPE
    , sl.LOCATION 'DOCK_DOOR'
    , CCR.CONTAINER_ID
    , SC.LOCATION 'FROM_LOC'
    , IWI.TO_LOC
    , SH.SHIP_TO
    , G.DESCRIPTION 'COUNTRY'

    , case  when SL.LEADING_STS = '201' then '[201] In Wave'
            when SL.LEADING_STS = '300' then '[300] Picking Pending'
            when SL.LEADING_STS = '301' then '[301] In Picking'
            when SL.LEADING_STS = '400' then '[400] Packing Pending'
            when SL.LEADING_STS = '401' then '[401] In Packing'
            when SL.LEADING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.LEADING_STS = '600' then '[600] Staging Pending'
            when SL.LEADING_STS = '650' then '[650] Loading Pending'
            when SL.LEADING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.LEADING_STS = '800' then '[800] Load Confirm Pending'
            when SL.LEADING_STS = '900' then '[900] Closed'
            END AS 'LEADING_STS'
    , case  when SL.TRAILING_STS = '201' then '[201] In Wave'
            when SL.TRAILING_STS = '300' then '[300] Picking Pending'
            when SL.TRAILING_STS = '301' then '[301] In Picking'
            when SL.TRAILING_STS = '400' then '[400] Packing Pending'
            when SL.TRAILING_STS = '401' then '[401] In Packing'
            when SL.TRAILING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.TRAILING_STS = '600' then '[600] Staging Pending'
            when SL.TRAILING_STS = '650' then '[650] Loading Pending'
            when SL.TRAILING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.TRAILING_STS = '800' then '[800] Load Confirm Pending'
            when SL.TRAILING_STS = '900' then '[900] Closed'
            END AS 'TRAILING_STS'
    , CCR.CONTAINER_TYPE
    , DIMS.WEIGHT
    , DIMS.LENGTH
    , DIMS.WIDTH
    , DIMS.HEIGHT

FROM
    IA_WORK_INSTRUCTION IWI
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, SHIP_TO, CARRIER, CARRIER_SERVICE, SHIPPING_LOAD_NUM, SHIP_TO_COUNTRY FROM SHIPMENT_HEADER) SH
    ON SH.INTERNAL_SHIPMENT_NUM = IWI.INTERNAL_NUM
JOIN
    (SELECT c1.INTERNAL_WORK_INST_NUM, c1.CONTAINER_ID, c1.URN, c1.INTERNAL_CONTAINER_NUM, c1.SPECIAL_COMMENT, sc1.container_type, H1.SHIPPING_LOAD_NUM
--  , ROW_NUMBER() OVER(PARTITION BY H1.SHIPPING_LOAD_NUM ORDER BY C1.CONTAINER_ID DESC) 'RNK'
    FROM CARTON_CALLOFF_REQUEST c1
    join shipping_container sc1
    on sc1.internal_container_num = c1.internal_container_num
    JOIN SHIPMENT_HEADER H1
    ON H1.INTERNAL_SHIPMENT_NUM = SC1.INTERNAL_SHIPMENT_NUM
    ) CCR
    ON CCR.INTERNAL_WORK_INST_NUM = IWI.PARENT_INSTR
JOIN
    (SELECT DISTINCT IDENTIFIER, DESCRIPTION FROM GENERIC_CONFIG_DETAIL WHERE RECORD_TYPE = 'COUNTRY') G
    ON G.IDENTIFIER = SH.SHIP_TO_COUNTRY
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, PARENT, LOCATION FROM SHIPPING_CONTAINER) SC
    ON SC.PARENT = CCR.INTERNAL_CONTAINER_NUM AND SC.INTERNAL_SHIPMENT_NUM = SH.INTERNAL_SHIPMENT_NUM
JOIN
    (SELECT DISTINCT INTERNAL_CONTAINER_NUM, WEIGHT, LENGTH, WIDTH, HEIGHT FROM SHIPPING_CONTAINER WHERE CONTAINER_ID IS NOT NULL) DIMS
    ON DIMS.INTERNAL_CONTAINER_NUM = CCR.INTERNAL_CONTAINER_NUM
JOIN
    (SELECT s1.INTERNAL_LOAD_NUM, s1.LEADING_STS, s1.TRAILING_STS, s1.LOAD_CLOSED, s1.ACTUAL_SHIP_DATE_TIME, s1.SCHEDULED_SHIP_DATE, s1.DOCK_DOOR, l1.location FROM SHIPPING_LOAD s1 join location l1 on l1.object_id = s1.dock_door WHERE L1.TEMPLATE_FIELD1 = 'DOOR') SL
    ON SL.INTERNAL_LOAD_NUM = SH.SHIPPING_LOAD_NUM

) AWI

WHERE AWI.WORK_TYPE = 'CALL OFF PICK'
</code>```

答:
You might try replacing the Partition and Over() with Group By.
Please let us know if you face any issues even after adding the Group By.</p>

```<code>SELECT DISTINCT AWI.*
FROM
(SELECT
    4 'SORT'
    , CCR.URN
    , CCR.SPECIAL_COMMENT
    , SL.SCHEDULED_SHIP_DATE
    , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE NULL END AS 'ACTUAL_SHIP_DATE'
    , SL.LOAD_CLOSED
    , SH.SHIPPING_LOAD_NUM 'SHIPPING_LOAD'
    , MAX(CCR.RNK)
    , SH.CARRIER
    , SH.CARRIER_SERVICE 'SERVICE'
    , IWI.PARENT_INSTR 'WORK_UNIT'
    , IWI.CONDITION 'WORK_STATUS'
    , IWI.WORK_TYPE
    , sl.LOCATION 'DOCK_DOOR'
    , CCR.CONTAINER_ID
    , SC.LOCATION 'FROM_LOC'
    , IWI.TO_LOC
    , SH.SHIP_TO
    , G.DESCRIPTION 'COUNTRY'

    , case  when SL.LEADING_STS = '201' then '[201] In Wave'
            when SL.LEADING_STS = '300' then '[300] Picking Pending'
            when SL.LEADING_STS = '301' then '[301] In Picking'
            when SL.LEADING_STS = '400' then '[400] Packing Pending'
            when SL.LEADING_STS = '401' then '[401] In Packing'
            when SL.LEADING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.LEADING_STS = '600' then '[600] Staging Pending'
            when SL.LEADING_STS = '650' then '[650] Loading Pending'
            when SL.LEADING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.LEADING_STS = '800' then '[800] Load Confirm Pending'
            when SL.LEADING_STS = '900' then '[900] Closed'
            END AS 'LEADING_STS'
    , case  when SL.TRAILING_STS = '201' then '[201] In Wave'
            when SL.TRAILING_STS = '300' then '[300] Picking Pending'
            when SL.TRAILING_STS = '301' then '[301] In Picking'
            when SL.TRAILING_STS = '400' then '[400] Packing Pending'
            when SL.TRAILING_STS = '401' then '[401] In Packing'
            when SL.TRAILING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.TRAILING_STS = '600' then '[600] Staging Pending'
            when SL.TRAILING_STS = '650' then '[650] Loading Pending'
            when SL.TRAILING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.TRAILING_STS = '800' then '[800] Load Confirm Pending'
            when SL.TRAILING_STS = '900' then '[900] Closed'
            END AS 'TRAILING_STS'
    , CCR.CONTAINER_TYPE
    , DIMS.WEIGHT
    , DIMS.LENGTH
    , DIMS.WIDTH
    , DIMS.HEIGHT

FROM
    IA_WORK_INSTRUCTION IWI
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, SHIP_TO, CARRIER, CARRIER_SERVICE, SHIPPING_LOAD_NUM, SHIP_TO_COUNTRY FROM SHIPMENT_HEADER) SH
    ON SH.INTERNAL_SHIPMENT_NUM = IWI.INTERNAL_NUM
JOIN
    (SELECT c1.INTERNAL_WORK_INST_NUM, c1.CONTAINER_ID, c1.URN, c1.INTERNAL_CONTAINER_NUM, c1.SPECIAL_COMMENT, sc1.container_type, H1.SHIPPING_LOAD_NUM
, ROW_NUMBER() OVER(PARTITION BY H1.SHIPPING_LOAD_NUM ORDER BY C1.CONTAINER_ID DESC) 'RNK'
    FROM CARTON_CALLOFF_REQUEST c1
    join shipping_container sc1
    on sc1.internal_container_num = c1.internal_container_num
    JOIN SHIPMENT_HEADER H1
    ON H1.INTERNAL_SHIPMENT_NUM = SC1.INTERNAL_SHIPMENT_NUM
    ) CCR
    ON CCR.INTERNAL_WORK_INST_NUM = IWI.PARENT_INSTR
JOIN
    (SELECT DISTINCT IDENTIFIER, DESCRIPTION FROM GENERIC_CONFIG_DETAIL WHERE RECORD_TYPE = 'COUNTRY') G
    ON G.IDENTIFIER = SH.SHIP_TO_COUNTRY
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, PARENT, LOCATION FROM SHIPPING_CONTAINER) SC
    ON SC.PARENT = CCR.INTERNAL_CONTAINER_NUM AND SC.INTERNAL_SHIPMENT_NUM = SH.INTERNAL_SHIPMENT_NUM
JOIN
    (SELECT DISTINCT INTERNAL_CONTAINER_NUM, WEIGHT, LENGTH, WIDTH, HEIGHT FROM SHIPPING_CONTAINER WHERE CONTAINER_ID IS NOT NULL) DIMS
    ON DIMS.INTERNAL_CONTAINER_NUM = CCR.INTERNAL_CONTAINER_NUM
JOIN
    (SELECT s1.INTERNAL_LOAD_NUM, s1.LEADING_STS, s1.TRAILING_STS, s1.LOAD_CLOSED, s1.ACTUAL_SHIP_DATE_TIME, s1.SCHEDULED_SHIP_DATE, s1.DOCK_DOOR, l1.location FROM SHIPPING_LOAD s1 join location l1 on l1.object_id = s1.dock_door WHERE L1.TEMPLATE_FIELD1 = 'DOOR') SL
    ON SL.INTERNAL_LOAD_NUM = SH.SHIPPING_LOAD_NUM

GROUP BY
     CCR.URN
     , CCR.SPECIAL_COMMENT
     , SL.SCHEDULED_SHIP_DATE
     , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE
        NULL END
     , SL.LOAD_CLOSED
     , SH.SHIPPING_LOAD_NUM
     , SH.CARRIER
     , SH.CARRIER_SERVICE
     , IWI.PARENT_INSTR
     , IWI.CONDITION
     , IWI.WORK_TYPE
     , sl.LOCATION
     , CCR.CONTAINER_ID
     , SC.LOCATION
     , IWI.TO_LOC
     , SH.SHIP_TO
     , G.DESCRIPTION
     , SL.LEADING_STS
     , TRAILING_STS
     , CCR.CONTAINER_TYPE
     , DIMS.WEIGHT
     , DIMS.LENGTH
     , DIMS.WIDTH
     , DIMS.HEIGHT
) AWI

WHERE AWI.WORK_TYPE = 'CALL OFF PICK'
</code>```

Body:
Let me preface this by saying I'm working in SSMS2016, and that my SQL is not the greatest.

I have a query with quite a few joins, which runs in slow-but-bearable 20-odd seconds.

I'd like to add a column which is basically the max value of the row_number in the CCR subquery (the 2 lines commented out in the code), however it massively impacts query time (60+ seconds for 1000 records). Is there any way I can get the required column without the huge jump in run time, and any way I can clean up the query in general?

SELECT DISTINCT AWI.*
FROM
(SELECT
    4 'SORT'
    , CCR.URN
    , CCR.SPECIAL_COMMENT
    , SL.SCHEDULED_SHIP_DATE
    , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE NULL END AS 'ACTUAL_SHIP_DATE'
    , SL.LOAD_CLOSED
    , SH.SHIPPING_LOAD_NUM 'SHIPPING_LOAD'
--  , MAX(CCR.RNK) OVER(PARTITION BY CCR.SHIPPING_LOAD_NUM) 'CTNS_ON_LOAD'
    , SH.CARRIER
    , SH.CARRIER_SERVICE 'SERVICE'
    , IWI.PARENT_INSTR 'WORK_UNIT'
    , IWI.CONDITION 'WORK_STATUS'
    , IWI.WORK_TYPE
    , sl.LOCATION 'DOCK_DOOR'
    , CCR.CONTAINER_ID
    , SC.LOCATION 'FROM_LOC'
    , IWI.TO_LOC
    , SH.SHIP_TO
    , G.DESCRIPTION 'COUNTRY'

    , case  when SL.LEADING_STS = '201' then '[201] In Wave'
            when SL.LEADING_STS = '300' then '[300] Picking Pending'
            when SL.LEADING_STS = '301' then '[301] In Picking'
            when SL.LEADING_STS = '400' then '[400] Packing Pending'
            when SL.LEADING_STS = '401' then '[401] In Packing'
            when SL.LEADING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.LEADING_STS = '600' then '[600] Staging Pending'
            when SL.LEADING_STS = '650' then '[650] Loading Pending'
            when SL.LEADING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.LEADING_STS = '800' then '[800] Load Confirm Pending'
            when SL.LEADING_STS = '900' then '[900] Closed'
            END AS 'LEADING_STS'
    , case  when SL.TRAILING_STS = '201' then '[201] In Wave'
            when SL.TRAILING_STS = '300' then '[300] Picking Pending'
            when SL.TRAILING_STS = '301' then '[301] In Picking'
            when SL.TRAILING_STS = '400' then '[400] Packing Pending'
            when SL.TRAILING_STS = '401' then '[401] In Packing'
            when SL.TRAILING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.TRAILING_STS = '600' then '[600] Staging Pending'
            when SL.TRAILING_STS = '650' then '[650] Loading Pending'
            when SL.TRAILING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.TRAILING_STS = '800' then '[800] Load Confirm Pending'
            when SL.TRAILING_STS = '900' then '[900] Closed'
            END AS 'TRAILING_STS'
    , CCR.CONTAINER_TYPE
    , DIMS.WEIGHT
    , DIMS.LENGTH
    , DIMS.WIDTH
    , DIMS.HEIGHT

FROM
    IA_WORK_INSTRUCTION IWI
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, SHIP_TO, CARRIER, CARRIER_SERVICE, SHIPPING_LOAD_NUM, SHIP_TO_COUNTRY FROM SHIPMENT_HEADER) SH
    ON SH.INTERNAL_SHIPMENT_NUM = IWI.INTERNAL_NUM
JOIN
    (SELECT c1.INTERNAL_WORK_INST_NUM, c1.CONTAINER_ID, c1.URN, c1.INTERNAL_CONTAINER_NUM, c1.SPECIAL_COMMENT, sc1.container_type, H1.SHIPPING_LOAD_NUM
--  , ROW_NUMBER() OVER(PARTITION BY H1.SHIPPING_LOAD_NUM ORDER BY C1.CONTAINER_ID DESC) 'RNK'
    FROM CARTON_CALLOFF_REQUEST c1
    join shipping_container sc1
    on sc1.internal_container_num = c1.internal_container_num
    JOIN SHIPMENT_HEADER H1
    ON H1.INTERNAL_SHIPMENT_NUM = SC1.INTERNAL_SHIPMENT_NUM
    ) CCR
    ON CCR.INTERNAL_WORK_INST_NUM = IWI.PARENT_INSTR
JOIN
    (SELECT DISTINCT IDENTIFIER, DESCRIPTION FROM GENERIC_CONFIG_DETAIL WHERE RECORD_TYPE = 'COUNTRY') G
    ON G.IDENTIFIER = SH.SHIP_TO_COUNTRY
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, PARENT, LOCATION FROM SHIPPING_CONTAINER) SC
    ON SC.PARENT = CCR.INTERNAL_CONTAINER_NUM AND SC.INTERNAL_SHIPMENT_NUM = SH.INTERNAL_SHIPMENT_NUM
JOIN
    (SELECT DISTINCT INTERNAL_CONTAINER_NUM, WEIGHT, LENGTH, WIDTH, HEIGHT FROM SHIPPING_CONTAINER WHERE CONTAINER_ID IS NOT NULL) DIMS
    ON DIMS.INTERNAL_CONTAINER_NUM = CCR.INTERNAL_CONTAINER_NUM
JOIN
    (SELECT s1.INTERNAL_LOAD_NUM, s1.LEADING_STS, s1.TRAILING_STS, s1.LOAD_CLOSED, s1.ACTUAL_SHIP_DATE_TIME, s1.SCHEDULED_SHIP_DATE, s1.DOCK_DOOR, l1.location FROM SHIPPING_LOAD s1 join location l1 on l1.object_id = s1.dock_door WHERE L1.TEMPLATE_FIELD1 = 'DOOR') SL
    ON SL.INTERNAL_LOAD_NUM = SH.SHIPPING_LOAD_NUM

) AWI

WHERE AWI.WORK_TYPE = 'CALL OFF PICK'

Body:
You might try replacing the Partition and Over() with Group By.
Please let us know if you face any issues even after adding the Group By.

SELECT DISTINCT AWI.*
FROM
(SELECT
    4 'SORT'
    , CCR.URN
    , CCR.SPECIAL_COMMENT
    , SL.SCHEDULED_SHIP_DATE
    , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE NULL END AS 'ACTUAL_SHIP_DATE'
    , SL.LOAD_CLOSED
    , SH.SHIPPING_LOAD_NUM 'SHIPPING_LOAD'
    , MAX(CCR.RNK)
    , SH.CARRIER
    , SH.CARRIER_SERVICE 'SERVICE'
    , IWI.PARENT_INSTR 'WORK_UNIT'
    , IWI.CONDITION 'WORK_STATUS'
    , IWI.WORK_TYPE
    , sl.LOCATION 'DOCK_DOOR'
    , CCR.CONTAINER_ID
    , SC.LOCATION 'FROM_LOC'
    , IWI.TO_LOC
    , SH.SHIP_TO
    , G.DESCRIPTION 'COUNTRY'

    , case  when SL.LEADING_STS = '201' then '[201] In Wave'
            when SL.LEADING_STS = '300' then '[300] Picking Pending'
            when SL.LEADING_STS = '301' then '[301] In Picking'
            when SL.LEADING_STS = '400' then '[400] Packing Pending'
            when SL.LEADING_STS = '401' then '[401] In Packing'
            when SL.LEADING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.LEADING_STS = '600' then '[600] Staging Pending'
            when SL.LEADING_STS = '650' then '[650] Loading Pending'
            when SL.LEADING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.LEADING_STS = '800' then '[800] Load Confirm Pending'
            when SL.LEADING_STS = '900' then '[900] Closed'
            END AS 'LEADING_STS'
    , case  when SL.TRAILING_STS = '201' then '[201] In Wave'
            when SL.TRAILING_STS = '300' then '[300] Picking Pending'
            when SL.TRAILING_STS = '301' then '[301] In Picking'
            when SL.TRAILING_STS = '400' then '[400] Packing Pending'
            when SL.TRAILING_STS = '401' then '[401] In Packing'
            when SL.TRAILING_STS = '500' then '[500] Pack and Hold Pending'
            when SL.TRAILING_STS = '600' then '[600] Staging Pending'
            when SL.TRAILING_STS = '650' then '[650] Loading Pending'
            when SL.TRAILING_STS = '700' then '[700] Ship Confirm Pending'
            when SL.TRAILING_STS = '800' then '[800] Load Confirm Pending'
            when SL.TRAILING_STS = '900' then '[900] Closed'
            END AS 'TRAILING_STS'
    , CCR.CONTAINER_TYPE
    , DIMS.WEIGHT
    , DIMS.LENGTH
    , DIMS.WIDTH
    , DIMS.HEIGHT

FROM
    IA_WORK_INSTRUCTION IWI
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, SHIP_TO, CARRIER, CARRIER_SERVICE, SHIPPING_LOAD_NUM, SHIP_TO_COUNTRY FROM SHIPMENT_HEADER) SH
    ON SH.INTERNAL_SHIPMENT_NUM = IWI.INTERNAL_NUM
JOIN
    (SELECT c1.INTERNAL_WORK_INST_NUM, c1.CONTAINER_ID, c1.URN, c1.INTERNAL_CONTAINER_NUM, c1.SPECIAL_COMMENT, sc1.container_type, H1.SHIPPING_LOAD_NUM
, ROW_NUMBER() OVER(PARTITION BY H1.SHIPPING_LOAD_NUM ORDER BY C1.CONTAINER_ID DESC) 'RNK'
    FROM CARTON_CALLOFF_REQUEST c1
    join shipping_container sc1
    on sc1.internal_container_num = c1.internal_container_num
    JOIN SHIPMENT_HEADER H1
    ON H1.INTERNAL_SHIPMENT_NUM = SC1.INTERNAL_SHIPMENT_NUM
    ) CCR
    ON CCR.INTERNAL_WORK_INST_NUM = IWI.PARENT_INSTR
JOIN
    (SELECT DISTINCT IDENTIFIER, DESCRIPTION FROM GENERIC_CONFIG_DETAIL WHERE RECORD_TYPE = 'COUNTRY') G
    ON G.IDENTIFIER = SH.SHIP_TO_COUNTRY
JOIN
    (SELECT INTERNAL_SHIPMENT_NUM, PARENT, LOCATION FROM SHIPPING_CONTAINER) SC
    ON SC.PARENT = CCR.INTERNAL_CONTAINER_NUM AND SC.INTERNAL_SHIPMENT_NUM = SH.INTERNAL_SHIPMENT_NUM
JOIN
    (SELECT DISTINCT INTERNAL_CONTAINER_NUM, WEIGHT, LENGTH, WIDTH, HEIGHT FROM SHIPPING_CONTAINER WHERE CONTAINER_ID IS NOT NULL) DIMS
    ON DIMS.INTERNAL_CONTAINER_NUM = CCR.INTERNAL_CONTAINER_NUM
JOIN
    (SELECT s1.INTERNAL_LOAD_NUM, s1.LEADING_STS, s1.TRAILING_STS, s1.LOAD_CLOSED, s1.ACTUAL_SHIP_DATE_TIME, s1.SCHEDULED_SHIP_DATE, s1.DOCK_DOOR, l1.location FROM SHIPPING_LOAD s1 join location l1 on l1.object_id = s1.dock_door WHERE L1.TEMPLATE_FIELD1 = 'DOOR') SL
    ON SL.INTERNAL_LOAD_NUM = SH.SHIPPING_LOAD_NUM

GROUP BY
     CCR.URN
     , CCR.SPECIAL_COMMENT
     , SL.SCHEDULED_SHIP_DATE
     , CASE WHEN SL.LOAD_CLOSED = 'Y' THEN SL.ACTUAL_SHIP_DATE_TIME ELSE
        NULL END
     , SL.LOAD_CLOSED
     , SH.SHIPPING_LOAD_NUM
     , SH.CARRIER
     , SH.CARRIER_SERVICE
     , IWI.PARENT_INSTR
     , IWI.CONDITION
     , IWI.WORK_TYPE
     , sl.LOCATION
     , CCR.CONTAINER_ID
     , SC.LOCATION
     , IWI.TO_LOC
     , SH.SHIP_TO
     , G.DESCRIPTION
     , SL.LEADING_STS
     , TRAILING_STS
     , CCR.CONTAINER_TYPE
     , DIMS.WEIGHT
     , DIMS.LENGTH
     , DIMS.WIDTH
     , DIMS.HEIGHT
) AWI

WHERE AWI.WORK_TYPE = 'CALL OFF PICK'


来源: stackexchange

问:
Parsing rss xml file in PYTHON
I am actually trying to extract data from RSS documents. I am using the following code to parse xml doc.

But wont work for this document http://www.mediafire.com/?hptptj8847awnn1 . Please help!!

```<code>#import easy to use xml parser called minidom:
import xml.dom.minidom as minidom
import csv

def getTags(xml):
"""
Print out all titles found in xml
"""

doc = minidom.parse(xml)

node = doc.documentElement
items = doc.getElementsByTagName("item")

titles = []
for item in items:
    titleObj = item.getElementsByTagName("title")[0]
    titles.append(titleObj)

print len(titles)

x = 0
for x in range(len(titles)):
    nodes = titles[x].childNodes
    for node in nodes:
        if node.nodeType == node.CDATA_SECTION_NODE:
            titletxt = node.data

        elif node.nodeType == node.TEXT_NODE:
            titletxt = node.data

if __name__ == "__main__":
    document = 'D2B0918.xml'
    getTags(document)
</code>```

答:
```<code>#-*-coding:utf8;-*-
#qpy:3
#qpy:console

import urllib.request
from xml.dom import minidom

def parse_feed(url):
    # This is what parse_feed returns.
    feed = type('Feed', (object,), {})
    feed.entries = []

    with urllib.request.urlopen(url) as res:
        dom = minidom.parseString(res.read().decode('latin-1'))
        feed.title = dom.getElementsByTagName('title')[0].firstChild.nodeValue
        feed.link = dom.getElementsByTagName('link')[0].getAttribute('href')
        feed.published = dom.getElementsByTagName('published')[0].firstChild.nodeValue

    for element in dom.getElementsByTagName('entry'):
        title = element.getElementsByTagName('title')[0].firstChild.nodeValue
        link = element.getElementsByTagName('link')[0].getAttribute('href')
        author = element.getElementsByTagName('name')[0].firstChild.nodeValue
        published = element.getElementsByTagName('published')[0].firstChild.nodeValue
        updated = element.getElementsByTagName('updated')[0].firstChild.nodeValue
        _id = element.getElementsByTagName('id')[0].firstChild.nodeValue
        category = element.getElementsByTagName('category')

        tags = []
        for node in category:
            tags.append(node.getAttribute('term'))

        article = element.getElementsByTagName('content')[0].firstChild.nodeValue

        entry_dict = dict(
                    title=title,
                    link=link,
                    author=author,
                    article=article,
                    tags=tags,
                    _id=_id)

        feed.entries.append(type('Entry', (feed,), entry_dict))

    return feed

# Example use.
feed_url = 'https://rickys-python-notes.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=1000'
feed = parse_feed(feed_url)
print(feed.title)
print(feed.updated)
for entry in feed.entries:
    print(entry.title)
    print(entry.link)
</code>```

ContentLicense: CC BY-SA 3.0

Title: Parsing rss xml file in PYTHON

Body:
I am actually trying to extract data from RSS documents. I am using the following code to parse xml doc.

But wont work for this document http://www.mediafire.com/?hptptj8847awnn1 . Please help!!

#import easy to use xml parser called minidom:
import xml.dom.minidom as minidom
import csv

def getTags(xml):
"""
Print out all titles found in xml
"""

doc = minidom.parse(xml)

node = doc.documentElement
items = doc.getElementsByTagName("item")

titles = []
for item in items:
    titleObj = item.getElementsByTagName("title")[0]
    titles.append(titleObj)

print len(titles)

x = 0
for x in range(len(titles)):
    nodes = titles[x].childNodes
    for node in nodes:
        if node.nodeType == node.CDATA_SECTION_NODE:
            titletxt = node.data

        elif node.nodeType == node.TEXT_NODE:
            titletxt = node.data

if __name__ == "__main__":
    document = 'D2B0918.xml'
    getTags(document)


问:
Cat file do some operation append from file output to first file
I'm trying to achieve the folowing:

```<code>  #!/bin/bash
  if [ -f file2 ]; then

  cat file2 > file1
  sed -i 's#some operation' file1
  cat file3 >> file1

  exit 0

  elif [ ! -f file2 ]; then

  cat file1 > file2
  sed -i 's#some operation' file1
  cat file3 >> file1

  exit 0

  else

  echo "something"
  exit 1

  fi
</code>```

Any ideas how to do it simpler ?
Without using that much cat and files ?

Thanks !

答:
For your original question:

```<code>cat file1 > file1.bak
sed -i 's# some operation' file1.bak
cat file2 >> file1.bak
echo -n > file1
cat file1.bak > file1
rm -f file1.bak
</code>```

the simpler answer is (as I said in the comments):

```<code>sed -i 's# some operation' file1
cat file2 >> file1
</code>```

For the edited question - bit more explanation.

In your condition:

```<code>#!/bin/bash
if [ -f file2 ]; then
...
elif [ ! -f file2 ]; then
...
else
...
fi
</code>```

The else will never happenes. If the file1 exists and is a regular file. the if-then will run, and if you negate the above will run the elif-then. E.g. you can simplify it to

```<code>if [ -f file2 ]; then
...
else
...
fi
</code>```

Now to the actions:

The:

```<code>cat file2 > file1
sed -i 's#some operation' file1
cat file3 >> file1
</code>```

is the same as:

```<code>sed 's#some operation' <file2 >file1
cat file3 >> file1
</code>```

and the:

```<code>cat file1 > file2
sed -i 's#some operation' file1
cat file3 >> file1
</code>```

is OK - you creating a backup (copy) of the file1 to file2. This could be written also as cp file1 file2.

Comparing the two sections, you in both doing the same thing:

```<code>cat file3 >> file1
</code>```

so, DRY (Don't Repeat Yourself) - and do this after the if, because it is common for both parts.

So, we get:

```<code>if [ -f file2 ]; then
    sed 's#some operation' <file2 >file1
else
    cp file1 file2  #make the copy
    sed -i 's#some operation' file1
fi
cat file3 >> file1
</code>```

Also, the

```<code>sed 's#some operation' <file2 >file1
#and
sed -i 's#some operation' file1
</code>```

are very similar - e.g. the result of the sed operations always going into the file1. Moreover, in the else you copying the file1 to file2 so the

```<code>cat file1 > file2
sed -i 's#some operation' file1
</code>```

could be written also as

```<code>cp file1 file2
sed 's#some operation' <file2 >file1
</code>```

and we got the indentical sed command for both cases, so - DRY again.

```<code>if [ -f file2 ]; then
    : #do nothing here...
else
    cp file1 file2  #make the copy
fi
sed 's#some operation' <file2 >file1
cat file3 >> file1
</code>```

But the do nothing part is unnecessary, so we got:

```<code>if [ ! -f file2 ]; then
    cp file1 file2  #make the copy
fi
sed 's#some operation' <file2 >file1
cat file3 >> file1
</code>```

this could be shortened using [[ condition ]] && ... but it is not needed for now. :)

Also it could be very helpful name your files more precisely. The file1 and so on - telling you nothing about the content.

Body:
I'm trying to achieve the folowing:

  #!/bin/bash
  if [ -f file2 ]; then

  cat file2 > file1
  sed -i 's#some operation' file1
  cat file3 >> file1

  exit 0

  elif [ ! -f file2 ]; then

  cat file1 > file2
  sed -i 's#some operation' file1
  cat file3 >> file1

  exit 0

  else

  echo "something"
  exit 1

  fi

Any ideas how to do it simpler ?
Without using that much cat and files ?

Thanks !

Body:
For your original question:

cat file1 > file1.bak
sed -i 's# some operation' file1.bak
cat file2 >> file1.bak
echo -n > file1
cat file1.bak > file1
rm -f file1.bak

the simpler answer is (as I said in the comments):

sed -i 's# some operation' file1
cat file2 >> file1

For the edited question - bit more explanation.

In your condition:

#!/bin/bash
if [ -f file2 ]; then
...
elif [ ! -f file2 ]; then
...
else
...
fi

The else will never happenes. If the file1 exists and is a regular file. the if-then will run, and if you negate the above will run the elif-then. E.g. you can simplify it to

if [ -f file2 ]; then
...
else
...
fi

Now to the actions:

The:

cat file2 > file1
sed -i 's#some operation' file1
cat file3 >> file1

is the same as:

sed 's#some operation' <file2 >file1
cat file3 >> file1

and the:

cat file1 > file2
sed -i 's#some operation' file1
cat file3 >> file1

is OK - you creating a backup (copy) of the file1 to file2. This could be written also as cp file1 file2.

Comparing the two sections, you in both doing the same thing:

cat file3 >> file1

so, DRY (Don't Repeat Yourself) - and do this after the if, because it is common for both parts.

So, we get:

if [ -f file2 ]; then
    sed 's#some operation' <file2 >file1
else
    cp file1 file2  #make the copy
    sed -i 's#some operation' file1
fi
cat file3 >> file1

Also, the

sed 's#some operation' <file2 >file1
#and
sed -i 's#some operation' file1

are very similar - e.g. the result of the sed operations always going into the file1. Moreover, in the else you copying the file1 to file2 so the

cat file1 > file2
sed -i 's#some operation' file1

could be written also as

cp file1 file2
sed 's#some operation' <file2 >file1

and we got the indentical sed command for both cases, so - DRY again.

if [ -f file2 ]; then
    : #do nothing here...
else
    cp file1 file2  #make the copy
fi
sed 's#some operation' <file2 >file1
cat file3 >> file1

But the do nothing part is unnecessary, so we got:

if [ ! -f file2 ]; then
    cp file1 file2  #make the copy
fi
sed 's#some operation' <file2 >file1
cat file3 >> file1

this could be shortened using [[ condition ]] && ... but it is not needed for now. :)

Also it could be very helpful name your files more precisely. The file1 and so on - telling you nothing about the content.


问:
Gradle Error in Android Studio: Plugin with id 'com.android.library' not found
When I am trying to Build an Android Library Project in Android Studio, I am getting the following Gradle error:

```<code>Gradle sync failed: Plugin with id 'com.android.library' not found.
</code>```

I am pretty new to Gradle and this is very confusing for me. Why is this happening?

The build.gradle file is as follows:

```<code>apply plugin: 'com.android.library'

android {
    compileSdkVersion 23
    buildToolsVersion "23.0.2"

    defaultConfig {
        minSdkVersion 8
        targetSdkVersion 23
        versionCode 1
        versionName "1.0"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:23.1.0'
}
</code>```

答: In my case was due to wrong sdk/ndk paths inside local.properties file, i moved the project from an hd to another and these settings were wrong, fixing the paths fixed the problem.</p>

Body:
When I am trying to Build an Android Library Project in Android Studio, I am getting the following Gradle error:

Gradle sync failed: Plugin with id 'com.android.library' not found.

I am pretty new to Gradle and this is very confusing for me. Why is this happening?

The build.gradle file is as follows:

apply plugin: 'com.android.library'

android {
    compileSdkVersion 23
    buildToolsVersion "23.0.2"

    defaultConfig {
        minSdkVersion 8
        targetSdkVersion 23
        versionCode 1
        versionName "1.0"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:23.1.0'
}


问:
ValueError while making db migrations in django
I am new to django and i tried changing tag field from char field to TaggableManager()

models.py

```<code>class UserBookmark(models.Model):

    user = models.ForeignKey(User)
    bookmark = models.URLField()
    tag = TaggableManager()

    def __str__(self):
        return '%i %s %s'%(self.id,self.user,self.bookmark)
</code>```

when i run python manage.py migrate, i get this error:

ValueError: Cannot alter field bookmark.UserBookmark.tags into
  bookmark.UserBookmark.tags - they are not compatible types (you cannot
  alter to or from M2M fields, or add or remove through= on M2M fields)

How can i remove this error?

答:
I was able to solve similar issue by deleting the migration files and sqlite db.

Make sure to backup your db before deleting

Body:
I am new to django and i tried changing tag field from char field to TaggableManager()

models.py

class UserBookmark(models.Model):

    user = models.ForeignKey(User)
    bookmark = models.URLField()
    tag = TaggableManager()

    def __str__(self):
        return '%i %s %s'%(self.id,self.user,self.bookmark)

when i run python manage.py migrate, i get this error:

ValueError: Cannot alter field bookmark.UserBookmark.tags into
  bookmark.UserBookmark.tags - they are not compatible types (you cannot
  alter to or from M2M fields, or add or remove through= on M2M fields)

How can i remove this error?


问:
Need help with While Loop
I am extremely new to programming so it's taking a lot to re-wire. my brain to think like a computer programmer.

I need to create a script in Python using a while loop that does this:

```
zebra arbez
ebraz zarbe
braze ezarb
razeb bezar
azebr rbeza
```

Keep in mind that the script should be able to do this with any word.  For example, right now a = 'zebra'.  If a = 'cat' the the script should look like:

```
cat tac
atc cta
tca cat
```

I've figured out how to do it with a for loop...just can't figure out exactly how to implement it with a while loop.

My for loop:

```<code>a = 'zebra'
for i in range(len(a)):
    print a[i:] + a[:i], (a[i:] + a[:i])[::-1]
</code>```

If anyone could help me out, or give me hints...I'd really appreciate it!
thanks!

答:
```<code>for (initialization_expression;loop_condition;increment_expression){
  // statements
}
</code>```

is basically just a nice way of writing

```<code>initialize_expression;
while(loop_condition){
    // statements
    increment_expression
}
</code>```

Deconstruct your for loops into this format and you should have your solution.

ContentLicense: CC BY-SA 2.5

Title: Need help with While Loop

Body:
I am extremely new to programming so it's taking a lot to re-wire. my brain to think like a computer programmer.

I need to create a script in Python using a while loop that does this:

zebra arbez
ebraz zarbe
braze ezarb
razeb bezar
azebr rbeza

Keep in mind that the script should be able to do this with any word.  For example, right now a = 'zebra'.  If a = 'cat' the the script should look like:

cat tac
atc cta
tca cat

I've figured out how to do it with a for loop...just can't figure out exactly how to implement it with a while loop.

My for loop:

a = 'zebra'
for i in range(len(a)):
    print a[i:] + a[:i], (a[i:] + a[:i])[::-1]

If anyone could help me out, or give me hints...I'd really appreciate it!
thanks!

Body:
for (initialization_expression;loop_condition;increment_expression){
  // statements
}

is basically just a nice way of writing

initialize_expression;
while(loop_condition){
    // statements
    increment_expression
}

Deconstruct your for loops into this format and you should have your solution.


问:
Ajax Call json_encode don't work
I hava a problem with my ajax call and the json response. The console tell me that my php file don't return a json format but i don't understand why. Here is my ajax function :

        function showEspece(espece, categorie, object)
  {
    $.ajax({
      type : 'POST',
      url: 'getespece.php',
      data: {espece: espece, categorie: categorie },
      dataType: 'json',
      success: function(data)
      {
        alert(data);
        var tableau = data;
        $('#output').html(tableau);
      },
      error: function(xhr, status, error) {
        console.log(xhr.responseText);
        console.log(error);
      }
    });
  }```
</div>
</div>
</p>

<p>And here is the php page call by the ajax function : </p>

```<?php
header('Content-type: application/json');
include("includes/connexionBD.php");

$requete = oci_parse($connect, "SELECT nomA, sexe, datenaissance FROM Animal WHERE categorie = '".$_POST['categorie']."' AND espece = '".$_POST['espece']."' ");

oci_execute($requete);

$test = oci_fetch_all($requete, $res);
$test1 = array();
$test1 = var_dump($res);

    echo json_encode($test1);
```

<p>?></p>

<p>My problem is that the ajax function always go to error and here is what i can read in the console : </p>

```    array(3) {
  ["NOMA"]=>
  array(3) {
    [0]=>
    string(6) "Chachi"
    [1]=>
    string(6) "Rafiki"
    [2]=>
    string(6) "Chakra"
  }
  ["SEXE"]=>
  array(3) {
    [0]=>
    string(1) "F"
    [1]=>
    string(1) "M"
    [2]=>
    string(1) "F"
  }
  ["DATENAISSANCE"]=>
  array(3) {
    [0]=>
    string(9) "05-MAY-15"
    [1]=>
    string(9) "07-JAN-15"
    [2]=>
    string(9) "17-SEP-17"
  }
}
null

SyntaxError: Unexpected token a in JSON at position 0
at parse (<anonymous>)
at Qb (jQuery.js:4)
at A (jQuery.js:4)
at XMLHttpRequest.<anonymous> (jQuery.js:4)
```

I have pass the day on it and i don't understand why it don't work. Can anyone help me ?

答:
You have some errors, var_dump is for print data and not to assign values to variables, your data are in $res not in $test or $test1, and please take carefully of data in your querys, check How can I prevent SQL injection in PHP?

```<code>header('Content-type: application/json');
include("includes/connexionBD.php");

$requete = oci_parse(
    $connect,
    "SELECT nomA, sexe, datenaissance FROM Animal WHERE categorie = '" . $_POST['categorie'] . "' AND espece = '"
    . $_POST['espece'] . "' "
);

oci_execute($requete);

oci_fetch_all($requete, $res);

echo json_encode($res);
</code>```

For html result you can try this:

First change your ajax request

```<code>$.ajax({
  type    : 'POST',
  url     : 'getespece.php',
  data    : {espece: espece, categorie: categorie},
  dataType: 'html',    //You need to change from json to html
  success : function (data) {
    alert(data);
    var tableau = data;
    $('#output').html(tableau);
  },
  error   : function (xhr, status, error) {
    console.log(xhr.responseText);
    console.log(error);
  }
});
</code>```

Then your PHP code

```<code>...
oci_execute($requete);

$test = oci_fetch_all($requete, $res);

header('Content-Type: text/html; charset=utf-8');
$table = '<table border="1">';
$rows = count($res);
for ($i = 0; $i < $rows; $i++){
    $cols = count($res[$i]);
    $table .= '<tr>';
    for ($j = 0; $j < $cols; $j++)
    {
        $table .= '<td>' . $res[$i][$j] . '</td>';
    }
    $table .= '</tr>';
}
$table .= '</table>';
echo $table;
</code>```

Title: Ajax Call json_encode don't work

Body:
I hava a problem with my ajax call and the json response. The console tell me that my php file don't return a json format but i don't understand why. Here is my ajax function :

        function showEspece(espece, categorie, object)
  {
    $.ajax({
      type : 'POST',
      url: 'getespece.php',
      data: {espece: espece, categorie: categorie },
      dataType: 'json',
      success: function(data)
      {
        alert(data);
        var tableau = data;
        $('#output').html(tableau);
      },
      error: function(xhr, status, error) {
        console.log(xhr.responseText);
        console.log(error);
      }
    });
  }

And here is the php page call by the ajax function :

<?php
header('Content-type: application/json');
include("includes/connexionBD.php");

$requete = oci_parse($connect, "SELECT nomA, sexe, datenaissance FROM Animal WHERE categorie = '".$_POST['categorie']."' AND espece = '".$_POST['espece']."' ");

oci_execute($requete);

$test = oci_fetch_all($requete, $res);
$test1 = array();
$test1 = var_dump($res);

    echo json_encode($test1);

?>

My problem is that the ajax function always go to error and here is what i can read in the console :

    array(3) {
  ["NOMA"]=>
  array(3) {
    [0]=>
    string(6) "Chachi"
    [1]=>
    string(6) "Rafiki"
    [2]=>
    string(6) "Chakra"
  }
  ["SEXE"]=>
  array(3) {
    [0]=>
    string(1) "F"
    [1]=>
    string(1) "M"
    [2]=>
    string(1) "F"
  }
  ["DATENAISSANCE"]=>
  array(3) {
    [0]=>
    string(9) "05-MAY-15"
    [1]=>
    string(9) "07-JAN-15"
    [2]=>
    string(9) "17-SEP-17"
  }
}
null

SyntaxError: Unexpected token a in JSON at position 0
at parse (<anonymous>)
at Qb (jQuery.js:4)
at A (jQuery.js:4)
at XMLHttpRequest.<anonymous> (jQuery.js:4)

I have pass the day on it and i don't understand why it don't work. Can anyone help me ?

Body:
You have some errors, var_dump is for print data and not to assign values to variables, your data are in $res not in $test or $test1, and please take carefully of data in your querys, check How can I prevent SQL injection in PHP?

header('Content-type: application/json');
include("includes/connexionBD.php");

$requete = oci_parse(
    $connect,
    "SELECT nomA, sexe, datenaissance FROM Animal WHERE categorie = '" . $_POST['categorie'] . "' AND espece = '"
    . $_POST['espece'] . "' "
);

oci_execute($requete);

oci_fetch_all($requete, $res);

echo json_encode($res);

For html result you can try this:

First change your ajax request

$.ajax({
  type    : 'POST',
  url     : 'getespece.php',
  data    : {espece: espece, categorie: categorie},
  dataType: 'html',    //You need to change from json to html
  success : function (data) {
    alert(data);
    var tableau = data;
    $('#output').html(tableau);
  },
  error   : function (xhr, status, error) {
    console.log(xhr.responseText);
    console.log(error);
  }
});

Then your PHP code

...
oci_execute($requete);

$test = oci_fetch_all($requete, $res);

header('Content-Type: text/html; charset=utf-8');
$table = '<table border="1">';
$rows = count($res);
for ($i = 0; $i < $rows; $i++){
    $cols = count($res[$i]);
    $table .= '<tr>';
    for ($j = 0; $j < $cols; $j++)
    {
        $table .= '<td>' . $res[$i][$j] . '</td>';
    }
    $table .= '</tr>';
}
$table .= '</table>';
echo $table;


问:
Open file on IntelliJ from iTerm 2
I have output in my iTerm that is like:

```<code>  File project/path/path/file.py:56:54 extra text information
</code>```

How can I open this file in IntelliJ with a single click?

答:
The accepted answer didn't quite work for me. I ended up using:</p>

```<code>open -a "IntelliJ IDEA" \1
</code>```

ContentLicense: CC BY-SA 4.0

Title: Open file on IntelliJ from iTerm 2

Body:
I have output in my iTerm that is like:

  File project/path/path/file.py:56:54 extra text information

How can I open this file in IntelliJ with a single click?

Body:
The accepted answer didn't quite work for me. I ended up using:

open -a "IntelliJ IDEA" \1


问:
C++ Stack Implementation (not working right)
Here's the previous thread where I got help with this same lab. My stack is misbehaving, to say the least, when I add an item to stack, to print out later, it doesn't seem to add right. I always print out plus'(+), not matter if I enter another operand(*,/,+).

I am using a stack to convert a, user inputed, infix express to postfix. It seems to work fine except printing out the operands in the stack at the end.

```<code>#include <iostream>;
#include <vector>
using namespace std;

class DishWell{
public:
    char ReturnFront(){
        return Well.front();
    }
    void Push(char x){
        Well.push_back(x);
    }
    void Pop(){
        Well.pop_back();
    }
    bool IsEmpty(){
        return Well.empty();
    }
private:
    vector<char> Well;
};

bool Precidence(char Input, char Stack){
    int InputPrecidence,StackPrecidence;
    switch (Input){
        case '*':
            InputPrecidence = 4;
            break;
        case '/':
            InputPrecidence = 4;
            break;
        case '+':
            InputPrecidence = 3;
            break;
        case '-':
            InputPrecidence = 3;
            break;
        case '(':
            InputPrecidence = 2;
            break;
        default:
            InputPrecidence = 0;
    }
switch (Stack){
    case '*':
        StackPrecidence = 4;
        break;
    case '/':
        StackPrecidence = 4;
        break;
    case '+':
        StackPrecidence = 3;
        break;
    case '-':
        StackPrecidence = 3;
        break;
    case '(':
        StackPrecidence = 2;
        break;
    default:
        StackPrecidence = 0;
}
if(InputPrecidence>StackPrecidence) return true;
else return false;
}

int main(int argc, char** argv) {
    DishWell DishTray;
    char Input;
    bool InputFlag;
    InputFlag = true;
    cout<<"Enter Input, invalid input will terminate"<<endl;
    while(InputFlag){
        cout<<"Input: ";
        cin>>Input;
        cout<<endl;
        if((((Input>='a'&&Input<='z')||(Input>='A'&&Input<='Z'))||Input>='0'&&Input<='9')))//If Digit or Number
            cout<<Input;
        if((Input=='*'||Input=='/'||Input=='+'||Input=='-')){//if operand
            if(DishTray.IsEmpty())
                DishTray.Push(Input);
            else if(Precidence(Input,DishTray.ReturnFront()))
                DishTray.Push(Input);
            else if(!Precidence(Input,DishTray.ReturnFront()))
                cout<<"Output: "<<Input<<endl;
        }
        else if(!((((Input>='a'&&Input<='z')||(Input>='A'&&Input<='Z'))||(Input>='0'&&Input<='9')))||((Input=='*'||Input=='/'||Input=='+'||Input=='-')))//if not digit/numer or operand
            InputFlag = false;
    }
    int counter = 0;
    while(!DishTray.IsEmpty()){
        counter++;
        cout<<counter<<" Element "<<DishTray.ReturnFront()<<endl;
        DishTray.Pop();
    }
    return 0;
</code>```

Thank you, Macaire Bell

答:
Your loop calls front(), but then calls pop_back().  This will always return the first element in the vector, until all elements are popped, since you are never erasing the front element.  Your ReturnFront() method should probably be:

```<code>char ReturnBack(){
    return Well.back();
}
</code>```

And then your loop at the end:

```<code>   while(!DishTray.IsEmpty()){
        counter++;
        cout<<counter<<" Element "<<DishTray.ReturnBack()<<endl;  // will return last element
        DishTray.Pop();  // actually pop the element printed
    }
</code>```

Body:
Here's the previous thread where I got help with this same lab. My stack is misbehaving, to say the least, when I add an item to stack, to print out later, it doesn't seem to add right. I always print out plus'(+), not matter if I enter another operand(*,/,+).

I am using a stack to convert a, user inputed, infix express to postfix. It seems to work fine except printing out the operands in the stack at the end.

#include <iostream>;
#include <vector>
using namespace std;

class DishWell{
public:
    char ReturnFront(){
        return Well.front();
    }
    void Push(char x){
        Well.push_back(x);
    }
    void Pop(){
        Well.pop_back();
    }
    bool IsEmpty(){
        return Well.empty();
    }
private:
    vector<char> Well;
};

bool Precidence(char Input, char Stack){
    int InputPrecidence,StackPrecidence;
    switch (Input){
        case '*':
            InputPrecidence = 4;
            break;
        case '/':
            InputPrecidence = 4;
            break;
        case '+':
            InputPrecidence = 3;
            break;
        case '-':
            InputPrecidence = 3;
            break;
        case '(':
            InputPrecidence = 2;
            break;
        default:
            InputPrecidence = 0;
    }
switch (Stack){
    case '*':
        StackPrecidence = 4;
        break;
    case '/':
        StackPrecidence = 4;
        break;
    case '+':
        StackPrecidence = 3;
        break;
    case '-':
        StackPrecidence = 3;
        break;
    case '(':
        StackPrecidence = 2;
        break;
    default:
        StackPrecidence = 0;
}
if(InputPrecidence>StackPrecidence) return true;
else return false;
}

int main(int argc, char** argv) {
    DishWell DishTray;
    char Input;
    bool InputFlag;
    InputFlag = true;
    cout<<"Enter Input, invalid input will terminate"<<endl;
    while(InputFlag){
        cout<<"Input: ";
        cin>>Input;
        cout<<endl;
        if((((Input>='a'&&Input<='z')||(Input>='A'&&Input<='Z'))||Input>='0'&&Input<='9')))//If Digit or Number
            cout<<Input;
        if((Input=='*'||Input=='/'||Input=='+'||Input=='-')){//if operand
            if(DishTray.IsEmpty())
                DishTray.Push(Input);
            else if(Precidence(Input,DishTray.ReturnFront()))
                DishTray.Push(Input);
            else if(!Precidence(Input,DishTray.ReturnFront()))
                cout<<"Output: "<<Input<<endl;
        }
        else if(!((((Input>='a'&&Input<='z')||(Input>='A'&&Input<='Z'))||(Input>='0'&&Input<='9')))||((Input=='*'||Input=='/'||Input=='+'||Input=='-')))//if not digit/numer or operand
            InputFlag = false;
    }
    int counter = 0;
    while(!DishTray.IsEmpty()){
        counter++;
        cout<<counter<<" Element "<<DishTray.ReturnFront()<<endl;
        DishTray.Pop();
    }
    return 0;

Thank you, Macaire Bell

Body:
Your loop calls front(), but then calls pop_back().  This will always return the first element in the vector, until all elements are popped, since you are never erasing the front element.  Your ReturnFront() method should probably be:

char ReturnBack(){
    return Well.back();
}

And then your loop at the end:

   while(!DishTray.IsEmpty()){
        counter++;
        cout<<counter<<" Element "<<DishTray.ReturnBack()<<endl;  // will return last element
        DishTray.Pop();  // actually pop the element printed
    }


问:
Giving custom path for native dll in Netbeans
I have a NetBeans platform application with 2 modules having different versions of same native library.

I added the native library inside release/module/lib folder under each module. [as per http://wiki.netbeans.org/DevFaqWrapperModules section: How do I include native libraries (.so or .dll) in my library wrapper module?]

But the problem is during the final build generation of the application, native libraries are copied to build\cluster\modules\lib folder. i.e only one version of my native library exists inside lib folder as both versions have same name. Now my question is can I specifically mention in Netbeans that jar (Version 1) should take native lib (ver1) and jar (Version 2) should refer to native lib (version 2).

答:
You can load a Native library file (.dll/Windows or .so/Linux) with two ways:

1) Load the file by providing the full path:

```<code>System.load("my/full/path/native.dll");
</code>```

2) If your native file is located inside your Java Library Path:

```<code>System.loadLibrary("native");
</code>```

Take notice that in the second case you only need to provide the name of your native file (without its extension).

The default Java Library Path depends on OS:

On Windows, it maps to PATH

On Linux, it maps to LD_LIBRARY_PATH

On OS X, it maps to DYLD_LIBRARY_PATH

If you want to set your own Java Library Path:

```<code>try {
        System.setProperty("java.library.path","YOUR/PATH");
        Field fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
        fieldSysPath.setAccessible(true);
        fieldSysPath.set(null, null);
    } catch (Exception ex) {
        System.out.println("Failed to set Java Library Path: " + ex.getMessage);
    }
</code>```

Body:
You can load a Native library file (.dll/Windows or .so/Linux) with two ways:

1) Load the file by providing the full path:

System.load("my/full/path/native.dll");

2) If your native file is located inside your Java Library Path:

System.loadLibrary("native");

Take notice that in the second case you only need to provide the name of your native file (without its extension).

The default Java Library Path depends on OS:

On Windows, it maps to PATH

On Linux, it maps to LD_LIBRARY_PATH

On OS X, it maps to DYLD_LIBRARY_PATH

If you want to set your own Java Library Path:

try {
        System.setProperty("java.library.path","YOUR/PATH");
        Field fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
        fieldSysPath.setAccessible(true);
        fieldSysPath.set(null, null);
    } catch (Exception ex) {
        System.out.println("Failed to set Java Library Path: " + ex.getMessage);
    }


问:
Qt two classes access each other
Let's say I have a Class A which inherits from QMainWindow, and a Class B. The code is like this:

In a.h:

```<code>#ifndef A_H
#define A_H

#include <QMainWindow>
#include "b.h"

class A : public QMainWindow
{
    Q_OBJECT

public:
    A(QWidget *parent = 0);
    ~A();

    B TestingB;

    int tryingNumA = 0;
    void getNumB() {
        qDebug() << TestingB.tryingNumB; //worked
    }
};

#endif // A_H
</code>```

In b.h:

```<code>#ifndef B_H
#define B_H

#include <QDebug>

class A;

class B
{
public:
    B();

    int tryingNumB = 1;

    A *testingA;
    void getNumA() {
        qDebug() << testingA->tryingNumA; //did not work, error: invalid use of incomplete type 'class A'
    }
};

#endif // B_H
</code>```

And it is easy to get Class B elements in Class A, but I also want to get Class A elements in Class B(I want these two Class can access each other), the code I tried did not work. Is that the reason because Class A inherits from QMainWindow?

In order to achieve that, what can I do?
Thanks.

答:
Try moving B::getNumA() to the implementation file. Thus you would have something like

```<code>// b.cpp
#include "b.h"
#include "a.h"

 void b::getNumA() {
    qDebug() << testingA->tryingNumA; //did not work, error: invalid use of incomplete type 'class A'
}
</code>```

The goal is to break the circular dependency between the headers.

Title: Qt two classes access each other

Body:
Let's say I have a Class A which inherits from QMainWindow, and a Class B. The code is like this:

In a.h:

#ifndef A_H
#define A_H

#include <QMainWindow>
#include "b.h"

class A : public QMainWindow
{
    Q_OBJECT

public:
    A(QWidget *parent = 0);
    ~A();

    B TestingB;

    int tryingNumA = 0;
    void getNumB() {
        qDebug() << TestingB.tryingNumB; //worked
    }
};

#endif // A_H

In b.h:

#ifndef B_H
#define B_H

#include <QDebug>

class A;

class B
{
public:
    B();

    int tryingNumB = 1;

    A *testingA;
    void getNumA() {
        qDebug() << testingA->tryingNumA; //did not work, error: invalid use of incomplete type 'class A'
    }
};

#endif // B_H

And it is easy to get Class B elements in Class A, but I also want to get Class A elements in Class B(I want these two Class can access each other), the code I tried did not work. Is that the reason because Class A inherits from QMainWindow?

In order to achieve that, what can I do?
Thanks.

Body:
Try moving B::getNumA() to the implementation file. Thus you would have something like

// b.cpp
#include "b.h"
#include "a.h"

 void b::getNumA() {
    qDebug() << testingA->tryingNumA; //did not work, error: invalid use of incomplete type 'class A'
}

The goal is to break the circular dependency between the headers.


问:
How can I make the Enter key submit a form while a select field is focussed, without using JavaScript?
When a <select> field is focussed, pressing the Enter key does not submit the form. Is it possible to make it submit the form, without using JavaScript?

A solution working in quirks mode is fine, too.

答:
It is not possible to alter this behaviour without using JavaScript. According to the discussion at this question, capturing the keyup event is the best option for browser compatibility.

You may want to reconsider altering this behaviour, as discussed at this question you may surprise your users by doing so.


问:
coreos dex compile error
I am trying to complete the coreos dex //github.com/coreos/dex.git. I have cloned and installed the dependency packages in Centos & with Go version 1.9.

When I run make its throwing below error message.

```<code> git clone https://github.com/coreos/dex.git
 cd dex
 make

# make
can't load package: package github.com/coreos/dex/cmd/dex: cannot find package "github.com/coreos/dex/cmd/dex" in any of:
        /usr/lib/golang/src/github.com/coreos/dex/cmd/dex (from $GOROOT)
        /root/work/src/github.com/coreos/dex/cmd/dex (from $GOPATH)
make: *** [bin/dex] Error 1
</code>```

using glide installed all the required dependency packages.

```<code>[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/beevik/etree
[INFO]  --> Exporting github.com/coreos/etcd
[INFO]  --> Exporting github.com/ghodss/yaml
[INFO]  --> Exporting github.com/coreos/go-oidc
[INFO]  --> Exporting github.com/golang/protobuf
[INFO]  --> Exporting github.com/cockroachdb/cockroach-go
[INFO]  --> Exporting github.com/gtank/cryptopasta
[INFO]  --> Exporting github.com/inconshreveable/mousetrap
[INFO]  --> Exporting github.com/jonboulle/clockwork
[INFO]  --> Exporting github.com/spf13/cobra
[INFO]  --> Exporting github.com/kylelemons/godebug
[INFO]  --> Exporting github.com/pquerna/cachecontrol
[INFO]  --> Exporting github.com/russellhaering/goxmldsig
[INFO]  --> Exporting github.com/mattn/go-sqlite3
[INFO]  --> Exporting github.com/sirupsen/logrus
[INFO]  --> Exporting github.com/gorilla/context
[INFO]  --> Exporting github.com/go-sql-driver/mysql
[INFO]  --> Exporting github.com/gorilla/handlers
[INFO]  --> Exporting github.com/gorilla/mux
[INFO]  --> Exporting github.com/lib/pq
[INFO]  --> Exporting github.com/spf13/pflag
[INFO]  --> Exporting golang.org/x/crypto
[INFO]  --> Exporting golang.org/x/net
[INFO]  --> Exporting google.golang.org/genproto
[INFO]  --> Exporting google.golang.org/appengine
[INFO]  --> Exporting golang.org/x/oauth2
[INFO]  --> Exporting golang.org/x/sys
[INFO]  --> Exporting gopkg.in/asn1-ber.v1
[INFO]  --> Exporting gopkg.in/ldap.v2
[INFO]  --> Exporting google.golang.org/grpc
[INFO]  --> Exporting golang.org/x/text
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  --> Exporting gopkg.in/square/go-jose.v2
[INFO]  Replacing existing vendor dependencies
</code>```

Any tips to resolve this compile issue?

答:
found the right steps to build the dex from here https://github.com/coreos/dex/blob/master/Documentation/getting-started.md</p>

```<code>$ go get github.com/coreos/dex
$ cd $GOPATH/src/github.com/coreos/dex
$ make
</code>```

Title: coreos dex compile error

Body:
I am trying to complete the coreos dex //github.com/coreos/dex.git. I have cloned and installed the dependency packages in Centos & with Go version 1.9.

When I run make its throwing below error message.

 git clone https://github.com/coreos/dex.git
 cd dex
 make

# make
can't load package: package github.com/coreos/dex/cmd/dex: cannot find package "github.com/coreos/dex/cmd/dex" in any of:
        /usr/lib/golang/src/github.com/coreos/dex/cmd/dex (from $GOROOT)
        /root/work/src/github.com/coreos/dex/cmd/dex (from $GOPATH)
make: *** [bin/dex] Error 1

using glide installed all the required dependency packages.

[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/beevik/etree
[INFO]  --> Exporting github.com/coreos/etcd
[INFO]  --> Exporting github.com/ghodss/yaml
[INFO]  --> Exporting github.com/coreos/go-oidc
[INFO]  --> Exporting github.com/golang/protobuf
[INFO]  --> Exporting github.com/cockroachdb/cockroach-go
[INFO]  --> Exporting github.com/gtank/cryptopasta
[INFO]  --> Exporting github.com/inconshreveable/mousetrap
[INFO]  --> Exporting github.com/jonboulle/clockwork
[INFO]  --> Exporting github.com/spf13/cobra
[INFO]  --> Exporting github.com/kylelemons/godebug
[INFO]  --> Exporting github.com/pquerna/cachecontrol
[INFO]  --> Exporting github.com/russellhaering/goxmldsig
[INFO]  --> Exporting github.com/mattn/go-sqlite3
[INFO]  --> Exporting github.com/sirupsen/logrus
[INFO]  --> Exporting github.com/gorilla/context
[INFO]  --> Exporting github.com/go-sql-driver/mysql
[INFO]  --> Exporting github.com/gorilla/handlers
[INFO]  --> Exporting github.com/gorilla/mux
[INFO]  --> Exporting github.com/lib/pq
[INFO]  --> Exporting github.com/spf13/pflag
[INFO]  --> Exporting golang.org/x/crypto
[INFO]  --> Exporting golang.org/x/net
[INFO]  --> Exporting google.golang.org/genproto
[INFO]  --> Exporting google.golang.org/appengine
[INFO]  --> Exporting golang.org/x/oauth2
[INFO]  --> Exporting golang.org/x/sys
[INFO]  --> Exporting gopkg.in/asn1-ber.v1
[INFO]  --> Exporting gopkg.in/ldap.v2
[INFO]  --> Exporting google.golang.org/grpc
[INFO]  --> Exporting golang.org/x/text
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  --> Exporting gopkg.in/square/go-jose.v2
[INFO]  Replacing existing vendor dependencies

Any tips to resolve this compile issue?

Body:
found the right steps to build the dex from here https://github.com/coreos/dex/blob/master/Documentation/getting-started.md

$ go get github.com/coreos/dex
$ cd $GOPATH/src/github.com/coreos/dex
$ make


问:
Default text on textarea jQuery?
I have a textarea that i would like to show some default text on page load. Once the textarea is clicked i would like to have the text disappear and possibly if user clicked in textarea and did not type anything in and then clicked out of textarea the default text will show again.

I have searched Google and on here but i can only seem to find tutorials relating to text boxes and NOT textareas, plus i already am using a class on the textarea so cannot depend on class for it to work.

Does anyone have some simple jQuery code they would like to share with me to do what i want above?

答:
I have searched Google and on here but
  i can only seem to find tutorials
  relating to text boxes

Whatever is used on a text-box can also be used on a textarea in jQuery. The val() method detects the control used and will use value attribute for input and inner text for textarea

Pick up any of those links and implement it

Title: Default text on textarea jQuery?


问:
Using Invoke-SQL query with Foreach loop
I have a list of instances I have imported into the

$SQLServerList

I want to have them look for the databases on each instance.

```<code>foreach ($server in $SQLServerList ){
    Invoke-Sqlcmd -Query "use master select name, database_id, create_date, recovery_model_desc from sys.databases"  -ServerInstance $server
}
</code>```

however this won't run what am i doing wrong?

答:
Given a CSV file with column 'Server' containing the server names, using

$SQLServerList = import-csv servers.csv

on the file will return an object with NoteProperty 'Server'. (Imagine the csv has multiple columns, you need a way to identify which one you're accessing, and the $SQLServerList object will contain a property for each column).

When using this in foreach ($server in $SQLServerList) , $SQLServerList and $server represent the whole object, which doesn't make much sense when passed to invoke-sqlcmd, so $Server.Server is required.

Alternatively, $SQLServerList = (import-csv servers.csv).Server returns a string array and should be usable directly in foreach ($server in $SQLServerList)

Body:
I have a list of instances I have imported into the

$SQLServerList

I want to have them look for the databases on each instance.

foreach ($server in $SQLServerList ){
    Invoke-Sqlcmd -Query "use master select name, database_id, create_date, recovery_model_desc from sys.databases"  -ServerInstance $server
}

however this won't run what am i doing wrong?


问:
Noda Timezone issue with IDateTimeZoneSource.MapTimeZoneId
I am trying to convert Windows to IANA time zone and below is the code that works most of the time. But when the windows time id = "Turks & Caicos" then below code returns NULL. I am using NODATIME Version 1.3.1

```<code>    Public Function ConvertID_WindowsToIana(strWindowsZoneID As String) As String
        If strWindowsZoneID.Equals("UTC", StringComparison.OrdinalIgnoreCase) Then
            Return "Etc/UTC"
        End If

        Dim tzdbSource = NodaTime.TimeZones.TzdbDateTimeZoneSource.[Default]
        Dim tzi = TimeZoneInfo.MapTimeZoneId(strWindowsZoneID)
        Return tzdbSource.CanonicalIdMap(tzi)
    End Function
</code>```

Here when tzi = -4:00 Turks & Caicos, in this case the return is NULL.

Now when I updated to version 2.2.3, thought that it will fix the stuff, but as per the documentation

And now that function is taking a "Position Of Current String" as parameter.

So my code will look like below:

```<code>  Public Function ConvertID_WindowsToIana(strWindowsZoneID As String) As String

        Dim tzi = TimeZoneInfo.FindSystemTimeZoneById(strWindowsZoneID)
        Dim tzid = tzdbSource.GetSystemDefaultId(WHAT TO PUT HERE)
        // some logic
    End Function
</code>```

Question 1: Why my first code does not work for "Turks and Caicos".

Question 2: I updated thinking it will fix that issue, but now that function is vanished and replaced, and I don't know what to use. Please help me.

P.S: For Eastern and many other time zones the First Code Snippet works fine.

答:
GetSystemDefaultId doesn't replace all the functionality of MapTimeZoneId - you don't want to call that here.

Instead, use tzdbSource.WindowsMapping to get a WindowsZones that you can then use to find the TZDB ID. Here's some sample code:

```<code>using System;
using NodaTime.TimeZones;

public class Test
{
    static void Main()
    {
        var source = TzdbDateTimeZoneSource.Default;

        string windowsId = "Turks And Caicos Standard Time";

        var mapping = source.WindowsMapping.PrimaryMapping;
        if (mapping.TryGetValue(windowsId, out string tzdbId))
        {
            Console.WriteLine($"Mapped to {tzdbId}");
        }
        else
        {
            Console.WriteLine("Not found");
        }
    }
}
</code>```

An alternative approach is to use Matt Johnson's TimeZoneConverter package, which can simplify things a bit more. If you're using Noda Time elsewhere, it probably makes sense to stick with Noda Time, but if you're only using it for time zone ID conversion, TimeZoneConverter is a better choice.

As for why "Turks and Caicos" doesn't work - I assume you mean "Turks And Caicos Standard Time" as the ID. That was only introduced into the CLDR mapping data in CLDR v30, which was released over a year later than 1.3.1. If you update to 1.3.5 or 1.4.2, both of those should be fine.

Body:
I am trying to convert Windows to IANA time zone and below is the code that works most of the time. But when the windows time id = "Turks & Caicos" then below code returns NULL. I am using NODATIME Version 1.3.1

    Public Function ConvertID_WindowsToIana(strWindowsZoneID As String) As String
        If strWindowsZoneID.Equals("UTC", StringComparison.OrdinalIgnoreCase) Then
            Return "Etc/UTC"
        End If

        Dim tzdbSource = NodaTime.TimeZones.TzdbDateTimeZoneSource.[Default]
        Dim tzi = TimeZoneInfo.MapTimeZoneId(strWindowsZoneID)
        Return tzdbSource.CanonicalIdMap(tzi)
    End Function

Here when tzi = -4:00 Turks & Caicos, in this case the return is NULL.

Now when I updated to version 2.2.3, thought that it will fix the stuff, but as per the documentation

And now that function is taking a "Position Of Current String" as parameter.

So my code will look like below:

  Public Function ConvertID_WindowsToIana(strWindowsZoneID As String) As String

        Dim tzi = TimeZoneInfo.FindSystemTimeZoneById(strWindowsZoneID)
        Dim tzid = tzdbSource.GetSystemDefaultId(WHAT TO PUT HERE)
        // some logic
    End Function

Question 1: Why my first code does not work for "Turks and Caicos".

Question 2: I updated thinking it will fix that issue, but now that function is vanished and replaced, and I don't know what to use. Please help me.

P.S: For Eastern and many other time zones the First Code Snippet works fine.

Body:
GetSystemDefaultId doesn't replace all the functionality of MapTimeZoneId - you don't want to call that here.

Instead, use tzdbSource.WindowsMapping to get a WindowsZones that you can then use to find the TZDB ID. Here's some sample code:

using System;
using NodaTime.TimeZones;

public class Test
{
    static void Main()
    {
        var source = TzdbDateTimeZoneSource.Default;

        string windowsId = "Turks And Caicos Standard Time";

        var mapping = source.WindowsMapping.PrimaryMapping;
        if (mapping.TryGetValue(windowsId, out string tzdbId))
        {
            Console.WriteLine($"Mapped to {tzdbId}");
        }
        else
        {
            Console.WriteLine("Not found");
        }
    }
}

An alternative approach is to use Matt Johnson's TimeZoneConverter package, which can simplify things a bit more. If you're using Noda Time elsewhere, it probably makes sense to stick with Noda Time, but if you're only using it for time zone ID conversion, TimeZoneConverter is a better choice.

As for why "Turks and Caicos" doesn't work - I assume you mean "Turks And Caicos Standard Time" as the ID. That was only introduced into the CLDR mapping data in CLDR v30, which was released over a year later than 1.3.1. If you update to 1.3.5 or 1.4.2, both of those should be fine.


问:
Restricting access to Excel source data

I have an Excel template which reads data from a source Excel file using vlookups and Index/Match functions. I there a way to prevent the end user from accessing the source data file/sheet? e.g. by storing the source file on a remote location and make the vlookups read from there..
</p>

答:
Depending on what resources are available to you, it may be difficult to prevent users from just going around the restrictions you put in place. Even if the data is in a database table you will need measures in place to prevent users from querying it outside of your Excel template. I don't know your situation, but ideally there would be someone (i.e. database administrator, infosec, back-end developer) who could help engineer a proper solution.

Having said that, I do believe your idea around using MS SQL Server could be a good way to go. You could create stored procedures instead of using sql queries to limit access. See this link for more details:
Managing Permissions with Stored Procedures in SQL Server

In addition, I would be worried about users figuring out other user IDs and arbitrarily accessing data. You could implement some sort of protection by having a mapping table so that there's no way to access information with user IDs. The table would be as follows:

Columns: randomKey, userId, creationDate

randomKey is just an x digit random number/letter sequence

creationDateTime is a time stamp and used for timeout purposes

Whenever someone needs a user id you would run a stored procedure that adds a record to the mapping table. You input the user id, the procedure creates a record and returns the key. You provide the user with the key which they enter in your template. A separate stored procedure takes the key and resolves to the user id (using the mapping table) and returns the requested information. These keys expire. Either they can be single use (the procedure deletes the record from the mapping table) or use a timeout (if creationDateTime is more than x hours/days old it will not return data).

For the keys, Mark Ransom shared an interesting solution for creating random IDs for which you could base your logic:
Generate 6 Digit unique number

Sounds like a lot of work, but if there is sensitivity around your data it's worth building a more robust process around it. There's probably a better way to approach this, but I hope it at least gives you food for thought.

Title: Restricting access to Excel source data


问:
Apache spark job failed immediately without retry, setting maxFailures doesn't work
I was testing a web crawling/scrapping program on Apache Spark locally on my computer.

the program use a few RDD transformations that takes a volatile function that sporadically fails. (The function's purpose is to transform URL links into web pages, sometimes the headless browser it invoked just blackout or got overloaded - I can't avoid that)

I heard that Apache Spark has powerful failover and retrying feature, any unsuccessful transformation or lost data can be recalculated from scratch from whatever resource it can find (sounds like magic right?) so I didn't put any failover or try-catch in my code.

This is my spark configuration:

```<code>val conf = new SparkConf().setAppName("MoreLinkedIn")
conf.setMaster("local[*]")
conf.setSparkHome(System.getenv("SPARK_HOME"))
conf.setJars(SparkContext.jarOfClass(this.getClass).toList)
conf.set("spark.task.maxFailures","40") //definitely enough
</code>```

Unfortunately the job failed after the majority of stages and individual tasks succeeded. The latest log in console shows:

```<code>Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 1.0:7 failed 1 times, most recent failure: Exception failure in TID 23 on host localhost: org.openqa.selenium.TimeoutException: Timed out after 50 seconds waiting for...
</code>```

Looks like Spark just give up cowardly after failed once. How do I configure it properly to make it more tenacious?

(my program can be downloaded from https://github.com/tribbloid/spookystuff, sorry for the scarce and disorganized code/documentation, I just start it for a few days)

ADD: if you want to try it yourself, The following code can demonstrate this problem:

```<code>def main(args: Array[String]) {
val conf = new SparkConf().setAppName("Spark Pi")
conf.setMaster("local[*]")
conf.setSparkHome(System.getenv("SPARK_HOME"))
conf.setJars(SparkContext.jarOfClass(this.getClass).toList)
conf.set("spark.task.maxFailures","400000")
val sc = new SparkContext(conf)
val slices = if (args.length > 0) args(0).toInt else 8
val n = 100000 * slices
val count = sc.parallelize(1 to n, slices).map { i =>
  val x = java.lang.Math.random()
  if (x > 0.9) throw new IllegalStateException("the map has a chance of 10% to fail")
  x
}.reduce(_ + _)
sc.stop()
println("finished")
}
</code>```

It should be noted that the same IllegalStateException got retried for 32 times in this post:
Apache Spark Throws java.lang.IllegalStateException: unread block data

答:
I know it's a very old question, but I had exactly same problem and came across this question while looking for a solution.

There are 3 master URL formats to submit a spark application in a local mode:

local - one thread (no parallelism), no retries

local[K] (or local[*]) - uses K (or number of cores) worker threads and sets task.maxFailures to 1 (see here)

local[K, F] (or local[*, F]) - sets the task.maxFailures=F, and this is what we were after.

Consult Spark documentation for details.

Body:
I was testing a web crawling/scrapping program on Apache Spark locally on my computer.

the program use a few RDD transformations that takes a volatile function that sporadically fails. (The function's purpose is to transform URL links into web pages, sometimes the headless browser it invoked just blackout or got overloaded - I can't avoid that)

I heard that Apache Spark has powerful failover and retrying feature, any unsuccessful transformation or lost data can be recalculated from scratch from whatever resource it can find (sounds like magic right?) so I didn't put any failover or try-catch in my code.

This is my spark configuration:

val conf = new SparkConf().setAppName("MoreLinkedIn")
conf.setMaster("local[*]")
conf.setSparkHome(System.getenv("SPARK_HOME"))
conf.setJars(SparkContext.jarOfClass(this.getClass).toList)
conf.set("spark.task.maxFailures","40") //definitely enough

Unfortunately the job failed after the majority of stages and individual tasks succeeded. The latest log in console shows:

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 1.0:7 failed 1 times, most recent failure: Exception failure in TID 23 on host localhost: org.openqa.selenium.TimeoutException: Timed out after 50 seconds waiting for...

Looks like Spark just give up cowardly after failed once. How do I configure it properly to make it more tenacious?

(my program can be downloaded from https://github.com/tribbloid/spookystuff, sorry for the scarce and disorganized code/documentation, I just start it for a few days)

ADD: if you want to try it yourself, The following code can demonstrate this problem:

def main(args: Array[String]) {
val conf = new SparkConf().setAppName("Spark Pi")
conf.setMaster("local[*]")
conf.setSparkHome(System.getenv("SPARK_HOME"))
conf.setJars(SparkContext.jarOfClass(this.getClass).toList)
conf.set("spark.task.maxFailures","400000")
val sc = new SparkContext(conf)
val slices = if (args.length > 0) args(0).toInt else 8
val n = 100000 * slices
val count = sc.parallelize(1 to n, slices).map { i =>
  val x = java.lang.Math.random()
  if (x > 0.9) throw new IllegalStateException("the map has a chance of 10% to fail")
  x
}.reduce(_ + _)
sc.stop()
println("finished")
}

It should be noted that the same IllegalStateException got retried for 32 times in this post:
Apache Spark Throws java.lang.IllegalStateException: unread block data


问:
PHP File Real Mime
I have done functions to upload files to DB. User (uploader) can upload pdf png or jpeg files. All others are not allowed. But there is way to 'hack' it. User can have a file (for example PHP file) on own computer. He will rename the file to 'test.jpg' from 'test.php'. Then he will upload renamed file as jpg and when the program will check file, it will upload it because it will return, that it is jpg (image/jpeg). Is there a way to check if file is realy jpeg/png/...?

E: I don't want to upload only images. File to upload generaly can be in another format, so I can not use function imagesize().

答: There is a native PHP function mime_content_type that returns the MIME content type for a file as determined by using information from the magic.mime file. </p>

Title: PHP File Real Mime


问:
Android Room: @Ignore vs Transient
Are those two interchangable in context of Room database entity, or, if not, what are the differences between them?</p>

答:
@Ignore is a Room-specific annotation, saying that Room should ignore that field or method.

transient is a Java construct, indicating that this field should not be serialized in standard Java serialization. Room happens to treat this similarly to @Ignore by default. Mostly, that is there for cases where you are inheriting from some class that happens to use transient and you do not control that class (e.g., it is from a library).

For your own code, if you are not using Java serialization, I recommend sticking with @Ignore for the fields. transient is not an available keyword for a method, so to tell Room to ignore certain constructors, you have no choice but to use @Ignore.

Title: Android Room: @Ignore vs Transient


问:
User (session) count in cluster
Is there a good way to get the logged in user count in a Java web application that is running in a cluster?

I wrote a simple HttpSessionListener with a static field, but I suppose this doesn't work in cluster. I can see there is a Spring Security solution, but I read in some forums that this is still not ok in cluster.

The product in which I have to implement this user count is trying to be application server independent, currently we support Tomcat, Weblogic and JBoss. At the moment I need a solution for Weblogic 10.3 clusters.

答:
A simple tutorial to demonstrate how to determine active users / sessions in a Java Web Application.

```<code>package com.hubberspot.javaee.listener;

import javax.servlet.annotation.WebListener;
import javax.servlet.http.HttpSessionEvent;
import javax.servlet.http.HttpSessionListener;

@WebListener
public class OnlineUsersCounter implements HttpSessionListener {

private static int numberOfUsersOnline;

 public OnlineUsersCounter() {
  numberOfUsersOnline = 0;
 }

 public static int getNumberOfUsersOnline() {
  return numberOfUsersOnline;
 }

    public void sessionCreated(HttpSessionEvent event) {

     System.out.println("Session created by Id : " + event.getSession().getId());
     synchronized (this) {
   numberOfUsersOnline++;
  }

    }

    public void sessionDestroyed(HttpSessionEvent event) {

     System.out.println("Session destroyed by Id : " + event.getSession().getId());
     synchronized (this) {
   numberOfUsersOnline--;
  }

    }

}
</code>```

Running the below servlet on three different browsers will provide output as : (see fig below)

```<code>package com.hubberspot.javaee;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

import com.hubberspot.javaee.listener.OnlineUsersCounter;

// @WebServlet annotation has a initParams field which takes
// in initialization parameters for a servlet.
// @WebInitParam annotation takes in a name and value for the
// initialization parameters for the current Servlet.

@WebServlet(name = "HelloWorldServlet" , urlPatterns = { "/HelloWorldServlet" }
, initParams = { @WebInitParam(name = "user" , value = "Jonty") })
public class HelloWorldServlet extends HttpServlet {

 protected void doGet(
   HttpServletRequest request,
   HttpServletResponse response
   ) throws ServletException, IOException {

  response.setContentType("text/html");

  PrintWriter out = response.getWriter();

  // sessionCreated method gets executed
  HttpSession session = request.getSession();

  session.setMaxInactiveInterval(60);

  try {
   out.println("<html>");
   out.println("<body>");
   out.println("<h2>Number of Users Online : "
      + OnlineUsersCounter.getNumberOfUsersOnline()
      + "</h2>");
   out.println("</body>");
   out.println("</html>");
  } finally {
   out.close();
  }

 }

}
</code>```

Output of the program :

Eclipse Browser ->

Firefox Browser ->

Internet Explorer Browser ->

Console Output ->

For more: http://www.hubberspot.com/2013/09/how-to-determine-active-users-sessions.html

Title: User (session) count in cluster

Body:
A simple tutorial to demonstrate how to determine active users / sessions in a Java Web Application.

package com.hubberspot.javaee.listener;

import javax.servlet.annotation.WebListener;
import javax.servlet.http.HttpSessionEvent;
import javax.servlet.http.HttpSessionListener;

@WebListener
public class OnlineUsersCounter implements HttpSessionListener {

private static int numberOfUsersOnline;

 public OnlineUsersCounter() {
  numberOfUsersOnline = 0;
 }

 public static int getNumberOfUsersOnline() {
  return numberOfUsersOnline;
 }

    public void sessionCreated(HttpSessionEvent event) {

     System.out.println("Session created by Id : " + event.getSession().getId());
     synchronized (this) {
   numberOfUsersOnline++;
  }

    }

    public void sessionDestroyed(HttpSessionEvent event) {

     System.out.println("Session destroyed by Id : " + event.getSession().getId());
     synchronized (this) {
   numberOfUsersOnline--;
  }

    }

}

Running the below servlet on three different browsers will provide output as : (see fig below)

package com.hubberspot.javaee;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

import com.hubberspot.javaee.listener.OnlineUsersCounter;

// @WebServlet annotation has a initParams field which takes
// in initialization parameters for a servlet.
// @WebInitParam annotation takes in a name and value for the
// initialization parameters for the current Servlet.

@WebServlet(name = "HelloWorldServlet" , urlPatterns = { "/HelloWorldServlet" }
, initParams = { @WebInitParam(name = "user" , value = "Jonty") })
public class HelloWorldServlet extends HttpServlet {

 protected void doGet(
   HttpServletRequest request,
   HttpServletResponse response
   ) throws ServletException, IOException {

  response.setContentType("text/html");

  PrintWriter out = response.getWriter();

  // sessionCreated method gets executed
  HttpSession session = request.getSession();

  session.setMaxInactiveInterval(60);

  try {
   out.println("<html>");
   out.println("<body>");
   out.println("<h2>Number of Users Online : "
      + OnlineUsersCounter.getNumberOfUsersOnline()
      + "</h2>");
   out.println("</body>");
   out.println("</html>");
  } finally {
   out.close();
  }

 }

}

Output of the program :

Eclipse Browser ->

Firefox Browser ->

Internet Explorer Browser ->

Console Output ->

For more: http://www.hubberspot.com/2013/09/how-to-determine-active-users-sessions.html


问:
Python - Getting all links from a div having a class
I am using BeautifulSoup to get all links of mobile phones from this url
http://www.gsmarena.com/samsung-phones-f-9-0-p2.php

My code for the following is :

```<code>import urllib2
from BeautifulSoup import BeautifulSoup

url = "http://www.gsmarena.com/samsung-phones-f-9-0-p2.php"
text = urllib2.urlopen(url).read();
soup = BeautifulSoup(text);

data = soup.findAll('div',attrs={'class':'makers'});
for i in data:
    print "http://www.gsmarena.com/" + i.ul.li.a['href'];
</code>```

But the returned list of urls is shorter than the expected output when i checked, this code outputs 3 values but the result should show much over 10 values

答:
If you have Python 3, you can use Simon's answer with the following change:</p>

```<code>from urllib.request import urlopen
from bs4 import BeautifulSoup

text = urlopen(base_url).read()
</code>```

Body:
I am using BeautifulSoup to get all links of mobile phones from this url
http://www.gsmarena.com/samsung-phones-f-9-0-p2.php

My code for the following is :

import urllib2
from BeautifulSoup import BeautifulSoup

url = "http://www.gsmarena.com/samsung-phones-f-9-0-p2.php"
text = urllib2.urlopen(url).read();
soup = BeautifulSoup(text);

data = soup.findAll('div',attrs={'class':'makers'});
for i in data:
    print "http://www.gsmarena.com/" + i.ul.li.a['href'];

But the returned list of urls is shorter than the expected output when i checked, this code outputs 3 values but the result should show much over 10 values

Body:
If you have Python 3, you can use Simon's answer with the following change:

from urllib.request import urlopen
from bs4 import BeautifulSoup

text = urlopen(base_url).read()


问:
Hive :Current token (VALUE_STRING) not numeric, can not use numeric value accessors, while trying to QUERY an external table in hive on nested json
Unable to query an external hive table on a nested json due to

```<code> Error: java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:  org.codehaus.jackson.JsonParseException: Current token (VALUE_STRING) not numeric, can not use numeric value accessors
</code>```

Json looks like-

Create table command used-

```<code>create external table s
(
magic String,
type String,
headers String,
messageSchemaId String,
messageSchema String,
message  struct<data:struct<s_ID:double,s_TYPE_ID:Int,NAME:String,DES   CR:String,ACTIVE_s:double,s_ID:double,s_ENABLED:Int,pharmacy_location:Int>,seData:struct<APPLICATION_ID:double,s_TYPE_ID:Int,NAME:String,DESCR:String,s_STAT:double,PROGRAM_ID:double,s_ENABLED:Int,s_location:Int>,headers:struct<operation:String, changeSequence:String, timestamp: String, streamPosition: String, transactionId: String>>
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
Stored as textfile
LOCATION '/user/eh2524/pt_rpt/MGPH.APPLICATION';
</code>```

For the same json I am able to create external table with -

```<code>CREATE EXTERNAL TABLE `MGPH_ZT`(
`jsonstr` string)
PARTITIONED BY (
`dt` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'/user/eh2524/pt_rpt/MGPH.APPLICATION/'
TBLPROPERTIES (
'transient_lastDdlTime'='1510776187')
</code>```

But to query the table created by above I have use jsontuple method like

```<code>select count(*) from pt_rpt_stg.hvf_modules j
lateral view json_tuple(j.jsonstr, 'message') m as message
lateral view json_tuple(m.message, 'data') d as datacntr
lateral view json_tuple(d.datacntr,'l_location') s as   pharmacy_location
where pharmacy_location is null;
</code>```

I want to create the table using Json serde so that my team can query it directly like we do for a normal hive table, right now it fails when you query it.

What did I try-

I checked if there is any /n in the json file but there was none, tried with single record as well.

Checked the table creation definition on (https://community.hortonworks.com/questions/29814/how-to-load-nested-json-file-in-hive-table-using-h.html) for a nested json but it seems correct as i have used required complex data types.

-

答: The problem is that you declaring pharmacy_location as int in your table definition, but your sample data is a string: "pharmacy_location": "93". If you change that in your  table definition, it should work.</p>

Body:
Unable to query an external hive table on a nested json due to

 Error: java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:  org.codehaus.jackson.JsonParseException: Current token (VALUE_STRING) not numeric, can not use numeric value accessors

Json looks like-

Create table command used-

create external table s
(
magic String,
type String,
headers String,
messageSchemaId String,
messageSchema String,
message  struct<data:struct<s_ID:double,s_TYPE_ID:Int,NAME:String,DES   CR:String,ACTIVE_s:double,s_ID:double,s_ENABLED:Int,pharmacy_location:Int>,seData:struct<APPLICATION_ID:double,s_TYPE_ID:Int,NAME:String,DESCR:String,s_STAT:double,PROGRAM_ID:double,s_ENABLED:Int,s_location:Int>,headers:struct<operation:String, changeSequence:String, timestamp: String, streamPosition: String, transactionId: String>>
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
Stored as textfile
LOCATION '/user/eh2524/pt_rpt/MGPH.APPLICATION';

For the same json I am able to create external table with -

CREATE EXTERNAL TABLE `MGPH_ZT`(
`jsonstr` string)
PARTITIONED BY (
`dt` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'/user/eh2524/pt_rpt/MGPH.APPLICATION/'
TBLPROPERTIES (
'transient_lastDdlTime'='1510776187')

But to query the table created by above I have use jsontuple method like

select count(*) from pt_rpt_stg.hvf_modules j
lateral view json_tuple(j.jsonstr, 'message') m as message
lateral view json_tuple(m.message, 'data') d as datacntr
lateral view json_tuple(d.datacntr,'l_location') s as   pharmacy_location
where pharmacy_location is null;

I want to create the table using Json serde so that my team can query it directly like we do for a normal hive table, right now it fails when you query it.

What did I try-

I checked if there is any /n in the json file but there was none, tried with single record as well.

Checked the table creation definition on (https://community.hortonworks.com/questions/29814/how-to-load-nested-json-file-in-hive-table-using-h.html) for a nested json but it seems correct as i have used required complex data types.

-


问:
Removing duplicates from a large table with NULL columns that also need to be taken into account
I am interested in removing duplicates from a very large table (40+ million records, 3+ GB in size), that also has to count NULL columns when determining duplicates.   5.5.56-MariaDB, tables are in InnoDB format.

The problem I'm running into is that some of the recommended methods will not work when it comes to NULL field comparison.    For example, I have a table that has about a dozen columns, many of which can contain NULL values, and I want NULLs treated as any other value, in that if a row has the same NULL values in the same columns, it's considered a dupe.

The technique below does NOT work:

```<code>create table tmp like mytable;
alter table tmp ADD UNIQUE INDEX(field1,field2, field3,...);
insert IGNORE into tmp (select * from mytable);
</code>```

At this point, tmp should have all dupes removed, but NULL columns are ignored in the Unique index, so it doesn't work.  If it did work, then I could rename the tables, or truncate the original table and copy over (assuming I wanted to preserve a primary key in the original table).

What is the most efficient way of deleting dupes in large files when dealing with NULL columns that should also be taken into consideration?

Additional notes:   I have a field called "recno" that is Auto Increment that needs to be ignored.

答:
Here's what I used that worked..  I ended up having to re-create the table and rename it, but the process time for this was about 90 minutes for a database of 3.1GB and 42 Million records (with about 977,000 dupes):</p>

```<code>drop table if exists tmp;
create table tmp AS
select distinct field1, field2, field3, ...
 from mytable;
-- add back my rowid
ALTER TABLE `tmp` ADD `recno` INT UNSIGNED NOT NULL AUTO_INCREMENT
-- add any indexes you were using

-- rename tables
rename table mytable to table_hold;
rename table tmp to mytable;
</code>```

Body:
I am interested in removing duplicates from a very large table (40+ million records, 3+ GB in size), that also has to count NULL columns when determining duplicates.   5.5.56-MariaDB, tables are in InnoDB format.

The problem I'm running into is that some of the recommended methods will not work when it comes to NULL field comparison.    For example, I have a table that has about a dozen columns, many of which can contain NULL values, and I want NULLs treated as any other value, in that if a row has the same NULL values in the same columns, it's considered a dupe.

The technique below does NOT work:

create table tmp like mytable;
alter table tmp ADD UNIQUE INDEX(field1,field2, field3,...);
insert IGNORE into tmp (select * from mytable);

At this point, tmp should have all dupes removed, but NULL columns are ignored in the Unique index, so it doesn't work.  If it did work, then I could rename the tables, or truncate the original table and copy over (assuming I wanted to preserve a primary key in the original table).

What is the most efficient way of deleting dupes in large files when dealing with NULL columns that should also be taken into consideration?

Additional notes:   I have a field called "recno" that is Auto Increment that needs to be ignored.

Body:
Here's what I used that worked..  I ended up having to re-create the table and rename it, but the process time for this was about 90 minutes for a database of 3.1GB and 42 Million records (with about 977,000 dupes):

drop table if exists tmp;
create table tmp AS
select distinct field1, field2, field3, ...
 from mytable;
-- add back my rowid
ALTER TABLE `tmp` ADD `recno` INT UNSIGNED NOT NULL AUTO_INCREMENT
-- add any indexes you were using

-- rename tables
rename table mytable to table_hold;
rename table tmp to mytable;