问:
Flutter Widget Test passes on device but not without one
I have a widget that displays a list of data from an api and I'm trying to write tests for it's various states starting with it's empty state.

Currently my test pumps the widget in question, the widget makes a network call which I'm mocking and then it checks to see if the empty state text is shown.

This test passes when I run the test on a device using

```<code> flutter run --flavor myTestApp -t test/booking/booking_list_widget_test.dart
</code>```

But fails when I run it from the IDE (IntelliJ) the failure exception is:

```<code>The following TestFailure object was thrown running a test:
Expected: exactly one matching node in the widget tree
Actual: _TextFinder:<zero widgets with text "You're all caught up." (ignoring offstage widgets)>
Which: means none were found but one was expected
</code>```

It seems to not be waiting for the Duration given to tester.pump and I've tried wrapping it in a tester.runAsync and using pump and settle etc but i cannot get it to pass.

Any ideas welcome I cant share the widget tree but I can share the test

```<code>  void main() {
    setupFirebaseAuthMocks();

    setUpAll(
      () async {
        SharedPreferences.setMockInitialValues(
          {
            Constants.USER_ID: '1234567890',
          },
        );
      },
    );

    testWidgets(
      'emptyListTest',
      (tester) async {
        await _initializeDependencies(tester);

        final dio = di.getIt.get<Dio>();
        final dioAdapter = DioAdapter(dio: dio);

        dioAdapter.onGet(
          '/url/user/1234567890',
          (server) => server.reply(
            200,
            BookingResponse(
              (b) => b
                ..data = <Booking>[].toBuiltList().toBuilder()
                ..pagination = Pagination((b) => b
                  ..last = ''
                  ..first = ''
                  ..next = ''
                  ..skip = 0
                  ..count = 0).toBuilder(),
            ),
          ),
        );

        final testApp = await tester.runAsync(
          () => wordskiiTestApp(
            widgetUnderTest: BookingView(),
          ),
        );

        await tester.pumpWidget(testApp!);
        await tester.pump(const Duration(seconds: 1));

        expect(find.text('AVAILABLE'), findsOneWidget);
        // debugDumpApp();

        expect(
          find.text(
            'You\'re all caught up.',
          ),
          findsOneWidget,
        );
      },
    );
  }

  Future<void> _initializeDependencies(WidgetTester tester) async {
    await tester.runAsync(di.getIt.reset);
    await tester.runAsync(di.initTesting);
    await tester.runAsync(di.getIt.allReady);
  }
</code>```

答:
On which Widget is 'You\'re all caught up' expected? I had similar issues encountered previously when I tried find.textContaining() on a ListView item. I was able to solve my issue by finding out which widget the Text should appear.

On mine, I had to use find.widgetWithText(InkWell, String). To find out the which Widget it is, you can tap on the widget displayed on screen when you've run the test (i.e. on an emulator). The logs should display the tapped Widget.

Body:
I have a widget that displays a list of data from an api and I'm trying to write tests for it's various states starting with it's empty state.

Currently my test pumps the widget in question, the widget makes a network call which I'm mocking and then it checks to see if the empty state text is shown.

This test passes when I run the test on a device using

 flutter run --flavor myTestApp -t test/booking/booking_list_widget_test.dart

But fails when I run it from the IDE (IntelliJ) the failure exception is:

The following TestFailure object was thrown running a test:
Expected: exactly one matching node in the widget tree
Actual: _TextFinder:<zero widgets with text "You're all caught up." (ignoring offstage widgets)>
Which: means none were found but one was expected

It seems to not be waiting for the Duration given to tester.pump and I've tried wrapping it in a tester.runAsync and using pump and settle etc but i cannot get it to pass.

Any ideas welcome I cant share the widget tree but I can share the test

  void main() {
    setupFirebaseAuthMocks();

    setUpAll(
      () async {
        SharedPreferences.setMockInitialValues(
          {
            Constants.USER_ID: '1234567890',
          },
        );
      },
    );

    testWidgets(
      'emptyListTest',
      (tester) async {
        await _initializeDependencies(tester);

        final dio = di.getIt.get<Dio>();
        final dioAdapter = DioAdapter(dio: dio);

        dioAdapter.onGet(
          '/url/user/1234567890',
          (server) => server.reply(
            200,
            BookingResponse(
              (b) => b
                ..data = <Booking>[].toBuiltList().toBuilder()
                ..pagination = Pagination((b) => b
                  ..last = ''
                  ..first = ''
                  ..next = ''
                  ..skip = 0
                  ..count = 0).toBuilder(),
            ),
          ),
        );

        final testApp = await tester.runAsync(
          () => wordskiiTestApp(
            widgetUnderTest: BookingView(),
          ),
        );

        await tester.pumpWidget(testApp!);
        await tester.pump(const Duration(seconds: 1));

        expect(find.text('AVAILABLE'), findsOneWidget);
        // debugDumpApp();

        expect(
          find.text(
            'You\'re all caught up.',
          ),
          findsOneWidget,
        );
      },
    );
  }

  Future<void> _initializeDependencies(WidgetTester tester) async {
    await tester.runAsync(di.getIt.reset);
    await tester.runAsync(di.initTesting);
    await tester.runAsync(di.getIt.allReady);
  }


来源: stackexchange

问:
How to prevent SliverAppBar from closing after scrolling up?
I have a SliverAppBar with 'snap' and 'floating' both set to true. When the user scrolls down, the AppBar disappears, which is expected. The problem occurs when the user scrolls up and then taps on a button in the AppBar. As you can see from the GIF below, the AppBar snaps shut instead of going to the appropriate page. I noticed that this only happens if the scroll view is still moving when the user taps the button. Does anyone know how I can fix this?

```<code>SliverAppBar(
  snap: true,
  floating: true,
  backgroundColor: Colors.white,
  forceElevated: true,
  elevation: 2.0,
  shadowColor: Colors.black26,
  leading: IconButton(
    onPressed: () {
      Navigator.push(
        context,
        PageTransition(
          child: const MyAppBarMenu(),
          type: PageTransitionType.leftToRightJoined,
          duration: const Duration(milliseconds: 400),
          reverseDuration: const Duration(milliseconds: 150),
          curve: Curves.easeOutExpo,
          childCurrent: Container(),
        ),
      );
    },
    icon: ValueListenableBuilder(
        valueListenable: AppData.isLoggedIn,
        builder: (context, bool isLoggedIn, child) {
          return ValueListenableBuilder(
              valueListenable: AppData.userInfo,
              builder: (context, Map<String, dynamic> userInfo, _) {
                if (isLoggedIn) {
                  String imgUrl = '';
                  String points = '';
                  if (userInfo.isNotEmpty) {
                    imgUrl = '${AppData.domain}/VIEWUSER/${userInfo['Photo']}';
                    points = userInfo['points'] ?? '';
                  }
                  return Badge(
                    showBadge: (points != '' && points != '0'),
                    badgeColor: Colors.white,
                    shape: BadgeShape.square,
                    animationDuration: const Duration(seconds: 0),
                    padding: const EdgeInsets.symmetric(vertical: 1.0, horizontal: 2.0),
                    borderRadius: BorderRadius.circular(16.0),
                    position: BadgePosition.bottomEnd(bottom: -2.0, end: -12.0),
                    badgeContent: Padding(
                      padding: const EdgeInsets.symmetric(horizontal: 2.0),
                      child: Text(points, style: MyFonts.blackText12bold),
                    ),
                    borderSide: const BorderSide(width: 0.5, color: Colors.black54),
                    child: Builder(builder: (context) {
                      if (userInfo['Photo'] != '') {
                        return ClipOval(
                          child: CachedNetworkImage(
                            httpHeaders: AppData.myHeaders,
                            imageUrl: imgUrl,
                            placeholder: (context, url) => const Icon(Icons.account_circle, color: Colors.grey),
                            errorWidget: (context, url, error) => const Icon(Icons.account_circle, color: Colors.grey),
                          ),
                        );
                      } else {
                        return const Icon(Icons.account_circle, color: Colors.grey);
                      }
                    }),
                  );
                } else {
                  return const Icon(Icons.account_circle, color: Colors.grey);
                }
              });
        }),
    iconSize: 32.0,
  ),
  title: OutlinedButton(
    style: OutlinedButton.styleFrom(
      primary: MyColors.theme,
      backgroundColor: Colors.white,
      side: const BorderSide(color: Colors.grey, width: 1.0),
      shape: const RoundedRectangleBorder(borderRadius: BorderRadius.all(Radius.circular(32.0))),
    ),
    onPressed: () {
      Navigator.pushNamed(context, '/search');
    },
    child: Row(
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: const [
        Text(
          '搜尋',
          style: TextStyle(color: Colors.grey),
        ),
        Icon(
          Icons.search_rounded,
          color: Colors.grey,
        ),
      ],
    ),
  ),
  actions: [
    Row(
      children: [
        IconButton(
          onPressed: () => Navigator.pushNamed(context, '/news'),
          icon: ValueListenableBuilder(
              valueListenable: AppData.unreadNewsIds,
              builder: (context, Set<String> unreadNewsIds, _) {
                return Badge(
                    showBadge: unreadNewsIds.isNotEmpty,
                    position: BadgePosition.topEnd(top: -4.0, end: -4.0),
                    badgeContent: Text(
                      (unreadNewsIds.length > 9) ? unreadNewsIds.length.toString() : ' ${unreadNewsIds.length.toString()} ',
                      style: MyFonts.whiteText12bold,
                    ),
                    padding: const EdgeInsets.all(2.0),
                    child: const Icon(Icons.notifications_none_rounded, color: Colors.grey));
              }),
          iconSize: 32.0,
        ),
        const MyCartButton(),
        const SizedBox(width: 8.0),
      ],
    )
  ],
);
</code>```

答:
From the official flutter documentation for sliverAppBar found here you could set the pinned property to true, floating to true, and snap to false.

here's a quick example:

```<code>SliverAppBar(
  expandedHeight: 150.0,
  pinned: true,
  snap: false,
  floating: true,
  flexibleSpace: const FlexibleSpaceBar(
    title: Text('Available seats'),
  ),
  actions: <Widget>[
    IconButton(
      icon: const Icon(Icons.add_circle),
      tooltip: 'Add new entry',
      onPressed: () { /* ... */ },
    ),
  ]
)
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I have a SliverAppBar with 'snap' and 'floating' both set to true. When the user scrolls down, the AppBar disappears, which is expected. The problem occurs when the user scrolls up and then taps on a button in the AppBar. As you can see from the GIF below, the AppBar snaps shut instead of going to the appropriate page. I noticed that this only happens if the scroll view is still moving when the user taps the button. Does anyone know how I can fix this?

SliverAppBar(
  snap: true,
  floating: true,
  backgroundColor: Colors.white,
  forceElevated: true,
  elevation: 2.0,
  shadowColor: Colors.black26,
  leading: IconButton(
    onPressed: () {
      Navigator.push(
        context,
        PageTransition(
          child: const MyAppBarMenu(),
          type: PageTransitionType.leftToRightJoined,
          duration: const Duration(milliseconds: 400),
          reverseDuration: const Duration(milliseconds: 150),
          curve: Curves.easeOutExpo,
          childCurrent: Container(),
        ),
      );
    },
    icon: ValueListenableBuilder(
        valueListenable: AppData.isLoggedIn,
        builder: (context, bool isLoggedIn, child) {
          return ValueListenableBuilder(
              valueListenable: AppData.userInfo,
              builder: (context, Map<String, dynamic> userInfo, _) {
                if (isLoggedIn) {
                  String imgUrl = '';
                  String points = '';
                  if (userInfo.isNotEmpty) {
                    imgUrl = '${AppData.domain}/VIEWUSER/${userInfo['Photo']}';
                    points = userInfo['points'] ?? '';
                  }
                  return Badge(
                    showBadge: (points != '' && points != '0'),
                    badgeColor: Colors.white,
                    shape: BadgeShape.square,
                    animationDuration: const Duration(seconds: 0),
                    padding: const EdgeInsets.symmetric(vertical: 1.0, horizontal: 2.0),
                    borderRadius: BorderRadius.circular(16.0),
                    position: BadgePosition.bottomEnd(bottom: -2.0, end: -12.0),
                    badgeContent: Padding(
                      padding: const EdgeInsets.symmetric(horizontal: 2.0),
                      child: Text(points, style: MyFonts.blackText12bold),
                    ),
                    borderSide: const BorderSide(width: 0.5, color: Colors.black54),
                    child: Builder(builder: (context) {
                      if (userInfo['Photo'] != '') {
                        return ClipOval(
                          child: CachedNetworkImage(
                            httpHeaders: AppData.myHeaders,
                            imageUrl: imgUrl,
                            placeholder: (context, url) => const Icon(Icons.account_circle, color: Colors.grey),
                            errorWidget: (context, url, error) => const Icon(Icons.account_circle, color: Colors.grey),
                          ),
                        );
                      } else {
                        return const Icon(Icons.account_circle, color: Colors.grey);
                      }
                    }),
                  );
                } else {
                  return const Icon(Icons.account_circle, color: Colors.grey);
                }
              });
        }),
    iconSize: 32.0,
  ),
  title: OutlinedButton(
    style: OutlinedButton.styleFrom(
      primary: MyColors.theme,
      backgroundColor: Colors.white,
      side: const BorderSide(color: Colors.grey, width: 1.0),
      shape: const RoundedRectangleBorder(borderRadius: BorderRadius.all(Radius.circular(32.0))),
    ),
    onPressed: () {
      Navigator.pushNamed(context, '/search');
    },
    child: Row(
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: const [
        Text(
          '搜尋',
          style: TextStyle(color: Colors.grey),
        ),
        Icon(
          Icons.search_rounded,
          color: Colors.grey,
        ),
      ],
    ),
  ),
  actions: [
    Row(
      children: [
        IconButton(
          onPressed: () => Navigator.pushNamed(context, '/news'),
          icon: ValueListenableBuilder(
              valueListenable: AppData.unreadNewsIds,
              builder: (context, Set<String> unreadNewsIds, _) {
                return Badge(
                    showBadge: unreadNewsIds.isNotEmpty,
                    position: BadgePosition.topEnd(top: -4.0, end: -4.0),
                    badgeContent: Text(
                      (unreadNewsIds.length > 9) ? unreadNewsIds.length.toString() : ' ${unreadNewsIds.length.toString()} ',
                      style: MyFonts.whiteText12bold,
                    ),
                    padding: const EdgeInsets.all(2.0),
                    child: const Icon(Icons.notifications_none_rounded, color: Colors.grey));
              }),
          iconSize: 32.0,
        ),
        const MyCartButton(),
        const SizedBox(width: 8.0),
      ],
    )
  ],
);

Body:
From the official flutter documentation for sliverAppBar found here you could set the pinned property to true, floating to true, and snap to false.

here's a quick example:

SliverAppBar(
  expandedHeight: 150.0,
  pinned: true,
  snap: false,
  floating: true,
  flexibleSpace: const FlexibleSpaceBar(
    title: Text('Available seats'),
  ),
  actions: <Widget>[
    IconButton(
      icon: const Icon(Icons.add_circle),
      tooltip: 'Add new entry',
      onPressed: () { /* ... */ },
    ),
  ]
)


问:
How does it work when executing the $regex query in Mongodb?
I have a question about Mongodb Index.

As far as I know, when the query is executed, it is found in the data sorted by the selected Index

```<code>db.HASH_TAG.find({"hashtag" : { $regex : "^Test"}}).sort({htseq:1}).hint("hashtag_1_htseq_1").explain("executionStats");
</code>```

========================= explain===============================

```<code>executionStages  :{
  "stage" : "SORT",
  "inputStage" : {
     "stage" : "SORT_KEY_GENERATOR",
     "inputStage" : {
             "stage" : "FETCH",
              "inputStage" : {
                        "stage" : "IXSCAN",
                        "indexName" : "hashtag_1_htseq_1",
                         "keyPattern" : {
                            "hashtag" : 1.0,
                            "htseq" : 1.0
                        },
                  }
          }
     }
}
</code>```

Please explain why SORT_KEY_GENERATOR occurs in the input stage.

答:
Your regular expression is not an equality match, therefore the query planner cannot guarantee ahead of time that the index will return the documents in sorted order.

The values in the index will be sorted by hashtag and then by htseq, the regex would match hashtags "Test", "Tester", "Tested", "Testing", etc., so the matched documents will still need to be sorted to be returned in order of htseq values.

You might be able to get index-supported sorting by anchoring both end of the regex like { $regex : "^Test$"}, but at that point you may as well just use equality match.

Body:
I have a question about Mongodb Index.

As far as I know, when the query is executed, it is found in the data sorted by the selected Index

db.HASH_TAG.find({"hashtag" : { $regex : "^Test"}}).sort({htseq:1}).hint("hashtag_1_htseq_1").explain("executionStats");

========================= explain===============================

executionStages  :{
  "stage" : "SORT",
  "inputStage" : {
     "stage" : "SORT_KEY_GENERATOR",
     "inputStage" : {
             "stage" : "FETCH",
              "inputStage" : {
                        "stage" : "IXSCAN",
                        "indexName" : "hashtag_1_htseq_1",
                         "keyPattern" : {
                            "hashtag" : 1.0,
                            "htseq" : 1.0
                        },
                  }
          }
     }
}

Please explain why SORT_KEY_GENERATOR occurs in the input stage.


问:
How to pass dictionary as command line argument to Python script?
How to pass dictionary as command line argument to Python script?
I need to get dictionary where key is string and value is list of some elements
– for example to look like:

```<code>command_line_arguments = {"names" : ["J.J.", "April"], "years" : [25, 29]}
</code>```

I have tried like

```<code>if __name__ == '__main__':
    args = dict([arg.split('=') for arg in sys.argv[2:]]) # also tried with 1 but doesn't work
    main(args)
</code>```

and I am calling script like

```<code>$ python saver.py names=["J.J.", "April"] years=[25, 29]
</code>```

but it doesn't work, dictionary has length 0 and need 2. Can anyone help me to pass and create dictionary in main.

答:
Pass the command line argument as below(mind the single and double inverted commas):

```<code>"{'k1':'v1','k2':['v1','v2',1,2,3]}"
</code>```

and then in your python code use eval() or ast.literal_eval()

```<code>import ast
if __name__=="__main__":
     cmd_arg=sys.argv[1]
     print(ast.literal_eval(cmd_arg))
     print(type(ast.literal_eval(cmd_arg)))

</code>```

Body:
How to pass dictionary as command line argument to Python script?
I need to get dictionary where key is string and value is list of some elements
– for example to look like:

command_line_arguments = {"names" : ["J.J.", "April"], "years" : [25, 29]}

I have tried like

if __name__ == '__main__':
    args = dict([arg.split('=') for arg in sys.argv[2:]]) # also tried with 1 but doesn't work
    main(args)

and I am calling script like

$ python saver.py names=["J.J.", "April"] years=[25, 29]

but it doesn't work, dictionary has length 0 and need 2. Can anyone help me to pass and create dictionary in main.

Body:
Pass the command line argument as below(mind the single and double inverted commas):

"{'k1':'v1','k2':['v1','v2',1,2,3]}"

and then in your python code use eval() or ast.literal_eval()

import ast
if __name__=="__main__":
     cmd_arg=sys.argv[1]
     print(ast.literal_eval(cmd_arg))
     print(type(ast.literal_eval(cmd_arg)))


问:
How to compare large text files?
I have a general question on your opinion about my "technique".

There are 2 textfiles (file_1 and file_2) that need to be compared to each other. Both are very huge (3-4 gigabytes, from 30,000,000 to 45,000,000 lines each).
My idea is to read several lines (as many as possible) of file_1 to the memory, then compare those to all lines of file_2. If there's a match, the lines from both files that match shall be written to a new file. Then go on with the next 1000 lines of file_1 and also compare those to all lines of file_2 until I went through file_1 completely.

But this sounds actually really, really time consuming and complicated to me.
Can you think of any other method to compare those two files?

How long do you think the comparison could take?
For my program, time does not matter that much. I have no experience in working with such huge files, therefore I have no idea how long this might take. It shouldn't take more than a day though. ;-) But I am afraid my technique could take forever...

Antoher question that just came to my mind: how many lines would you read into the memory? As many as possible? Is there a way to determine the number of possible lines before actually trying it?
I want to read as many as possible (because I think that's faster) but I've ran out of memory quite often.

Thanks in advance.

EDIT
I think I have to explain my problem a bit more.

The purpose is not to see if the two files in general are identical (they are not).
There are some lines in each file that share the same "characteristic".
Here's an example:
file_1 looks somewhat like this:

```<code>mat1 1000 2000 TEXT      //this means the range is from 1000 - 2000
mat1 2040 2050 TEXT
mat3 10000 10010 TEXT
mat2 20 500 TEXT
</code>```

file_2looks like this:

```<code>mat3 10009 TEXT
mat3 200 TEXT
mat1 999 TEXT
</code>```

TEXT refers to characters and digits that are of no interest for me, mat can go from mat1 - mat50 and are in no order; also there can be 1000x mat2 (but the numbers in the next column are different). I need to find the fitting lines in a way that: matX is the same in both compared lines an the number mentioned in file_2 fits into the range mentioned in file_1.
So in my example I would find one match: line 3 of file_1and line 1 of file_2 (because both are mat3 and 10009 is between 10000 and 10010).
I hope this makes it clear to you!

So my question is: how would you search for the matching lines?

Yes, I use Java as my programming language.

EDIT
I now divided the huge files first so that I have no problems with being out of memory. I also think it is faster to compare (many) smaller files to each other than those two huge files. After that I can compare them the way I mentioned above. It may not be the perfect way, but I am still learning ;-)
Nonentheless all your approaches were very helpful to me, thank you for your replies!

答:
I have never worked with such huge files but this is my idea and should work.

You could look into hash. Using SHA-1 Hashing.

Import the following

```<code>import java.io.FileInputStream;
import java.security.MessageDigest;
</code>```

Once your text file etc has been loaded have it loop through each line and at the end print out the hash. The example links below will go into more depth.

```<code>StringBuffer myBuffer = new StringBuffer("");
//For each line loop through
    for (int i = 0; i < mdbytes.length; i++) {
        myBuffer.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1));
    }
System.out.println("Computed Hash = " + sb.toString());
</code>```

SHA Code example focusing on Text File

SO Question about computing SHA in JAVA (Possibly helpful)

Another sample of hashing code.

Simple read each file seperatley, if the hash value for each file is the same at the end of the process then the two files are identical. If not then something is wrong.

Then if you get a different value you can do the super time consuming line by line check.

Overall, It seems that reading line by line by line by line etc would take forever. I would do this if you are trying to find each individual difference. But I think hashing would be quicker to see if they are the same.

SHA checksum

ContentLicense: CC BY-SA 3.0

Title: How to compare large text files?

Body:
I have a general question on your opinion about my "technique".

There are 2 textfiles (file_1 and file_2) that need to be compared to each other. Both are very huge (3-4 gigabytes, from 30,000,000 to 45,000,000 lines each).
My idea is to read several lines (as many as possible) of file_1 to the memory, then compare those to all lines of file_2. If there's a match, the lines from both files that match shall be written to a new file. Then go on with the next 1000 lines of file_1 and also compare those to all lines of file_2 until I went through file_1 completely.

But this sounds actually really, really time consuming and complicated to me.
Can you think of any other method to compare those two files?

How long do you think the comparison could take?
For my program, time does not matter that much. I have no experience in working with such huge files, therefore I have no idea how long this might take. It shouldn't take more than a day though. ;-) But I am afraid my technique could take forever...

Antoher question that just came to my mind: how many lines would you read into the memory? As many as possible? Is there a way to determine the number of possible lines before actually trying it?
I want to read as many as possible (because I think that's faster) but I've ran out of memory quite often.

Thanks in advance.

EDIT
I think I have to explain my problem a bit more.

The purpose is not to see if the two files in general are identical (they are not).
There are some lines in each file that share the same "characteristic".
Here's an example:
file_1 looks somewhat like this:

mat1 1000 2000 TEXT      //this means the range is from 1000 - 2000
mat1 2040 2050 TEXT
mat3 10000 10010 TEXT
mat2 20 500 TEXT

file_2looks like this:

mat3 10009 TEXT
mat3 200 TEXT
mat1 999 TEXT

TEXT refers to characters and digits that are of no interest for me, mat can go from mat1 - mat50 and are in no order; also there can be 1000x mat2 (but the numbers in the next column are different). I need to find the fitting lines in a way that: matX is the same in both compared lines an the number mentioned in file_2 fits into the range mentioned in file_1.
So in my example I would find one match: line 3 of file_1and line 1 of file_2 (because both are mat3 and 10009 is between 10000 and 10010).
I hope this makes it clear to you!

So my question is: how would you search for the matching lines?

Yes, I use Java as my programming language.

EDIT
I now divided the huge files first so that I have no problems with being out of memory. I also think it is faster to compare (many) smaller files to each other than those two huge files. After that I can compare them the way I mentioned above. It may not be the perfect way, but I am still learning ;-)
Nonentheless all your approaches were very helpful to me, thank you for your replies!

Body:
I have never worked with such huge files but this is my idea and should work.

You could look into hash. Using SHA-1 Hashing.

Import the following

import java.io.FileInputStream;
import java.security.MessageDigest;

Once your text file etc has been loaded have it loop through each line and at the end print out the hash. The example links below will go into more depth.

StringBuffer myBuffer = new StringBuffer("");
//For each line loop through
    for (int i = 0; i < mdbytes.length; i++) {
        myBuffer.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1));
    }
System.out.println("Computed Hash = " + sb.toString());

SHA Code example focusing on Text File

SO Question about computing SHA in JAVA (Possibly helpful)

Another sample of hashing code.

Simple read each file seperatley, if the hash value for each file is the same at the end of the process then the two files are identical. If not then something is wrong.

Then if you get a different value you can do the super time consuming line by line check.

Overall, It seems that reading line by line by line by line etc would take forever. I would do this if you are trying to find each individual difference. But I think hashing would be quicker to see if they are the same.

SHA checksum


问:
jQuery: how to sort list text taking into account numerical values
I am using the following script (jQuery 2.1.1) to sort a list with text and numerical values. In theory, sorting works because between "iPhone 11" and "iPhone 6", "iPhone 1" comes before "iPhone 6" but in the real world that does not make sense because 6<11 and "iPhone 6"<"iPhone 11".

```<code>$('ul').children("li").sort(function(a, b) {
var A = $(a).text().toUpperCase();
var B = $(b).text().toUpperCase();
return (A < B) ? -1 : (A > B) ? 1 : 0;
}).appendTo('ul');
</code>```

How sorting should be:

```<code>iPhone 6
iPhone 8
iPhone 11
iPhone 12
samsung galaxy s8
samsung galaxy s10
samsung galaxy s20
</code>```

what sorting returns:

```<code>iPhone 11
iPhone 12
iPhone 6
iPhone 8
samsung galaxy s10
samsung galaxy s20
samsung galaxy s8
</code>```

What would the best approach be?
I am thinking of stripping all text and sort based on the integers left but if I did that, in the current list that would make the "galaxy" models appear between "iphone" models (which also makes no sense) like so:

```<code>iPhone 6
iPhone 8
samsung galaxy s8
samsung galaxy s10
iPhone 11
iPhone 12
samsung galaxy s20
</code>```

答:
Since you have to sort the text where it includes numbers, then do:

```<code>return A.localeCompare(B, false, {
  numeric: true
})
</code>```

You can find more information on localCompare and the options here

So you code will look like:

```<code>$('ul').children("li").sort(function(a, b) {
  var A = $(a).text().toUpperCase();
  var B = $(b).text().toUpperCase();
  return A.localeCompare(B, false, {
    numeric: true
  })
}).appendTo('ul');
</code>```

Demo

$('ul').children("li").sort(function(a, b) {
  var A = $(a).text().toUpperCase();
  var B = $(b).text().toUpperCase();
  return A.localeCompare(B, false, {
    numeric: true
  })
}).appendTo('ul');```
<pre class="snippet-code-html lang-html prettyprint-override"><code><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<ul>
  <li>iPhone 6</li>
  <li>iPhone 8</li>
  <li>samsung galaxy s8</li>
  <li>samsung galaxy s10</li>
  <li>iPhone 11</li>
  <li>iPhone 12</li>
  <li>samsung galaxy s20</li>
</ul></code>```

Body:
I am using the following script (jQuery 2.1.1) to sort a list with text and numerical values. In theory, sorting works because between "iPhone 11" and "iPhone 6", "iPhone 1" comes before "iPhone 6" but in the real world that does not make sense because 6<11 and "iPhone 6"<"iPhone 11".

$('ul').children("li").sort(function(a, b) {
var A = $(a).text().toUpperCase();
var B = $(b).text().toUpperCase();
return (A  B) ? 1 : 0;
}).appendTo('ul');

How sorting should be:

iPhone 6
iPhone 8
iPhone 11
iPhone 12
samsung galaxy s8
samsung galaxy s10
samsung galaxy s20

what sorting returns:

iPhone 11
iPhone 12
iPhone 6
iPhone 8
samsung galaxy s10
samsung galaxy s20
samsung galaxy s8

What would the best approach be?
I am thinking of stripping all text and sort based on the integers left but if I did that, in the current list that would make the "galaxy" models appear between "iphone" models (which also makes no sense) like so:

iPhone 6
iPhone 8
samsung galaxy s8
samsung galaxy s10
iPhone 11
iPhone 12
samsung galaxy s20

Body:
Since you have to sort the text where it includes numbers, then do:

return A.localeCompare(B, false, {
  numeric: true
})

You can find more information on localCompare and the options here

So you code will look like:

$('ul').children("li").sort(function(a, b) {
  var A = $(a).text().toUpperCase();
  var B = $(b).text().toUpperCase();
  return A.localeCompare(B, false, {
    numeric: true
  })
}).appendTo('ul');

Demo

$('ul').children("li").sort(function(a, b) {
  var A = $(a).text().toUpperCase();
  var B = $(b).text().toUpperCase();
  return A.localeCompare(B, false, {
    numeric: true
  })
}).appendTo('ul');

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<ul>
  <li>iPhone 6</li>
  <li>iPhone 8</li>
  <li>samsung galaxy s8</li>
  <li>samsung galaxy s10</li>
  <li>iPhone 11</li>
  <li>iPhone 12</li>
  <li>samsung galaxy s20</li>
</ul>


问:
Node.js: How to better write a MySQL batch operation code without too many callbacks?
I am not coming from a javascript background, however involved in writing a Node.js backend. There are situations where I have to do multiple database entry in a row and rollback if they failed. Below is such an example. (Node.js v14.17.5)

```<code>const mysql = require('mysql2');
const errorCodes = require('source/error-codes');
const PropertiesReader = require('properties-reader');

const prop = PropertiesReader('properties.properties');

const con = mysql.createConnection({
    host: prop.get('server.host'),
    user: prop.get("server.username"),
    password: prop.get("server.password"),
    port: prop.get("server.port"),
    database: prop.get("server.dbname")
});

exports.createExtraGig = (event, context, callback) => {

    context.callbackWaitsForEmptyEventLoop = false;

    if (event.body == null && event.body == undefined) {
        var response = errorCodes.missing_parameters;
        callback(null, response)
    } else {
        let body = JSON.parse(event.body)
        console.log("body", body);

        // Check for validations
        if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null
        ) {
            console.log("fire 1");
            var response = errorCodes.not_null_parameters;
            callback(null, response)
        } else {
            if (
                isNaN(body.idjob_room) ||
                isNaN(body.idgig_fund_status) ||
                isNaN(body.cost) ||
                isNaN(body.idjob) ||
                isNaN(body.end_date)) {
                var response = errorCodes.missing_fields;
                callback(null, response)
            }
        }

        // Validations passed. Proceed
        let idjob_room = Number(body.idjob_room);
        let idgig_fund_status = Number(body.idgig_fund_status);
        let title = body.title;
        let cost = Number(body.cost);
        let end_date = new Date(body.end_date);
        let idjob = Number(body.idjob);

        console.log("title: " + title);

        con.beginTransaction(function(err){
            if (err) {
                var response = errorCodes.save_failed;
                callback(null, response);
            }

            const sql = "INSERT INTO gig_service (idjob_room, idgig_fund_status, title, cost, end_date, is_deleted, is_main_offer) VALUES(?,?,?,?,?,?,?)";
            con.query(sql, [idjob_room, idgig_fund_status, title, cost, end_date, false, false], function(err, gigResult) {
                if (err) {
                    con.rollback(function() {
                        var response = errorCodes.save_failed;
                        callback(null, response);
                    });

                }

                const jobUpdateSql = "UPDATE job SET end_date=? WHERE idjob=?";
                con.execute(jobUpdateSql, [end_date, idjob], function(err, jobResult) {

                    con.commit(function(err) {
                        if (err) {
                            return con.rollback(function() {
                                var response = errorCodes.save_failed;
                                callback(null, response);
                            });
                        }
                        console.log('success!');

                        var response = {
                            "statusCode": 200,
                            "headers": {
                                "Content-Type": "application/json"
                            },
                            "body": JSON.stringify({
                                "idgig_service": gigResult.insertId
                            }),
                            "isBase64Encoded": false
                        };
                        callback(null, response)
                    });

                });
            });
        }

        )}
};
</code>```

However this is a maintenance hell, because the more transactions I have, more if-else conditions and more callbacks i have to make.

Looking at posts such as this tells me there are better ways, but i am really not sure.

How can I better write my Node.js code without having too many rollback callbacks? Like one rollback callback is "any" fails?

(Note: exports.createExtraGig = (event, context, callback) =>  line is important because this is an aws lambda function)

UPDATE

Below is the updated code

```<code>const mysql = require('mysql2');
const errorCodes = require('source/error-codes');
const PropertiesReader = require('properties-reader');

const prop = PropertiesReader('properties.properties');

const con =  mysql.createConnection({
    host: prop.get('server.host'),
    user: prop.get("server.username"),
    password: prop.get("server.password"),
    port: prop.get("server.port"),
    database: prop.get("server.dbname")
});

exports.createExtraGig = async (event, context)=>{
    context.callbackWaitsForEmptyEventLoop = false;

    if (event.body == null && event.body == undefined) {
        var response = errorCodes.missing_parameters;
        return response;
    } else {
        let body = JSON.parse(event.body)
        console.log("body", body);

        // Check for validations
        if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null
        ) {
            console.log("fire 1");
            var response = errorCodes.not_null_parameters;
            return response;
        } else {
            if (
                isNaN(body.idjob_room) ||
                isNaN(body.idgig_fund_status) ||
                isNaN(body.cost) ||
                isNaN(body.idjob) ||
                isNaN(body.end_date)) {
                var response = errorCodes.missing_fields;
                return response;
            }
        }

        // Validations passed. Proceed
        let idjob_room = Number(body.idjob_room);
        let idgig_fund_status = Number(body.idgig_fund_status);
        let title = body.title;
        let cost = Number(body.cost);
        let end_date = new Date(body.end_date);
        let idjob = Number(body.idjob);

        console.log("title: " + title);

        try {
            await con.promise().beginTransaction();

            const sql = "INSERT INTO gig_service (idjob_room, idgig_fund_status, title, cost, end_date, is_deleted, is_main_offer) VALUES(?,?,?,?,?,?,?)";
            const gigResult = await con.promise().query(sql,[idjob_room, idgig_fund_status, title, cost, end_date, false, false]);
            console.log(gigResult);

            const jobUpdateSql = "UPDATE job SET end_date=? WHERE idjob=?";
            await con.promise().query(jobUpdateSql,[end_date, idjob]);
            await con.promise().commit();

            var response = {
                "statusCode": 200,
                "headers": {
                    "Content-Type": "application/json"
                },
                "body": JSON.stringify({
                    "idgig_service": gigResult.insertId
                }),
                "isBase64Encoded": false
            };
            return response;

          } catch (error) {
              console.log(error);
              await con.promise().rollback();
              return  errorCodes.save_failed;
          }
        }
};
</code>```

答:
You can use promise with mysql.

```<code>const mysql = require('mysql2/promise');

// get the promise implementation, we will use bluebird
const bluebird = require('bluebird');

// create the connection, specify bluebird as Promise
const connection = await mysql.createConnection({host:'localhost', user: 'root', database: 'test', Promise: bluebird});
</code>```

Then... (Ref: Async/Await )

```<code>try {

  const resultsFromQuery1 = await con.query(sql1)
  const resultsFromQuery2 = await con.query(sql2)

} catch (err) {
 con.rollback(function() {
      var response = errorCodes.save_failed;
      callback(null, response);
                    });
}
</code>```

You can also make your codes cleaner by using helpers like .some or .map

```<code>   if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null)

   //and the following chunk

      isNaN(body.idjob_room) ||
            isNaN(body.idgig_fund_status) ||
            isNaN(body.cost) ||
            isNaN(body.idjob) ||
            isNaN(body.end_date))
</code>```

can be written like

```<code>   let { idgig_fund_status, title, cost, idjob, end_date } = body;
   if ( [idgig_fund_status, title, cost, idjob, end_date].some(x => x === null )) {
        var response = errorCodes.not_null_parameters;
        callback(null, response)
   } //can write similar for isNan checks
</code>```

The following can also be written

```<code>    let idjob_room = Number(body.idjob_room);
    let idgig_fund_status = Number(body.idgig_fund_status);
    let title = body.title;
    let cost = Number(body.cost);
    let end_date = new Date(body.end_date);
    let idjob = Number(body.idjob);

 //becomes...

 [idjob_room, idgig_fund_status, cost, idjob] = [idgig_fund_status, cost, end_date, idjob].map(x => x+);
 end_date = new Date(end)date;
 //title is already deconstructed above.

</code>```

Body:
I am not coming from a javascript background, however involved in writing a Node.js backend. There are situations where I have to do multiple database entry in a row and rollback if they failed. Below is such an example. (Node.js v14.17.5)

const mysql = require('mysql2');
const errorCodes = require('source/error-codes');
const PropertiesReader = require('properties-reader');

const prop = PropertiesReader('properties.properties');

const con = mysql.createConnection({
    host: prop.get('server.host'),
    user: prop.get("server.username"),
    password: prop.get("server.password"),
    port: prop.get("server.port"),
    database: prop.get("server.dbname")
});

exports.createExtraGig = (event, context, callback) => {

    context.callbackWaitsForEmptyEventLoop = false;

    if (event.body == null && event.body == undefined) {
        var response = errorCodes.missing_parameters;
        callback(null, response)
    } else {
        let body = JSON.parse(event.body)
        console.log("body", body);

        // Check for validations
        if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null
        ) {
            console.log("fire 1");
            var response = errorCodes.not_null_parameters;
            callback(null, response)
        } else {
            if (
                isNaN(body.idjob_room) ||
                isNaN(body.idgig_fund_status) ||
                isNaN(body.cost) ||
                isNaN(body.idjob) ||
                isNaN(body.end_date)) {
                var response = errorCodes.missing_fields;
                callback(null, response)
            }
        }

        // Validations passed. Proceed
        let idjob_room = Number(body.idjob_room);
        let idgig_fund_status = Number(body.idgig_fund_status);
        let title = body.title;
        let cost = Number(body.cost);
        let end_date = new Date(body.end_date);
        let idjob = Number(body.idjob);

        console.log("title: " + title);

        con.beginTransaction(function(err){
            if (err) {
                var response = errorCodes.save_failed;
                callback(null, response);
            }

            const sql = "INSERT INTO gig_service (idjob_room, idgig_fund_status, title, cost, end_date, is_deleted, is_main_offer) VALUES(?,?,?,?,?,?,?)";
            con.query(sql, [idjob_room, idgig_fund_status, title, cost, end_date, false, false], function(err, gigResult) {
                if (err) {
                    con.rollback(function() {
                        var response = errorCodes.save_failed;
                        callback(null, response);
                    });

                }

                const jobUpdateSql = "UPDATE job SET end_date=? WHERE idjob=?";
                con.execute(jobUpdateSql, [end_date, idjob], function(err, jobResult) {

                    con.commit(function(err) {
                        if (err) {
                            return con.rollback(function() {
                                var response = errorCodes.save_failed;
                                callback(null, response);
                            });
                        }
                        console.log('success!');

                        var response = {
                            "statusCode": 200,
                            "headers": {
                                "Content-Type": "application/json"
                            },
                            "body": JSON.stringify({
                                "idgig_service": gigResult.insertId
                            }),
                            "isBase64Encoded": false
                        };
                        callback(null, response)
                    });

                });
            });
        }

        )}
};

However this is a maintenance hell, because the more transactions I have, more if-else conditions and more callbacks i have to make.

Looking at posts such as this tells me there are better ways, but i am really not sure.

How can I better write my Node.js code without having too many rollback callbacks? Like one rollback callback is "any" fails?

(Note: exports.createExtraGig = (event, context, callback) =>  line is important because this is an aws lambda function)

UPDATE

Below is the updated code

const mysql = require('mysql2');
const errorCodes = require('source/error-codes');
const PropertiesReader = require('properties-reader');

const prop = PropertiesReader('properties.properties');

const con =  mysql.createConnection({
    host: prop.get('server.host'),
    user: prop.get("server.username"),
    password: prop.get("server.password"),
    port: prop.get("server.port"),
    database: prop.get("server.dbname")
});

exports.createExtraGig = async (event, context)=>{
    context.callbackWaitsForEmptyEventLoop = false;

    if (event.body == null && event.body == undefined) {
        var response = errorCodes.missing_parameters;
        return response;
    } else {
        let body = JSON.parse(event.body)
        console.log("body", body);

        // Check for validations
        if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null
        ) {
            console.log("fire 1");
            var response = errorCodes.not_null_parameters;
            return response;
        } else {
            if (
                isNaN(body.idjob_room) ||
                isNaN(body.idgig_fund_status) ||
                isNaN(body.cost) ||
                isNaN(body.idjob) ||
                isNaN(body.end_date)) {
                var response = errorCodes.missing_fields;
                return response;
            }
        }

        // Validations passed. Proceed
        let idjob_room = Number(body.idjob_room);
        let idgig_fund_status = Number(body.idgig_fund_status);
        let title = body.title;
        let cost = Number(body.cost);
        let end_date = new Date(body.end_date);
        let idjob = Number(body.idjob);

        console.log("title: " + title);

        try {
            await con.promise().beginTransaction();

            const sql = "INSERT INTO gig_service (idjob_room, idgig_fund_status, title, cost, end_date, is_deleted, is_main_offer) VALUES(?,?,?,?,?,?,?)";
            const gigResult = await con.promise().query(sql,[idjob_room, idgig_fund_status, title, cost, end_date, false, false]);
            console.log(gigResult);

            const jobUpdateSql = "UPDATE job SET end_date=? WHERE idjob=?";
            await con.promise().query(jobUpdateSql,[end_date, idjob]);
            await con.promise().commit();

            var response = {
                "statusCode": 200,
                "headers": {
                    "Content-Type": "application/json"
                },
                "body": JSON.stringify({
                    "idgig_service": gigResult.insertId
                }),
                "isBase64Encoded": false
            };
            return response;

          } catch (error) {
              console.log(error);
              await con.promise().rollback();
              return  errorCodes.save_failed;
          }
        }
};

Body:
You can use promise with mysql.

const mysql = require('mysql2/promise');

// get the promise implementation, we will use bluebird
const bluebird = require('bluebird');

// create the connection, specify bluebird as Promise
const connection = await mysql.createConnection({host:'localhost', user: 'root', database: 'test', Promise: bluebird});

Then... (Ref: Async/Await )

try {

  const resultsFromQuery1 = await con.query(sql1)
  const resultsFromQuery2 = await con.query(sql2)

} catch (err) {
 con.rollback(function() {
      var response = errorCodes.save_failed;
      callback(null, response);
                    });
}

You can also make your codes cleaner by using helpers like .some or .map

   if (body.idjob_room == null ||
            body.idgig_fund_status == null ||
            body.title == null ||
            body.cost == null ||
            body.idjob == null ||
            body.end_date == null)

   //and the following chunk

      isNaN(body.idjob_room) ||
            isNaN(body.idgig_fund_status) ||
            isNaN(body.cost) ||
            isNaN(body.idjob) ||
            isNaN(body.end_date))

can be written like

   let { idgig_fund_status, title, cost, idjob, end_date } = body;
   if ( [idgig_fund_status, title, cost, idjob, end_date].some(x => x === null )) {
        var response = errorCodes.not_null_parameters;
        callback(null, response)
   } //can write similar for isNan checks

The following can also be written

    let idjob_room = Number(body.idjob_room);
    let idgig_fund_status = Number(body.idgig_fund_status);
    let title = body.title;
    let cost = Number(body.cost);
    let end_date = new Date(body.end_date);
    let idjob = Number(body.idjob);

 //becomes...

 [idjob_room, idgig_fund_status, cost, idjob] = [idgig_fund_status, cost, end_date, idjob].map(x => x+);
 end_date = new Date(end)date;
 //title is already deconstructed above.


问:
verify that CUBLAS is installed
How to check if cuBLAS is installed. Is there a simple way to do it using command line without actually running any line of cuda code</p>

答:
On Windows 10, it's in file C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include\cublas_api.h

for example:

#define CUBLAS_VER_MAJOR 10
#define CUBLAS_VER_MINOR 2
#define CUBLAS_VER_PATCH 3
#define CUBLAS_VER_BUILD 254
#define CUBLAS_VERSION  (CUBLAS_VER_MAJOR * 1000 + \
                         CUBLAS_VER_MINOR *  100 + \
                         CUBLAS_VER_PATCH)
```

It shows cublas is 10.2.3 build 254

Title: verify that CUBLAS is installed

Body:
On Windows 10, it's in file C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include\cublas_api.h

for example:

#define CUBLAS_VER_MAJOR 10
#define CUBLAS_VER_MINOR 2
#define CUBLAS_VER_PATCH 3
#define CUBLAS_VER_BUILD 254
#define CUBLAS_VERSION  (CUBLAS_VER_MAJOR * 1000 + \
                         CUBLAS_VER_MINOR *  100 + \
                         CUBLAS_VER_PATCH)

It shows cublas is 10.2.3 build 254


问:
Defining Rules in Scrapy while accessing through flask
I am running scrapy spider using flask and crochet. In this I am using Rule Link extractor for defining the rules. In the rule, I am setting allow_domains which is passed from flask application.

spider.py

```<code>class myCrawler(CrawlSpider):
    name = 'symphony'
    base_url=''
    start_urls = []
    allowed_domains = ''

    def __init__(self, category='', **kwargs):
        super().__init__(**kwargs)
        self.base_url = category
        self.allowed_domains = ['.'.join(urlparse(self.base_url).netloc.split('.')[-2:])]
        self.start_urls.append(self.base_url)
        print(f"Base url is {self.base_url} and allowed domain is {self.allowed_domains}")

    custom_settings = {
        # in order to reduce the risk of getting blocked
        'DOWNLOADER_MIDDLEWARES': {'sitescrapper.sitescrapper.middlewares.RotateUserAgentMiddleware': 400, },
        'COOKIES_ENABLED': False,
        'CONCURRENT_REQUESTS': 6,
        'DOWNLOAD_DELAY': 2,
        # Duplicates pipeline
        'ITEM_PIPELINES': {'sitescrapper.sitescrapper.pipelines.DuplicatesPipeline': 300},

        # In order to create a CSV file:
        'FEEDS': {'csv_file.csv': {'format': 'csv'}}
    }

    rules = (
        Rule(
            LinkExtractor(allow_domains='.'.join(urlparse(self.base_url).netloc.split('.')[-2:])),
            process_links=process_links,
            callback='parse_item',
            follow=True
        ),
    )
</code>```

Here I gave LinkExtractor(allow_domains='.'.join(urlparse(self.base_url).netloc.split('.')[-2:])). But self is not defined there which throws an error. So how can I assign the value of expression '.'.join(urlparse(self.base_url).netloc.split('.')[-2:]) to the variable allow_domains [which is also same as self.allowed_domains] Or else is there any better method to do this

答:
The problem here is that CrawlSpider constructor (init) is also handling the rules parameter, so if we need to assign them, we'll have to do it before calling the default constructor.

```<code>class myCrawler(CrawlSpider):
    name = 'symphony'
    rotate_user_agent = True
    base_url=''
    start_urls = []
    allowed_domains = ''

    def __init__(self, category='', **kwargs):
        self.base_url = category
        self.allowed_domains = ['.'.join(urlparse(self.base_url).netloc.split('.')[-2:])]
        self.start_urls.append(self.base_url)
        print(f"Base url is {self.base_url} and allowed domain is {self.allowed_domains}")

        self.rules = (
            Rule(
                LinkExtractor(allow_domains=self.allowed_domains),
                process_links=process_links,
                callback='parse_item',
                follow=True
            ),
        )
        super().__init__(**kwargs)

    custom_settings = {
            # in order to reduce the risk of getting blocked
            'DOWNLOADER_MIDDLEWARES': {'sitescrapper.sitescrapper.middlewares.RotateUserAgentMiddleware': 400, },
            'COOKIES_ENABLED': False,
            'CONCURRENT_REQUESTS': 6,
            'DOWNLOAD_DELAY': 2,

            # Duplicates pipeline
            'ITEM_PIPELINES': {'sitescrapper.sitescrapper.pipelines.DuplicatesPipeline': 300},

            # In order to create a CSV file:
            'FEEDS': {'csv_file.csv': {'format': 'csv'}}
        }
</code>```

Body:
I am running scrapy spider using flask and crochet. In this I am using Rule Link extractor for defining the rules. In the rule, I am setting allow_domains which is passed from flask application.

spider.py

class myCrawler(CrawlSpider):
    name = 'symphony'
    base_url=''
    start_urls = []
    allowed_domains = ''

    def __init__(self, category='', **kwargs):
        super().__init__(**kwargs)
        self.base_url = category
        self.allowed_domains = ['.'.join(urlparse(self.base_url).netloc.split('.')[-2:])]
        self.start_urls.append(self.base_url)
        print(f"Base url is {self.base_url} and allowed domain is {self.allowed_domains}")

    custom_settings = {
        # in order to reduce the risk of getting blocked
        'DOWNLOADER_MIDDLEWARES': {'sitescrapper.sitescrapper.middlewares.RotateUserAgentMiddleware': 400, },
        'COOKIES_ENABLED': False,
        'CONCURRENT_REQUESTS': 6,
        'DOWNLOAD_DELAY': 2,
        # Duplicates pipeline
        'ITEM_PIPELINES': {'sitescrapper.sitescrapper.pipelines.DuplicatesPipeline': 300},

        # In order to create a CSV file:
        'FEEDS': {'csv_file.csv': {'format': 'csv'}}
    }

    rules = (
        Rule(
            LinkExtractor(allow_domains='.'.join(urlparse(self.base_url).netloc.split('.')[-2:])),
            process_links=process_links,
            callback='parse_item',
            follow=True
        ),
    )

Here I gave LinkExtractor(allow_domains='.'.join(urlparse(self.base_url).netloc.split('.')[-2:])). But self is not defined there which throws an error. So how can I assign the value of expression '.'.join(urlparse(self.base_url).netloc.split('.')[-2:]) to the variable allow_domains [which is also same as self.allowed_domains] Or else is there any better method to do this

Body:
The problem here is that CrawlSpider constructor (init) is also handling the rules parameter, so if we need to assign them, we'll have to do it before calling the default constructor.

class myCrawler(CrawlSpider):
    name = 'symphony'
    rotate_user_agent = True
    base_url=''
    start_urls = []
    allowed_domains = ''

    def __init__(self, category='', **kwargs):
        self.base_url = category
        self.allowed_domains = ['.'.join(urlparse(self.base_url).netloc.split('.')[-2:])]
        self.start_urls.append(self.base_url)
        print(f"Base url is {self.base_url} and allowed domain is {self.allowed_domains}")

        self.rules = (
            Rule(
                LinkExtractor(allow_domains=self.allowed_domains),
                process_links=process_links,
                callback='parse_item',
                follow=True
            ),
        )
        super().__init__(**kwargs)

    custom_settings = {
            # in order to reduce the risk of getting blocked
            'DOWNLOADER_MIDDLEWARES': {'sitescrapper.sitescrapper.middlewares.RotateUserAgentMiddleware': 400, },
            'COOKIES_ENABLED': False,
            'CONCURRENT_REQUESTS': 6,
            'DOWNLOAD_DELAY': 2,

            # Duplicates pipeline
            'ITEM_PIPELINES': {'sitescrapper.sitescrapper.pipelines.DuplicatesPipeline': 300},

            # In order to create a CSV file:
            'FEEDS': {'csv_file.csv': {'format': 'csv'}}
        }


问:
Use cssText to change child style from parent
I have a data object that contains data to find a specific element that exist within a DOM Tree and after I find that element I want to change it css by adding inline css.

```<code>    {
      outerHtml: '<a class="btn btn-pill">\n                        <svg xmlns="http://www.w3.org/2000/svg" class="icon" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"><path stroke="none" d="M0 0h24v24H0z" fill="none"></path>',
      selector: ".col-auto.ml-auto.d-print-none>.d-flex>.btn.btn-pill",
      style: `background-color: #206bc4; color: white; svg { color: white }`,
    }

</code>```

I'm utilizing this data and injecting the style data in to the element once I find it. As you can see in my outerHTML, the parent prop also has a child and in my style I have the css to change the svg color as well. But when using my code below. It only changes the style of the a tag as opposed to the svg as well.

Note: the outerHTML & selector are just for finding the correct element to change. Sort of like unique identifiers

All I'm doing in the below code is using my selectors to find the correct element and then add the CSS to the final node.

```<code>export function findAndInject(remove) {
  const data = getData();
  let selector = data.selector.split(',');
  let select;
  for (const prop in selector) {
    if (prop == 0) {
      select = document.querySelector(selector[prop]);
    } else {
      select = select.querySelector(selector[prop]);
    }
  }
  if (remove) {
    select.style.cssText = null;
  } else {
    select.style.cssText = data.style;
  }
}
</code>```

I know the issue is probably that I'm doing style.cssText but is there any other way I can add an inline style that can also be added to the child without creating a stylesheet or using style tags?

Before Injection:

After Injection:

The Plus icon should be white.

答:
The short answer is: No, it is not possible to add inline styles that can also be applied to the child.  For a good explanation as to why "why", you can refer to this post (https://stackoverflow.com/a/5293299/3180489) which succinctly explains (with references):

Neither selectors (including pseudo-elements), nor at-rules, nor any
other CSS construct are allowed.

Think of inline styles as the styles applied to some anonymous
super-specific ID selector: those styles only apply to that one very
element with the style attribute. (They take precedence over an ID
selector in a stylesheet too, if that element has that ID.)
Technically it doesn't work like that; this is just to help you
understand why the attribute doesn't support pseudo-class or
pseudo-element styles (it has more to do with how pseudo-classes and
pseudo-elements provide abstractions of the document tree that can't
be expressed in the document language).

With that being said, for this particular case you may be able to modify your SVG icons to "inherit" the color from the parent element, if you choose to go that route.

Body:
I have a data object that contains data to find a specific element that exist within a DOM Tree and after I find that element I want to change it css by adding inline css.

    {
      outerHtml: '\n                        <svg xmlns="http://www.w3.org/2000/svg" class="icon" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"><path stroke="none" d="M0 0h24v24H0z" fill="none"></path>',
      selector: ".col-auto.ml-auto.d-print-none>.d-flex>.btn.btn-pill",
      style: `background-color: #206bc4; color: white; svg { color: white }`,
    }

I'm utilizing this data and injecting the style data in to the element once I find it. As you can see in my outerHTML, the parent prop also has a child and in my style I have the css to change the svg color as well. But when using my code below. It only changes the style of the a tag as opposed to the svg as well.

Note: the outerHTML & selector are just for finding the correct element to change. Sort of like unique identifiers

All I'm doing in the below code is using my selectors to find the correct element and then add the CSS to the final node.

export function findAndInject(remove) {
  const data = getData();
  let selector = data.selector.split(',');
  let select;
  for (const prop in selector) {
    if (prop == 0) {
      select = document.querySelector(selector[prop]);
    } else {
      select = select.querySelector(selector[prop]);
    }
  }
  if (remove) {
    select.style.cssText = null;
  } else {
    select.style.cssText = data.style;
  }
}

I know the issue is probably that I'm doing style.cssText but is there any other way I can add an inline style that can also be added to the child without creating a stylesheet or using style tags?

Before Injection:

After Injection:

The Plus icon should be white.


问:
Simultaneous animation when entering editing mode of UITableViewCell
when entering the editing mode in a UITableViewCell through setEditing:animated:, there is an (implicit) animation where the cells move right and the delete "buttons" appear at the left.

How do I manage, probably with Core Animation features, to run a second animation simultaneously to the first animation when entering editing mode, i.e. with the the same starting time, duration, timing curve, and so on? (I'm actually trying to change the width of a cell's sublayer synchronously.)

I studied the Core Animation Programming Guide and tried some of the techniques in a custom UITableViewCell class, for example by overriding willTransitionToState:. In principle, everything works well, however, I can only manage to have the animations one after the other, but not simultaneously.

Any advice is greatly appreciated.

答:
Alright, it's much simpler than I initially thought. The behavior I sought can rather easily be achieved by correctly initializing the layers (or views) in the custom UITableViewCell. However, I realized that the order and, e.g., whether you manipulate bounds or frame properties is quite important.

The following code snippet does the job for a cell in a grouped UITableView. In this example, a CAGradientLayer (as an example for a CALayer) is added as a sublayer to a UIView, which itself is added as a subview to self.contentView of the cell. The subview's frame is tweaked to fit into the contentView. The code goes, e.g., into initWithStyle:reuseIdentifier: of a custom UITableViewCell:

```<code>CGRect realFrame = self.contentView.frame;
realFrame.size.width += 20;
UIView *gradientView = [[UIView alloc] initWithFrame:realFrame];

CAGradientLayer *gradientLayer = [[CAGradientLayer alloc] init];
gradientLayer.anchorPoint = CGPointMake(0.0, 0.0);
gradientLayer.frame = gradientView.frame;
gradientLayer.position = CGPointMake(0.0, 0.0);
[gradientLayer setColors:[NSArray arrayWithObjects:(id)[[UIColor colorWithRed:0.0 green:0.0 blue:0.0 alpha:1.0] CGColor], (id)[[UIColor clearColor] CGColor], nil]];
gradientLayer.startPoint = CGPointMake(0.5, 0.0);
gradientLayer.endPoint = CGPointMake(0.5, 1.0);

gradientView.layer.masksToBounds = YES;
gradientView.layer.cornerRadius = 8.0;

[gradientView.layer insertSublayer:gradientLayer atIndex:0];
[self.contentView addSubview:gradientView];

gradientView.autoresizingMask = UIViewAutoresizingFlexibleWidth;

[gradientView release];
[gradientLayer release];
</code>```

Body:
Alright, it's much simpler than I initially thought. The behavior I sought can rather easily be achieved by correctly initializing the layers (or views) in the custom UITableViewCell. However, I realized that the order and, e.g., whether you manipulate bounds or frame properties is quite important.

The following code snippet does the job for a cell in a grouped UITableView. In this example, a CAGradientLayer (as an example for a CALayer) is added as a sublayer to a UIView, which itself is added as a subview to self.contentView of the cell. The subview's frame is tweaked to fit into the contentView. The code goes, e.g., into initWithStyle:reuseIdentifier: of a custom UITableViewCell:

CGRect realFrame = self.contentView.frame;
realFrame.size.width += 20;
UIView *gradientView = [[UIView alloc] initWithFrame:realFrame];

CAGradientLayer *gradientLayer = [[CAGradientLayer alloc] init];
gradientLayer.anchorPoint = CGPointMake(0.0, 0.0);
gradientLayer.frame = gradientView.frame;
gradientLayer.position = CGPointMake(0.0, 0.0);
[gradientLayer setColors:[NSArray arrayWithObjects:(id)[[UIColor colorWithRed:0.0 green:0.0 blue:0.0 alpha:1.0] CGColor], (id)[[UIColor clearColor] CGColor], nil]];
gradientLayer.startPoint = CGPointMake(0.5, 0.0);
gradientLayer.endPoint = CGPointMake(0.5, 1.0);

gradientView.layer.masksToBounds = YES;
gradientView.layer.cornerRadius = 8.0;

[gradientView.layer insertSublayer:gradientLayer atIndex:0];
[self.contentView addSubview:gradientView];

gradientView.autoresizingMask = UIViewAutoresizingFlexibleWidth;

[gradientView release];
[gradientLayer release];


问:
Invalid Tilemap Layer ID
I was doing my game in phaser 3 when I went to add a tilemap created in tiled, I followed several tutorials, I did and redid the code several times and even then it always gives me the same invalid ID error.

unsuccessful attempts:

Tried to manually make the tilemap;

Tried using JSON files;

I tried to use a single layer;

I tried to use the names according to the tiled layers;

I tried using different names;

here follows the code:

```<code>import './phaser'

const config = {
  type: Phaser.AUTO,
  parent: "phaser-example",
  width: 1824,
  height: 960,
  physics: {
      default: 'arcade'
  },
  scene: {
      preload: preload,
      create: create
  }
};

export const game = new Phaser.Game(config)

function preload()
{
  this.load.image('terrain', './complementos/tilemaps/terrainBasic.png')
  this.load.tilemapTiledJSON('map', './complementos/tilemaps/basicLand.json')
}

function create()
{
  const teste = [[0,1,2],[0,1,2],[0,1,2]]

  const map = this.make.tilemap(teste)

  map.addTilesetImage('terrainBasic','terrain', 32, 32)

  const layer = map.createLayer(0,'land', 0, 0)
}
</code>```

I don't know if it changes much but I was using react together with phaser 3, but my react code only had styling.

答:
The problem is, that your are not providig the correct parameters for the tilemap function. You have to pass a Phaser.Tilemaps.MapData checkout the documentation, here https://photonstorm.github.io/phaser3-docs/Phaser.Tilemaps.Tilemap.html or you could checkout the examples on the phaser.io website.

Checkout especially, some Demos on the official website, they show many different ways how to use tiles(maps) (with working examples): https://phaser.io/examples/v3/category/tilemap

Here is a snippet, showing how to use a array, for the tiles:

var config = {
    type: Phaser.AUTO,
    width: 5*16,
    height: 3*16,
    zoom:2,
    scene: {
        preload: preload,
        create: create
    }
};

var game = new Phaser.Game(config);

function preload ()
{
    this.load.image('tiles', 'https://labs.phaser.io/assets/tilemaps/tiles/super-mario.png');
}

function create ()
{
    // Load a map from a 2D array of tile indices
    var level = [ [ 0, 0, 0, 0, 0],[ 0, 14, 13, 14, 0],[ 0, 0, 0, 0, 0]];

    var map = this.make.tilemap({ data: level, tileWidth: 16, tileHeight: 16 });
    var tiles = map.addTilesetImage('tiles');
    var layer = map.createLayer(0, tiles, 0, 0);
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><script src="https://cdn.jsdelivr.net/npm/phaser@3.55.2/dist/phaser.js"></script></code>```

Title: Invalid Tilemap Layer ID

Body:
I was doing my game in phaser 3 when I went to add a tilemap created in tiled, I followed several tutorials, I did and redid the code several times and even then it always gives me the same invalid ID error.

unsuccessful attempts:

Tried to manually make the tilemap;

Tried using JSON files;

I tried to use a single layer;

I tried to use the names according to the tiled layers;

I tried using different names;

here follows the code:

import './phaser'

const config = {
  type: Phaser.AUTO,
  parent: "phaser-example",
  width: 1824,
  height: 960,
  physics: {
      default: 'arcade'
  },
  scene: {
      preload: preload,
      create: create
  }
};

export const game = new Phaser.Game(config)

function preload()
{
  this.load.image('terrain', './complementos/tilemaps/terrainBasic.png')
  this.load.tilemapTiledJSON('map', './complementos/tilemaps/basicLand.json')
}

function create()
{
  const teste = [[0,1,2],[0,1,2],[0,1,2]]

  const map = this.make.tilemap(teste)

  map.addTilesetImage('terrainBasic','terrain', 32, 32)

  const layer = map.createLayer(0,'land', 0, 0)
}

I don't know if it changes much but I was using react together with phaser 3, but my react code only had styling.

Body:
The problem is, that your are not providig the correct parameters for the tilemap function. You have to pass a Phaser.Tilemaps.MapData checkout the documentation, here https://photonstorm.github.io/phaser3-docs/Phaser.Tilemaps.Tilemap.html or you could checkout the examples on the phaser.io website.

Checkout especially, some Demos on the official website, they show many different ways how to use tiles(maps) (with working examples): https://phaser.io/examples/v3/category/tilemap

Here is a snippet, showing how to use a array, for the tiles:

var config = {
    type: Phaser.AUTO,
    width: 5*16,
    height: 3*16,
    zoom:2,
    scene: {
        preload: preload,
        create: create
    }
};

var game = new Phaser.Game(config);

function preload ()
{
    this.load.image('tiles', 'https://labs.phaser.io/assets/tilemaps/tiles/super-mario.png');
}

function create ()
{
    // Load a map from a 2D array of tile indices
    var level = [ [ 0, 0, 0, 0, 0],[ 0, 14, 13, 14, 0],[ 0, 0, 0, 0, 0]];

    var map = this.make.tilemap({ data: level, tileWidth: 16, tileHeight: 16 });
    var tiles = map.addTilesetImage('tiles');
    var layer = map.createLayer(0, tiles, 0, 0);
}

<script src="https://cdn.jsdelivr.net/npm/phaser@3.55.2/dist/phaser.js"></script>


问:
How do I append a value to an array in a JavaScript object while mapping through the object its contained in?
I have a JavaScript object of the following structure:

```<code>results = [
   {
      date: ... // holds a string representing a date
      timeslots: [
         {
           slotsAvailable: ... // holds an int
           time: ... // holds a string representing time
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   },
   {
      date: ...
      timeslots: [
         {
           slotsAvailable: ...
           time: ...
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   },
   {
      date: ...
      timeslots: [
         {
           slotsAvailable: ...
           time: ...
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   }
]
</code>```

The object starts off empty and I fill it through the following code:

```<code>    "timeslots": [
        {
            "slotsAvailable": 1,
            "time": "2022-02-11T14:00:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-13T13:40:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-15T17:40:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-15T18:20:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-16T14:00:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-16T14:20:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-17T21:40:00-0500"
        }
    ]

    var results = []

    let date = new Date(timeslots[0].time)
    for(let i = 0; i < 8; i++) {
        results = [...results, {date: new Date(date), timeslots: []}] // Date function called b/c otherwise value is passed by reference
        date.setDate(date.getDate() + 1)
    }

    console.log(results)

    for(const i in timeslots) {
        let slot = timeslots[i]

        results.map(result => {
            return result.date.getDate() === new Date(slot.time).getDate()
                   ? {...result, timeslots: [...result.timeslots, slot]} // this line doesn't work
                   : result
        })
    }

    console.log(results)
</code>```

The results object correctly populates with 8 dates in the first loop, but after the second loop, all of the timeslots arrays are still empty. The intended result is that the final object contains an array of dates where each timeslot is placed in the object corresponding to its date.

I believe the error is with the line in the second loop where I spread in the existing array and append a new value to the array, but I'm not sure how to fix this.

Suggestions?

答:
Is this how you expect your output to be?

const timeslots= [{"slotsAvailable": 1,"time": "2022-02-11T14:00:00-0500"},{"slotsAvailable": 1,"time": "2022-02-13T13:40:00-0500"},{"slotsAvailable": 1,"time": "2022-02-15T17:40:00-0500"},{"slotsAvailable": 1,"time": "2022-02-15T18:20:00-0500"},{"slotsAvailable": 1,"time": "2022-02-16T14:00:00-0500"},{"slotsAvailable": 1,"time": "2022-02-16T14:20:00-0500"},{"slotsAvailable": 1,"time": "2022-02-17T21:40:00-0500"}]

const results = timeslots.reduce((acc, curr) => {
  const date = curr.time.split('T')[0];
  const time = curr.time.split('T')[1];
  const dateTime = date + ' ' + time;
  const dateTimeObject = new Date(dateTime);
  const dateString = dateTimeObject.toDateString();
  const timeString = dateTimeObject.toLocaleTimeString();
  const timeObject = {
    slotsAvailable: curr.slotsAvailable,
    time: timeString,
  };
  if (acc.length === 0) {
    acc.push({
      date: dateString,
      timeslots: [timeObject],
    });
  } else {
    const lastIndex = acc.length - 1;
    const lastDate = acc[lastIndex].date;
    if (lastDate === dateString) {
      acc[lastIndex].timeslots.push(timeObject);
    } else {
      acc.push({
        date: dateString,
        timeslots: [timeObject],
      });
    }
  }
  return acc;
}, []);

console.log(JSON.stringify(results, null, 2));```
<pre class="snippet-code-css lang-css prettyprint-override"><code>.as-console-wrapper{max-height:100%  !important; top:0px}</code>```

Body:
I have a JavaScript object of the following structure:

results = [
   {
      date: ... // holds a string representing a date
      timeslots: [
         {
           slotsAvailable: ... // holds an int
           time: ... // holds a string representing time
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   },
   {
      date: ...
      timeslots: [
         {
           slotsAvailable: ...
           time: ...
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   },
   {
      date: ...
      timeslots: [
         {
           slotsAvailable: ...
           time: ...
         },
         {
           slotsAvailable: ...
           time: ...
         }
      ]
   }
]

The object starts off empty and I fill it through the following code:

    "timeslots": [
        {
            "slotsAvailable": 1,
            "time": "2022-02-11T14:00:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-13T13:40:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-15T17:40:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-15T18:20:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-16T14:00:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-16T14:20:00-0500"
        },
        {
            "slotsAvailable": 1,
            "time": "2022-02-17T21:40:00-0500"
        }
    ]

    var results = []

    let date = new Date(timeslots[0].time)
    for(let i = 0; i < 8; i++) {
        results = [...results, {date: new Date(date), timeslots: []}] // Date function called b/c otherwise value is passed by reference
        date.setDate(date.getDate() + 1)
    }

    console.log(results)

    for(const i in timeslots) {
        let slot = timeslots[i]

        results.map(result => {
            return result.date.getDate() === new Date(slot.time).getDate()
                   ? {...result, timeslots: [...result.timeslots, slot]} // this line doesn't work
                   : result
        })
    }

    console.log(results)

The results object correctly populates with 8 dates in the first loop, but after the second loop, all of the timeslots arrays are still empty. The intended result is that the final object contains an array of dates where each timeslot is placed in the object corresponding to its date.

I believe the error is with the line in the second loop where I spread in the existing array and append a new value to the array, but I'm not sure how to fix this.

Suggestions?

Body:
Is this how you expect your output to be?

const timeslots= [{"slotsAvailable": 1,"time": "2022-02-11T14:00:00-0500"},{"slotsAvailable": 1,"time": "2022-02-13T13:40:00-0500"},{"slotsAvailable": 1,"time": "2022-02-15T17:40:00-0500"},{"slotsAvailable": 1,"time": "2022-02-15T18:20:00-0500"},{"slotsAvailable": 1,"time": "2022-02-16T14:00:00-0500"},{"slotsAvailable": 1,"time": "2022-02-16T14:20:00-0500"},{"slotsAvailable": 1,"time": "2022-02-17T21:40:00-0500"}]

const results = timeslots.reduce((acc, curr) => {
  const date = curr.time.split('T')[0];
  const time = curr.time.split('T')[1];
  const dateTime = date + ' ' + time;
  const dateTimeObject = new Date(dateTime);
  const dateString = dateTimeObject.toDateString();
  const timeString = dateTimeObject.toLocaleTimeString();
  const timeObject = {
    slotsAvailable: curr.slotsAvailable,
    time: timeString,
  };
  if (acc.length === 0) {
    acc.push({
      date: dateString,
      timeslots: [timeObject],
    });
  } else {
    const lastIndex = acc.length - 1;
    const lastDate = acc[lastIndex].date;
    if (lastDate === dateString) {
      acc[lastIndex].timeslots.push(timeObject);
    } else {
      acc.push({
        date: dateString,
        timeslots: [timeObject],
      });
    }
  }
  return acc;
}, []);

console.log(JSON.stringify(results, null, 2));

.as-console-wrapper{max-height:100%  !important; top:0px}


问:
reading a .osm file in pandas, python using xml.etree.ElementTree
Hi I am trying to read a .osm file in pandas using xml.etree.ElementTree.
How to fetch the id inside node and the values of tag k = "" v = ""

Input:

```<code><osm version="0.6" generator="CGImap 0.8.6 (895960 spike-06.openstreetmap.org)" copyright="OpenStreetMap and contributors" attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
 <bounds minlat="12.5770000" minlon="-2.9532000" maxlat="12.6328000" maxlon="-2.8514000"/>
 <node id="902955577" visible="true" version="6" changeset="90170478" timestamp="2020-08-31T06:29:59Z" user="pizzaiolo" uid="1772368" lat="12.6049118" lon="-2.9023135">
  <tag k="admin_level" v="8"/>
  <tag k="capital" v="8"/>
  <tag k="description" v="Commune rurale"/>
  <tag k="is_in" v="Toma;Nayala;Boucle du Mouhoun;Burkina Faso;Africa"/>
  <tag k="is_in:continent" v="Africa"/>
  <tag k="is_in:country" v="Burkina Faso"/>
  <tag k="is_in:country_code" v="BF"/>
  <tag k="is_in:department" v="Toma"/>
  <tag k="is_in:province" v="Nayala"/>
  <tag k="is_in:province_code" v="40"/>
  <tag k="is_in:region" v="Boucle du Mouhoun"/>
  <tag k="is_in:region_code" v="02"/>
  <tag k="name" v="Goa"/>
  <tag k="name:fr" v="Goa"/>
  <tag k="note" v="Inserted by script"/>
  <tag k="place" v="village"/>
  <tag k="population" v="651"/>
  <tag k="source" v="NGA GEOnet Names Server (GNS)"/>
  <tag k="source:population" v="INSD.BF"/>
  <tag k="wikidata" v="Q13414957"/>
 </node>
</code>```

Code:

```<code>from xml.etree.ElementTree import parse
import pandas as pd

mydata = parse('map.osm')
print(mydata,'\n')

my_id=[]

for item in mydata.iterfind('node'):
    my_id.append(item.findtext('id'))

print(my_id)
</code>```

Output:
<xml.etree.ElementTree.ElementTree object at 0x0000018504B10160>

[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]

答: findtext looks for an element, but id is not an element; it's an attribute. You want item.attrib['id'], or perhaps item.get('id').

Body:
Hi I am trying to read a .osm file in pandas using xml.etree.ElementTree.
How to fetch the id inside node and the values of tag k = "" v = ""

Input:

<osm version="0.6" generator="CGImap 0.8.6 (895960 spike-06.openstreetmap.org)" copyright="OpenStreetMap and contributors" attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
 <bounds minlat="12.5770000" minlon="-2.9532000" maxlat="12.6328000" maxlon="-2.8514000"/>
 <node id="902955577" visible="true" version="6" changeset="90170478" timestamp="2020-08-31T06:29:59Z" user="pizzaiolo" uid="1772368" lat="12.6049118" lon="-2.9023135">
  <tag k="admin_level" v="8"/>
  <tag k="capital" v="8"/>
  <tag k="description" v="Commune rurale"/>
  <tag k="is_in" v="Toma;Nayala;Boucle du Mouhoun;Burkina Faso;Africa"/>
  <tag k="is_in:continent" v="Africa"/>
  <tag k="is_in:country" v="Burkina Faso"/>
  <tag k="is_in:country_code" v="BF"/>
  <tag k="is_in:department" v="Toma"/>
  <tag k="is_in:province" v="Nayala"/>
  <tag k="is_in:province_code" v="40"/>
  <tag k="is_in:region" v="Boucle du Mouhoun"/>
  <tag k="is_in:region_code" v="02"/>
  <tag k="name" v="Goa"/>
  <tag k="name:fr" v="Goa"/>
  <tag k="note" v="Inserted by script"/>
  <tag k="place" v="village"/>
  <tag k="population" v="651"/>
  <tag k="source" v="NGA GEOnet Names Server (GNS)"/>
  <tag k="source:population" v="INSD.BF"/>
  <tag k="wikidata" v="Q13414957"/>
 </node>

Code:

from xml.etree.ElementTree import parse
import pandas as pd

mydata = parse('map.osm')
print(mydata,'\n')

my_id=[]

for item in mydata.iterfind('node'):
    my_id.append(item.findtext('id'))

print(my_id)

Output:
<xml.etree.ElementTree.ElementTree object at 0x0000018504B10160>

[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]


问:
Non-resolvable parent POM and Could not transfer artifact from/to central repository
enter image description hereGetting below error in eclipse for spring boot project

Non-resolvable parent POM and Could not transfer artifact from/to central repository and transfer failed for spring-boot-starter-parent-2.1.2.pom

I have tried maven->update project->force update project and tried to delete m2 folder.but failed
Can anyone help on this

答:
Try the following</p>
```<code><parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.4.0</version>
</parent>
</code>```

Body:
Try the following

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.4.0</version>
</parent>


问:
How to play a video from a particular timestamp when user clicks on a button in Brightcove
I am working on Brightcove to play a video. I need to play a video from a particular timestamp within the video. For eg: if the user clicks on a button the video should play from  5 min 30 sec of the video. I think the idea is clear. I am using brightcove/react-player-loader to play video.</p>
```<code><ReactPlayerLoader
    onSuccess={onVideoSuccess}
    accountId={accountID}
    videoId={mediaID}
    playerId={playerID}
    attrs={{ className: "brightcove", autoPlay: "muted" }}
/>
</code>```

答:
```<code>import { useRef } from 'react'
const mediaRef = useRef()

const gotoTimeStamp = (time) => {
    mediaRef.current.player.currentTime(time)
  }

return (

 <ReactPlayerLoader
   ref={mediaRef}
    onSuccess={onVideoSuccess}
    accountId={accountID}
    videoId={mediaID}
    playerId={playerID}
   attrs={{ className: 'brightcove', autoPlay: 'muted />

 <button onClick={() => gotoTimeStamp(1434)}>
   Click me
  </button>
)
</code>```

Body:
I am working on Brightcove to play a video. I need to play a video from a particular timestamp within the video. For eg: if the user clicks on a button the video should play from  5 min 30 sec of the video. I think the idea is clear. I am using brightcove/react-player-loader to play video.

<ReactPlayerLoader
    onSuccess={onVideoSuccess}
    accountId={accountID}
    videoId={mediaID}
    playerId={playerID}
    attrs={{ className: "brightcove", autoPlay: "muted" }}
/>


问:
How to fix print((double parentheses)) after 2to3 conversion?
When migrating my project to Python 3 (2to3-3.7 -w -f print *), I observed that a lot of (but not all) print statements became print((...)), so these statements now print tuples instead of performing the expected behavior. I gather that if I'd used -p, I'd be in a better place right now because from __future__ import print_function is at the top of every affected module.

I'm thinking about trying to use sed to fix this, but before I break my teeth on that, I thought I'd see if anyone else has dealt with this before. Is there a 2to3 feature to clean this up?

I do use version control (git) and have commits immediately before and after (as well as the .bak files 2to3 creates), but I'm not sure how to isolate the changes I've made from the print situations.

答:
This is an option that can be used if you have a mix of print statements and functions in your code. 2to3 doesn't cope well with complex content in the print function so in an editor with regular expression search and replace, you can clean these up in a semi-manual way. For instance in vscode, an example of a search string would be

```<code>print\(\((.*\.format.*)\)\)\n
</code>```

Replace with

```<code>print\($1\)
</code>```

Adjust the content for other cases.

Body:
This is an option that can be used if you have a mix of print statements and functions in your code. 2to3 doesn't cope well with complex content in the print function so in an editor with regular expression search and replace, you can clean these up in a semi-manual way. For instance in vscode, an example of a search string would be

print\(\((.*\.format.*)\)\)\n

Replace with

print\($1\)

Adjust the content for other cases.


问:
Plotting a secondary x-axis in ggplot based on another column in the data
I am struggling to find if it is possible to plot a secondary x-axis (on the top of the linegraph if possible) using an entirely different column in the dataset. I've been looking for a while, so apologies if this has been answered already. For example:

```<code>Day <- c(1:10)
Revenue <- c(100,200,300,400,550,600,750,800,975,1000)
Date <- c("1/1","1/5","1/10","1/11","1/14","1/15","1/18","1/21","1/28","1/31")
a <- data.frame(Day,Revenue,Date)
</code>```

I can plot "Day" on the x-axis and "Revenue" on the y-axis:

```<code>ggplot(a,(aes(Day,Revenue))) + geom_line(aes(y = Revenue))
</code>```

However, is it possible to have the corresponding "Date" included at the top of the graph on a secondary x-axis that matches the primary x-axis ("Day")? Thank you in advance for any help on this!

答:
One option to achieve your desired result would be to make use of dup_axis and use a labeller function which assigns the value from your Date column to values of Day chosen as breaks for the primary scale:

library(ggplot2)

dup_labels <- function(x) a$Date[match(x, a$Day)]

ggplot(a,(aes(Day,Revenue))) + geom_line(aes(y = Revenue)) +
  scale_x_continuous(breaks = 1:10, sec.axis = dup_axis(labels = dup_labels))
```

Body:
I am struggling to find if it is possible to plot a secondary x-axis (on the top of the linegraph if possible) using an entirely different column in the dataset. I've been looking for a while, so apologies if this has been answered already. For example:

Day <- c(1:10)
Revenue <- c(100,200,300,400,550,600,750,800,975,1000)
Date <- c("1/1","1/5","1/10","1/11","1/14","1/15","1/18","1/21","1/28","1/31")
a <- data.frame(Day,Revenue,Date)

I can plot "Day" on the x-axis and "Revenue" on the y-axis:

ggplot(a,(aes(Day,Revenue))) + geom_line(aes(y = Revenue))

However, is it possible to have the corresponding "Date" included at the top of the graph on a secondary x-axis that matches the primary x-axis ("Day")? Thank you in advance for any help on this!


问:
aiohttp terminate web.app_run

from aiohttp import web

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

app = web.Application()
app.add_routes([web.get('/', handle),
                web.get('/{name}', handle)])

if __name__ == '__main__':
    web.run_app(app)
```
<p>The above example is from oficial documentation.</p>
<p>The question is how can i terminate the <code>web.run_app(app)</code> operations with no <code>KeyBoardInterrupt</code> (Ctrl+C)?</p>
<p>I am looking something like:</p>
```await app.shutdown()
await app.cleanup()
```

but i don't know where can i put this code and how can i use it.

答:
Based on @atline answer i wrote this code:

import signal
from aiohttp import web
from aiohttp.web_runner import GracefulExit

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from multiprocessing import cpu_count
import requests
import time

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

async def shutdown(request):
    print("will shutdown now")
    raise GracefulExit()

app = web.Application()
app.add_routes([web.get('/shutdown', shutdown),
                web.get('/', handle),
                web.get('/{name}', handle)])

def init_aiohttp():
    web.run_app(app, host="127.0.0.1", port="8080", ssl_context=None)

if __name__ == '__main__':
    executor = ProcessPoolExecutor()
    for i in range(0, 1):
        aiohttp = executor.submit(init_aiohttp)

    total_time = 0
    while(True):
        time.sleep(1)
        total_time += 1
        if total_time == 5:
            try:
                requests.get('http://127.0.0.1:8080/shutdown')
            except Exception as e:
                print(e)

        print(total_time)
```

The above code:

Starts an aiohttp server.

Close the server after 5 second (fetching the shutdown url: http://127.0.0.1:8080/shutdown)

Hope that will help someone in the future.

Title: aiohttp terminate web.app_run

Body:
from aiohttp import web

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

app = web.Application()
app.add_routes([web.get('/', handle),
                web.get('/{name}', handle)])

if __name__ == '__main__':
    web.run_app(app)

The above example is from oficial documentation.

The question is how can i terminate the web.run_app(app) operations with no KeyBoardInterrupt (Ctrl+C)?

I am looking something like:

await app.shutdown()
await app.cleanup()

but i don't know where can i put this code and how can i use it.

Body:
Based on @atline answer i wrote this code:

import signal
from aiohttp import web
from aiohttp.web_runner import GracefulExit

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from multiprocessing import cpu_count
import requests
import time

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

async def shutdown(request):
    print("will shutdown now")
    raise GracefulExit()

app = web.Application()
app.add_routes([web.get('/shutdown', shutdown),
                web.get('/', handle),
                web.get('/{name}', handle)])

def init_aiohttp():
    web.run_app(app, host="127.0.0.1", port="8080", ssl_context=None)

if __name__ == '__main__':
    executor = ProcessPoolExecutor()
    for i in range(0, 1):
        aiohttp = executor.submit(init_aiohttp)

    total_time = 0
    while(True):
        time.sleep(1)
        total_time += 1
        if total_time == 5:
            try:
                requests.get('http://127.0.0.1:8080/shutdown')
            except Exception as e:
                print(e)

        print(total_time)

The above code:

Starts an aiohttp server.

Close the server after 5 second (fetching the shutdown url: http://127.0.0.1:8080/shutdown)

Hope that will help someone in the future.