codecov[bot]: # [Codecov](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=h1) Report
> Merging [#475](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=desc) into [master](https://codecov.io/gh/regro/regolith/commit/bf28e43ecac7e5469af4d3ab2efc25937db65034&el=desc) will **increase** coverage by `0.24%`.
> The diff coverage is `92.85%`.

[![Impacted file tree graph](https://codecov.io/gh/regro/regolith/pull/475/graphs/tree.svg?width=650&height=150&src=pr&token=YOWfAqEPVS)](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #475      +/-   ##
==========================================
+ Coverage   64.10%   64.34%   +0.24%     
==========================================
  Files          54       55       +1     
  Lines        4948     4990      +42     
==========================================
+ Hits         3172     3211      +39     
- Misses       1776     1779       +3     
```


| [Impacted Files](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [regolith/helpers/l\_generalhelper.py](https://codecov.io/gh/regro/regolith/pull/475/diff?src=pr&el=tree#diff-cmVnb2xpdGgvaGVscGVycy9sX2dlbmVyYWxoZWxwZXIucHk=) | `92.68% <92.68%> (ø)` | |
| [regolith/helper.py](https://codecov.io/gh/regro/regolith/pull/475/diff?src=pr&el=tree#diff-cmVnb2xpdGgvaGVscGVyLnB5) | `100.00% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=footer). Last update [bf28e43...2186a09](https://codecov.io/gh/regro/regolith/pull/475?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

dy2403: Hi @sbillinge!
I have a question about the -k and exactly it should retrieve. (From UC4)
Are keys like name, position, etc.? Should it be retrieved from schemas?
Or should I establish the keys available for each collection in the function?

sbillinge: > Hi @sbillinge!
> I have a question about the -k and exactly it should retrieve. (From UC4)
> Are keys like name, position, etc.? Should it be retrieved from schemas?
> Or should I establish the keys available for each collection in the function?

-k there stands for --keys and it lists all the keys that I can give to the lister either to search over, or to return.  Otherwise I may not remember what I have available to search over.
dy2403: @sbillinge ready for review and merge!
dy2403: I do have a prum! Should I finish it when it is merged, or when we guarantee that it is working when we test it?
dy2403: Hi prof. @sbillinge,
I think this is ready to review and merge!
diff --git a/news/generallister.rst b/news/generallister.rst
new file mode 100644
index 00000000..5847f0f5
--- /dev/null
+++ b/news/generallister.rst
@@ -0,0 +1,12 @@
+**Added:**
+ * a helper for listing filtered data from collections in the database.
+
+**Changed:** None
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None
\ No newline at end of file
diff --git a/regolith/helper.py b/regolith/helper.py
index ca11f826..49c54542 100644
--- a/regolith/helper.py
+++ b/regolith/helper.py
@@ -16,6 +16,7 @@ from regolith.helpers import u_milestonehelper as u_milestone
 from regolith.helpers import u_contacthelper as u_contact
 from regolith.helpers import l_todohelper as l_todo
 from regolith.helpers import u_finishprumhelper as u_finishprum
+from regolith.helpers import l_generalhelper as l_general
 
 HELPERS = {
     "hello": (hello.HelloHelper, hello.subparser),
@@ -34,6 +35,7 @@ HELPERS = {
     "u_milestone": (u_milestone.MilestoneUpdaterHelper, u_milestone.subparser),
     "l_todo": (l_todo.TodoListerHelper, l_todo.subparser),
     "finish_prum": (u_finishprum.FinishprumUpdaterHelper, u_finishprum.subparser),
+    "lister": (l_general.GeneralListerHelper, l_general.subparser),
 }
 
 
diff --git a/regolith/helpers/l_generalhelper.py b/regolith/helpers/l_generalhelper.py
new file mode 100644
index 00000000..21ebc971
--- /dev/null
+++ b/regolith/helpers/l_generalhelper.py
@@ -0,0 +1,100 @@
+"""Helper for listing filtered data from collections in the database.
+"""
+
+from regolith.helpers.basehelper import SoutHelperBase
+from regolith.fsclient import _id_key
+from regolith.tools import (
+    all_docs_from_collection,
+    get_pi_id,
+    search_collection,
+    collection_str,
+)
+HELPER_TARGET = "lister"
+
+def subparser(subpi):
+    subpi.add_argument(
+        "coll",
+        help='Collection that the lister is going to list from.')
+    subpi.add_argument(
+        "-f","--kv_filter",
+        nargs="+",
+        help="Search the given collection by key-value pairs. "
+             "e.g. 'regolith helper lister -f name simon' will "
+             "return the id of all the people who's name contains simon.")
+    subpi.add_argument(
+        "-r",  "--return_fields",
+        nargs="+",
+        help="Specify from which keys to print values. "
+             "e.g. 'regolith helper lister people -r name status title' "
+             "will return the name, status, and title of all the people "
+             "in the collection. If no argument is given the default is just the id.")
+    subpi.add_argument(
+        "-k", "--keys",
+        action="store_true",
+        help="List of the available keys in the collection.")
+    return subpi
+
+class GeneralListerHelper(SoutHelperBase):
+    """Helper for listing filtered data from collections in the database.
+    """
+    # btype must be the same as helper target in helper.py
+    btype = HELPER_TARGET
+
+    def construct_global_ctx(self):
+        """Constructs the global context"""
+        super().construct_global_ctx()
+        gtx = self.gtx
+        rc = self.rc
+        needed_dbs = [rc.coll]
+        if "groups" in needed_dbs:
+            rc.pi_id = get_pi_id(rc)
+        try:
+            if not rc.database:
+                rc.database = rc.databases[0]["name"]
+        except BaseException:
+            pass
+        colls = [
+            sorted(
+                all_docs_from_collection(rc.client, collname), key=_id_key
+            )
+            for collname in needed_dbs
+        ]
+        for db, coll in zip(needed_dbs, colls):
+            gtx[db] = coll
+        gtx["all_docs_from_collection"] = all_docs_from_collection
+        gtx["float"] = float
+        gtx["str"] = str
+        gtx["zip"] = zip
+
+    def sout(self):
+        rc = self.rc
+        coll = self.gtx[rc.coll]
+        keys = sorted(set([i for k in coll for i in k.keys()]))
+        list_keys = []
+        if rc.kv_filter:
+            list_keys.extend([rc.kv_filter[n] for n in range(len(rc.kv_filter)) if n % 2 == 0])
+        if rc.return_fields:
+            list_keys.extend(rc.return_fields)
+        for i in list_keys:
+            if i not in keys:
+                raise ValueError(f"{i} is not a valid key. Please choose a valid key from: {keys}")
+        if len(coll) == 0:
+            raise RuntimeError('This collection is empty or does not exist. Please inform a valid collection.')
+        if rc.kv_filter:
+            filter_result = search_collection(coll, rc.kv_filter).strip()
+            if filter_result == '':
+                print("There are no results that match your search.")
+            else:
+                if rc.return_fields:
+                    print("Results of your search:\n"
+                            f"{(search_collection(coll, rc.kv_filter, rc.return_fields).strip())}")
+                else:
+                    print(f"Results of your search:\n{filter_result}")
+        if rc.return_fields and not rc.kv_filter:
+            print(f"Results of your search:\n{collection_str(coll, rc.return_fields).strip()}")
+        if rc.keys:
+            print(f"Available keys:\n{keys}")
+            return
+        if not rc.kv_filter and not rc.return_fields:
+            print(f"Results of your search:\n{collection_str(coll).strip()}")
+        return
diff --git a/regolith/schemas.py b/regolith/schemas.py
index a38ad6bb..0a537df7 100644
--- a/regolith/schemas.py
+++ b/regolith/schemas.py
@@ -1468,7 +1468,7 @@ SCHEMAS = {
             "type": "string",
         },
         "institution": {
-            "description": "name of the inistitution",
+            "description": "name of the institution",
             "required": True,
             "type": "string",
         },

helper: general lister
UC1:
1. simon types ``regolith helper list people``
1. lister lists the ids of all the people in people

UC2:
1. S types ``rh list people --kv_filter name simon``
1. l lists the id of all people in people who's name contains simon

UC3:
1. S types ``rh list people --kv_filter name simon -r name position``
1. l lists the id, name and position of all people in people who's name contains simon

UC4:
1. S types ``rh list people -k``
1. l returns a list of the available keys and their description (if available) in collection people
diff --git a/tests/test_helpers.py b/tests/test_helpers.py
index 8a86bcd3..8b0a8b25 100644
--- a/tests/test_helpers.py
+++ b/tests/test_helpers.py
@@ -159,6 +159,36 @@ helper_map = [
      "20sb_firstprojectum     status:finished\n"
      "Please rerun the helper specifying the complete ID.\n"
      ),
+
+    #UC1
+    (["helper", "lister", "people"],
+     "Results of your search:\nsbillinge    \nscopatz\n"),
+    (["helper", "lister", "people", "--kv_filter", "name", "simon"],
+     "Results of your search:\n"
+     "sbillinge\n"),
+    (["helper", "lister", "people", "--kv_filter", "name", "simon", "--return_fields", "name", "position"],
+     "Results of your search:\nsbillinge    name: Simon J. L. Billinge    position: professor\n"),
+    (["helper", "lister", "people", "--keys"],
+     "Available keys:\n"
+     "['_id', 'activities', 'aka', 'appointments', 'avatar', 'bio', 'bios', "
+     "'committees', 'education', 'email', 'employment', 'facilities', 'funding', "
+     "'github_id', 'google_scholar_url', 'hindex', 'home_address', 'initials', "
+     "'membership', 'miscellaneous', 'name', 'office', 'orcid_id', 'position', "
+     "'publicity', 'research_focus_areas', 'service', 'skills', 'teaching', "
+     "'title', 'todos']\n"),
+    (["helper", "lister", "people", "--kv_filter", "name", "simon", "--keys"],
+     "Results of your search:\nsbillinge\n"
+     "Available keys:\n"
+     "['_id', 'activities', 'aka', 'appointments', 'avatar', 'bio', 'bios', "
+     "'committees', 'education', 'email', 'employment', 'facilities', 'funding', "
+     "'github_id', 'google_scholar_url', 'hindex', 'home_address', 'initials', "
+     "'membership', 'miscellaneous', 'name', 'office', 'orcid_id', 'position', "
+     "'publicity', 'research_focus_areas', 'service', 'skills', 'teaching', "
+     "'title', 'todos']\n"
+     ),
+    (["helper", "lister", "people", "--kv_filter", "name", "simon", "position", "singer"],
+     "There are no results that match your search.\n"
+     ),
 ]
 
 

pip install -e .[dev]
parse_log_pytest
requirements.txt
pytest coverage
apt-get update apt-get install -y gcc
3.8
requirements/run.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
Cerberus==1.3.7
coverage==7.6.1
dnspython==2.6.1
docutils==0.20.1
et_xmlfile==2.0.0
exceptiongroup==1.2.2
iniconfig==2.1.0
Jinja2==3.1.6
MarkupSafe==2.1.5
nameparser==1.1.3
numpy==1.24.4
openpyxl==3.1.5
packaging==24.2
pandas==2.0.3
pluggy==1.5.0
pymongo==4.10.1
pytest==8.3.5
python-dateutil==2.9.0.post0
pytz==2025.2
-e git+https://github.com/regro/regolith.git@3d3476a6880b52ee19ff94f91f4ed1a0cbcbcea2#egg=regolith
rever==0.3.3
ruamel.yaml==0.18.10
ruamel.yaml.clib==0.2.8
six==1.17.0
tomli==2.2.1
tzdata==2025.2
xonsh==0.14.0

name: regolith
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - pip=24.2=py38h06a4308_0
  - python=3.8.20=he870216_0
  - readline=8.2=h5eee18b_0
  - setuptools=75.1.0=py38h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - wheel=0.44.0=py38h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - cerberus==1.3.7
      - coverage==7.6.1
      - dnspython==2.6.1
      - docutils==0.20.1
      - et-xmlfile==2.0.0
      - exceptiongroup==1.2.2
      - iniconfig==2.1.0
      - jinja2==3.1.6
      - markupsafe==2.1.5
      - nameparser==1.1.3
      - numpy==1.24.4
      - openpyxl==3.1.5
      - packaging==24.2
      - pandas==2.0.3
      - pluggy==1.5.0
      - pymongo==4.10.1
      - pytest==8.3.5
      - python-dateutil==2.9.0.post0
      - pytz==2025.2
      - rever==0.3.3
      - ruamel-yaml==0.18.10
      - ruamel-yaml-clib==0.2.8
      - six==1.17.0
      - tomli==2.2.1
      - tzdata==2025.2
      - xonsh==0.14.0
prefix: /opt/conda/envs/regolith

tests/test_helpers.py::test_helper_python[hm37] tests/test_helpers.py::test_helper_python[hm38] tests/test_helpers.py::test_helper_python[hm39] tests/test_helpers.py::test_helper_python[hm40] tests/test_helpers.py::test_helper_python[hm41] tests/test_helpers.py::test_helper_python[hm42]
tests/test_helpers.py::test_helper_python[hm4]
tests/test_helpers.py::test_helper_python[hm0] tests/test_helpers.py::test_helper_python[hm1] tests/test_helpers.py::test_helper_python[hm2] tests/test_helpers.py::test_helper_python[hm3] tests/test_helpers.py::test_helper_python[hm5] tests/test_helpers.py::test_helper_python[hm6] tests/test_helpers.py::test_helper_python[hm7] tests/test_helpers.py::test_helper_python[hm8] tests/test_helpers.py::test_helper_python[hm9] tests/test_helpers.py::test_helper_python[hm10] tests/test_helpers.py::test_helper_python[hm11] tests/test_helpers.py::test_helper_python[hm12] tests/test_helpers.py::test_helper_python[hm13] tests/test_helpers.py::test_helper_python[hm14] tests/test_helpers.py::test_helper_python[hm15] tests/test_helpers.py::test_helper_python[hm16] tests/test_helpers.py::test_helper_python[hm17] tests/test_helpers.py::test_helper_python[hm18] tests/test_helpers.py::test_helper_python[hm19] tests/test_helpers.py::test_helper_python[hm20] tests/test_helpers.py::test_helper_python[hm21] tests/test_helpers.py::test_helper_python[hm22] tests/test_helpers.py::test_helper_python[hm23] tests/test_helpers.py::test_helper_python[hm24] tests/test_helpers.py::test_helper_python[hm25] tests/test_helpers.py::test_helper_python[hm26] tests/test_helpers.py::test_helper_python[hm27] tests/test_helpers.py::test_helper_python[hm28] tests/test_helpers.py::test_helper_python[hm29] tests/test_helpers.py::test_helper_python[hm30] tests/test_helpers.py::test_helper_python[hm31] tests/test_helpers.py::test_helper_python[hm32] tests/test_helpers.py::test_helper_python[hm33] tests/test_helpers.py::test_helper_python[hm34] tests/test_helpers.py::test_helper_python[hm35] tests/test_helpers.py::test_helper_python[hm36]
regolith/helper.py regolith/schemas.py news/generallister.rst regolith/helpers/l_generalhelper.py
regolith/helper.py regolith/schemas.py news/generallister.rst regolith/helpers/l_generalhelper.py