feat: switch from docker sdk to gradio dsk, add [email protected]

README.md

@@ -3,141 +3,53 @@ title: SamGIS - LISA on CUDA
 emoji: 🗺️
 colorFrom: red
 colorTo: blue
-sdk: docker
-pinned: false
+sdk: gradio
+sdk_version: 4.40.0
+app_file: app.py
+pinned: true
 license: mit
 ---
 
-# [LISA](https://github.com/dvlab-research/LISA) and [SamGIS](https://github.com/trincadev/samgis-be)
+# [LISA](https://github.com/dvlab-research/LISA) + [SamGIS](https://github.com/trincadev/samgis-be) on a dedicated CUDA GPU
 
-[LISA](https://github.com/dvlab-research/LISA) (Reasoning Segmentation via Large Language Model) applied to geospatial data thanks to [SamGIS](https://github.com/trincadev/samgis-be).
+This project aims to permit use of [LISA](https://github.com/dvlab-research/LISA) (Reasoning Segmentation via Large Language Model) applied to geospatial data thanks to [SamGIS](https://github.com/trincadev/samgis-be). In this space I adapted LISA to HuggingFace [lisa-on-cuda](https://huggingface.co/spaces/aletrn/lisa-on-cuda) ZeroGPU space.
 
-## Segment Anything models
+This [home page project](https://huggingface.co/spaces/aletrn/samgis-lisa-on-zero) is a plane Gradio interface that take a json in input to translate it to a geojson. More information about these API implementation [here](
+https://aletrn-samgis-lisa-on-zero.hf.space/docs). On this [blog page](https://trinca.tornidor.com/projects/lisa-adapted-for-samgis) you can find more details, including some request and response examples with the geojson map representations.
 
-It's possible to prepare the model files using https://github.com/vietanhdev/samexporter/ or using the ones 
-from https://huggingface.co/aletrn/sam-quantized (copy them within the folder `/machine_learning_models`).
+You can also find the alternative map interface [here](https://aletrn-samgis-lisa-on-zero.hf.space/lisa/) useful to create on the fly the payload requests and to represent the geojson response.
 
-## SamGIS - HuggingFace version
+## Custom environment variables for HuggingFace CUDA Space
 
-The SamGIS HuggingSpace url is https://huggingface.co/spaces/aletrn/samgis-lisa-on-cuda.
-Build the docker image this way:
+Fundamental environment variables you need are:
 
 ```bash
-# clean any old active containers
-docker stop $(docker ps -a -q); docker rm $(docker ps -a -q)
-
-# build the base docker image from the repository root folder using ARGs:
-# - DEPENDENCY_GROUP=fastapi used by poetry
-# VITE__MAP_DESCRIPTION, VITE__SAMGIS_SPACE used by 'docker build'
-(
-  set -o allexport && source <(cat ./static/.env|grep VITE__) && set +o allexport;
-  env|grep VITE__;
-  docker build . -f dockerfiles/dockerfile-samgis-base --progress=plain \
-  --build-arg DEPENDENCY_GROUP=fastapi \
-  --build-arg VITE__MAP_DESCRIPTION=${VITE__MAP_DESCRIPTION} \
-  --build-arg VITE__SAMGIS_SPACE=${VITE__SAMGIS_SPACE} \
-  --tag registry.gitlab.com/aletrn/gis-prediction
-)
-
-# build the image, use the tag "samgis-huggingface"
-docker build . --tag example-docker-namespace/samgis-huggingface --progress=plain
-```
-
-Run the container (keep it on background) and show logs
-
-```bash
-docker run  -d --name samgis-huggingface -p 7860:7860 example-docker-namespace/samgis-huggingface; docker logs -f samgis-huggingface
+XDG_CACHE_HOME="/data/.cache"
+PROJECT_ROOT_FOLDER="/home/user/app"
+WORKDIR="/home/user/app"
 ```
 
-Test it with curl using a json payload:
+Derived ones:
 
 ```bash
-URL=http://localhost:7860/infer_samgis
-curl -d@./events/payload_point_eolie.json -H 'accept: application/json' ${URL}
+MPLCONFIGDIR="/data/.cache/matplotlib"
+TRANSFORMERS_CACHE="/data/.cache/transformers"
+PYTORCH_KERNEL_CACHE_PATH="/data/.cache/torch/kernels"
+FASTAPI_STATIC="/home/user/app/static"
+VIS_OUTPUT="/home/user/app/vis_output"
+MODEL_FOLDER="/home/user/app/machine_learning_models"
+FOLDERS_MAP='{"WORKDIR":"/home/user/app","XDG_CACHE_HOME":"/data/.cache","PROJECT_ROOT_FOLDER":"/home/user/app","MPLCONFIGDIR":"/data/.cache/matplotlib","TRANSFORMERS_CACHE":"/data/.cache/transformers","PYTORCH_KERNEL_CACHE_PATH":"/data/.cache/torch/kernels","FASTAPI_STATIC":"/home/user/app/static","VIS_OUTPUT":"/home/user/app/vis_output"}'
 ```
 
-or better visiting the swagger page on http://localhost:7860/docs
-
-### Dependencies installation and local tests
-The docker build process needs only the base dependency group plus the `aws_lambda` or `fastapi` optional one.
-Install also the `test` and/or `docs` groups if needed.
-
-### Tests
+The function `build_frontend()` from lisa_on_cuda package create all the folders required for this project using the environment variable `FOLDERS_MAP`. That's useful for cache folders (XDG_CACHE_HOME, MPLCONFIGDIR, TRANSFORMERS_CACHE, PYTORCH_KERNEL_CACHE_PATH) because missing these can slow down the inference process. Also you could keep these folders in a permanent storage disk mounted on a custom path.
 
-Tests are defined in the `tests` folder in this project. Use PIP to install the test dependencies and run tests.
+To change the base relative url for custom frontend add the VITE_PREFIX environment variable, e.g.:
 
 ```bash
-python -m pytest --cov=samgis_lisa_on_cuda --cov-report=term-missing && coverage html
+VITE_INDEX_URL="/custom-url"
 ```
 
-### How to update the static documentation with sphinx
-
-This project documentation uses sphinx-apidoc: it's a tool for automatic generation of Sphinx sources that, using the autodoc
-extension, document a whole package in the style of other automatic API documentation tools. See the 
-[documentation page](https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html) for details.
-Run the command from the project root:
-
-```bash
-# missing docs folder (run from project root) initialize this way
-cd docs && sphinx-quickstart -p SamGIS -r 1.0.0 -l python --master index
-
-# update docs folder (from project root)
-sphinx-apidoc -f -o docs samgis
-```
-
-Then it's possible to generate the HTML pages
-
-```bash
-cd docs && make html && ../
-
-# to clean old files
-cd docs && make clean html && cd ../
-```
-
-The static documentation it's now ready at the path `docs/_build/html/index.html`.
-
-To create a work in progress openapi json or yaml file use
-
-- `extract-openapi-fastapi.py`
-- `extract-openapi-lambda.py` (useful to export the json schema request and response from lambda app api)
-
-### Handle dynamic folder creation
-
-it's possible to dynamically create a new support folder adding it to a json string env FOLDERS_MAP, e.g.:
-
-```json
-{
-    "WORKDIR": "/var/task",
-    "XDG_CACHE_HOME": "/data",
-    "PROJECT_ROOT_FOLDER": "/data",
-    "MPLCONFIGDIR": "/data/.cache/matplotlib",
-    "TRANSFORMERS_CACHE": "/data/.cache/transformers",
-    "PYTORCH_KERNEL_CACHE_PATH": "/data/.cache/torch/kernels",
-    "FASTAPI_STATIC": "/var/task/static",
-    "VIS_OUTPUT": "/data/vis_output"
-}
-```
-
-The python script create_folders_and_variables_if_not_exists.py will read this env variable, removing any files that exists with these pathnames and assert the correct creation of all the folders. Also these folders must exist as env variables, so the script assert that an env variable exists with its path.
-
-It's possible to use the project in a bare metal installation (not within a docker container). To do this
-
-- download this project
-- prepare a virtualenv and install the python dependencies
-- install nodejs LTS
-- create a .env_source file (in this case `HOME=/home/jovyan`)
-
-  ```bash
-  export FOLDERS_MAP='{"WORKDIR":"/home/jovyan/workspace/samgis-lisa-on-cuda","XDG_CACHE_HOME":"/home/jovyan/.cache","PROJECT_ROOT_FOLDER":"/home/jovyan/","MPLCONFIGDIR":"/home/jovyan/.cache/matplotlib","TRANSFORMERS_CACHE":"/home/jovyan/.cache/transformers","PYTORCH_KERNEL_CACHE_PATH":"/home/jovyan/.cache/torch/kernels","FASTAPI_STATIC":"/home/jovyan/workspace/samgis-lisa-on-cuda/static","VIS_OUTPUT":"/home/jovyan/workspace/samgis-lisa-on-cuda/vis_output"}'
-  export WORKDIR="$HOME/workspace/samgis-lisa-on-cuda"
-  export XDG_CACHE_HOME="$HOME/.cache"
-  export PROJECT_ROOT_FOLDER="$HOME/"
-  export MPLCONFIGDIR="$HOME/.cache/matplotlib"
-  export TRANSFORMERS_CACHE="$HOME/.cache/transformers"
-  export PYTORCH_KERNEL_CACHE_PATH="$HOME/.cache/torch/kernels"
-  export FASTAPI_STATIC="$HOME/workspace/samgis-lisa-on-cuda/static"
-  export VIS_OUTPUT="$HOME/workspace/samgis-lisa-on-cuda/vis_output"
-  export WRITE_TMP_ON_DISK=${VIS_OUTPUT}
-  ```
+## About HuggingFace space dependencies
 
-- execute the script `baremetal_entrypoint.sh` instead than `docker_entrypoint.sh`.
+For this demo simply installing `samgis-lisa` already brings all the needed dependencies. 
+Now `lisa.lisa_predict()` has the optional argument `inference_decorator` useful in case of use on ZeroGPU hardware or similar.

app.py

@@ -0,0 +1,259 @@
+import json
+import os
+from pathlib import Path
+from typing import Callable, NoReturn
+
+from asgi_correlation_id import CorrelationIdMiddleware
+import gradio as gr
+from starlette.responses import JSONResponse
+import structlog
+import uvicorn
+from dotenv import load_dotenv
+from fastapi import FastAPI, HTTPException, Request, status
+from fastapi.exceptions import RequestValidationError
+from fastapi.responses import FileResponse, HTMLResponse
+from fastapi.staticfiles import StaticFiles
+from fastapi.templating import Jinja2Templates
+from pydantic import ValidationError
+from samgis_core.utilities import create_folders_if_not_exists
+from samgis_core.utilities import frontend_builder
+from samgis_core.utilities.session_logger import setup_logging
+from samgis_web.utilities.constants import GRADIO_EXAMPLES_TEXT_LIST, GRADIO_MARKDOWN, GRADIO_EXAMPLE_BODY_STRING_PROMPT
+from samgis_web.utilities.type_hints import StringPromptApiRequestBody
+
+
+load_dotenv()
+project_root_folder = Path(globals().get("__file__", "./_")).absolute().parent
+workdir = Path(os.getenv("WORKDIR", project_root_folder))
+model_folder = Path(project_root_folder / "machine_learning_models")
+
+log_level = os.getenv("LOG_LEVEL", "INFO")
+setup_logging(log_level=log_level)
+app_logger = structlog.stdlib.get_logger()
+app_logger.info(f"PROJECT_ROOT_FOLDER:{project_root_folder}, WORKDIR:{workdir}.")
+
+folders_map = os.getenv("FOLDERS_MAP", "{}")
+markdown_text = os.getenv("MARKDOWN_TEXT", "")
+examples_text_list = os.getenv("EXAMPLES_TEXT_LIST", "").split("\n")
+example_body = json.loads(os.getenv("EXAMPLE_BODY", "{}"))
+mount_gradio_app = bool(os.getenv("MOUNT_GRADIO_APP", ""))
+
+static_dist_folder = workdir / "static" / "dist"
+input_css_path = os.getenv("INPUT_CSS_PATH", "src/input.css")
+vite_gradio_url = os.getenv("VITE_GRADIO_URL", "/gradio")
+vite_index_url = os.getenv("VITE_INDEX_URL", "/")
+vite_samgis_url = os.getenv("VITE_SAMGIS_URL", "/samgis")
+vite_lisa_url = os.getenv("VITE_LISA_URL", "/lisa")
+fastapi_title = "samgis-lisa-on-zero2"
+app = FastAPI(title=fastapi_title, version="1.0")
+
+
+@app.middleware("http")
+async def request_middleware(request, call_next):
+    from samgis_web.web.middlewares import logging_middleware
+
+    return await logging_middleware(request, call_next)
+
+
+def get_example_complete(example_text):
+    example_dict = dict(**GRADIO_EXAMPLE_BODY_STRING_PROMPT)
+    example_dict["string_prompt"] = example_text
+    return json.dumps(example_dict)
+
+
+def get_gradio_interface_geojson(fn_inference: Callable):
+    with gr.Blocks() as gradio_app:
+        gr.Markdown(GRADIO_MARKDOWN)
+
+        with gr.Row():
+            with gr.Column():
+                text_input = gr.Textbox(lines=1, placeholder=None, label="Payload input")
+                btn = gr.Button(value="Submit")
+            with gr.Column():
+                text_output = gr.Textbox(lines=1, placeholder=None, label="Geojson Output")
+
+        gr.Examples(
+            examples=[
+                get_example_complete(example) for example in GRADIO_EXAMPLES_TEXT_LIST
+            ],
+            inputs=[text_input],
+        )
+        btn.click(
+            fn_inference,
+            inputs=[text_input],
+            outputs=[text_output]
+        )
+    return gradio_app
+
+
+def handle_exception_response(exception: Exception) -> NoReturn:
+    import subprocess
+    project_root_folder_content = subprocess.run(
+        f"ls -l {project_root_folder}/", shell=True, universal_newlines=True, stdout=subprocess.PIPE
+    )
+    app_logger.error(f"project_root folder 'ls -l' command output: {project_root_folder_content.stdout}.")
+    workdir_folder_content = subprocess.run(
+        f"ls -l {workdir}/", shell=True, universal_newlines=True, stdout=subprocess.PIPE
+    )
+    app_logger.error(f"workdir folder 'ls -l' command stdout: {workdir_folder_content.stdout}.")
+    app_logger.error(f"workdir folder 'ls -l' command stderr: {workdir_folder_content.stderr}.")
+    app_logger.error(f"inference error:{exception}.")
+    raise HTTPException(
+        status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Internal server error on inference"
+    )
+
+
+@app.get("/health")
+async def health() -> JSONResponse:
+    from samgis_web.__version__ import __version__ as version_web
+    from samgis_core.__version__ import __version__ as version_core
+    from lisa_on_cuda.__version__ import __version__ as version_lisa_on_cuda
+    from samgis_lisa.__version__ import __version__ as version_samgis_lisa
+
+    app_logger.info(f"still alive, version_web:{version_web}, version_core:{version_core}.")
+    app_logger.info(f"still alive, version_lisa_on_cuda:{version_lisa_on_cuda}, version_samgis_lisa:{version_samgis_lisa}.")
+    return JSONResponse(status_code=200, content={"msg": "still alive..."})
+
+
+def infer_lisa_gradio(request_input: StringPromptApiRequestBody) -> str:
+    from samgis_lisa.io_package.wrappers_helpers import get_parsed_bbox_points_with_string_prompt
+    from samgis_lisa.prediction_api import lisa
+    from samgis_lisa.utilities.constants import LISA_INFERENCE_FN
+
+    app_logger.info("starting lisa inference request...")
+
+    try:
+        import time
+
+        time_start_run = time.time()
+        body_request = get_parsed_bbox_points_with_string_prompt(request_input)
+        app_logger.info(f"lisa body_request:{body_request}.")
+        try:
+            source = body_request["source"]
+            source_name = body_request["source_name"]
+            app_logger.debug(f"body_request:type(source):{type(source)}, source:{source}.")
+            app_logger.debug(f"body_request:type(source_name):{type(source_name)}, source_name:{source_name}.")
+            app_logger.debug(f"lisa module:{lisa}.")
+            output = lisa.lisa_predict(
+                bbox=body_request["bbox"], prompt=body_request["prompt"], zoom=body_request["zoom"],
+                source=source, source_name=source_name, inference_function_name_key=LISA_INFERENCE_FN
+            )
+            duration_run = time.time() - time_start_run
+            app_logger.info(f"duration_run:{duration_run}.")
+            body = {
+                "duration_run": duration_run,
+                "output": output
+            }
+            dumped = json.dumps(body)
+            app_logger.info(f"json.dumps(body) type:{type(dumped)}, len:{len(dumped)}.")
+            app_logger.debug(f"complete json.dumps(body):{dumped}.")
+            return dumped
+        except Exception as inference_exception:
+            app_logger.error(f"inference_exception:{inference_exception}.")
+            app_logger.error(f"inference_exception, request_input:{request_input}.")
+            raise HTTPException(status_code=500, detail="Internal Server Error")
+    except ValidationError as va1:
+        app_logger.error(f"validation error: {str(va1)}.")
+        app_logger.error(f"ValidationError, request_input:{request_input}.")
+        raise RequestValidationError("Unprocessable Entity")
+
+
+@app.post("/infer_lisa")
+def infer_lisa(request_input: StringPromptApiRequestBody) -> JSONResponse:
+    dumped = infer_lisa_gradio(request_input=request_input)
+    app_logger.info(f"json.dumps(body) type:{type(dumped)}, len:{len(dumped)}.")
+    app_logger.debug(f"complete json.dumps(body):{dumped}.")
+    return JSONResponse(status_code=200, content={"body": dumped})
+
+
+@app.exception_handler(RequestValidationError)
+def request_validation_exception_handler(request: Request, exc: RequestValidationError) -> JSONResponse:
+    from samgis_web.web import exception_handlers
+
+    return exception_handlers.request_validation_exception_handler(request, exc)
+
+
+@app.exception_handler(HTTPException)
+def http_exception_handler(request: Request, exc: HTTPException) -> JSONResponse:
+    from samgis_web.web import exception_handlers
+
+    return exception_handlers.http_exception_handler(request, exc)
+
+
+create_folders_if_not_exists.folders_creation(folders_map)
+write_tmp_on_disk = os.getenv("WRITE_TMP_ON_DISK", "")
+app_logger.info(f"write_tmp_on_disk:{write_tmp_on_disk}.")
+if bool(write_tmp_on_disk):
+    try:
+        assert Path(write_tmp_on_disk).is_dir()
+        app.mount("/vis_output", StaticFiles(directory=write_tmp_on_disk), name="vis_output")
+        templates = Jinja2Templates(directory=str(project_root_folder / "static"))
+
+        @app.get("/vis_output", response_class=HTMLResponse)
+        def list_files(request: Request):
+
+            files = os.listdir(write_tmp_on_disk)
+            files_paths = sorted([f"{request.url._url}/{f}" for f in files])
+            print(files_paths)
+            return templates.TemplateResponse(
+                "list_files.html", {"request": request, "files": files_paths}
+            )
+    except (AssertionError, RuntimeError) as rerr:
+        app_logger.error(f"{rerr} while loading the folder write_tmp_on_disk:{write_tmp_on_disk}...")
+        raise rerr
+
+frontend_builder.build_frontend(
+    project_root_folder=workdir,
+    input_css_path=input_css_path,
+    output_dist_folder=static_dist_folder
+)
+app_logger.info("build_frontend ok!")
+
+templates = Jinja2Templates(directory="templates")
+
+app.mount("/static", StaticFiles(directory=static_dist_folder, html=True), name="static")
+# important: the index() function and the app.mount MUST be at the end
+# samgis.html
+app.mount(vite_samgis_url, StaticFiles(directory=static_dist_folder, html=True), name="samgis")
+
+
+@app.get(vite_samgis_url)
+async def samgis() -> FileResponse:
+    return FileResponse(path=str(static_dist_folder / "samgis.html"), media_type="text/html")
+
+
+# lisa.html
+app.mount(vite_lisa_url, StaticFiles(directory=static_dist_folder, html=True), name="lisa")
+
+
+@app.get(vite_lisa_url)
+async def lisa() -> FileResponse:
+    return FileResponse(path=str(static_dist_folder / "lisa.html"), media_type="text/html")
+
+
+# index.html (lisa.html copy)
+app.mount(vite_index_url, StaticFiles(directory=static_dist_folder, html=True), name="index")
+
+
+@app.get(vite_index_url)
+async def index() -> FileResponse:
+    return FileResponse(path=str(static_dist_folder / "index.html"), media_type="text/html")
+
+
+app_logger.info(f"creating gradio interface...")
+gr_interface = get_gradio_interface_geojson(infer_lisa_gradio)
+app_logger.info(f"gradio interface created, mounting gradio app on url {vite_gradio_url} within FastAPI...")
+app = gr.mount_gradio_app(app, gr_interface, path=vite_gradio_url)
+app_logger.info("mounted gradio app within fastapi")
+
+# add the CorrelationIdMiddleware AFTER the @app.middleware("http") decorated function to avoid missing request id
+app.add_middleware(CorrelationIdMiddleware)
+
+
+if __name__ == '__main__':
+    try:
+        uvicorn.run(host="0.0.0.0", port=7860, app=app)
+    except Exception as ex:
+        app_logger.error(f"fastapi/gradio application {fastapi_title}, exception:{ex}.")
+        print(f"fastapi/gradio application {fastapi_title}, exception:{ex}.")
+        raise ex

packages.txt

@@ -0,0 +1,2 @@
+npm
+nodejs

requirements.txt

@@ -0,0 +1 @@
+samgis-lisa==1.0.1

resources

@@ -0,0 +1 @@
+static/resources