README.md

---
title: >-
  S2 Chunking: A Hybrid Framework for Document Segmentation Through Integrated
  Spatial and Semantic Analysis
emoji: 🧠
colorFrom: yellow
colorTo: indigo
sdk: static
pinned: true
license: cc0-1.0
short_description: 's2 chunking: a hybrid framework for document segmentation'
---
# 📄 S2 Chunking: A Hybrid Framework for Document Segmentation Through Integrated Spatial and Semantic Analysis


🧠 Authors: Prashant Verma 
📅 Published: Jan 2025  
🔗 [Read on arXiv](https://arxiv.org/abs/2501.05485)

## 🔍 Abstract

Document chunking is a critical task in natural language processing (NLP) that involves dividing a document into meaningful segments. Traditional methods often rely solely on semantic analysis, ignoring the spatial layout of elements, which is crucial for understanding relationships in complex documents. This paper introduces a novel hybrid approach that combines layout structure, semantic analysis, and spatial relationships to enhance the cohesion and accuracy of document chunks. By leveraging bounding box information (bbox) and text embeddings, our method constructs a weighted graph representation of document elements, which is then clustered using spectral clustering. Experimental results demonstrate that this approach outperforms traditional methods, particularly in documents with diverse layouts such as reports, articles, and multi-column designs. The proposed method also ensures that no chunk exceeds a specified token length, making it suitable for use cases where token limits are critical (e.g., language models with input size limitations)


## 🔧 Code & Models

The code used in this paper is available in this Space. You can also try the demo.