In this work, we present a novel architecture in a field programmable gate array (FPGA) for accelerating image processing algorithms based on conformal geometric algebra (CGA). This implementation specifically accelerates the execution of the Conformal Geometric Algebra Voting Scheme for detection of circles and lines in images. All geometric operations, such as meet, join, and transformation entities, are computed in hardware using FPGA. Other non-geometric operations, such as DBSCAN, are also executed in the FPGA. The full voting scheme consists of two main stages: a local stage computed using neighborhoods in the image and a global stage using the results of the local voting stage. In this implementation, we focused on the most computationally demanding stage: local voting. The top level design consists of five main hardware cores and an ARM processor interconnected using the AXI4 BUS. The cores exchange data using memory sections in a DRAM directly connected to the processing system side in the FPGA, which can be accessed by the hardware cores through the memory controller using a High performance AXI BUS. The design has been validated by comparing the results of the FPGA with the results previously obtained in a software reference application using real image data.