Selected research and engineering projects, in reverse chronological order.

2026.03

OCRGate: OCR + VLM-Based Financial Document Verification System

Built an automated document verification pipeline that combines OCR, rule-based validation, and selective VLM fallback for robust financial document processing.

Python FastAPI PyTorch Transformers Streamlit Docker

Details

Role

• Designed a 4-stage validation pipeline for document classification, field verification, and quality assessment.
• Developed an OCR-first + selective VLM fallback framework using PaddleOCR, Qwen-VL, and confidence-based routing.
• Built an end-to-end service with FastAPI, SSE streaming, and a web-based review interface for real-time document inspection.

Results

• Reduced manual review requests by 39% using confidence-based OCR–VLM routing.
• Achieved a 94.7% Safe Pass Rate.

2025.01 – present

Fine-Tuning Hunyuan3D 2.1 for Industrial Plant Domain

Build a domain-specialized image-to-3D generation model tailored for industrial plant environments, where complex mechanical structures require higher geometric accuracy and visual consistency compared to generic pretrained models.

PyTorch Diffusers

Details

Role

• Led geometry-aware fine-tuning of Hunyuan3D for industrial image-to-3D generation, addressing structural defects, topology failures, and reconstruction artifacts.
• Designed a two-stage supervision framework combining surface normal alignment and SDF-based geometric refinement without modifying the base architecture.
• Built a 1,000-asset industrial benchmark from ABC and Objaverse datasets through semantic and shape-aware filtering for evaluation and validation.

Results

• Improved reconstruction quality by +0.34 IoU and +0.35 F-score while reducing Chamfer Distance and improving semantic similarity on industrial assets.

2025.07 – present

Structure-Aware Fine-Grained 3D Multimodal Embedding

Develop a structure-aware 3D multimodal embedding that maintains semantic alignment while capturing intra-class geometric variations, addressing the limitations of existing approaches that fail to differentiate fine-grained structural differences within the same category.

PyTorch HuggingFace Transformers

Details

Role

• Designed a geometry-aware intra-class hard negative sampling strategy to encourage the model to distinguish structurally different samples within the same semantic class.
• Built a GPT-based text generation pipeline that produces multi-view, part-level descriptions to ensure structural cues are reflected in the text modality.
• Proposed and implemented a joint learning framework combining multimodal contrastive loss (3D–text, 3D–image) with a structure-aware triplet loss applied on the 3D branch to enhance geometry-sensitive representation learning.

Results

• Improved FG3D chair fine-grained classification Top-1 accuracy by +2% compared to the OpenShape baseline.

2024.01 – 2024.12

Retrieval-Augmented 3D Mesh Generation Pipeline

Address limitations of end-to-end image-to-3D generation models (slow inference and insufficient asset quality for industrial use) by building a retrieval-based approach that first finds the closest 3D mesh from a large-scale database using multimodal embeddings, then generates texture map conditioned on the retrieved mesh.

PyTorch Diffusers

Details

Role

• Developed an image-to-3D retrieval pipeline using OpenShape multimodal embeddings to effectively match input images with the most similar 3D mesh assets.
• Evaluated multiple texture generation approaches, selected the best-performing model, and integrated it with the retrieval module to build a full end-to-end generation system.

Results

• 1 domestic conference paper, 1 international conference paper, and 1 domestic patent.

2023.01 – 2023.12

Webtoon Generation Automation System

Develop a system enabling webtoon creators to generate character images based on prompts/images through generative models, including style-conditioned generation trained on a specific artist's drawing style.

PyTorch HuggingFace Diffusers ControlNet

Details

Role

• Fine-tuned Stable Diffusion 1.5 using a webtoon character dataset via DreamBooth to learn consistent artist styles.
• Integrated ControlNet to support pose, facial expression, and background editing.
• Developed key modules of a web-based creative workflow tool supporting conti → sketch and sketch → coloring pipelines.

Results

• Achieved +0.1 LPIPS improvement and +0.71 recall in character identity recognition compared to the baseline model.
• Technology showcased at the 2023 Bucheon International Comics Festival (BICOF).
• 1 domestic journal paper, 1 domestic conference paper, 1 international conference paper, and 1 domestic patent.

2022.07 – 2022.08

Contrastive Learning for Knowledge-Grounded Dialogue Generation

Improve the ability of knowledge-grounded dialogue models to distinguish relevant information from irrelevant knowledge using a contrastive learning approach.

PyTorch

Details

Role

• Built the experimental environment for contrastive learning and adversarial perturbation–based training.
• Supported benchmarking and ablation studies on the Wizard of Wikipedia dataset, including evaluation and comparative analysis against baseline models.

Results

• Achieved measurable improvement with +6% increase in KF1 over the baseline.

2021.06 – 2022.12

AI-Based Landmine Detection Model Development

Build a 5-class object detection model capable of identifying landmine type and location from Ground-Penetrating Radar (GPR) imagery to support real-world field detection and reduce manual inspection workload.

PyTorch OpenCV

Details

Role

• Fine-tuned a Faster R-CNN object detection model using a custom GPR dataset.
• Designed preprocessing and augmentation strategies optimized for unique signal and texture characteristics in GPR imagery.

Results

• Successfully achieved the target recall score.