Back to AI News
arxiv

Automated Keyword Generation Using LLMs in a High-Volume Data Virtualization Service

This paper presents a solution for automatically generating keywords for virtual datasets in a data virtualization service (DVS) using large language models (LLMs). The authors propose five strategies for selecting text data for LLM processing and evaluate them across three LLM generators and two embedding techniques in five application case studies. Results show that strategy performance depends on text description quality, leading to a multi-strategy implementation in a new keyword generation service. The work highlights the importance of automated keyword generation for dataset-rich, frequently accessed DVS environments.

AI Brief

The paper offers a concrete, recent approach to automating metadata generation for data discovery, which is valuable for practitioners managing large-scale data virtualization systems.

arXiv:2610.41893v1 Announce Type: new Abstract: In modern data infrastructure, a data virtualization service (DVS) allows users to create many versions of source datasets for different processing tasks. For example, a multivariate or multi-modality dataset can be used for different machine learning tasks (e.g., classification, detection, segmentation, forecasting, and so on). Data selection, participation, integration, and preprocessing in different tasks result in many virtual datasets [Khan et al., SSC 2026]. Meanwhile, different users need to discover existing virtual datasets frequently and easily to avoid duplicate efforts. To enable an effective search engine in a DVS, it is desirable to automatically generate keywords for each virtual dataset. In this work, we present a solution for generating keywords for virtual datasets using large language models (LLMs). We propose five strategies for selecting different collections of text data for processing by the LLMs. We applied the five strategies to virtual datasets created in five application case studies with different source datasets. We conducted experiments with three LLM generators and two embedding techniques. The experiments show that the performances of the five strategies depend on the quality of the text descriptions. The results help us adopt a multi-strategy implementation of a new keyword generation service (KGS) as part of an extended DVS. The work demonstrates that the capability of automated keyword generation is indispensable for a high-volume DVS that is dataset-rich and frequently accessed.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!