student
Russian Federation
student
Russian Federation
employee
Russian Federation
VAK Russia 2.3.6
UDC 004.658
This paper examines the challenges associated with the use of ETL and ELT processes within data processing systems. These processes are of pivotal significance in the realms of data integration and analysis. However, a number of challenges that necessitate a robust foundation in scientific and theoretical understanding accompanies the implementation of these processes. Purpose: to examine the challenges associated with the development of data processing systems, with a particular focus on the significant time input associated with configuration and optimization, and issues concerning data quality, scalability, and security. Results: a number of recommendations and solutions have been proposed for choosing between ETL and ELT, for the optimization of processes, and for insured data quality and security. Practical significance: new approaches have been developed to improve the efficiency of work with corporate data storage. Discussion: the necessity to optimize ETL and ELT processes, to improve data quality and security, and to make a justified choice of data storage architecture for effective information management is emphasized.
ETL, ELT, data integration, data quality issues, scalability, data security, architecture selection, process optimization, data storage
1. Ostudina K. A., Kireev V. S. Arkhitektura khraneniya i obrabotki tekstovykh otkrytykh dannykh [Architecture of Storage and Processing of Textual Open Data], Intellektualnye tekhnologii v nauke i obrazovanii: materialy Mezhdunarodnoy nauchno-prakticheskoy konferentsii [Intelligent Technologies in Science and Education: Materials of the International Scientific and Practical Conference], Novocherkassk, Russia, November 24–25, 2023. Novocherkassk, Lik Publishing House, 2023, Pp. 102–108. (In Russian) EDN: https://elibrary.ru/BNMYYT
2. Sokolov V. A., Ponomareva K. A. ETL-protsessy kak sposob organizatsii importa dannykh dlya obespecheniya funktsiy upravleniya [ETL processes as a way of organizing data import to ensure management functions], Prospekt Svobodnyy — 2023: materialy XIX Mezhdunarodnoy nauchnoy konferentsii studentov, aspirantov i molodykh uchenykh [Prospect Svobodny — 2023: Proceedings of the XIX International Scientific Conference for Undergraduate, Postgraduate, PhD Students and Early Career Researchers], Krasnoyarsk, Russia, April 24–29, 2023. Krasnoyarsk, Siberian Federal University, 2023, Pp. 3350–3353. (In Russian) EDN: https://elibrary.ru/KZSQXL
3. Upaeva P. V. Optimizatsiya ETL-protsessov: obzor otechestvennogo rynka [Optimization of ETL Processes: An Review of the Domestic Market], Vestnik Nauki, 2024, No. 6 (75), Vol. 4. Pp. 1218–1224. (In Russian) EDN: https://elibrary.ru/ALSVLJ
4. Pyanzin A. M., Laisha A. K., Anosov M. S. Obzor sovremennykh resheniy dlya khraneniya i strukturirovaniya dannykh v additivnykh tekhnologiyakh [Overview of Modern Solutions for Data Storage and Structuring in Additive Technologies], Tekhnologii additivnogo proizvodstva [Additive Fabrication Technologies], 2025, Vol. 3, No. 1, Pp. 43–48. (In Russian) EDN: https://elibrary.ru/CICUVT
5. Blokhin V. V., Losev V. S. Avtomatizatsiya protsessa aktualizatsii dannykh v vitrinakh: kak uprostit rabotu analitikov [Automation of the Process of Data Acquisition in Storefronts: How to Simplify theWork of Analysts], Finansovaya ekonomika [Financial Economics], 2024, No. 10, Pp. 100–103. (In Russian) EDN: https://elibrary.ru/EJXSHW
6. Tavtorkin N. O., Kulyashova N. M. ETL-protsessy v rabote s dannymi [ETL processes in working with data], Budushchee nauki — 2024: sbornik nauchnykh statey 11-y Mezhdunarodnoy molodezhnoy nauchnoy konferentsii [The Future of Science — 2024: Collection of Scientific Articles of the 11th International Youth Scientific Conference], Kursk, Russia, April 18–19, 2024, Vol. 4. Kursk, Universitetskaya Kniga Publishing House, 2024, Pp. 175–178. (In Russian) EDN: https://elibrary.ru/TASEXJ
7. Makarov V. V., Volchik O. V. Upravlenie dannymi v sistemakh menedzhmenta kachestva [Data Management in Quality Management Systems], Ekonomika i Kachestvo Sistem Svyazi, 2024, No. 4 (34), Pp. 4–13. (In Russian) EDN: https://elibrary.ru/ODTKQU
8. Labinskiy A. Yu. Programmnye sredstva obrabotki bolshikh obemov dannykh [Processing Software Big Data], Prirodnye i tekhnogennye riski (fiziko-matematicheskie i prikladnye aspekty) [Natural and Man-Made Risks (Physico-Mathematical and Applied Aspects)], 2024, No. 4, Pp. 45–52. DOI:https://doi.org/10.61260/2307-7476-2024-2023-4-45-52. (In Russian) EDN: https://elibrary.ru/JSUJTW
9. Mandrika O. S., Ermakov S. G. Modeli postroeniya korporativnogo khranilishcha dannykh [Models for Building an Enterprise Data Warehouse], Informatizatsiya i svyaz [Informatization and Communication], 2024, No. 2, Pp. 22–26. DOI:https://doi.org/10.34219/2078-8320-2024-15-2-22-26. (In Russian) EDN: https://elibrary.ru/KFXRLZ
10. Yakovenko E. S. Osnovnye ponyatiya i primery programmnykh instrumentov v sfere obrabotki bolshikh dannykh [Basic concepts and examples of software tools in the field of big data processing], Nauchnye issledovaniya v sovremennom mire: opyt, problemy i perspektivy razvitiya: sbornik nauchnykh statey po materialam XIV Mezhdunarodnoy nauchnoprakticheskoy konferentsii [Scientific Research in the Modern World: Experience, Problems and Development Prospects: Proceedings of the XIV International Scientific and Practical Conference], Ufa, Russia, April 19, 2024. Ufa, NIC Vestnik Nauki Publishing House, 2024, Pp. 563–567. (In Russian) EDN: https://elibrary.ru/EUIOWC
