Nowadays, the set of data has been increasing together with its value. In order to systematically manage and utilize large amount of stored data, the concept called Data Architecture needed to be introduced.
Data Architecture is the concept of planning and set the structure of data designed by organizations to store useful data. Data Architecture works as a blueprint to illustrate the flow of collected data within the organizations through each factors. In some cases, the complexity of Data Architecture structure of each organizations might have differences when compared to others based on their functions and goals.
3 Essential Components of Data Architecture:
- Data Warehouse
- Data Mart
- Data Lake
Data Warehouse
Data Warehouse collects relative data set from different sources within the organizations which it works as a single central storage. After collecting, data flows through ETL process (Extract-Transform-Load). During this process, data will be transformed repeatedly to match the expected result.
Data Mart
Data Mart is similar to Data Warehouse but Data Mart is consist of smaller data set, it works as a minor subset of arranged data to allow users in those organizations gain access and utilize the data in those particular tasks. Dara Mart is suitable for supporting organizations, businesses, departments, and teams by enabling time-saving access to the data when dealing with large-sale data set.
Data Lake
Data Lake is the area where larger amount of raw data is stored without being transformed or categorized from its origin, while Data Warehouse stores processed data. High flexibility in storing data in Data Lake is beneficial to Data Scientists, Data Engineers, and Developers to explore and utilize the data in machine learning works.
Benefits and Importances of Data Architecture
- Facilitating the organizations and internal users to access the data with ease and also match their functions.
- Assisting in systematically storing data.
- Assisting in managing Data Life Cycle to be more convenient in the future
- Increasing data security and privacy.
Focal-point for engaging in Data Architecture
- Complexity of data usage by general users (non-developers), because stored data are diverse by their functions used by various users which are not only from IT department.
- Storing data on Cloud-native/ Cloud-Enable system. This will reduce limitations and allow data to be accessed with convenience.
- Designing the structure of data to match users’ objectives to reduce the difficulties in data usage. As a consequence of varied users, the data is needed to be categorized to fit users’ tasks properly.
