MINISTRY OF EDUCATION AND TRAINING
HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION

FACULTY FOR HIGH-QUALITY TRAINING



CAPSTONE PROJECT
INFORMATION TECHNOLOGY

STUDY LAMBDA ARCHITECTURE AND BUILD A
DEMO APPLICATION

LECTURER: M.S. Quach Dinh Hoang
STUDENTS: Nguyen Thanh Phat - 19110101
Tran Duc Tuan

- 19110140

Ho Chi Minh City, August 2023
1

THE SOCIALIST REPUBLIC OF VIETNAM
Independence – Freedom– Happiness
-Ho Chi Minh City, August 11, 2023

GRADUATION PROJECT ASSIGNMENT
Student name: Nguyen Thanh Phat

Student ID: 19110101

Student name: Tran Duc Tuan

Student ID: 19110140

Major: Software Engineering

Class: 19110CLA4

Advisor: M.S. Quach Dinh Hoang
Date of assignment: 06/03/2023

Date of submission: 19/06/2023

1. Project title: Study Lambda architecture and build a demo application
2. Initial materials provided by the advisor: Advisor suggests me to read about some
reference sources or articles about Lambda architecture which will help me to do this
project
3. Content of the project: Combine Big Data tools together and get advantage of them.
Then learning about Lambda architecture and how it resolves problems about scalability
and complexity of Big Data systems.
4. Final product: A model that applies Lambda architecture to stream and process a
massive amount of data and a dashboard illustrates these data under charts.

CHAIR OF THE PROGRAM

ADVISOR

(Sign with full name)

(Sign with full name)

3

THE SOCIALIST REPUBLIC OF VIETNAM
Independence – Freedom– Happiness
-Ho Chi Minh City, August 11, 2023

ADVISOR’S EVALUATION SHEET
Student name: Nguyen Thanh Phat
Student ID: 19110101
Student name: Tran Duc Tuan
Student ID: 19110140
Major: Software Engineering
Project title: Study Lambda architecture and build a demo application
Advisor: M.S. Quach Dinh Hoang
EVALUATION
1. Content of the project:

...........................................................................................................................................
...........................................................................................................................................
...........................................................................................................................................
2. Strengths:
...........................................................................................................................................
..........................................
...

...
doing this project, big data tools consume
too much RAM and the Lambda architecture is too complicated to a complete
system which satisfies the desired properties in big data systems. Besides, it needs to have
many machines to scale up and the API which produces a huge amount of real-time data
per second to fetch into the system.
5.3 Further development direction
In the future, if we had more time and resources for this project, we could have improved
and accomplished the following work:
• Deploy the architecture into cloud services such as Amazon Web Server EC2 or
containerize it in Kubernetes.
• Deeper research on Lambda architecture and improve the desired properties in Big
Data systems such as scalability, performance or fault tolerance.
•

Replace all Big Data tools like Airflow, Spark and Kafka to Amazon service tools
to a serverless architecture and leverage all benefits of using cloud.
61

REFERENCES
[1] Nathan Marz and James Warren, Big Principles and Best Practices of Scalable
Realtime Data Systems, April 29, 2015.
[2] Twitter API Documentation, [Online]. Available:
https://developer.twitter.com/en/docs/platformoverview#:~:text=The%20Twitter%20API%20is%20a,Spaces.
[3] Twitter rate limits, [Online]. Available:
https://developer.twitter.com/en/docs/twitter-api/rate-limits.
[4] Real Estate Sales 2001-2020 GL, [Online]. Available: https://data.ct.gov/Housingand-Development/Real-Estate-Sales-2001-2020-GL/5mzw-sjtu.
[5] SnapLogic, What is the Snowflake Data Platform?, 14 January 2022. [Online].
Available: https://www.snaplogic.com/blog/snowflake-data-platform.
[6] Key Concepts &amp Architecture Snowflake, [Online]. Available:
https://docs.snowflake.com/en/user-guide/intro-key-concepts.
[7] Apache Airflow documentation, [Online]. Available:
https://airflow.apache.org/docs/apache-airflow/stable/index.html.
[8] A. Airflow, Running Airflow in Docker, [Online]. Available:
https://airflow.apache.org/docs/apache-airflow/stable/howto/dockercompose/index.html.
[9] abhilash, What is lambda architecture?, [Online]. Available:
https://www.educative.io/answers/what-is-lambda-architecture.
[10] N. Vaidya, Apache Spark Architecture – Spark Cluster Architecture Explained,
[Online]. Available: https://www.edureka.co/blog/spark-architecture/.

62

[11] P. Pro, Apache Kafka Architecture and Its Components-The A-Z Guide, [Online].
Available: https://www.projectpro.io/article/apache-kafka-architecture-/442.

63