Loading...

End To End Data Engineering Project With Snowflake | Parquet, JSON & CSV Data Files

2.2万回視聴・高評価数472 472________

In this exciting hands on tutorial , we will deep dive into the world of End-to-End Data Engineering Project using Snowflake Snowpark (Python). We will explore the Snowpark's File API & Data Frame API capabilities to tackle Amazon sales order data project for mobile products. With step-by-step demonstrations and insightful explanations, this content will guide you through the entire journey, from data ingestion and transformation to advanced analytics and visualisation. Whether you're a data enthusiast, aspiring data engineer, or simply curious about the world of data engineering, this video is a must-watch. Get ready to unlock the full potential of Snowflake Snowpark and revolutionise your data projects.

Once you complete this end-to-end snowpark python based data engineering project (ETL), you would be able answer of following questions

1. How to load large amount of data set from local machine to internal stages?
2. How to load delta data set from local machine to internal stages?
3. How to use copy command using Snowpark File API?
4. How to use Snowpark data frame API for complex transformation?
5. How to draw simple dashboard using snowsight dashboard along with filters?

🚀🚀 Social Links 🚀🚀
-----------------------------------------------------------
✏ Instagram: www.instagram.com/learn_dataengineering/
✏ Twitter: twitter.com/de_simplified
✏ Facebook: fb.me/learndataengineering
✏ GitHub : github.com/TopperTips
✏ Website: toppertips.com/

🚀🚀 Data Engineering Simplified Snowflake Cheat Sheets 🚀🚀
------------------------------------------------------------------------------------------------------
➥ Complete Snowflake Tutorial Cheat Sheet - rebrand.ly/maj4l6x
➥ Complete Snowflake JSON Guide Cheat Sheet - rebrand.ly/d52cef

🚀🚀 Snowpark Medium Article For Source Code 🚀🚀
---------------------------------------------------------------------------------------
medium.com/@data-engineering-simplified/744c1d5a8d…
gitlab.com/toppertips/snowflake-work/-/tree/main/s…

🚀🚀 Snowpark End 2 End Use Case Video Sections 🚀🚀
----------------------------------------------------------------------------
00:00:00 Introduction
00:02:04 Welcome To DE Simplified
00:02:37 Sample Code - Git Location
00:04:06 Logical Data Flow
00:06:11 Layered Design Approach Using Schema
00:08:34 Snowflake Logical Data Model
00:09:49 Amazon Retail Sales - Mobile Data Set
00:11:30 Step-01 - Create User & VWH
00:14:30 Step-02 - Create DB & Schemas
00:14:30 Step-03 - Create Internal Stage
00:17:47 Git Location (Data Set)
00:19:27 Data Processing Rule
00:14:30 Step-04 - Load Data To Stage Location
00:24:27 Step-05 - File Format Creation
00:30:07 Step-06 - Loading Forex Data
00:31:39 Step-07 - Internal Stage To Source Schema
00:41:53 Step-08 - Source To Curated
00:53:20 Step-09 - Curated To Consumption (Dim Modelling)
01:08:59 Step-10 - Snowsight dashboard
01:10:47 Step-11 - Delta Processing
01:17:47 Step-12 - Additional Dashboard


🚀 🚀 Everything About Snowpark Playlist 🚀 🚀
--------------------------------------------------------------------------------
➥ What Is Snowpark, Introduction & Architecture? (01/15    • #01 | What is Snowpark in Snowflake  )
➥ What is NOT Snowpark? (02/15    • #02 | What Is NOT Snowpark? | Learn o...  .)
➥Who Should Learn Snowpark? (03/15    • #03  | Who Should Learn Snowpark?  )
➥Where To Learn Snowpark From? (04/15 coming soon...)
➥Can A SQL Developer Learn Snowpark? (05/15 coming soon...)
➥Can we write snowpark in Python, Scala or Java Programming Languages? (06/15 coming soon...)
➥Can I run snowpark in Databricks notebooks? (07/15 coming soon...)
➥Can Snowpark run inside Snowflake Cloud Data Warehouse?(08/15 coming soon...)
➥Will Apache Spark Survive? (09/15 coming soon...)
➥Snowpark and Future of ADF, Databricks & Azure Synapse Analytics? (10/15 coming soon...)
➥Apache Spark to Snowflake Snowpark Migration Roadmap? (11/15 coming soon...)
➥Snowpark Python Vs Snowpark Scala Vs. Snowpark Java? (12/15 coming soon...)

🚀 🚀 Other Playlist By Data Engineering Simplified 🚀 🚀
----------------------------------------------------------------------------------------------
➥ Complete Snowflake Master Class 🌐 bit.ly/snowflake-tutorial
➥ Data Loading In Snowflake Master Class 🌐 bit.ly/load-data-into-snowflake
➥ SnowPro Certification Mock Test Papers 🌐 bit.ly/snowpro-mock-test
➥ SnowPro Certification Guide 🌐 bit.ly/snowpro-certification-v1

#snowpark
#snowflake
#snowflaketutorial
#snowflakedatawrehouse
#snowflakecomputing
#clouddatawarehouse
#snowparktutorial



Disclaimer: All snowflake-related learning materials and tutorial videos published in this channel are the personal opinions of the data engineering simplified team and they're neither authorised by nor ass

コメント