Introduction to Big Data Testing using Hive and HQL
Course Summary
This one day course of lectures and hands-on training is designed to provide students with the foundation necessary for testing Big Data. The course covers several common transformation tests and the HQL syntax required to retrieve the data in order to perform the test.
Intended Audience
- Data Quality Teams
- Analysts
- Automation Engineers
- Quality Assurance Analysts
- Project Managers
- anyone involved with providing software quality for Big Data
- Course Objectives
- Course Outline
- Prerequisites
At the end of the course, you will be able to:
- understand Big Data structures and architectures
- implement a successful process for Big Data testing
- create and execute the most common transformation tests.
- use ETL mapping documents to structure your tests.
- understand the basic challenges to overcome when performing Big Data testing.
Big Data Overview
- Understanding Hadoop Architecture
- Understanding the challenges of Big Data Testing
- Understanding ETL Mapping Documents
- Overview of Transformation Types
- Big Data Testing Comparison Methods
One-to-One Mapping Transformation Test
- HQL commands for returning data
- Compare Source table to Target table
One-to-One Mapping with Filters Transformation Test
- HQL commands for filtering data
- Logical operators for data filtering
- HQL commands for sorting data
- Compare source table to a subset in target table
Join Mapping Transformation Test
- HQL commands for joining data from different tables with matching records
- HQL commands for joining data from different tables with matching record
- Using an Alias
- Compare records from a source in multiple target tables
Field Merging and Splitting Transformation Test
- String concatenation
- Utilization of HQL String functions
- Compare multiple source columns to a single target column.
Type Casting with Formatting Transformation Test
- Data Field Casting
- String Type Conversion Functions
- Date Math functions
- Compare source columns to a target column where casting and formatting is required.
Translation and Lookup Transformation Test
- Using Case statements
- Utilizing Lookup tables
- Decoding field values
- Compare a source column where a transitioned target column
Statistical Tests
- Aggregate Functions
- Compare a target field that is comprised of an aggregation of a source field.
Understanding of basic software testing processes
Students also registered for...
Big Data Immersion
Learn More
Advanced Big Data Testing using Hive and HQL
Learn More