Learning Outcomes

Understand how parallel & distributed databases work

You understand

• the need for parallelism in data processing
• the different types of parallelism
• the different parallel processing architectures
• how to measure the performance of parallel processing
• the different data partitioning methods
• basic parallel query processing
• basic parallel programming using threads

Referencing modules: Parallel Databases

Understand how data storage is organized

You understand

• how to use an array of disks to store more data
• how to deal with disk failures
• how to scale storage systems to handle more and more data
• the different ways of connecting disk arrays to processing units
• how database records are organized and stored on disks

Referencing modules: Storage

Understand query processing in parallel databases

You understand

• the query processing process including query optimization
• how query execution plans are represented
• how query execution plans are manipulated algebraically to handle parallelism
• how different query operators work especially in a parallel environment
• how to analyze query execution plans and estimate their costs

Referencing modules: Query Processing

Understand concurrency control in parallel databases

You understand

• transaction semantics in a parallel/distributed setting
• theoretical model for transactions
• the different types of concurrency control algorithms
• locking-based concurrency control
• timestamp-based concurrency control

Referencing modules: Concurrency Control

Understand reliability in parallel databases

You understand

• Reliability and availability concepts
• Failures and recovery
• Local reliability protocols
• Distributed reliability protocols

Referencing modules: DBMS Reliability

Understand database replication

You understand

• consistency requirements for replication
• replication protocols for maintaining consistency
• lazy versus eager replication
• centralized versus distributed replication

Referencing modules: DBMS Replication

Understand Basic Peer-to-peer Concepts

You understand

• basic peer-to-peer concepts
• how distributed hash tables work
• how peer-to-peer concepts are applied in applications like bittorrent and bitcoin

Referencing modules: P2P Systems

Understand NoSQL systems

You understand

• what are NoSQL systems
• the difference between SQL and NoSQL systems
• the implications of NoSQL to application development

Referencing modules: NoSQL Systems

Understand Business Intelligence and Data Warehousing

You understand

• business intelligence, data integration & data warehousing
• star and snowflake schema used in data warehousing
• OLAP queries and their application to reports and dashboards
• the process of setting up a BI system
• how to integrate data from diverse data sources into a data warehouse.

Referencing modules: Data Warehousing

Understand How to Engineer Big Data Systems using Lambda Architecture

You understand

• the limitations of incremental architectures
• the desirable properties of big data systems
• what the lambda architecture is and how it relates to big data systems
• the different trade-offs
• the assumptions made on the data and the application state
• the data model used in lambda architecture

Referencing modules: Lambda Architecture

Understand How to Engineer Batch Processing Systems

You understand

• the requirements of the batch layer in lambda architecture
• how distributed file systems work
• how map-reduce works
• the trade-offs between alternatives
• how to model queries using pipe diagrams
• how to translate pipe diagrams to Map-Reduce jobs.
• how to model queries using JCascalog
• the relationship between queries and batch views

Referencing modules: Batch Layer : Map-Reduce

Understand How to Engineer the Serving Layer

You understand

• the requirements of the serving layer in lambda architecture
• the trade-offs between alternatives

Referencing modules: Serving Layer

Understand How to Engineer the Speed Layer

You understand

• the requirements of the speed layer in lambda architecture
• the trade-offs between alternatives

Referencing modules: Speed Layer

Understand the key ethical questions on Big Data

You understand

• what is ethics
• the different approaches to values to deal with ethical questions
• what are the ethical questions in Big Data

Referencing modules: Big Data Ethics

Apply parallel data processing concepts

You are able to

• install, configure and use data management tools
• extract, transform, load large data sets
• query and update large data sets to extract relevant information
• compare different data management solutions quantitatively

Referencing modules: Project