Distributed Data Management (WT 2018/19) - tele-TASK

0

The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization.Big Data Analytics is a multi-million dollar market that grows constantly! Data and the ability to control and use it is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements.In this lecture, we take a look a various technologies involved in building distributed, data-intensive systems. We discuss theoretical concepts (data models, encoding, replication, ...) as well as some of their practical implementations (Akka, MapReduce, Spark, ...). Since workload distribution is a concept which is useful for many applications, we focus in particular on data analytics.

Recent Episodes
  • Lecture Summary
    Feb 5, 2019 – 01:32:33
  • Distributed Query Optimization (1)
    Jan 22, 2019 – 01:26:03
  • Distributed Query Optimization (2)
    Jan 22, 2019 – 01:23:23
  • Processing Streams
    Jan 15, 2019 – 01:33:58
  • Stream Processing
    Jan 14, 2019 – 01:27:31
  • Transactions
    Jan 8, 2019 – 01:29:57
  • Consistency and Consensus
    Jan 7, 2019 – 01:30:02
  • Distributed Systems
    Dec 18, 2018 – 01:33:19
  • Spark - Hands On
    Dec 17, 2018 – 01:28:41
  • Apache Spark
    Dec 11, 2018 – 01:29:38
  • Beyond MapReduce
    Dec 10, 2018 – 01:29:41
  • Distributed File Systems and MapReduce
    Dec 4, 2018 – 01:27:15
  • Batch Processing
    Dec 3, 2018 – 01:29:20
  • Partitioning
    Nov 27, 2018 – 01:19:06
  • Replication
    Nov 26, 2018 – 01:26:56
  • Storage and Retrieval
    Nov 20, 2018 – 01:27:01
  • Data Models and Query Languages
    Nov 13, 2018 – 01:24:05
  • Patterns
    Nov 12, 2018 – 01:29:00
  • Akka Actor-Programming Part 2
    Nov 6, 2018 – 01:29:57
  • Akka Actor-Programming Hands-on
    Nov 5, 2018 – 01:28:43
  • Models of Dataflow
    Oct 30, 2018 – 01:27:18
  • Encoding and Evolution
    Oct 29, 2018 – 01:11:08
  • Data Warehouses
    Oct 23, 2018 – 01:17:07
  • Distributed DBMS
    Oct 22, 2018 – 01:29:01
  • Foundations
    Oct 16, 2018 – 01:23:06
  • Introduction
    Oct 15, 2018 – 01:12:41
Recent Reviews
Similar Podcasts
Disclaimer: The podcast and artwork on this page are property of the podcast owner, and not endorsed by UP.audio.