Computing with Data
elgeish
585.6K views
02 Chapter 3
Bash - Introduction Bash - Variables Bash - Pipes Bash - Loops Bash - Conditional Logic Bash - Aliases Command Prompt - Introduction Command Prompt - Variables Command Prompt - Pipes Command Prompt - Loop Command Prompt - Conditional Logic PowerShell - Introduction PowerShell - Pipes PowerShell - Variables PowerShell - Loops PowerShell - Conditional Logic Processes in Linux - Part I Processes in Linux - Part II Processes in Windows Files in Linux Displaying Files Moving, Copying, and Removing Files and Directories Wildcard Characters Soft Links Listing Directory Contents The PATH Environment Variable Compression in Linux Bash Initialization File Script Files Files in Windows Hierarchy of Directories Compression in Windows Users and Permissions in Linux Users and Permissions in Windows Redirecting Input and Output in Linux Redirecting Input and Output in Windows Working on Remote Linux Computers
03 Chapter 4
Compilation Variables Scope Operators Type Conversions References Pointers Arrays Preprocessor and Namespaces - Part I Preprocessor and Namespaces - Part II Strings - Part I Strings - Part II Input - Part I Input - Part II Input - Part III Output If-Else Clauses While-Loops For-Loops - Part I For-Loops - Part II For-Loops - Part III Functions Return Value Function Parameters Function Definition and Function Declaration Scope of Function Variables Pointer Parameters Reference Parameters Recursion - Part I Recursion - Part II Recursion - Part III Recursion - Part IV Passing Arguments to Main Overloading Functions Structs - Part I Structs - Part II Structs - Part III Structs - Part IV Structs - Part V Classes Constructors - Part I Constructors - Part II The Destructor The Copy Constructor The Converting Constructor The Implicit Converting Constructor The Explicit Converting Constructor Operator Overloading Friend Functions and Classes Inheritance Polymorphism Static Binding Virtual Functions Static Variables and Functions - Part I Static Variables and Functions - Part II Dynamic Memory Allocation - Part I Dynamic Memory Allocation - Part II Dynamic Memory Allocation - Part III Smart Pointers - Part I Smart Pointers - Part II Smart Pointers - Part III Template Functions - Part I Template Functions - Part II Template Classes Vectors Arrays Sets Maps - Part I Maps - Part II Unordered Maps
05 Chapter 6
Getting Started Objects Importing Modules Executing Shell Commands Scalar Data Types Strings Duck Typing Tuples Lists Ranges Slicing Sets Dictionaries Counters Dictionaries with Default Values Hashable Objects List Comprehensions Set Comprehensions Dictionary Comprehensions Nested Comprehensions Control Flow The Empty Statement Functions - Part I Functions - Part II Functions - Part III Classes Inheritance The Empty Class The NumPy Package Linear Algebra Random Number Generation The SciPy Package The pandas Package The scikit-learn Package Clustering Classification Regression Reading and Writing Data in Text Format Reading and Writing Ndarrays Reading and Writing Dataframes Material Differences between Python 3 and 2
07 Chapter 8
Graphing Data in R Datasets Packages Strip Plots Histograms Line Plots Kernel Functions Smoothing Histograms Using Gaussian Kernels Smoothing Histograms Using qplot Smoothing Histograms Using ggplot Scatter Plots Smoothing Scatter Plots Facets All-Pairs Relationships Contour Plots Box Plots qq-Plots Devices Data Preparation Graphing Data in Python Line Plots Histograms Contour Plots Surface Plots
08 Chapter 9
Missing Data in R - Part I Missing Data in R - Part II Missing Data in Python Outliers Skewness and Power Transformation - Part I Skewness and Power Transformation - Part II Binning Indicator Variables Random Sampling, Partitioning, and Shuffling Concatenations and Joins Reshaping Data The Split-Apply-Combine Framework
09 Chapter 10
Thread Safety - Part I Thread Safety - Part II Volatility - Part I Volatility - Part II Synchronization Ineffectual Synchronization - Part I Ineffectual Synchronization - Part II Ineffectual Synchronization - Part III Synchronization vs. Volatility Starvation Deadlocks - Part I Deadlocks - Part II Deadlocks - Part II The Producer-Consumer Problem - Part I The Producer-Consumer Problem - Part II The Producer-Consumer Problem - Part III The Producer-Consumer Problem - Part IV The Producer-Consumer Problem - Part V The Producer-Consumer Problem - Part VI Reader-Writer Locks - Part I Reader-Writer Locks - Part II Reader-Writer Locks - Part III Reader-Writer Locks - Part IV Reentrant Locks - Part I Reentrant Locks - Part II Reentrant Locks - Part III Reentrant Locks - Part IV Thread Pools - Part I Thread Pools - Part II Thread Pools - Part III Scheduled Executor Services Futures Streams ParSeq - Part I ParSeq - Part II ParSeq - Part III ParSeq - Part IV File Locks Distributed Locks Linearizable Counters
qq-Plots
Quantile-quantile plots, also known as qq-plots, are useful for comparing two datasets, one of which may be sampled from a certain distribution. They are essentially scatter plots of the quantiles of one dataset vs. the quantiles of another:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
set.seed(1) # constant seed for reproducibility
df = data.frame(samples = c(rnorm(200, 1, 1),
rnorm(200, 0, 1),
rnorm(200, 0, 2)))
df$parameter[1:200] = "$\\mathcal{N}(1, 1)$"
df$parameter[201:400] = "$\\mathcal{N}(0, 1)$"
df$parameter[401:600] = "$\\mathcal{N}(0, 2)$"
qplot(samples,
bins = 30,
facets = parameter~.,
geom = "histogram",
main = "Datasets",
data = df)
# We compute below the qq-plots of these three datasets (y axis)
# vs. a sample drawn from the N(0, 1) distribution (x axis)
ggplot(df, aes(sample = samples)) +
stat_qq() +
facet_grid(.~parameter) +
ggtitle("Quantile-Quantile of Datasets")
# Compare two distributions and show how the $t$-distribution has tails that
# are heavier than the Gaussian N(0, 1) distribution
x_grid = seq(-6, 6, length.out = 200)
df = data.frame(density = dnorm(x_grid, 0, 1))
df$tdensity = dt(x_grid, 1.5)
df$x = x_grid
ggplot(df, aes(x = x, y = density)) +
geom_area(fill = I("grey")) +
geom_line(aes(x = x, y = tdensity)) +
ggtitle("$\\mathcal{N}(0, 1)$ (shaded) and $t$-distribution with 1.5 Degrees of Freedom")
df$samples = rnorm(200, 0, 1)
pm = list(df = 1.5)
ggplot(df, aes(sample = samples)) +
stat_qq(distribution = qt, dparams = pm) +
ggtitle("Quantile-Quantile of the Two Distributions Above")
Create your playground on Tech.io
This playground was created on Tech.io, our hands-on, knowledge-sharing platform for developers.