Embedding
Last updated: 2020-07-11
Checks: 7 0
Knit directory: requestival/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20200529) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version d563c54. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Ignored: code/_spotify_secrets.R
Ignored: data/.DS_Store
Ignored: data/raw/.DS_Store
Ignored: data/raw/requestival_24_files/
Ignored: data/raw/requestival_25_files/
Ignored: data/raw/requestival_26_files/
Ignored: data/raw/requestival_27_files/
Ignored: data/raw/requestival_28_files/
Ignored: data/raw/requestival_29_files/
Ignored: data/raw/requestival_30_files/
Ignored: data/raw/requestival_31_files/
Ignored: output/01-scraping.Rmd/
Ignored: output/02-tidying.Rmd/
Ignored: output/03-augmentation.Rmd/
Ignored: output/04-exploration.Rmd/
Ignored: output/05-embedding.Rmd/
Ignored: renv/library/
Ignored: renv/staging/
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/05-embedding.Rmd) and HTML (docs/05-embedding.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | d563c54 | Luke Zappia | 2020-07-11 | Add embedding |
source(here::here("code", "setup.R"))Introduction
After checking the individual variables the first thing I like to do with a new dataset is to perform some dimensionality reduction to get an overiew of it as a whole.
requestival <- read_tsv(
PATHS$augmented,
col_types = cols(
.default = col_double(),
DateTime = col_datetime(format = ""),
Song = col_character(),
Artist = col_character(),
Release = col_character(),
IsUnearthed = col_logical(),
UnearthedURL = col_character(),
SpotifyQuery = col_character(),
SpotifyURL = col_character(),
YouTubeQuery = col_character(),
YouTubeURL = col_character(),
SpotifyID = col_character(),
HasSpotify = col_logical(),
AlbumDate = col_date(format = ""),
Explicit = col_logical(),
IsMajor = col_logical()
)
) %>%
mutate(DateTime = with_tz(DateTime, "Australia/Sydney"))Chunk time: 0.22 secs
1 PCA
Let’s perform a PCA of the dataset. We need numeric values so we can only really do this on the songs that we have Spotify information for.
features <- c("Duration", "Loudness", "Tempo", "Valence", "Energy",
"Danceability", "Speechiness", "Acousticness", "Liveness")
features_mat <- requestival %>%
filter(HasSpotify) %>%
select(SpotifyID, !!features) %>%
distinct() %>%
column_to_rownames("SpotifyID") %>%
as.matrix() %>%
scale()Chunk time: 0.02 secs
I have selected the features that describe the basic characteristics of each song. These are: Duration, Loudness, Tempo, Valence, Energy, Danceability, Speechiness, Acousticness and Liveness. Most of these features are scores between zero and one but a few have very different ranges so I have centered and scaled each of them.
pca <- prcomp(features_mat)Chunk time: 0.11 secs
Let’s have a look at the loadings for each principle component.
feature_loadings <- pca$rotation %>%
as.data.frame() %>%
rownames_to_column("Feature") %>%
mutate(Feature = factor(Feature, levels = features)) %>%
pivot_longer(-Feature, names_to = "PC", values_to = "Loading")
ggplot(feature_loadings,
aes(x = fct_rev(Feature), y = Loading, fill = Feature)) +
geom_col() +
geom_hline(yintercept = 0) +
coord_flip() +
scale_fill_brewer(palette = "Set1", guide = FALSE) +
facet_wrap(~ PC) +
labs(
title = "PC loadings",
x = NULL
) +
theme(
strip.text = element_text(colour = "white"),
strip.background = element_rect(fill = "black"),
panel.border = element_rect(fill = NA)
)
Chunk time: 1.52 secs
Here is the scatter plot of the first two components.
pca_points <- pca$x %>%
as.data.frame() %>%
rownames_to_column("SpotifyID")
ggplot(pca_points, aes(x = PC1, y = PC2)) +
geom_point()
Chunk time: 0.33 secs
There’s not a lot of separation between points here. Perhaps thing will be clearer if we do a non-linear embedding based on the PCA?
2 t-SNE
Let’s see what a t-SNE plot looks like.
tsne <- Rtsne(pca$x, pca = FALSE)
tsne_points <- tsne$Y %>%
as.data.frame() %>%
set_names(c("TSNE1", "TSNE2")) %>%
mutate(SpotifyID = pca_points$SpotifyID)
ggplot(tsne_points, aes(x = TSNE1, y = TSNE2)) +
geom_point()
Chunk time: 2.64 secs
That maybe does a better job of showing the structure in the data? What if we overlay some of the information we have.
2.1 Features
requestival <- left_join(requestival, tsne_points, by = "SpotifyID")
plot_data <- requestival %>%
filter(HasSpotify) %>%
select(-DateTime) %>%
distinct(SpotifyID, .keep_all = TRUE)
plot_features <- c("IsUnearthed", "AlbumDate", "Duration", "Explicit",
"Popularity", "IsMajor", "Loudness", "Tempo", "Valence",
"Energy", "Danceability", "Speechiness", "Acousticness",
"Liveness")
src_list <- map(plot_features, function(.feature) {
if (is.logical(plot_data[[.feature]][1])) {
colour_scale <- 'scale_colour_manual(values = c("grey", "red"))'
} else if (is.Date(plot_data[[.feature]][1])) {
colour_scale <- 'scale_colour_date()'
} else {
colour_scale <- 'scale_colour_viridis_c()'
}
knit_expand(text = c(
"### {{.feature}} {.unnumbered}",
"```{r}",
"ggplot(plot_data) +",
"aes(x = TSNE1, y = TSNE2, colour = {{.feature}}) +",
"geom_point(size = 2) +",
colour_scale,
"```",
""
))
})
out <- knit_child(text = unlist(src_list), options = list(cache = FALSE))Chunk time: 0.43 secs
IsUnearthed
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = IsUnearthed) +
geom_point(size = 2) +
scale_colour_manual(values = c("grey", "red"))
Chunk time: 0.43 secs
AlbumDate
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = AlbumDate) +
geom_point(size = 2) +
scale_colour_date()
Chunk time: 0.38 secs
Duration
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Duration) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.39 secs
Explicit
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Explicit) +
geom_point(size = 2) +
scale_colour_manual(values = c("grey", "red"))
Chunk time: 0.39 secs
Popularity
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Popularity) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.39 secs
IsMajor
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = IsMajor) +
geom_point(size = 2) +
scale_colour_manual(values = c("grey", "red"))
Chunk time: 0.39 secs
Loudness
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Loudness) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.39 secs
Tempo
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Tempo) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.36 secs
Valence
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Valence) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.38 secs
Energy
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Energy) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.37 secs
Danceability
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Danceability) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.38 secs
Speechiness
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Speechiness) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.39 secs
Acousticness
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Acousticness) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.37 secs
Liveness
ggplot(plot_data) +
aes(x = TSNE1, y = TSNE2, colour = Liveness) +
geom_point(size = 2) +
scale_colour_viridis_c()
Chunk time: 0.38 secs
requestival %>% filter(HasSpotify) %>% mutate(Day = day(DateTime), Time = (60 * hour(DateTime) + minute(DateTime)) * 60) %>% select(Day, Time, Danceability, Speechiness, Acousticness, Liveness) %>% pivot_longer(c(-Time, -Day), names_to = “Feature”, values_to = “Score”) -> pp
ggplot(pp, aes(x = Time, y = Score, colour = Feature, fill = Feature)) + geom_point(alpha = 0.25, size = 1) + geom_smooth() + facet_grid(Day ~ Feature, scales = “free”) + scale_x_time() + scale_colour_brewer(palette = “Set1”, guide = FALSE) + scale_fill_brewer(palette = “Set1”, guide = FALSE)
sessioninfo::session_info()─ Session info ───────────────────────────────────────────────────────────────
setting value
version R version 4.0.0 (2020-04-24)
os macOS Catalina 10.15.5
system x86_64, darwin17.0
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Europe/Berlin
date 2020-07-11
─ Packages ───────────────────────────────────────────────────────────────────
! package * version date lib source
P assertthat 0.2.1 2019-03-21 [?] CRAN (R 4.0.0)
P backports 1.1.7 2020-05-13 [?] CRAN (R 4.0.0)
P base64enc 0.1-3 2015-07-28 [?] CRAN (R 4.0.0)
P blob 1.2.1 2020-01-20 [?] CRAN (R 4.0.0)
P broom 0.5.6 2020-04-20 [?] CRAN (R 4.0.0)
P cellranger 1.1.0 2016-07-27 [?] standard (@1.1.0)
P cli 2.0.2 2020-02-28 [?] CRAN (R 4.0.0)
P colorspace 1.4-1 2019-03-18 [?] standard (@1.4-1)
P conflicted * 1.0.4 2019-06-21 [?] standard (@1.0.4)
P crayon 1.3.4 2017-09-16 [?] CRAN (R 4.0.0)
P DBI 1.1.0 2019-12-15 [?] CRAN (R 4.0.0)
P dbplyr 1.4.4 2020-05-27 [?] CRAN (R 4.0.0)
P digest 0.6.25 2020-02-23 [?] CRAN (R 4.0.0)
P dplyr * 0.8.5 2020-03-07 [?] CRAN (R 4.0.0)
P ellipsis 0.3.1 2020-05-15 [?] CRAN (R 4.0.0)
P evaluate 0.14 2019-05-28 [?] standard (@0.14)
P fansi 0.4.1 2020-01-08 [?] CRAN (R 4.0.0)
P farver 2.0.3 2020-01-16 [?] CRAN (R 4.0.0)
P forcats * 0.5.0 2020-03-01 [?] CRAN (R 4.0.0)
P fs * 1.4.1 2020-04-04 [?] CRAN (R 4.0.0)
P generics 0.0.2 2018-11-29 [?] standard (@0.0.2)
P genius 2.2.2 2020-05-28 [?] CRAN (R 4.0.0)
P ggplot2 * 3.3.1 2020-05-28 [?] CRAN (R 4.0.0)
P git2r 0.27.1 2020-05-03 [?] CRAN (R 4.0.0)
P glue * 1.4.1 2020-05-13 [?] CRAN (R 4.0.0)
P gtable 0.3.0 2019-03-25 [?] standard (@0.3.0)
P haven 2.3.0 2020-05-24 [?] CRAN (R 4.0.0)
P here * 0.1 2017-05-28 [?] standard (@0.1)
P hms 0.5.3 2020-01-08 [?] CRAN (R 4.0.0)
P htmltools 0.4.0 2019-10-04 [?] standard (@0.4.0)
P httpuv 1.5.3.1 2020-05-26 [?] CRAN (R 4.0.0)
P httr 1.4.1 2019-08-05 [?] standard (@1.4.1)
P janeaustenr 0.1.5 2017-06-10 [?] CRAN (R 4.0.0)
P jsonlite 1.6.1 2020-02-02 [?] CRAN (R 4.0.0)
P knitr * 1.28 2020-02-06 [?] CRAN (R 4.0.0)
P labeling 0.3 2014-08-23 [?] standard (@0.3)
P later 1.0.0 2019-10-04 [?] standard (@1.0.0)
P lattice 0.20-41 2020-04-02 [3] CRAN (R 4.0.0)
P lifecycle 0.2.0 2020-03-06 [?] CRAN (R 4.0.0)
P lubridate * 1.7.8 2020-04-06 [?] CRAN (R 4.0.0)
P magrittr 1.5 2014-11-22 [?] CRAN (R 4.0.0)
P Matrix 1.2-18 2019-11-27 [3] CRAN (R 4.0.0)
P memoise 1.1.0 2017-04-21 [?] standard (@1.1.0)
P modelr 0.1.8 2020-05-19 [?] CRAN (R 4.0.0)
P munsell 0.5.0 2018-06-12 [?] standard (@0.5.0)
P nlme 3.1-148 2020-05-24 [3] CRAN (R 4.0.0)
P pillar 1.4.4 2020-05-05 [?] CRAN (R 4.0.0)
P pkgconfig 2.0.3 2019-09-22 [?] CRAN (R 4.0.0)
P plyr 1.8.6 2020-03-03 [?] CRAN (R 4.0.0)
P promises 1.1.0 2019-10-04 [?] standard (@1.1.0)
P purrr * 0.3.4 2020-04-17 [?] CRAN (R 4.0.0)
P R6 2.4.1 2019-11-12 [?] CRAN (R 4.0.0)
P RColorBrewer 1.1-2 2014-12-07 [?] standard (@1.1-2)
P Rcpp 1.0.4.6 2020-04-09 [?] CRAN (R 4.0.0)
P readr * 1.3.1 2018-12-21 [?] standard (@1.3.1)
P readxl 1.3.1 2019-03-13 [?] standard (@1.3.1)
P reprex 0.3.0 2019-05-16 [?] standard (@0.3.0)
P reshape2 1.4.4 2020-04-09 [?] CRAN (R 4.0.0)
P rlang 0.4.6 2020-05-02 [?] CRAN (R 4.0.0)
P rmarkdown 2.1 2020-01-20 [?] CRAN (R 4.0.0)
P rprojroot 1.3-2 2018-01-03 [?] CRAN (R 4.0.0)
P rstudioapi 0.11 2020-02-07 [?] CRAN (R 4.0.0)
P Rtsne * 0.15 2018-11-10 [?] CRAN (R 4.0.0)
P rvest * 0.3.5 2019-11-08 [?] standard (@0.3.5)
P scales 1.1.1 2020-05-11 [?] CRAN (R 4.0.0)
sessioninfo 1.1.1 2018-11-05 [3] CRAN (R 4.0.0)
P SnowballC 0.7.0 2020-04-01 [?] CRAN (R 4.0.0)
P spotifyr * 2.1.1 2019-07-13 [?] CRAN (R 4.0.0)
P stringi 1.4.6 2020-02-17 [?] CRAN (R 4.0.0)
P stringr * 1.4.0 2019-02-10 [?] CRAN (R 4.0.0)
P tibble * 3.0.1 2020-04-20 [?] CRAN (R 4.0.0)
P tidyr * 1.1.0 2020-05-20 [?] CRAN (R 4.0.0)
P tidyselect 1.1.0 2020-05-11 [?] CRAN (R 4.0.0)
P tidytext 0.2.4 2020-04-17 [?] CRAN (R 4.0.0)
P tidyverse * 1.3.0 2019-11-21 [?] standard (@1.3.0)
P tokenizers 0.2.1 2018-03-29 [?] CRAN (R 4.0.0)
P vctrs 0.3.0 2020-05-11 [?] CRAN (R 4.0.0)
P viridisLite 0.3.0 2018-02-01 [?] standard (@0.3.0)
P whisker 0.4 2019-08-28 [?] standard (@0.4)
P withr 2.2.0 2020-04-20 [?] CRAN (R 4.0.0)
P workflowr 1.6.2 2020-04-30 [?] CRAN (R 4.0.0)
P xfun 0.14 2020-05-20 [?] CRAN (R 4.0.0)
P xml2 * 1.3.2 2020-04-23 [?] CRAN (R 4.0.0)
P yaml 2.2.1 2020-02-01 [?] CRAN (R 4.0.0)
[1] /Users/luke.zappia/Documents/Projects/requestival/renv/library/R-4.0/x86_64-apple-darwin17.0
[2] /private/var/folders/rj/60lhr791617422kqvh0r4vy40000gn/T/RtmpiOA0NZ/renv-system-library
[3] /Library/Frameworks/R.framework/Versions/4.0/Resources/library
P ── Loaded and on-disk path mismatch.Chunk time: 0.21 secs