Retrieving our Data

The Quantmod function, getSymbols takes a stock ticker parameter, a flag to determine whether we want a variable containing the data generated automatically without arrow assignment (auto.assign), and the start date of the time series of stock prices that we want to retrieve (from).

stock_xts <- getSymbols("SPY", auto.assign = FALSE, from="2018-01-01", to="2019-09-07")

Quantmod returns an XTS object, and not a dataframe, as we see when we call class on this object. XTS objects are similar in nature to dataframes and tibbles, but are optimized for time series data, and unlike tibbles and dataframes, are not compatible with tidyverse functions.

class(stock_xts)

## [1] "xts" "zoo"

In order to use tidyverse (dplyr) functions on this data, we will first need to convert it to a dataframe using the fortify.zoo function. Note that while it is possible to convert an xts object to a dataframe with the as_data_frame or as.data.frame functions, these will drop the index, whereas fortify.zoo retains the index as the first column when it is converted.

stock_df <- fortify.zoo(stock_xts)
class(stock_df)

## [1] "data.frame"

Calculating and Transforming Columns

Once we have our stock dataframe, we need to produce year and month columns that we will use to facet (split) the plot by; a wkday column that will form the y axis of our calendar heatmap; a day column representing day of month (the values from which will appear within the squares of the plot); a wk column (week of year) that will form the x axis; and finally a returns column that will show the percentage change in value from the previous day when we map it to a color palette. We will then select the columns we created, leaving the original ones out.

The entirety of this chunk of code (and output) will look like the following (we will break this down in the next section):

stock_df %>%
  mutate(year = year(Index),
         month = month(Index, label = TRUE),
         wkday = fct_relevel(wday(Index, label=TRUE),
                             c("Mon", "Tue","Wed","Thu","Fri","Sat","Sun")
                             ),
         day = day(Index),
         wk = format(Index, "%W"),
         returns = (.[[5]] - lag(.[[5]]))/lag(.[[5]])) %>%
  select(year, month, wkday, day, wk, returns) 

The lubridate package provides some handy functions that I use on an almost daily basis: year, month, wday, and day. Pass these functions a date or datetime column and they give you the year, the month (specify label = TRUE to get the abbreviated text form e.g., ‘Jan’, ‘Feb’), weekday, and day of month associated with each date.

Note that we use fct_relevel on the output of our wday call in order to specify the order of the days. On the y axis of the calendar heatmap, we want the days to start from monday at the bottom, and not Sunday (which is the default ordering that wday produces).

         wkday = fct_relevel(wday(Index, label=TRUE),
                             c("Mon", "Tue","Wed","Thu","Fri","Sat","Sun")
                             ),

The second important thing to notice is the call to format which can be used to extract and rearrange components of dates (among other uses). Here, we are passing it the Index column, like we did with the lubridate functions above it, and the parameter "%W". This unix strftime code indicates that we want the week number of the date, with the weeks beginning on monday (Notice that in the dataframe, above, week 2 appears on day 8 which is a Monday as indicated in the wkday column).

         wk = format(Index, "%W"),

Finally, we calculate the daily returns. Since quantmod returns an object that has column names with the ticker in the names (e.g., SPY.Close), it would be difficult if you decided later to change the ticker in the initial getSymbols call to see the returns for, say, Disney, since you would then need to go into mutate and change the column names there, each time, as well. Because of this, we will reference the closing price columns by position when calculating the returns column. The dots you see stand in for the dataframe, itself, and we use the double bracket indexing we use for lists and vectors to get the fifth column. The dot placeholder is a powerful concept to understand when using pipes - you can find out more about it here.

In order to compare the difference in price from one date to another, we will use the lag function. lag shifts the column down by one position so that the value originally in position n is now found in position n+1. The returns column calculates the difference between SPY.Closing on day n+1 and SPY.Closing on day n, then divides this by the price on day n. 

         returns = (.[[5]] - lag(.[[5]]))/lag(.[[5]])) 

Building the Plot

Once those columns have been created, we will select and pipe them to ggplot. The wk column, representing weeks of the year, is passed to the x axis, while the wkday column, representing the day of the week, is passed to the y axis. We set fill=returns since we are coloring an area by this variable. geom_tile is added, with color='black' since we want the borders of each square to be black (remember that fill colors in areas such as bars, and color adds color to single dimension objects such as dots and lines). We then add geom_text with label=day in order to have the day of the month overlayed on each square that has a value (return) associated with it.

  select(year, month, wkday, day, wk, 5, returns) %>% 
    ggplot(aes(wk, wkday, fill=returns)) +
      geom_tile(color='black') +
      geom_text(aes(label=day), size=3) + 
      labs(x='', 
           y='',
           title="SPY") +
      scale_fill_distiller(type="div" 
                           ,palette=7
                           ,na.value = 'white' 
                           ,limits=c(-.055, .055)
                           ,labels = percent
                           ,direction=1
                           ) +
      theme(panel.background = element_blank(),
            axis.ticks = element_blank(),
            axis.text.x = element_blank(),
            strip.background = element_rect("grey92")
            ) +
      facet_grid(year~month, scales="free", space="free")

There are a couple of key parts of this section of code to pay attention to: the call to scale_fill_distiller and facet_grid.

Summary

We now have a calendar heatmap of daily stock returns for the SPY ETF across 2018 and 2019. We can easily see that no weekday patterns are present, that there was a high degree of volatility from October through early January, and that February 5 and 8 were brutal days for this ETF (and the market in general since it tracks the S&P 500).

Calendar heatmaps can be a great way of displaying your data if you want others to be able to easily note precise values at the day level as well as to detect patterns across the week, month, and year in a timeseries. With the flexibility of ggplot and other tidyverse packages, this can be applied to a wide variety of data sets, addressing many different goals.

The code:

library(tidyverse)
library(lubridate)
library(quantmod)
library(scales)
library(zoo)
library(kableExtra)

stock_xts <- getSymbols("SPY", auto.assign = FALSE, from="2018-01-01")
stock_df <- fortify.zoo(stock_xts)

stock_df %>%
  mutate(year = year(Index),
         month = month(Index, label = TRUE),
         wkday = fct_relevel(wday(Index, label=TRUE),
                             c("Mon", "Tue","Wed","Thu","Fri","Sat","Sun")
                             ),
         day = day(Index),
         wk = format(Index, "%W"),
         returns = (.[[5]] - lag(.[[5]]))/lag(.[[5]])) %>%
  select(year, month, wkday, day, wk, returns) %>% 
    ggplot(aes(wk, wkday, fill=returns)) +
      geom_tile(color='black') +
      geom_text(aes(label=day), size=3) + 
      labs(x='', 
           y='',
           title="SPY") +
      scale_fill_distiller(type="div" 
                           ,palette=7
                           ,na.value = 'white' 
                           ,limits=c(-.055, .055)
                           ,labels = percent
                           ,direction=1
                           ) +
      theme(panel.background = element_blank(),
            axis.ticks = element_blank(),
            axis.text.x = element_blank(),
            strip.background = element_rect("grey92")
            ) +
      facet_grid(year~month, scales="free", space="free")