Goodbye, Hugo Apéro

In 2019 I created my first website using blogdown and the Hugo Academic theme, and in 2021 I migrated that site to the Hugo Apéro theme created by Alison Hill. I even documented that trek in great detail 😅 to provide something resembling a paved path for others to use on their journey adopting this beautiful and functional theme. Over the past two years, my Apéro site became a wonderful place to cultivate a digital garden¹. Now the time has come to say thank you, say goodbye, and travel down a new path. Enter Quarto.

The cost of porting

Gaining the aforementioned Quarto functionality did not come without a cost. To me, the biggest cost was the aesthetics – the Hugo Apéro theme is quite lovely! – but I customized my Quarto site enough to reflect the design aspects I loved the most. More on styling later. Some other costs included:

• Loss of the sidebar layout. The Apéro theme has options for adding a nicely styled sidebar that I had been using for my blog. It included a featured image and details about the individual post like tags, categories, length, etc. All of this information is available in Quarto posts, it’s just organized a little differently.

  Example of the sidebar

  

• Loss of Utterances comments. Any interactions with folks through Utteranc.es comments did not transfer over, and I’ll miss them. The few comments on my site have been very kind (see example below) and I’ve really enjoy reading them!

  Example of Utterances comments

  

  NoteUpdate: Oct. 12, 2023

  Thanks to Emily Riederer, I’m happy to report this is a non-issue! See her tips in the comments at the end of the post, or on GitHub.

• Hugo Apéro collections. The Apéro theme had a way of bundling together posts that were related into a collection of posts. Garrick and I used this feature for our user!2021 xaringan workshop and it worked well to organize learning materials for different sections of our workshop. If I wanted something like this in the future, I would probably use Quarto’s sidebar navigation feature like Andrew Bray did in his Rmd to Quarto workshop for rstudio::conf(2022).

• Name pronunciation on the About page. Apéro had a built-in option for adding an audio clip under your name that you could use to help people learn how your name is pronounced. I never used this feature myself, but if I wanted to, I could look to Emil Hvitfeldt’s site as an example.

• Redirects. The Apéro theme gave me the option to define the combination of date information and URL slugs for each of my posts, and I had them set up as year-slug (e.g., blog/2023-hello-quarto). Quarto does not offer that functionality and strictly creates URLs for posts based on the filepath (e.g., blog/2023-09-29-hello-quarto). In order to continue using my year-month-day-slug naming convention for my post folders, I had to set up redirects from the old URLs to the new ones so that there wouldn’t be broken links sprinkled throughout the internet. More on redirects later.

Setting up redirects

index.qmd

manual_redirects <-
  readr::read_table(here::here("static", "_manualredirects.txt"),
                    col_names = FALSE) |> 
  dplyr::mutate(redirect = paste0(X1, " ", X2, " ", X3))

manual_redirects <- manual_redirects$redirect

head(manual_redirects)

[1] "https://silvia.rbind.io/authors/silvia/avatar.png https://silviacanelon.com/about/sidebar/avatar.png 301!"
[2] "https://silvia.rbind.io/post/* https://silviacanelon.com/blog/:splat 301!"                                
[3] "https://silvia.rbind.io/blog/* https://silviacanelon.com/blog/:splat 301!"                                
[4] "https://silvia.rbind.io/talk/* https://silviacanelon.com/talk/:splat 301!"                                
[5] "https://silvia.rbind.io/project/* https://silviacanelon.com/project/:splat 301!"                          
[6] "http://silvia.rbind.io/* https://silviacanelon.com/:splat 301!"                                           

index.qmd

# function: obtain list of post paths
list_paths <- function(folder) {
  posts <-
    list.dirs(
    path = c(here::here(folder)),
    full.names = FALSE,
    recursive = FALSE
    ) |> 
    tibble::as_tibble_col(column_name = "path")  |>
    dplyr::mutate(folder = folder)
}

# define post folders
folders <- c("blog", "project", "publication", "talk")

# list post paths by folder
posts <- purrr::map(folders, list_paths) |> purrr::list_rbind()

head(posts)

# extract short paths and create redirects
posts <- 
  posts |> 
  dplyr::mutate(
    # extract the year-slugs
    short_path = stringr::str_remove(path, "(?!\\d{4}-)\\d{2}-\\d{2}-(?!\\d)"),
    # create short paths
    short_path = paste0(folder, "/", short_path),
    # create lines to insert to a netlify _redirect file
    redirects = paste0("/", short_path, " ", "/", folder, "/", path)
    )

head(posts)

index.qmd

# extract redirects
redirects <- posts$redirects

# combine with manual redirects
redirects_combined <- c(manual_redirects, redirects)

# write the _redirect file
writeLines(redirects_combined, here::here("_site", "_redirects"))

Fin

Now that it’s been over a year since Quarto made its debut, there are so many wonderful blog posts walking folks through how to create a Quarto site. My hope is that the notes I took in this blog post help provide some stepping stones for folks making the transition from blogdown sites, and offer some ideas for what is possible with the flexibility of this new publishing framework. I, for one, am looking forward to growing my digital garden with Quarto tools 🌱.

Tutorial start

Aside from the tidyverse and here packages, I used a handful of R packages to bring this map project together.

Scraping the data

get_robotstxt("https://centercityphila.org/explore-center-city/ccd-sips/sips-list-view")

[robots.txt]
--------------------------------------

# robots.txt overwrite by: on_suspect_content

User-agent: *
Allow: /



[events]
--------------------------------------

requested:   https://centercityphila.org/explore-center-city/ccd-sips/sips-list-view/robots.txt 
downloaded:  https://centercityphila.org/explore-center-city/ccd-sips/sips-list-view/robots.txt 

$on_not_found
$on_not_found$status_code
[1] 404


$on_file_type_mismatch
$on_file_type_mismatch$content_type
[1] "text/html; charset=utf-8"


$on_suspect_content
$on_suspect_content$parsable
[1] FALSE

$on_suspect_content$content_suspect
[1] TRUE


[attributes]
--------------------------------------

problems, cached, request, class

# define the page
url <- "https://centercityphila.org/explore-center-city/ccd-sips/sips-list-view?page=1"

# read the page html
html1 <- read_html(url)

# extract table info
table1 <- 
  html1 |> 
  html_node("table") |> 
  html_table()
table1 |> head(3) |> kableExtra::kable()

# extract hyperlinks to specific restaurant/bar specials
links <- 
  html1 |> 
  html_elements(".o-table__tag.ccd-text-link") |> 
  html_attr("href") |> 
  as_tibble()
links |> head(3) |> kableExtra::kable()

# add full hyperlinks to the table info
table1Mod <-
  bind_cols(table1, links) |> 
  mutate(Specials = paste0(url, value)) |> 
  select(-c(`CCD SIPS Specials`, value))
table1Mod |> head(3) |> kableExtra::kable()

getTables <- function(pageNumber) {
  Sys.sleep(2)
  
  url <- paste0("https://centercityphila.org/explore-center-city/ccd-sips/sips-list-view?page=", pageNumber)
  
  html <- read_html(url)
  
  table <- 
    html |> 
    html_node("table") |>
    html_table()
  
  links <- 
    html |> 
    html_elements(".o-table__tag.ccd-text-link") |> 
    html_attr("href") |> 
    as_tibble()
  
  tableSpecials <<-
    bind_cols(table, links) |> 
    mutate(Specials = paste0(url, value)) |> 
    select(-c(`CCD SIPS Specials`, value))
}

# get remaining tables
table2 <- map_df(2:3, getTables) 

# combine all tables
table <- bind_rows(table1Mod, table2)
table |> head(3) |> kableExtra::kable()

# save full table to file
write_rds(
  table,
  file = here("content/blog/2022-05-31-ccd-sips/specialsScraped.Rds")
  )

Geocoding addresses

The next step was to use geocoding to convert the restaurant/bar addresses to geographical coordinates (longitude and latitude) that I could map. I used the ggmap package and the Google Geocoding API service because this was a small project (59 addresses/requests) which wouldn’t make a dent in the free credit available on the platform.

The last time I geocoded addresses was for an almost identical project in 2019 and I had issues using the same API key from back then, so I made a new one. I restricted my new key to the Geocoding and Geolocation APIs.

# register my API key
# ggmap::register_google(key = "[your key]")

# geocode addresses
specials_ggmap <- 
  table |> 
  mutate_geocode(Address)

# rename new variables
specials <- 
  specials_ggmap |> 
  rename(Longitude = lon,
         Latitude = lat) 
specials |> head(3) |> kableExtra::kable()

I made sure to save the new data frame with geographical coordinates as an .Rds object so I wouldn’t have to geocode the data again! This would be particularly important if I was working on a large project.

# save table with geocoded addresses to file
write_rds(
  specials,
  file = here("content/blog/2022-05-31-ccd-sips/specialsGeocoded.Rds"))

Building the map

To build the map, I used the leaflet package. Some of the resources I found helpful, in addition to the package documentation:

• Scrape website data with the new R package rvest (+ a postscript on interacting with web pages with RSelenium) · Hollie at ZevRoss – how to style pop-ups
• Leaflet Map Markers in R · Jindra Lacko – how to customize marker icons
• A guide to basic Leaflet accessibility · Leaflet – accessibility considerations. Though it’s unclear to me how these features built into the Leaflet library translate over to the leaflet R package. For example, I couldn’t find an option for adding alt-text or a title to each marker, but maybe I wasn’t looking in the right place within the documentation.

# style pop-ups for the map with inline css styling

# marker for the restaurants/bars
popInfoCircles <- paste("<h2 style='font-family: Red Hat Text, sans-serif; font-size: 1.6em; color:#43464C;'>", "<a style='color: #00857A;' href=", specials$Specials, ">", specials$Name, "</a></h2>","<p style='font-family: Red Hat Text, sans-serif; font-weight: normal; font-size: 1.5em; color:#9197A6;'>", specials$Address, "</p>")

# marker for the center of the map
popInfoMarker<-paste("<h1 style='padding-top: 0.5em; margin-top: 1em; margin-bottom: 0.5em; font-family: Red Hat Text, sans-serif; font-size: 1.8em; color:#43464C;'>", "<a style='color: #00857A;' href='https://centercityphila.org/explore-center-city/ccdsips'>", "Center City District Sips 2022", "</a></h1><p style='color:#9197A6; font-family: Red Hat Text, sans-serif; font-size: 1.5em; padding-bottom: 1em;'>", "Philadelphia, PA", "</p>")

# custom icon for the center of the map
awesome <-
  makeAwesomeIcon(
    icon = "map-pin",
    iconColor = "#FFFFFF",
    markerColor = "darkblue",
    library = "fa"
  )

leaflet(data = specials, 
        width = "100%", 
        height = "850px",
        # https://stackoverflow.com/a/42170340
        options = tileOptions(minZoom = 15,
                              maxZoom = 19)) |>
  # add map markers ----
  addCircles(
    lat = ~ specials$Latitude, 
    lng = ~ specials$Longitude, 
    fillColor = "#009E91", #olivedrab goldenrod
    fillOpacity = 0.6, 
    stroke = F,
    radius = 12, 
    popup = popInfoCircles,
    label = ~ Name,
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      ))

leaflet(data = specials, 
        width = "100%", 
        height = "850px",
        # https://stackoverflow.com/a/42170340
        options = tileOptions(minZoom = 15,
                              maxZoom = 19)) |>
  # add map markers ----
  addCircles(
    lat = ~ specials$Latitude, 
    lng = ~ specials$Longitude, 
    fillColor = "#009E91", #olivedrab goldenrod
    fillOpacity = 0.6, 
    stroke = F,
    radius = 12, 
    popup = popInfoCircles,
    label = ~ Name,
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      )) |>
  # add map tiles in the background ----
  addProviderTiles(providers$CartoDB.Positron)

leaflet(data = specials, 
        width = "100%", 
        height = "850px",
        # https://stackoverflow.com/a/42170340
        options = tileOptions(minZoom = 15,
                              maxZoom = 19)) |>
  # add map markers ----
  addCircles(
    lat = ~ specials$Latitude, 
    lng = ~ specials$Longitude, 
    fillColor = "#009E91", #olivedrab goldenrod
    fillOpacity = 0.6, 
    stroke = F,
    radius = 12, 
    popup = popInfoCircles,
    label = ~ Name,
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      )) |>
  # add map tiles in the background ----
  addProviderTiles(providers$CartoDB.Positron) |>
  # set the map view
  setView(mean(specials$Longitude), 
          mean(specials$Latitude), 
          zoom = 16)

leaflet(data = specials, 
        width = "100%", 
        height = "850px",
        # https://stackoverflow.com/a/42170340
        options = tileOptions(minZoom = 15,
                              maxZoom = 19)) |>
  # add map markers ----
  addCircles(
    lat = ~ specials$Latitude, 
    lng = ~ specials$Longitude, 
    fillColor = "#009E91", #olivedrab goldenrod
    fillOpacity = 0.6,
    stroke = F,
    radius = 12, 
    popup = popInfoCircles,
    label = ~ Name,
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      )) |>
  # add map tiles in the background ----
  addProviderTiles(providers$CartoDB.Positron) |>
  # set the map view
  setView(mean(specials$Longitude), 
          mean(specials$Latitude), 
          zoom = 16) |>
  # add marker at the center ----
  addAwesomeMarkers(
    icon = awesome,
    lng = mean(specials$Longitude), 
    lat = mean(specials$Latitude), 
    label = "Center City District Sips 2022",
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      ),
    popup = popInfoMarker,
    popupOptions = popupOptions(maxWidth = 250))

leaflet(data = specials, 
        width = "100%", 
        height = "850px",
        # https://stackoverflow.com/a/42170340
        options = tileOptions(minZoom = 15,
                              maxZoom = 19)) |>
  # add map markers ----
  addCircles(
    lat = ~ specials$Latitude, 
    lng = ~ specials$Longitude, 
    fillColor = "#009E91", #olivedrab goldenrod
    fillOpacity = 0.6, 
    stroke = F,
    radius = 12, 
    popup = popInfoCircles,
    label = ~ Name,
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      )) |>
  # add map tiles in the background ----
  addProviderTiles(providers$CartoDB.Positron) |>
  # set the map view
  setView(mean(specials$Longitude), 
          mean(specials$Latitude), 
          zoom = 16) |>
  # add marker at the center ----
  addAwesomeMarkers(
    icon = awesome,
    lng = mean(specials$Longitude), 
    lat = mean(specials$Latitude), 
    label = "Center City District Sips 2022",
    labelOptions = labelOptions(
      style = list(
        "font-family" = "Red Hat Text, sans-serif",
        "font-size" = "1.2em")
      ),
    popup = popInfoMarker,
    popupOptions = popupOptions(maxWidth = 250)) |> 
  # add fullscreen control button ----
  leaflet.extras::addFullscreenControl()

Creating the map with Quarto

The first time around, I created a standalone map by first running an R script with the necessary code, and then exporting the HTML output as a webpage. This worked well enough, except that I realized:

1. The title of the map webpage (the name that is displayed on a browser tab) was just “map” because the name of the HTML file was map.html. I wanted something more descriptive.
2. The map wasn’t mobile-responsive. In other words, the map markers and text looked too small when viewed on a mobile device.

pagetitle: "Philly CCD Sips 2022 Map"

<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />

# make mobile-responsive
meta_viewport(
  width = "device-width",
  initial_scale = "1.0",
  maximum_scale = "1.0",
  user_scalable = "no"
  )

# tags for social media
meta_social(
  title = "Philly CCD Sips 2022 Interactive Map",
  url = "https://www.silviacanelon.com/blog/2022-ccd-sips/map.html",
  image = "https://github.com/spcanelon/silvia/blob/main/content/blog/2022-05-31-ccd-sips/featured.png?raw=true",
  image_alt = "Map of Philly's Center City with a pop-up saying Center City District Sips 2022",
  og_type = "website",
  og_author = "Silvia Canelón",
  twitter_card_type = "summary_large_image",
  twitter_creator = "@spcanelon"
)

format: 
  html:
    page-layout: custom

format:
  html:
    page-layout: custom
    self-contained: true

What to expect

A brief walkthrough of the steps I took to deploy Umami web analytics for my personal website, as documented in a short Twitter thread.

07:50pmMonths ago I removed GA from my #RStats #blogdown site & this weekend I added http://umami.is 🍚 (@caozilla) as an open source, privacy-friendly, web analytics alternative

I was intimidated by the self-hosting aspect, but the docs + @Railway made it possible! Steps in 🧵

Installation

07:50pmSteps I followed:

1. Install Railway CLI with Homebrew https://docs.railway.app/develop/cli

2. Install PostgreSQL with Homebrew https://wiki.postgresql.org/wiki/Homebrew

3. Fork Umami repo & follow steps in “Running on Railway from a forked repository” at https://umami.is/docs/running-on-railway

4. Clone repo locally w git

Railway

07:50pm5. Link local setup to Railway project in the terminal w/ railway link <projectid>. Project ID is in the Railway dashboard under Setup

6. Create PostgreSQL tables using railway run in local umami directory + steps in “Create database tables” at https://umami.is/docs/running-on-railway

07:50pm7. Deploy with railway up! 🚄

8. Follow steps in Umami Getting Started docs https://umami.is/docs/login to login & add website

9. Add tracking code to website. In my #HugoApero #blogdown site I added it to layouts/partials/head.html. My example at https://github.com/spcanelon/silvia/blob/7d407b5967ae5d1bfe9df97e9a395fd02adeb985/layouts/partials/head.html#L21-L29

Tracker Configuration

07:50pm10. In order to not track my own visits to my site, I followed the tip in @DeepankarBhade’s post https://dpnkr.in/blog/self-host-umami and disabled Umami from my browser’s local storage. He kindly explained the steps to me in this thread 😅 https://twitter.com/DeepankarBhade/status/1480214508987551750?s=20

Pricing

07:50pmA note about Railway pricing https://docs.railway.app/reference/limits:

I’m using the free tier, the Starter Plan, which has $5 of credits. In the past 2 days I’ve used $0.7258 of my credits & it’s estimated I’ll use $3.04 by the end of the month. My site receives relatively low traffic, so YMMV

07:51pmThere is a free-ish $10 credit Railway plan available also, where you would only get billed for any usage above $10

For a fully free & more adventurous experience you could give up the convenience of Railway & self-host! See the Umami docs for options https://umami.is/docs/hosting

GoatCounter

07:51pmI’ll leave you with another great free, open source, privacy-friendly option, which is GoatCounter 🐐 https://www.goatcounter.com/. And @mattdray wrote a blogdown post about it! https://twitter.com/mattdray/status/1306353556706992128?s=20

For more convos about GA web analytics alternatives, see https://twitter.com/ma_salmon/status/1379363183526285312?s=20

Updating Umami

01:52pmNote to self – how to update http://umami.is with new releases:

Recent update to v1.25.0 https://github.com/mikecao/umami/releases/tag/v1.25.0

1. Fetch upstream from my umami fork
2. Locally in terminal, change to my umami directory
3. git pull, npm install, railway up 🚄

🚀 #WebAnalytics

General resources

• Note: There is an extensive list of data viz accessibility resources at https://github.com/dataviza11y/resources

• Article by Doug Schepers that provides great background on data viz accessibility, explains how data visualization itself is assistive technology, and offers considerations for a variety of readers: Why Accessibility Is at the Heart of Data Visualization

• The Chartability methodology helps data viz practitioners audit their data viz, and it’s language/tool-agnostic. Chartability was created by Frank Elavsky with input from the broader community and is the most thorough set of standards I’ve come across.

• Øystein Moseng provides some basic practices to consider in the post 10 Guidelines for DataViz Accessibility. Among those included are these two which I don’t see covered as often:

  • Do not rely on color alone to convey information. I’ve also heard this referred to as “color independence” and in some cases as “dual-encoding” of information

  • Prefer simple, familiar visualizations over complex novelties.

    

• Talk at the Data Visualization Society’s Outlier conference earlier this year, by Frank Elavsky, Larene Le Gassick, and Sarah Fossheim: Are your visualizations excluding people?

• Guidelines by Amy Cesal on how to write alt text for data visualizations: Writing Alt Text for Data Visualization

  

• Post by Lisa Charlotte Muth on how to pick colors for your data viz that everyone can appreciate: How to pick more beautiful colors for your data visualizations - Datawrapper Blog

  

• Post by Gareth Ford Williams from The Readability Group about how to make more informed font choices: A Guide to Understanding What Makes a Typeface Accessible

  • Related is a talk from The Readability Group sharing findings from a survey study about font preferences including 2000+ participants. Among these were participants with dyslexia characteristics and participants with poor near vision: Don’t Believe the Type!

    

• Sarah Fossheim authored a post titled An intro to designing accessible data visualizations which uses real-word applications to demonstrate 10 different accessibility practices.

  Sarah also provides a code-through showing how to make screenreader-friendly graphs using D3.js in their post How to create a screen reader accessible graph like Apple’s with D3.js

  

• Amber Thomas provides an example of how to make scrollytales more accessible in a piece created with Ofunne Amaka for The Pudding: The Naked Truth

  I'm excited to release this project for many reasons, but one among them is that I introduced lots of new (to me) #a11y features. My favorite: the ability to turn off scrollytelling 💖 Would love to hear what folks think! pic.twitter.com/eDSQr9RjFe

  — Amber Thomas (@ProQuesAsker) March 25, 2021

• Chris DeMartini has a series of blog posts documenting his journey auditing one of his public Tableau visualizations for accessibility:

  1. A Tableau Accessibility Journey
  2. Focus Order
  3. Color Contrast and Font Size
  4. Keyboard Accessibility
     
     

  

R resources

• Liz Hare and I gave a talk earlier this year on alt text in data viz shared as a part of TidyTuesday on Twitter. After web-scraping alt text from TidyTuesday tweets we found that only 3% of data viz tweets had alt text to accompany them (over the first 3 years of the TidyTuesday project). Links to the video recording, slides, and resources at https://github.com/spcanelon/csvConf2021. The talk includes guidelines on what makes effective alt text for data viz (complementary to those Amy Cesal includes in her post).

  

• Post from RStudio on how to add alt text to visualizations produced in R Markdown using code chunk option fig.alt: New in knitr: Improved accessibility with image alt text.
  New in knitr v1.35: You can now add code chunk options inside the code chunk!

  Example updated thanks to a heads-up from Garrick:

  #| fig.cap: Bigger flippers, bigger bills 
  #| fig.alt: Scatterplot of flipper length by bill length of 3 penguin species, where we show penguins with bigger flippers have bigger bills.
  
  ggplot(data = penguins, aes(x = flipper_length_mm,
                              y = bill_length_mm,
                              color = species)) +
    geom_point(aes(shape = species), alpha = 0.8) +
    scale_color_manual(
      values = c("darkorange","purple","cyan4")) 

  Code chunk adapted from the RStudio blog post

  The previous example included fig.cap and fig.alt in the code chunk heading:

  

• The ggpattern R package developed by Mike FC supports filling ggplot2 geometries with patterns. If used judiciously, patterns can help make visualizations more accessible by providing an additional way to encode information without relying on color. Below is one example using ggpattern v0.2.2:

  library(ggpattern)
  
  penguinColors <- c("darkorange", "purple", "cyan4")
  
  ggplot(penguins, aes(species)) +
  geom_bar_pattern(aes(fill = species,
                      pattern = species,
                      pattern_color = species),
    fill = penguinColors,
    alpha = 0.1,
    linewidth = 1,
    color = penguinColors,
    pattern_color = penguinColors,
    pattern_fill = penguinColors,
    pattern_spacing = 0.025
  ) +
  theme_minimal() +
  theme(legend.position = 'none')

  

• The Highcharter R package developed by Joshua Kunst adds interactivity to data viz using Highcharts JavaScript components designed with web accessibility in mind. The package has a learning curve, but lucky for us Mara Averick wrote an excellent series of blog posts on using the Highcharts accessibility module with {highcharter}.

  

My audit results

I relied on WebAIM’s Web Accessibility Evaluation Tool (WAVE) to help me get started. I installed the Firefox extension to conduct my audit. Here’s what I found at a high level, and each section of the post contains an itemized list of failures for the page along with a screenshot:

• Most errors corresponded with missing alt-text for any images that are used to decorate page sidebars or populate thumbnails

• Some of my alt-text descriptions are too long! 😆 I’ll use this example to highlight some of the super helpful features of the WAVE tool. Below is a screenshot of the tool when I select the alert next to the super long alt-text description for a hex logo 🤦‍♀️

  WAVE tool reference and code accompanying an alert

  

• My blog and talk listings have redundant links because these layouts have thumbnails enabled and both the thumbnail and the title of the post are active links

• There are 5 alerts (combined) in the navigation bar and footer that appear on every page. These correspond to redundant title links. For example, the link for my Blog has the name “Blog” and also the title “Blog.” This is something I can fix easily in the config.toml configuration file. I was wondering how these two fields were different in that file and now I know!

• The blog post example I documented here contained 177 low contrast errors! These were all related to the arduino syntax highlighting in the various code chunks that I included. The code chunks currently feature primarily gray code against a white background.

Next steps

I’m definitely not an accessibility expert by any stretch of the imagination. But the great thing is, I don’t have to be an expert in order to correct some of these failures! :muscle:

I’m going to need some more Hugo knowledge to correct some of the other ones but I think I’ve become familiar enough with the Hugo Apéro theme building this site that I might be able to poke around and make some changes. In either case, I’ll be documenting my changes in this a11y series of blog posts, so stay tuned and follow along with me!

Update (June 7, 2021): I’ve opened issue #28 and have the first changes in branch wave-audit-1 available to preview before merging.

I’ll be learning from the WCAG primer from TetraLogical, recommended by the a11y Project, and the WCAG resources from WebAIM.

And last but not least:

Even if I manage to correct all of these errors and alerts, it doesn’t mean my site is fully accessible.

It just means it’s more accessible than when I began. There are numerous accessibility considerations that are very hard to test for failure (e.g. cognitive accessibility, flexibility, etc.) that I continue to learn about. Also, accessibility is changing all of the time, and we need to be ready to change with it. So just like with R, the learning doesn’t stop! 🤓

Home page

Link: https://silvia.rbind.io/

Audit results:

• 1 error
  • 1 x Missing alternative text
• 6 alerts
  • 1 x Suspicious link text
  • 5 x Redundant title text

About page

Link: https://silvia.rbind.io/about/

Audit results:

• 4 errors
  • 3 x Linked image missing alternative text
  • 1 x Empty link
• 19 alerts
  • 2 x Skipped heading level
  • 1 x Possible heading
  • 4 x Suspicious link text
  • 7 x Redundant link
  • 5 x Redundant title text

Blog page

Link: https://silvia.rbind.io/blog/

Audit results:

• 6 errors
  • 1 x Missing alternative text
  • 5 x Linked image missing alternative text
• 10 alerts
  • 5 x Redundant link
  • 5 x Redundant title text

Talk page

Link: https://silvia.rbind.io/talk/

Audit results:

• 5 errors
  • 5 x Linked image missing alternative text
• 13 alerts
  • 5 x Linked image missing alternative text
  • 2 x Skipped heading level
  • 5 x Redundant link
  • 5 x Redundant title text
  • 1 x YouTube video

Publication page

Link: https://silvia.rbind.io/publication/

Audit results:

• 0 errors
• 5 alerts
  • 5 x Redundant title text

Project page

Link: https://silvia.rbind.io/project/

Audit results:

• 3 errors
  • 3 x Empty link
• 5 alerts
  • 5 x Redundant title text

What to expect

By the end of the tutorial you will have switched your website from using the Hugo Academic theme to using the new Hugo Apéro theme designed by Alison Hill

Specifically you will be able to migrate your blog, publications, and talks. If you need to migrate courses, I recommend taking a look at how Alison and Kelly Bodwin organized their courses and workshops into projects using this theme. I didn’t have projects prior to converting my site, but after creating a few projects post-Apéro I’m confident any projects you’ve created pre-Apéro will carry over easily.

If you like videos, Alison recorded a walkthrough of this conversion process using Julia Silge’s site as an example.

What not to expect

A tutorial on how to create a Hugo Apéro site from scratch – but don’t worry! Alison covers this in a workshop she gave for R-Ladies Tunis and in the Get started series of blog posts included in the documentation site.

The Plan

I was lucky that Alison had already started converting her own personal site because she gave me a template and example to follow!

We’ll follow the steps below throughout the tutorial and each of the six steps comes with its own commit in my git history, so you can see exactly what I changed and when.

Then we’ll reuse and migrate your existing content, set up a contact form, tidy up your directory, explore some resources for customizing your new site, and end with the grand finale: deploying your new site!

Prework

1. Install theme alongside Academic, change in config.toml

Follow along with me at commit cc5d24

The first step is to install all of the Hugo Apéro theme files to the theme/ folder in your site directory:

blogdown::install_theme(theme = "hugo-apero/hugo-apero",
                        update_config = FALSE, 
                        force = TRUE)

trying URL 'https://github.com/hugo-apero/hugo-apero/archive/main.tar.gz'
downloaded 21.4 MB

Do not forget to change the 'theme' option in 'config.toml' to "hugo-apero"
Warning message:
The theme has provided an example site. You should read the theme's documentation and at least take a look at the config file config.toml (or .yaml) of the example site, because not all Hugo themes work with any config files. 

As indicated in console output, modify the config.toml file so it points to your new theme folder instead of hugo-academic:

#theme = "hugo-academic"
theme = "hugo-apero"

At this point you will probably start to get some error messages like the one below. Don’t panic! Let’s get through the rest of the steps first. I’m including my errors in this post in case they are helpful/validating for you!

Could not build site because certain shortcodes weren't found

Error: Error building site: "/Users/silvia/Documents/Website/silvia/content/home/demo.md:58:1": failed to extract shortcode: template for shortcode "alert" not found

2. Copy all Academic shortcodes to layouts/ root (remove later)

Follow along with me at commit f3c7d53

Copy the shortcodes

• From themes/hugo-academic/layouts/shortcodes/

• To layouts/shortcodes/

My error message:

Error: Error building site: TOCSS: failed to transform "style.main.scss" (text/x-scss): SCSS processing failed: file "stdin", line 7, col 24: Invalid CSS after "...textFontFamily:": expected expression (e.g. 1px, bold), was "<no value>;"

3. Remove all assets

Follow along with me at commit 3843c76

Before deleting anything, I recommend making a backup of your entire website folder, just in case.

In the assets/ root folder, delete:

• the images/ folder which might contain your site icon

• the scss/ folder which might contain your custom.scss file

My error message:

Error: Error building site: TOCSS: failed to transform "style.main.scss" (text/x-scss): SCSS processing failed: file "stdin", line 7, col 24: Invalid CSS after "...textFontFamily:": expected expression (e.g. 1px, bold), was "<no value>;"

4. Remove all custom layouts

Follow along with me at commit 1ad7e3d

I had a couple of partials that I deleted from the layouts/ folder:

• partials/site_footer.html which provided a custom footer for my website
• partials/widgets/about.html which included the custom formatting for certificates in the Education section of the About page of my Academic site

My error message:

Error: Error building site: TOCSS: failed to transform "style.main.scss" (text/x-scss): SCSS processing failed: file "stdin", line 7, col 24: Invalid CSS after "...textFontFamily:": expected expression (e.g. 1px, bold), was "<no value>;"

5. Copy over Apéro example site config.toml file

Follow along with me at commit db37289

Rename config.toml in the root folder to config_old.toml

Copy config.toml

• From themes/hugo-apero/exampleSite/

• To your root directory (in my case it was silvia/)

My error message:

Error: Error building site: failed to render pages: render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "/Users/silvia/Documents/Website/silvia/themes/hugo-apero/layouts/partials/head.html:14:53": execute of template failed: template: partials/head.html:14:53: executing "partials/head.html" at <js>: can't evaluate field Build in type string

6. Remove Academic config/ directory

Follow along with me at commit 5541f38

Delete the config/ folder from your root directory (in my case silvia/)

I learned the hard way that the error below was due to not using an updated version of Hugo, which is why I included that step in the Prework. All this to say, I’m hoping you don’t see the error below!

My error message:

Error: Error building site: failed to render pages: render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "/Users/silvia/Documents/Website/silvia/themes/hugo-apero/layouts/partials/head.html:14:53": execute of template failed: template: partials/head.html:14:53: executing "partials/head.html" at <js>: can't evaluate field Build in type string

Migrating the content

Assuming you have made it this far and are able to at least serve a live site that uses the new Hugo Apéro theme, you are ready to start migrating your content! :tada:

File are organized differently in Hugo Apéro and the next steps detail the high-level changes I made to get my content to fit the new structure. The goal was to have my site parallel the Hugo Apéro example site and Alison’s personal site.

├── config.toml
├── content
    ├── _index.md
    ├── about
    ├── blog
    ├── collection
    ├── contributors.md
    ├── elements
    ├── form
    ├── license.md
    ├── project
    └── talk

.
├── config.toml
├── content
    ├── authors
    ├── courses
    ├── home
    ├── license.md
    ├── post
    ├── project
    ├── publication
    ├── slides
    └── talk

.
├── config.toml
├── content
    ├── _index.md       # <-- new!
    ├── about           # <-- new!
    ├── blog            # <-- renamed (formerly post)
    ├── collection      # <-- new!
    ├── form            # <-- new!
    ├── license.md
    ├── project
    ├── publication
    └── talk

[[menu.header]]
  name = "Blog"
  title = "Blog"
  url = "/blog/"
  weight = 2            # <-- item 2 in the navigation bar

[[menu.header]]
  name = "Publications"
  title = "Publications"
  url = "/publication/"
  weight = 4            # <-- item 4 in the navigation bar

Final touches

.
├── config                # <-- this folder
├── resources             # <-- this folder
├── data                  # <-- this folder
├── assets
│   └── images            # <-- this folder
├── static
│   ├── img
│   │   └── headers       # <-- this folder
│   ├── publications      # <-- this folder
│   └── rmarkdown-libs    # <-- this folder
├── layouts
│   └── shortcodes        # <-- custom partials in this folder
└── themes
    └── hugo-academic     # <-- this folder

Customizing your site

Hopefully all of that wasn’t terrible, and if it was, please know I’m rooting for you. You’re doing great! :raised_hands:

Now you get to enjoy the fun part which is customizing your site! The theme documentation goes through this in detail:

• Set up your social | Hugo Apéro
• Style your site typography | Hugo Apéro
• Style your site colors | Hugo Apéro

Deploying your new site

Once you’re happy with your new Apéro site, the last step is to merge your apero branch with the primary branch of your website repository. But first, a few steps:

1. Optional: Create a branch of your primary branch and call it hugo-academic so that you have a snapshot of your Academic files right before the merge. Since we set up Netlify to deploy all of our branches, there will now be a live link for this new branch that you can visit whenever you feel like time traveling back to your old site. For me this link is https://hugo-academic–silvia.netlify.app/

2. Switch back to your apero branch and update the baseURL field in config.toml to your regular website path. In my case:

   baseURL = "https://silvia.rbind.io/"

   Then commit and push this change to your apero branch.

3. Merge your apero branch with your primary branch. I usually use git commands in a combination of the RStudio terminal and the Git pane, but for this big merge I felt more comfortable doing it on github.com! :sweat_smile: Do what feels most comfortable for you.

4. Resolve any merge conflicts (I had a few!) in the git tool of your choosing. These are the git commands GitHub recommended:

   git fetch origin       # makes sure local files were recent
   git checkout apero     # moves you to your `apero` branch
   git merge main         # attempts a merge with your `main` branch

   When you’re finished, commit your changes and push. Then follow these next steps, also recommended by GitHub:

   git checkout main       # moves you to your `main` branch
   git merge --no-ff apero # creates a new commit for the merge

   This step will sort of replace all of the files that both themes had in common with the apero version (e.g. config.toml, netlify.toml, content/publication), and leave the old Academic files alone. So you will have to delete these extra Academic files (again!). I’m not sure how to avoid this – maybe it’s not an issue when you don’t have merge conflicts? I don’t know :thinking:

5. Tidy up your directory (again?)

   Go through the steps above to clean out any residual Academic files from your directory. Make sure to check your content/ folders for any example files from Academic that might still be hanging around and delete them.

   Then run blogdown::serve_site() to build your new Apéro site locally. Go through the site and make sure everything looks the way it should and that links are generally pointing to the right places.

   When you’re satisfied, commit the changes to your primary branch!
   There may be a lot of files that were deleted and added during the switch to Apéro and, while not generally recommended, I used the git add . command to stage all of the changes at once, commited the changes, and then pushed. I did this after thoroughly looking through the list of changed files so I knew what was happening.

6. Wait a couple of minutes for the changes to get pushed to your primary branch (e.g. main) and then wait patiently for Netlify to build your site after the merge.

7. Celebrate and share your brand new site! 🎉 🥳 🍾
   If you share on Twitter, use the #HugoApero hashtag so the Hugo Apéro squad can clap for you!

Choose your own adventure

• Option 1: Start at the beginning of the workflow to make a slide deck using the R Markdown template built into the xaringan package. The built-in template doubles as documentation for the xaringan package, so it is a great way to familiarize yourself with the package features, but it also includes a lot of content that you will probably want to remove and modify when creating your presentation.

• Option 2: Start with an R-Ladies themed xaringan template (embedded below). This is an example slide deck originally created as a teaching tool to highlight some of the main features of the xaringan package, and to demo some customization that incorporates the R-Ladies CSS theme built into xaringan. Please feel welcome to use/modify it to suit your needs! When you are ready, you can follow the steps immediately below 👇 to download the files to your machine, and then skip down to Initialize version control with git.

usethis::use_course(
  repo_spec = "spcanelon/RLadies-xaringan-template",
  destdir = "filepath/for/your/presentation"
  )

Note: After copying the files to your machine you’ll probably want to rename the file folder to whatever makes sense for your presentation.

Try navigating through the slides ☝️ with your left/right arrow keys and press the letter “P” on your keyboard to see some notes in Presenter View

The Ten-Step Workflow

install.packages("xaringan")
install.packages("usethis")

usethis::create_project("filepath/for/your/presentation/repo-name")

xaringan::infinite_moon_reader()

usethis::use_git()

usethis::use_github()

# Step 1: Stage all modified files
git add .

# Step 2: Describe the changes you made to your files
git commit -m "<A brief but descriptive commit message>"

# Step 3: Push the changes to your GitHub repository
git push

usethis::use_github_pages(branch = "master")

Bonus steps

11. Go to the repository home page and find the About section on the right hand side. Add a description of your presentation and the link to your slides, that way your presentation is easily available to anyone visiting your repo.

12. Check out Garrick Aden-Buie’s blog post Sharing Your xaringan Slides to learn how to create a social media card for your slides and use your new link to share your slides in more places (e.g. embedded on a website, etc.)

13. This GitHub Pages workflow is not exclusive to xaringan slides! Try it out with any other HTML file.

Learn more

Why did I get trained?

Teaching R was not something I thought I’d ever get into but a couple of reasons were nudging me towards getting some formal training.

I wanted to be able to…

1. …support my coworkers in their own data analysis projects and be better prepared to introduce R to undergraduate students coming through and hoping to apply learned skills elsewhere
2. …incorporate data analysis into the research efforts of my local organizing collective, and was feeling ill-equipped to lead our group. Most collective members have very little experience with data analysis, if any, and none have programming experience

After the RStudio Instructor training I can confidently say I feel prepared for both of these challenges!

Choose your own adventure

1. Jump down to a panel on the RStudio Certification hosted by the MiR Community
2. Jump down to a list of teaching exam examples and Tidyverse sample exam solutions shared openly by other instructors
3. Keep reading to find out what I learned before I decided to pursue the certification and what the certification experience was like for me.

Deciding to become certified

Certification components

The certification itself has three components:

1. Instructor training covering modern teaching methods. Materials for this training are available here and are based on the book Teaching Tech Together written by Greg (and currently undergoing translation into Spanish by a team of volunteers!).

2. A 90-minute teaching exam based on content from the instructor training. This includes a 15-minute demonstration lesson and a written component.

3. A 90-minute Tidyverse exam to test your knowledge of the subject matter in R for Data Science (make sure to check the program FAQs for information about the Shiny exam)

Learning the pedagogy

When I completed the instructor training on pedagogy it took place completely online over three days. The first two days were dedicated to the materials linked in the earlier section and were taught by Greg. Despite the training starting at 7am for me, I really enjoyed it! There were 12 of us in the cohort and there were a handful of opportunities for us to do activities in small groups and multiple opportunities to practice the teaching tools that will help us better support R learners in our own teaching.

Some of my favorite topics were mental models, cognitive capacity, and formative assessments. At the heart of all three is the concept map, which serves as a guide for reverse-designing a lesson and delivering it effectively. You can find examples of concept maps here as slides and here on GitHub. I’ve found concept maps to come in handy in so many other places in my life and I even made one to help me plan out this blog post. Check it out:

The third day of training, our cohort had the pleasure of learning all about using the learnr 📦 to build interactive R tutorials, from Mine Çetinkaya-Rundel:

• Course website: rstd.io/rit
• Slides: building interactive tutorials in R
• RStudio Cloud project: rstd.io/rit-cloud
• Source materials: rstudio-education/instructor-training

Preparing for the exams

Additional resources

Where I started

With Alison Hill’s Up and Running with Blogdown post! Super helpful, though because I came to it 2.5 years late, it was more like ‘up and running with lots of water breaks’ because I had to stop and account for changes made to the Hugo Academic theme in the meantime.

• For example, prior to Academic version 4.6, custom CSS was added using the plugins_css option in params.toml, but in current version 4.8, the theme supports SCSS (a superset of CSS) and a custom.scss file is added in the assets/scss/ folder.

The going futher section in Alison’s post specifically talks about customizing the out-of-the-box theme and Alison directly links to her custom CSS file, which I closely referred to when changing colors in my custom SCSS file.

Alison’s CSS helped me customize everything from text colors and fonts to alert colors and borders for Academic’s light mode. At this point I had the light mode looking the way I wanted but the dark mode still used out-of-the-box colors for the most part and they just didn’t go.

So I decided not to enable the dark-mode option in params.toml until I could figure out how to customize my stylesheet accordingly. That time has come because it turns out it’s pretty straightforward!

The Blogdown book does an excellent job summarizing what you need to know about CSS. This post builds on that a little by incorporating features made possible by SCSS including variables.

Choose your own adventure

1. Check out the SCSS file linked above to see how I customized my theme.
2. Keep reading to follow along and learn with me .

Why Atom?

Atom is a text editor that I like to use to modify any files in my website directory. I say modify because I much prefer to create content for posts like this one using the Blogdown addins in RStudio. It’s such a breeze! But here’s why I think Atom is a fantastic companion to the RStudio IDE:

1. Seeing your directory files is made friendly in Tree View when you open your website folder as a ‘project folder.’ You can just click on the file once to preview it and double-click if you want to actually open it. I find this to be really helpful when I can’t remember which file contains what I’m looking for because I don’t have to actually open each file to see what’s in it.
2. Searching for something specific is easy with the built-in search tool. I’ve used this many times when I can’t remember where that specific parameter (e.g. fonts) or file type (e.g. .scss) is located within my directory. I also find it super handy to find specific HTML color codes or color variables. This brings me to my most favorite feature in Atom…
3. The pigments package! Like R, Atom has a universe of handy packages and pigments will display colors behind the code text that represents it. It’s kind of magical and has been a real game changer for me. Having this package will make customizing your scss file much easier on the eyes and brain.

• You may also choose to install the highlight-selected package to make highlighted text easier to spot throughout your file and the file-icons package to make it easier to discern between different file types in your tree view. You can install packages in Atom > Preferences > Install.

Adding dark mode definitions to stylesheet

Looking through GitHub issues I found that including two sub-themes, one for light mode and one for dark mode, may be considered for Academic in the future but is not a priority. A related issue shared that if you use a custom CSS, you can define .dark .<SELECTOR-NAME> {...} for dark mode.

Not knowing what all of the options were for dark mode, I took a look at the dark.scss dark mode stylesheet built in to Academic in themes/hugo-academic/assets/scss/academic/ folder. You can see the pigments package in action here! It makes it easier to identify elements with colors that you want to change. I’m looking at you inline code text color rgb(139, 233, 253)

.dark pre,
.dark code {
  color: rgb(139, 233, 253); // cyan code text color
  background-color: rgb(68, 71, 90); // light purple code background
}

Another way to find the css for the element you want to change is to use your browser’s web inspector tool. In the screenshot below from Hugo Academic’s customization documentation (available in the Internet Archive) I right-clicked on themes/academic/data/themes/minimal.toml and then clicked on “inspect element.” The element code was .dark pre, .dark code which you might recognize from the code in dark.scss.

You can copy and paste this new .dark theme code into a new (or existing) custom.scss file saved in the assets/scss/ folder of your directory to start seeing how changes you make affect the look of your site.

Choosing dark theme colors

Alright. Now that we have some new dark theme code in our custom file, we can move on to the exciting task of choosing colors! If you’re looking for some inspiration, Coolors has some dark color palettes:

Coolors is also a great place to save colors into custom palettes. Here’s a basic example of mine:

If you’re like me and want to design a dark theme that uses the primary and secondary colors you picked for your light theme, keep reading! I’m gonna share highlights from what I learned about dark theme design and how to ensure your color combinations meet Web Content Accessibility Guidelines (WCAG).

The bulk of what I learned came from the extensive Material Design resources, including (of course) the dark theme foundational material, which will look familiar if you’ve ever used an Android device.

Color variables

One of the cool features of SCSS is the ability to use variables defined by prefix $ (i.e. $spc-primary: #122140;. You can set your variables at the top of your SCSS file and use the variable name throughout the file. This makes it super easy to try out different dark mode colors without having to use find-and-replace. If you’re using the pigments package in Atom, it’ll even show you the variable colors in a dropdown!

Tip: I like using a bright color like red

to test color changes for different CSS elements because stands out and makes it easier to see what was modified.

I defined my primary colors and variants, text colors, and various background colors as variables. And although dark gray (#121212) is recommended as a standard dark theme background, you can also create a custom background color by mixing it with your primary color. In my case I used mix(#122140, #121212, 40) to create a color that was 40% my primary color and 60% dark gray.

I controlled white text opacity by using rgba() in CSS which accepts an alpha value between 0 and 1.

Now, ideally I’d be able to define the result of this mix() as a color variable, but some CSS properties like background-color don’t recognize it. So I had to convert the result to a color code and define that as a variable instead. pigments made this conversion easy when I toggled the menu (Shift+Command+P on a Mac) and searched for the pigments: convert options.

This mix() function also came in handy when defining background surfaces with semi-transparent white overlays.

// COLOR CALCULATIONS
// below from mix($spc-primary, $spc-bkg-dark-std, 25)
// custom bkg
$spc-bkg-dark: #12151d;
// below from mix($spc-primary, $spc-bkg-dark-std, 40)
// 00dp surface
$spc-00dp: #121824;     
// below from mix($spc-00dp, white, 95)
// 00dp with 5% white overlay
$spc-01dp: #1d232e;
// below from mix($spc-00dp, white, 93)
// 00dp with 7% white overlay
$spc-02dp: #222833;
// below from mix($spc-00dp, white, 92)
// 00dp with 8% white overlay
$spc-03dp: #242a35;
// below from mix($spc-00dp, white, 91)
// 00dp with 9% white overlay
$spc-04dp: #272c37;

And that’s a wrap. Have fun building your new custom dark theme!

What to expect

The code provided in the R Markdown file requires the tidyverse, JSONlite, and lubridate packages as well as a JSON file containing your Trello data. It will generate the following:

• tidy_cards.csv file ready to import to Airtable
• attachments folder containing any content you’ve attached to your cards using a URL.
• attachment_errors.csv file containing the card name, or task, as well as the URL/file path of any attachments that failed to download (i.e. links to files stored locally), so that you exactly which links you’ll need to locate manually.

By the end of the tutorial you’ll have imported a CSV file into Airtable and configured Airtable to match your Trello board. Specifically, you’ll be able to migrate labels, lists, attachments, card names, card descriptions, card due dates, date last modified, and whether the cards were marked “complete” or archived.

What not to expect

• Rendered markdown from your card descriptions. Airtable does support markdown in its “long text” field type, but it does not automatically render from imported text.

Choose your own adventure

1. Download the R Markdown file or R script linked above and run with it. Jump down to section Formatting in Airtable for pointers on how to get your Airtable base configured like your Trello board.
2. Keep reading for a code-through of the R Markdown file, using the example featured in the picture above, with tips and resources along the way.

Follow along with me!

The Trello board used for this code-through is this Program Management Template created by Soniya Ahuja. To download the Trello board data, go to Menu > More > Print and Export and then right-click on “Export as JSON” to save the JSON file in your working directory. Or save a little time and download the program-mgmt.json file linked above .

You may also choose to read through Making Sense of Trello’s JSON Export.

Last but not least – we’ll be keeping things tidy, with the help of the tidyverse collection of R packages, to ensure each variable is in its own column, observations are in rows, and values are in cells. You can read more about tidy data in Chapter 12 of R for Data Science.

Importing JSON file

# loading libraries
library(jsonlite)
library(lubridate)
library(tidyverse)

trello <- stream_in(file("program-mgmt.json")) # produces a data frame

glimpse(trello)

A lot of information about the Trello board is contained in JSON data. We’ll be parsing information about the cards as well as the lists and then joining them together by a common identifier id or idShort.

Parsing information

Parsing information from JSON data was not something I was familiar with and found myself coming back to Kan Nishida’s post Working with JSON data in a very simple way time and time again throughout this process.

# selecting cards information
cards <- trello$cards # list of 1

# flattening
cards_flat <- flatten(cards) #list of 37

# tibble time
cards_flat_tbl <- as_tibble(cards_flat) # 32 obs of 37 variables
glimpse(cards_flat_tbl)

# selecting wanted variables
cards_trim <- cards_flat_tbl %>%
  select(id, idShort, idList, dateLastActivity, name, desc, dueComplete, due,
         labels, attachments, shortUrl, closed) %>%
  arrange(desc(dateLastActivity))

# extracting labels details
labels_info <- cards_trim %>%
  select(id, idShort, labels) %>%
  unnest() %>% # no arguments because the nested items don't have names
  rename(labelName = name) %>%
  select(id, idShort, labelName) %>%
  group_by(id, idShort) %>%
  summarize(labelList = list(labelName)) %>%
  mutate(labelList = as.character(labelList)) %>%
  mutate(labelList_tidy = str_remove_all(labelList, pattern = "\"")) %>%
  mutate(labelList_tidy = str_remove_all(labelList_tidy, pattern ="c\\(")) %>%
  mutate(labelList_tidy = str_remove_all(labelList_tidy, pattern ="\\)")) %>%
  unique()

knitr::kable(labels_info %>% head(n = 3L))

# joining back with main cards data frame
ct_labels <- left_join(cards_trim %>% select(-labels), labels_info %>% select(-labelList))

# expanding the attachment lists into separate url records
att_urls <- ct_labels %>%
  select(idShort, attachments) %>%
  unnest() %>%
  select(idShort, url) %>%
  mutate(url = as.character(url),
         attachmentError = 'FALSE')
knitr::kable(att_urls %>% head(n = 3L))

# creating directory for attachments
dirAttachments <- "attachments/"
dir.create(dirAttachments)

# downloading urls and checking for errors using try()
for (i in 1:length(att_urls$url)){
  locAttachments <-
        paste(dirAttachments, "/", att_urls$idShort[i], "_", basename(att_urls$url[i]), sep = "")
  step_to_try <- try(attachment_check <- download.file(att_urls$url[i], destfile = locAttachments))
  if("try-error" %in% class(step_to_try)) {
    cat("Error row: ", i, "\n", "Error message: ", step_to_try[1], sep = "")
    att_urls$attachmentError[i] = 'TRUE'
  }
}

# preparing data frame for export to CSV
attachment_errors <- att_urls %>%
  filter(attachmentError == TRUE) %>%
  rename(Task_Id = idShort, Attachment_URL = url, Attachment_Error = attachmentError)

# exporting to CSV
write.csv(attachment_errors, file = "attachment_errors.csv")

# creates individual directory folders for each card id
for (i in 1:length(att_urls$url)){
  dirAttachments <- paste(dirFiles, "attachments", att_urls$idShort[i], sep = "/")
  dir.create(dirAttachments) # creates directory for each unique card id
  locAttachments <- paste(dirAttachments, basename(url[i]), sep = "/")
  download.file(url[i], destfile = locAttachments)
}

# converts the attachment column to a categorical variable in the main cards+labels data frame
ct_labels <- ct_labels %>%
  mutate(attachments = ifelse(idShort %in% att_urls$idShort, TRUE, FALSE))

# selecting lists information
lists <- trello$lists # list of 1 data frame
glimpse(lists)

# flattening
lists_flat <- lists[[1]] # 17 obs of 9 variables

# selecting wanted variables
lists_trim <- lists_flat %>%
  select(id, name, closed) %>%
  rename(idList = id, nameList = name, closedList = closed)
knitr::kable(lists_trim %>% head(n = 3L))

# joining back with main cards+labels data frame
ct_labels_list <- left_join(ct_labels, lists_trim) %>%
  select(id:shortUrl, labelList_tidy:nameList, closed, closedList)

Exporting to CSV

# changing variable names
tidy_cards <- ct_labels_list %>%
  select(-id, -idList, -closedList) %>%
  rename(Task = name, Task_ID = idShort, Notes = desc, Done = dueComplete, Date_Due = due,
         Labels = labelList_tidy, Trello_List = nameList, Trello_Last_Modified = dateLastActivity,
         Trello_Url = shortUrl, Trello_Attachments = attachments, Archived = closed) %>%
  select(Task, Task_ID, Notes, Done, Date_Due, Labels, Trello_List, Trello_Last_Modified,
         Trello_Url, Trello_Attachments, Archived) %>%
  mutate(Trello_Last_Modified = as_datetime(Trello_Last_Modified, tz = ""),
         Date_Due = as_datetime(Date_Due, tz = ""),
         Done = ifelse(is.na(Date_Due) == TRUE, 'NA', Done)) # ensures only undone tasks assigned a due date get marked as "FALSE"

# determining the number of unique tasks
length(unique(tidy_cards$Task_ID))

# final look at tidy_cards
glimpse(tidy_cards)

write.csv(tidy_cards, file = "tidy_cards.csv")

Formatting in Airtable

Import CSV into Airtable as a new base. Add a base > Import a spreadsheet > CSV file. Upload the cards_tidy.csv file. Pick a name, icon, and color for your brand new base! More detailed instructions can be found in this Airtable article.

Moving forward