author: Karishma Srikanth, Aaron Chafetz

Efficient and scalable analytics workflows are critical for an adaptive and data-driven organization. How can we scale systems to support an office charged with implementing USAID’s $6 billion HIV/AIDS program? Our team leveraged R and global health APIs to build more efficient workflows through automation by developing custom R packages to access health program data. Our investment in creating an automated data infrastructure with flexible, open-source tools like R enabled us to build reproducible workflows for analysts in over 50 partner countries. We would like to share our experience in a federal agency integrating APIs with R to develop scalable data pipelines, as inspiration for organizations facing similar resource & data challenges.

Presentation

From August 28-September 1, 2023, our division put on the USAID Global Data Skills Workshop in Bangkok, Thailand. For two and a half days, participants broke out into three different tracks by software preference (Excel, Tableau, R).

Goals - Understand how to optimize data lifecycles and workflows, build data skills in data visualization and statistical softwares and facilitate the sharing of critical data insights and solutions by Mission-based colleagues.

Objectives - Learn how to execute related and scalable functions in one of 3 tracks focused on Excel, Tableau or R/RStudio and principles on the following topics:

1. Data collection, pipelines, processing, and management; examples include working with imperfect data by learning how to profile and clean your data, and transforming a raw dataset into tidy data;
2. Data Visualization/Summaries, such as understanding the basic concepts of a plot and how to select the right visualization for a given analysis, learning how to layer information from multi-dimensional data to build upon the plot/s, and applying the OHA Style Guide to develop impactful visualizations;
3. Data Storytelling for data-driven, purposeful deliverables that translate into action.

The below materials were used for the R tracks. All material have a CC-BY-NC license,

…[enabling] reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator. CC BY-NC includes the following elements:

• BY: credit must be given to the creator.
• NC: Only noncommercial uses of the work are permitted.

The contents of the R training presentations were created specifically for the 2023 USAID Global Data Skills Workshop. If you plan on using and/or adapting any of the content from this session, please do the following:

• contact the content creators, Aaron Chafetz (achafetz@usaid.gov) and Tim Essam (tessam@usaid.gov) as well as the OHA/SIEI Capacity Building group (gh.oha.siei.capacitybuilding@usaid.gov) for tracking purposes;
• ensure you have an attributions slide that states, “Content adapted from: Chafetz, A and T Essam. (2023 August). USAID Global Data Skills Workshop.”; and
• provide a link to this page for the original source material

Material

(internal)

Working with Data in R

Data Visualizaton Fundamentals

Data Visualization in R

Pulling It All Together

Working from your USAID laptop, Government Furnished Equipment (GFE), you can install R and RStudio without having to submit a ticket to the M/CIO Help Desk. R is the open-source statistics package that we use for our work and is the engine that powers RStudio Desktop, the user interface or integrated development environment (IDE). RStudio Desktop or another IDE such as Visual Studio Code is not required to use R, but will vastly improve your experience.

To install both R and RStudio Desktop on your GFE, go to Software Center on your computer (Start > Microsoft Endpoint manager > Software Center) Once there, you can select the Application called “R for Windows” and click “Install”. After that completes, you can then select “RStudio Desktop” and then “Install”. If you run into any issues, first try restarting your machine and if that fails, you can contact M/CIO Help Desk.

If working on a personal machine, you can install R from CRAN. Select “Download R for Windows” and then “base” and follow the instructions for installing that pop up when you launch the .exe file from your downloads. RStudio Desktop can be installed Posit’s website by selecting “Download RStudio Desktop for Windows” and then following the setup instructions.

Installing Rtools

If you are working on a GFE, you will need to submit a ticket to M/CIO Help Desk to install Rtools on your machine. If you are installing from your personal machine, you will need to download and install the version of Rtools based on the R version you are using. You can determine what version of R you are using by opening up a new instance of R or RStudio and the version will be the very first thing that appears in your console.

RStudio Global Options

It’s best practice to start with a clean session each time you load up RStudio, so you will want to adjust some default options in your IDE. To access these, in the menu bar at the top, navigate to Tools > Global Options. Here are the places you will want to make changes to the default options before you hit “Apply”:

• Uncheck “Result most recent opened project at startup”
• Uncheck “Restore .RData into Workspace at startup”
• Change dropdown to “Never” for “Save workspace to .RData on exit”
• Uncheck “Always save history (even when not saving .RData)

Storing Snippets

Rstudio code snippets are predefined code shortcuts that can be used to quickly insert commonly used code blocks. The use of snippets can improve coding efficiency, reduce the time spent copying and pasting code from other scripts, and improve the readability of your code by providing a standardized format for your analytical scripts.

To create your own snippets in Rstudio, go to “Tools > Global Options > Code > Edit Snippets. This will open a file where you can define your custom snippets using a simple syntax.

Rstudio comes bundled with a set of built-in snippets that you may have already used without even realizing it. The snippets are not limited to just R, but to all of the different languages you can code in within the Rstudio IDE. In the example below, this snippet sets up the formatting of a script for you. To create a new snippet, follow the syntax in the window and click save. Your snippet is now available for use.

For example, if we wanted to create snippet to insert a new object that represents the time right now, we could use the following snippet:

snippet time "insert time right now" Sys.time() Close your snippet window by hitting save, and return to the console.

When we start typing we will then see the following appear.

If we hit Tab, the Sys.time() function will be inserted in the console window. When we hit enter, Rstudio will report the current time. While this may not be that useful, you can imagine how useful this may be if you need to insert the date or a repeated chunk of code.

####Types of snippets There are broadly four different types of snippets available for use or creation. Predefined snippets: RStudio comes with several built-in snippets for common programming tasks. These snippets cover a wide range of R code structures and functions, such as for loops, if statements, function definitions, and more. Triggering snippets: Snippets are triggered by typing a specific keyword followed by pressing the “Tab” key. For example, if you type “for” and then press “Tab,” RStudio will automatically expand the snippet into a basic for loop structure. Tab stops: Snippets may contain tab stops (usually denoted by the ${1}), indicated by numbers or placeholders. These allow you to quickly navigate through the different sections of the snippet by pressing the “Tab” key. For example, if you have a placeholder for a variable name, pressing “Tab” will move the cursor to that position, allowing you to enter the desired variable name. Dynamic snippets: Snippets can be dynamic and include placeholders that are automatically filled with values based on the context. For example, the # DATE: `r Sys.Date()` code chunk will insert today’s date into your script.

By using RStudio snippets, you can streamline your coding workflow, reduce repetitive typing, and improve overall productivity when working with R code. We highly encourage you to take advantage of snippets and share your discoveries with the team.

Link to SI Snippets

Additional Resources

• Installing Rtools - Jeffrey Leek
• RBBS - 0 Software and Account Setup: Setting up Rtools - Aaron Chafetz

Installing Source Sans Pro

To create standard visualizations across our SI team, we rely on one of USAID’s alternate fonts, Sans Source Pro. This typeface is not only not native to R, nor is it a standard to Windows, but is an open source typeface available from Google Fonts.

To install the font on your GFE, you can find it in Software Center (Start > Microsoft Endpoint manager > Software Center). Once there, you can select the Application called “Source Sans Pro” and click “Install”.

To install it on your computer, navigate to the typeface on Google Fonts and click the “Download family”. After the folder finishes downloading, unzip it.

Accessing Fonts in R

To use non-native fonts in R, you must run a program called extrafonts. You will need to run the following code below to install all the fonts on your computer (if desired) and the one you just downloaded/added. You will only need to import fonts only once on your machine. However, to use these fonts with any plotting in R, you will need to load the extrafont as with any other package.

#load library (install if these are not already installed)
library(extrafont)  #install.packages("extrafont")
library(remotes)  #install.packages("remotes")

#downgrade a package dependency for extrafont
#https://stackoverflow.com/questions/61204259/how-can-i-resolve-the-no-font-name-issue-when-importing-fonts-into-r-using-ext/68642855#68642855
install_version("Rttf2pt1", version = "1.3.8")

#import all Windows fonts
  font_import()
  
#restart your R session - CTRL + SHIFT + F10

#check that your fonts are now accessible in R
library(extrafont)
fonts()

Additional Resources

• USAID Graphic Standards Manual
• Google Fonts: Sans Source Pro
• Stack Overflow: Resolve the “No Font Name Issue”
• glitr package

Overview

Our goal today’s session is to explore and develop reproducible examples as a better way to ask for help when you run into a problem with R.

Recording

USAID staff can use this link to access today’s recording (not available to external users).

Material

Amongst the plethora of articles the Washington Post published on the topic last week, there was one in particular that caught my eye. Luis Melgar and Hamza Shaban wrote a very short article, but included a couple of pretty fantabulous graphics. I want to focus on the one included below.

In their plot, Melgar and Shaban show how many banks have failed since 2001, as depicted by different sized points based on the size of their assets. This beeswarm plot is essentially a scatter plot that has one “real” axis. In this case, we have the date on the x-axis that is “real”. If we plotted these along a single x-axis, we wouldn’t be able to see how many banks failed because they could be overlapping points. To solve this problem, a beeswarm plot uses an artificial y-axis to distribute the points so they are not all on top of one another. All these points fanned out like this give the impression of a beeswarm, hence the name.

We can see that a lot of banks failed during the Great Recession, and Melgar and Shaban even annotated the plot to give you additional context that 400 banks failed during this period. The authors also used an encoding of size to show the magnitude. Washington Mutual in 2008 is the largest failure (and circle) followed by the recent failures of SVB and Signature Bank last week. These three banks are encoded with color as well to make them stand out against all the other bank failures.

This plot certainly could have been a bar chart, showing either how many banks failed in a given year or the value in assets, but this gives us a much richer understanding of what is going on. The same is also true with our PEPFAR data. We typically aggregate our site level data up higher orders of magnitude (e.g. national/sub-national units, agency, partner), which can be useful, but we lose out on a lot of great detail and stories by masking this more granular data. I encourage you all to go beyond reporting out aggregated data and to explore the stories contained within your finer level of detail data.

Happy plotting!

What stood out to me was the similarity of the icons used. These icons clearly come from the same “family”, which we can identify from the line weighting, the style, level of detail, stroke v fill, etc. All too often when we see icons used in the visualization space, they are pulled from Google or are combined with icons that just don’t quite fit the same style.

Icons, when used appropriately, can make it easier for us to communicate concepts and reduce the amount of noise going on in a visual or dashboard. Yes, icons can be used as the visualization itself, an Isotype chart, but I’m thinking mainly about the little part of the visualization. In the two visuals below from a very old Essam + Chafetz collaboration (in our Stata and pre-OHA years) the icons add to what could otherwise be a pretty plain line chart and heat map, by helping draw the reader’s attention and making it easier for them to associate data with an icon rather than just text.

This is also true in a dashboard. We want to use icons across the document that make it easier for the audience to navigate and interpret, while minimizing the amount of text and distraction. But we have to make sure those icons work well together and work well with the feel of what is a part of.

Next time you are designing a visualization or dashboard, think about how you can incorporate a family of icons to accentuate your work.

Happy plotting!

How cool is this? If you were to plot all the countries’ lines on one chart, you would have what is affectionately known as a spaghetti plot. But by breaking out the trends by country, you can quickly and more easily compare how each individual is doing. And to take this a step further than the normal small multiples you typically see, the plot also had the additional context piece of all the other countries faintly plotted in the background.

If you haven’t checked out Holtz’s R graphic gallery before, it is an amazing resource with how to build all different sorts of plots (in R for this particular site) from the ground up. So you can not only be inspired by the different graph types, but also learn to recreate them with your own work.

When you are creating your next small multiples graph, I would highly encourage you to consider adding in additional contextual info by drawing in the other comparison points in a light color/high transparency in the background.

Happy plotting!

Adding his sort of guided annotation may not be necessary for every plot, but it’s useful to consider your audience and the complexity of the visualization from their standpoint. A presentation that either needs to stand on its own without you presenting or material being sent to leadership may be prime for including an additional level of annotated text.

Happy plotting!

Happy plotting!