Turtletopia: a blog about programming in R

Shiny Reactivity Tricks, pt. II: Reactives Factories

Wed, 31 Aug 2022 00:00:00 +0000

Preface

There was no post last week because the holiday season/master’s thesis finishing season/parental pet care season is in full swing, and I’ve been completely crushed. This week isn’t easy either, but I didn’t want to leave you empty-handed when there’s so much to talk about!

This time I wanted to present you with another installment in the shiny tricks series. I’m going to show something that isn’t quite as ‘tricky’ but can still be non-obvious to those who don’t use reactivity and environments proficiently. I will tell you about factories of reactives. And the tale starts with an example.

Basic example

Remark: As I often use the shiny::reactive() function in this article, it will probably not have escaped you that I always wrap the first argument of this function in curly brackets. By doing so, I make it clear that the function’s argument is an expression, not its value. Most of the time, this is not necessary, but it is a habit of mine and helps me keep the code readable.

Suppose we have an application that accepts several numeric inputs that perform very complex transformations, then plots and outputs the result. Let’s start with an example with two inputs.

library(shiny)

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      numericInput("a", "input a", 1, 0, 100, 1),
      numericInput("b", "input b", 1, 0, 100, 1)
    ),
    mainPanel(
      textOutput("text_a_b"),
      plotOutput("plot_a_b")
    )
  )
)

server <- function(input, output, session) {
  temp_val <- reactive({
    c(
      input[["a"]] + input[["b"]],
      input[["a"]]^3 * input[["b"]],
      27 - (input[["b"]] + input[["a"]])
    )
  })

  output[["text_a_b"]] <- renderText(
    paste(temp_val(), collapse = " -- ")
  )
  output[["plot_a_b"]] <- renderPlot(
    plot(temp_val(), temp_val())
  )
}

shinyApp(ui, server)

Basic example app.

This application takes two inputs and performs an incredibly complex calculation that, in effect, returns a three-element vector. This vector, an intermediate value, is later used for the two outputs.

Of course, again, the calculations themselves, the inputs and outputs, are not relevant or even meaningful here. It’s just my usual problem finding simple examples to demonstrate not-so-simple concepts. The only important thing is a reactive intermediate value calculated from inputs.

So far, nothing complicated. So let’s add a little spice.

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      numericInput("a", "input a", 1, 0, 100, 1),
      numericInput("b", "input b", 1, 0, 100, 1),
      numericInput("c", "input c", 1, 0, 100, 1),
      numericInput("d", "input d", 1, 0, 100, 1),
      numericInput("e", "input e", 1, 0, 100, 1)
    ),
    mainPanel(
      textOutput("text_a_b"),
      plotOutput("plot_a_b"),
      plotOutput("plot_b_d"),
      tableOutput("table_c_d"),
      textOutput("table_a_e")
    )
  )
)

server <- function(input, output, session) {
  temp_val_1 <- reactive({
    c(
      input[["a"]] + input[["b"]],
      input[["a"]]^3 * input[["b"]],
      27 - (input[["b"]] + input[["a"]])
    )
  })

  temp_val_2 <- reactive({
    c(
      input[["b"]] + input[["d"]],
      input[["b"]]^2 * input[["d"]],
      8 - (input[["d"]] + input[["b"]])
    )
  })

  temp_val_3 <- reactive({
    c(
      input[["c"]] + input[["d"]],
      input[["c"]] * input[["d"]],
      1 - (input[["d"]] + input[["c"]])
    )
  })

  temp_val_4 <- reactive({
    c(
      input[["a"]] + input[["e"]],
      input[["a"]]^4 * input[["e"]],
      64 - (input[["e"]] + input[["a"]])
    )
  })

  output[["text_a_b"]] <- renderText(
    paste(temp_val_1(), collapse = " -- ")
  )
  output[["plot_a_b"]] <- renderPlot(
    plot(temp_val_1(), temp_val_1())
  )
  output[["plot_b_d"]] <- renderPlot(
    plot(c(1, 1, 1), temp_val_2())
  )
  output[["table_c_d"]] <- renderTable(
    data.frame(x = temp_val_3(), e = exp(temp_val_3()))
  )
  output[["table_a_e"]] <- renderText(temp_val_4())
}

I modified the UI and the server to do more things. Everything is centered around the very complex transformation as before, but we have more inputs, more outputs, and more temporary values. And above all: more iterations. As you may have already realized, I’m a prominent opponent of repetition in code: if there’s copy-paste somewhere, it can probably be done better. And that’s what we’re going to try to address. This code has two primary repeating rhythms: inputs and the calculation of temporary values. We can fix the first repetition very quickly, and I’ve already done similar things in the last part of the article series, so I won’t go into it this time. Let’s deal with the more exciting thing, which is the server-side repetitions.

Solution 1: wrap a function with a reactive.

The thing that jumps to mind almost immediately is the creation of a function that generates these temporary values.


magic_func <- function(name_1, name_2, param) c(
  name_1 + name_2,
  name_1^param * name_2,
  param^3 - (name_2 + name_1)
)

server <- function(input, output, session) {
  temp_val_1 <- reactive({ magic_func(input[["a"]], input[["b"]], 3) })
  temp_val_2 <- reactive({ magic_func(input[["b"]], input[["d"]], 2) })
  temp_val_3 <- reactive({ magic_func(input[["c"]], input[["d"]], 1) })
  temp_val_4 <- reactive({ magic_func(input[["a"]], input[["e"]], 4) })

  # here, put the other part of the server code...
}

This already looks much, much better! It’s more readable and less error-proof, so the advantages alone. And that’s basically where we could leave it. But if it satisfied us, we wouldn’t read any further.

Solution 2a and 2b: extract input.

Another thing that we can improve in this solution is to refer to the input object every time there are multiple arguments to the magic_func() function. So maybe we can move this reference inside the function?


magic_func <- function(name_1, name_2, param) c(
  input[[name_1]] + input[[name_2]],
  input[[name_1]]^param * input[[name_2]],
  param^3 - (input[[name_2]] + input[[name_1]])
)

server <- function(input, output, session) {
  temp_val_1 <- reactive({ magic_func("a", "b", 3) })
  temp_val_2 <- reactive({ magic_func("b", "d", 2) })
  temp_val_3 <- reactive({ magic_func("c", "d", 1) })
  temp_val_4 <- reactive({ magic_func("a", "e", 4) })
  
  # here, put the other part of the server code...
}

Output error message.

Unfortunately, we will find that this is not enough. The input object is for the interior of the function unknown. That may seem unnatural if you understand the basics of how scopes work in R – if the object is not found in the current environment, the parent environment is checked (I talked about this in the last part of the series, I refer you to it again). Here, however, this situation is not so evident because the server environment is not in the search path of the environment in which the function is defined.

That can be resolved in two ways: either move the function definition inside the server or pass an input object as an additional parameter to the function. I leave the proof to the reader.

Solution 3: wrap a reactive with a function (factory).

However, we could go even further and reduce code redundancy. And this is where we want to create a reactive factory.

What is a factory of reactives? It is simply a function that creates a reactive. Every shiny user should be familiar with the primary function used to create and register reactives – namely, shiny::reactive().

But this is the most general factory that exists – you pass it any expression, turning it into a reactive expression. We would like to have something more concrete, i.e. instead of writing reactive({ fun(parameters) }) writing reactive_fun(parameters).

In our case, we want to replace

reactive({ magic_func("a", "b", 3) })

with

r_magic_func("a", "b", 3)

Notice the r_ I added to the function name. Suggesting that this function returns a reactive is a good idea.

So, let’s try to make such a function!

r_magic_func <- function(name_1, name_2, param) reactive({
  c(
    input[[name_1]] + input[[name_2]],
    input[[name_1]]^param * input[[name_2]],
    param^3 - (input[[name_2]] + input[[name_1]])
  )
})

Does it work? Yes… at least: almost. It will not work unless we consider our previous solution’s experiences: it needs to be either defined in the server function or take input as a separate parameter.

But what if we do not want to do either of those? We are lazy and do not want to type all the parameters that will look the same either way. Those “inputs” typed every time are annoying, but if, in addition, we have some other reactive values that we would have to pass on each time, this would be very annoying. On the other hand, defining functions within a server works for small apps but is terrible if you have an extensive, packaged application that wants to keep its code tidy and test its functions with unit tests. So how do we get around these obstacles?

reactive has additional parameters that are worth looking at. Of particular interest to us is the env parameter. It indicates the environment in which the expression passed as the first parameter should be evaluated. That is usually set as the environment calling the reactive function. So this is the reason why our original factory is not able to see the input object – it is not in the environment calling the reactive. So what happens if we replace this environment with the environment which calls our factory function?

r_magic_func <- function(name_1, name_2, param) reactive({
  c(
    input[[name_1]] + input[[name_2]],
    input[[name_1]]^param * input[[name_2]],
    param^3 - (input[[name_2]] + input[[name_1]])
  )
}, env = rlang::caller_env())

We got a different error! That’s progress!

Why are we getting this error this time? Since we replaced the evaluation environment with the caller environment, it can see all objects available in the server environment. Still, it cannot see the objects available inside the r_magic_func – namely, name_1 (pun intended) and other parameters.

Our favorite package comes to our rescue: rlang! We can use a similar idea with an injection of values just as in the previous tutorial.

r_magic_func <- function(name_1, name_2, param) rlang::inject({
  reactive({
    c(
      input[[!!name_1]] + input[[!!name_2]],
      input[[!!name_1]]^(!!param) * input[[!!name_2]],
      (!!param)^3 - (input[[!!name_2]] + input[[!!name_1]])
    )
  }, env = rlang::caller_env())
})

And it fulfills all of our objectives! One more thing to do: if we want to take care of the r_magic_func, we should ensure that this factory can also be called within other functions. So, instead of fixing the env parameter of reactive as the rlang::caller_env(), we should make it a default parameter of r_magic_func, similarly to reactive itself does it.

Discussion & summary

My example will probably seem too artificial and contrived to some of you. I am not writing this for the sake of the brilliance of the example but rather to teach the concept behind it. Probably many of you will find a suitable application for this method. Some may agitate that hiding the reactive behind a function is superfluous, but, in my opinion, it is just another step towards code readability. This kind of factory can be further generalized and improved, which I will discuss next time.

One more thing: modules. I still deliberately avoid using modules. What I am doing here could also be done using modules, although I think it is much less intuitive than in the last part of the series. I will also write about modules another time (ah, too many of these topics!).

I have shown you how to use tools to change the evaluation environment to control code execution and, as a result, reduce unnecessary repetitions. This is only the second step on the long road to becoming a master at using the language to its full potential. If you have any questions or comments, contact us on Twitter or make an issue on Github.

Show Pride on Your Plots

Fri, 12 Aug 2022 00:00:00 +0000

This will be a short post, since I’m busy playing blues this week. At least the topic’s short!

It all began with a post by RainbowR on Twitter:

Alright, RainbowRs, show us your blogs/websites! Quarto/distill/blogdown/whatever - we love to see it! https://t.co/SivyCaY0yi
— RainbowR (@R_LGBTQ) August 5, 2022

As a very curious person, I had to check their GitHub account. And I found a repository curiously named pridepalettes. I was a little disappointed to find out it’s just a single script with little reusability. I hoped for a package!

A quick research showed me a few more repos with pride colors, but none I could download from CRAN. In fact, only one was a package at all!

So I took matters into my own hands and created gglgbtq.

It would be easy to just scrape the colors of multiple flags, slap a palette_lgbtq() function on top of ’em, and call it a day. But it would be barely usable. Have you ever used white for a group with default ggplot2 settings? It’d be barely visible.

# I swear it's the last time I'd use iris dataset
library(ggplot2)
library(gglgbtq)
ggplot(iris, aes(x = Sepal.Length, y = Petal.Length, color = Species)) +
  geom_point(size = 2) +
  scale_color_manual(values = palette_lgbtq("genderqueer"))

This is why I included theme_lgbtq(). Each palette has its own custom theme for increased readability (although many themes are reused across multiple palettes). Check out the difference!

# Is it considered "another time" or just "the last use part II"?
ggplot(iris, aes(x = Sepal.Length, y = Petal.Length, color = Species)) +
  geom_point(size = 2) +
  scale_color_manual(values = palette_lgbtq("genderqueer")) +
  theme_lgbtq("genderqueer")

The difference isn’t much, but I found this gray to be the perfect balance for most palettes that include white (which is like 85% or so of all LGBT+ flags). But there are palettes without white, like this one:

# I spent like 10 minutes sifting through the base datasets
# but - behold! - a non-iris dataset at last
ggplot(warpbreaks, aes(x = wool, y = breaks, fill = tension)) +
  geom_bar(stat = "identity", position = "dodge") +
  scale_fill_manual(values = palette_lgbtq("pansexual")) +
  theme_lgbtq("pansexual")

For now I included 16 palettes, but expect more (and if you want a specific flag to be included in the next release, write an issue on GitHub, strike a message on Twitter, or send a pigeon with a letter):

The Pride Palettes (edition gglgbtq 0.1.0)

Like what you see? You can get it from CRAN or GitHub, whichever suits you best:

install.packages("gglgbtq")
remotes::install_github("turtletopia/gglgbtq")

Show us your pride plots on Twitter, we’d love to see them!

Sonnet to infix function

Wed, 10 Aug 2022 00:00:00 +0000

Some poetry for the programmers

Blest be the day, and blest be the month and year,

Season and hour and very moment blest,

The lovely IDE where first possessed

By two percent signs I found me prisoner; (…)

Francesco Petrarch, Sonnet 61. Translated by Joseph Auslander. Possibly with some spicing it up by me

Subject and addressee of the poem

Custom infix functions are one of my favorite features in R. This article is my love letter to them. But first, a quick recap.

For those unfamiliar with the terminology, infix function is a function fun which is called using infix notation, e.g., x fun y instead of fun(x, y). Those functions are also called infix operators by base R, and I will use those terms and name infixes interchangeably. There are a lot of infix operators in base R used very frequently, i.e., arithmetic or logical operators. We use them so often that we usually forget that they are functions. And that we can call them just like regular functions.

23 + 19

## [1] 42

`+`(23, 19)

## [1] 42

As you may have noticed, when calling such a function in a non-infix-manner, we need to use backticks around the operator to avoid calling it regularly.

To prove that they share a lot of typical behavior with functions, I am going to demonstrate to you how to redefine them just like regular functions. We can, for example, make the + operator work like a multiplication.

`+` <- function(lhs, rhs) lhs * rhs
23 + 19

## [1] 437

Let it ring out loud: you should not do that at all. Overriding the default behavior of functions may be dangerous, as it interferes with other chunks of code using the operator. I showed this trick only for demonstration purposes.

(Side note: you might be aware that some packages change the default behavior of operators, e.g., ggplot2 has its own + for joining plot elements. However, this is a slightly different situation. They do not create a whole new function but only add a method for generic function. They do not override the behavior, but more like overload it.)

Additional remark to make: in this assignment call, backticks are also necessary. An alternative is using quotes, but it is inadvisable, and its presence in the language is entirely due to legacy reasons. You can read more about it in Hadley’s book. In another chapter, you can find more about infix functions.

Once again: do not override built-in infix functions. BUT. Create custom infix functions!

R allows for creating your operators using double percent signs. A few shipped with the base R packages, including inclusion operator %in% (pun intended) or matrix multiplication operator %*%. But You can create more. And we will explore it in the next section.

Enumeration is an important stylistic device

Piping operators

The most widely known example is the piping operator: %>%. I assume that you are familiar with it. If not, you better get to know the dplyr package. The pipe comes from magrittr package, but dplyr shows its natural strength and the most significant advantage of using custom operators: custom infixes make code more readable. When we apply operations one by one, it makes more sense to write those operations in order of application, not inverse order, as we do without infix operators. The operator’s shape is also essential, as it suggests the direction of the data flow. For me, it is more natural to read and understand

x %>% 
  fun_1 %>% 
  fun_2 %>% 
  fun_3

rather than

fun_3(fun_2(fun_1(x)))

It became so popular that R 4.1.0 included its pipe, |>. However, I will not dwell more on that because others have already said a lot about pipes. I only want to mention, what is often forgotten by newcomers, that magrittr offer more than one type of pipe. Go check them out if you don’t know them!

NULL-default operator

Another tremendous and straightforward function is %||%. It is exported by the rlang package (it’s the turtleblog after all”, so rlang mention is obligatory), but other packages borrow this idea to avoid dependency. The code is straightforward and clearly explains what the function is intended to do.

`%||%` <- function(lhs, rhs) if (is.null(lhs)) rhs else lhs

34 %||% 8

## [1] 34

NULL %||% 10

## [1] 10

This one comes in handy when you have optional NULLs returned by a function or possibly NULL parameters of the function or named function arguments. In the code, it looks way more straightforward than the expansion of the function definition itself. I usually do not export such a helper function to the end user. But it is worth repeating that readability of your code for yourself should be as important as readability for others. If such infixes help you, then you should use them.

NULL-propagating operator

Let’s stay with NULL-related functions for a little longer. Now it is time for my creation. When I build packages or shiny apps, it happens quite often that I need to apply some function to some object, but if it is NULL, I need to return NULL. For those situations, I created the %?>% operator:

`%?>%` <- function(lhs, rhs) if (is.null(lhs)) NULL else rhs(lhs)

7 %?>% exp

## [1] 1096.633

NULL %?>% exp

## NULL

This one is some modification of a standard pipe but serves a specific purpose. It simplified my code significantly, and I love it for its conciseness.

Motif inclusion operator

Now, an example from one of our packages, tidysq. It is a package for the tidy processing of biological sequences. Here we implemented the %has% operator that checks for the presence of specific motifs in sequences.

library(tidysq)

sq(c("AAAA", "AGCA", "CGCG", "TTCG")) %has% "GC"

## [1] FALSE  TRUE  TRUE FALSE

Laura and I found it very in line with the tidyverse philosophy of code being readable and understandable by others. It can be especially seen in dplyr processing pipes.

library(dplyr)

tibble(
  id = 1:4,
  sequence = c("LVGWEK", "KLLCVN", "ER", "LLLY")
 ) %>%
  mutate(sequence = sq(sequence)) %>%
  filter(sequence %has% "LL")

## # A tibble: 2 × 2
##      id sequence  
##   <int> <ami_bsc> 
## 1     2 KLLCVN <6>
## 2     4 LLLY   <4>

Ternary if operator

Finally, one fancy trick! Have you programmed in any language that contains ternary if operator? E.g., in C++, it looks like this:

(1 > 0) ? 'A' : 'B'

// would return 'A'

If the expression on LHS of ? is TRUE, a value between ? and : is returned. Otherwise, the value after : is returned. Why use it over regular if-else? It is an expression, while standard if in C and C++ is not an expression. Thus, the user can assign the result of this expression to any variable.

Do you miss it in R? Probably not. In R, if is an expression, so we have:

if (1 > 0) "A" else "B"

## [1] "A"

But what if we really wanted to have something in R more like in C++? Here you go!

`%?%` <- function(lhs, rhs) {
  values <- rlang::enexpr(rhs)
  if (values[[1]] != as.symbol(":"))
    stop("RHS for `%?%` operator has to be in the form of '*:*' where '*' are any expressions")
  if (lhs) rlang::eval_bare(values[[2]]) else rlang::eval_bare(values[[3]])
}

(1 > 0) %?% "A" : "B"

## [1] "A"

I did it only because I can.

As you can see, I also abused the : operator; not overrode: abused. Inside the function, the RHS argument is in the form expr:expr. Normally that would result in range. But here I used the non-standard evaluation, using rlang::enexpr() and ralng::eval_bare() to actually stop R from doing what it usually does. I will probably go into detail about it some other time; right now, enjoy the fanciness.

Remark about precedence

One thing to remember: operators have specific precedence. Those that have been programming for longer know that well from an autopsy. Those that have just started programming will get to know that. Custom infix functions obey those rules as well, and you sometimes may get surprised. You can check that the operators I provided may sometimes be misleading and require parentheses to work correctly. It is worth checking out R operator precedence manual page when in doubt.

The strange sibling of my love

As a side note, I wanted to mention one operator I was unaware of before writing the article. Or: I was not fully aware of what lies underneath it.

Have you come across := operator in R? :=, with no % around it? You probably have seen if you are using data.table or some rlang magic. So have I. However, I took it for granted that its operator existence is due to non-standard evaluation since it is always used within some specific context (similarly to my previously shown %?% operator). I was astonished to find out that I can define this.

You can write something like this.

`:=` <- function(lhs, rhs) lhs + 5 * rhs
2 := 2

## [1] 12

And it works.

I told you earlier that overriding built-in operators is a big no-no. What is the difference with this one? Well, there is no such operator in base R! It turns out that := is reserved for legacy reasons, and the parser still treats it as a single operator. (You can try creating similar functions for other strings – you won’t make R treat them as infix operators without percent signs, they need to be reserved.) Yet := has no definition in base. You can read more on the SO thread about this operator.

If you want to be fancy, you can assign a classic assignment operator to this one and make others wonder how.

`:=` <- `<-`
a := 3

(But seriously: don’t do it.)

Send the letter!

The ability to create your custom inter-argument operators is an exquisite addition to the language. They can make the code much more readable and, usually, shorter. I strongly encourage you to take advantage of the language’s possibilities and play around with it. Since it’s your working tool, let’s make using it as enjoyable as possible!

Case Study: Modularizing a Package

Thu, 04 Aug 2022 00:00:00 +0000

“Would it be possible…?”, “I think it would be nice if…”, “Can you implement…?”.

User feedback is a reliable source of valuable ideas for package improvement, but it’s easy to get too eager and implement everything the users want, especially when you’ve only started making a name for yourself. I and Dominik have fallen victim to that too.

Origins of deepdep

Our package deepdep was initially created as a university project. There were four of initial authors: the two of us and our colleagues, Hubert and Szymon. The teacher had a use case in mind (creating layered dependency plots) and we wanted to implement all that could get us a good grade. So we added everything more or less related to dependency plots that we could implement at the time.

Fast forward a few years and a question came up regarding using a repository mirror other than the CRAN mirror we hardcoded. The function in question? get_available_packages(). We’ve exchanged a few messages and it turned out that get_available_packages() only served as a safeguard against using other mirrors within deepdep() itself.

In fact, the whole backend that downloads the data needed a rewrite. get_dependencies() tried to provide a unified API for retrieving dependencies from different sources and get_descriptions() did the same for DESCRIPTION… but they ended up messy and counterintuitive. The user could only get data from CRAN, CRAN with Bioconductor, or from the local library that was first in .localPaths(). No handling Bioconductor only, no using CRAN as a fallback for local library, no querying other repositories (e.g. R-universe). The functions had to grow a lot if we wanted them to be as universal as possible.

The other issue made us realize that the plotting feature is optional to some; that the key feature is collecting dependency data in a table, which only needs a small fraction of dependencies (httr and jsonlite). We moved a lot of previous Imports to Suggests (ggplot2, ggraph, graphlayouts, igraph, and scales), lightening deepdep significantly… but that’s a topic for another post.

It was time to ask ourselves: “what does «deepdep» mean to us?”. The answer was: “it’s a package that helps with analyzing and visualizing hierarchy of package dependencies”. No more, no less. The functions that extracted dependencies of a package or a DESCRIPTION file were just tools to accomplish that goal. They were exported because “we couldn’t let a good function go to waste”, not because they presented a functionality we wanted to provide. If a user would want to use one of these, they’d have to install the whole deepdep; it would be like installing ggplot2 for cut_interval() and cut_width() instead of plotting.

The time has come for a separation.

Separation of woodendesc

The idea is to modularize – to allow the user to install what they want. If they want to retrieve a list of dependencies for one package or a list of available packages in a repository, they should not need to install deepdep. They should be able to install a separate package that deepdep imports: woodendesc.

This package is a complete rewrite of these functionalities, but much more flexible and much more potent. To show the difference, this is how you’d get packages available on CRAN and Bioconductor in old deepdep:

# Hey, a wild R 4.1.0 pipe appeared!
deepdep::get_available_packages(bioc = TRUE) |>
  head()

## [1] "A3"        "a4"        "a4Base"    "a4Classif" "a4Core"    "a4Preproc"

You can’t do much more than that. The only other option is to get locally available packages. This is the signature of the function:

get_available_packages <- function(
  bioc = FALSE, local = FALSE, reset_cache = FALSE
) {
  # Implementation goes here
}

But woodendesc goes three steps further. There are functions for many different sources of packages, each of them optimized for minimal network usage and maximal cache utilization:

# Simple CRAN extractor
woodendesc::wood_cran_packages()
# Allows `release` parameter to query old releases
woodendesc::wood_bioc_packages()
# The user can specify different paths
woodendesc::wood_local_packages()
# Functions below not possible in old deepdep:
woodendesc::wood_runiverse_packages("turtletopia")
woodendesc::wood_url_packages("http://www.omegahat.net/R")
woodendesc::wood_core_packages()

And if you’d want a single function like get_available_packages()? Easy, just call wood_packages() with specified repos (by default it only queries CRAN):

woodendesc::wood_packages(c("bioc", "cran")) |>
  head()

## [1] "A3"        "a4"        "a4Base"    "a4Classif" "a4Core"    "a4Preproc"

You can do it with all the sources above and even pass most parameters:

woodendesc::wood_packages(
  c("bioc@1.5", "core", "runiverse@turtletopia",
    "http://www.omegahat.net/R", "local#all")
) |>
  head()

## [1] "00LOCK-lubridate" "aCGH"             "affy"             "affycomp"        
## [5] "affydata"         "affylmGUI"

Now, you can see why’d we separate these functionalities into a new package. There are analogous functions for version codes and dependencies (about 20 functions total!) and they’d overwhelm the original intent of deepdep. Adding woodendesc as a dependency of a deepdep costs nothing because the alternative is to include this code within deepdep itself – so it’d have to be tested and maintained anyways.

But sometimes modularizing is a bit extra.

A new package is not always the answer

If you have a function if your package that doesn’t fit the general idea, don’t rush to move it into a separate package. There’s one important question to ask before:

“Will it be used by anything else than my package?”

And don’t be proactive here. If your answer is: “not right now, but perhaps in the future…”, just wait for the future. Keep the function in the package until the time comes and simply remove or deprecate it then (depending on how popular it gets).

There’s one such functionality in deepdep: get_downloads() and plot_downloads(). Analyzing download statistics is not exactly the goal of deepdep, but there’s no point in making it into a separate package; these two don’t introduce any new dependencies nor do they crowd the namespace. And no one expressed any interest in having it separate from deepdep yet.

Besides, nobody creates a package around a single function.

Modularization of tidysq

You might have noticed that woodendesc consists of functions that served as a backbone of deepdep while querying and plotting download statistics are more of an extension. There’s one package we’ve created that was planned to be extended since the beginning: tidysq.

It’s a package that compresses biological sequences (e.g. DNA/RNA) by coding each letter with fewer bits (3 in DNA/RNA case). We’ve included a few basic operations like reversing, subsetting, translating to amino acids, and reading a FASTA file – the most common file format for biological sequences. We’ve intentionally omitted many more advanced functions, though.

Why? Because there are countless functions and algorithms we could implement and that’d make tidysq huge. Instead, we’d gone the route of modularization. The idea is to have tidysq with the base functionality and several packages depending on tidysq, oriented towards certain aspects of working with biological sequences.

For example, if we were to create a set of read_x() and write_x() functions for various formats like FASTQ or BAM/SAM, we’d place it in a separate package that’d have tidysq in Depends (and LinkingTo) fields. We’d call it something like “tidysqfiles” to signify that it’s an extension to tidysq.

(We may or may not be working on such a package.)

If you want to see a real-life example of a package ecosystem, see mlr3 and mlr3verse.

Summary

In short, there are two cases where modularization should be considered:

the backend to the main functionality grows and overshadows the rest of the package – create a set of logically related backend functions, move them into a new package, and add that package to Imports of the old one;
there’s an optional functionality that requires additional imports or significantly increases the weight of the package – collect several such functionalities so that they are somewhat related, move them into a new package, and add the old package to the Imports/Depends field of the new one.

Be wary of separation if the only use case for the new package is to be imported by the old one. Avoid it if there are too few functionalities for a new package. Sometimes copying a function or two isn’t a sin.

Do you want to borrow a code that shows an install prompt for a missing package?

Ice Cream for R Programmers

Thu, 28 Jul 2022 00:00:00 +0000

What and why ice cream is.

This post - perhaps contrary to the headline - will not be about eating ice cream. It will, however, be about something we all do too. About something, we do too often, even though we shouldn’t. About debugging with a print().

Let’s face it; it happens to us. We have a rich arsenal to fight bugs. We have error messages, advanced tracebacks from rlang, classic browser() and debug() functions, logging utilities or IDE built-in breakpoints. And yet. In our code, there happen to be code snippets scattered here and there like the following:

some_advanced_function <- function(x) {
  print("HERE")
  y <- other_function(x)
  print(y)
  more_calculations()
  print("I HOPE IT GETS THERE!!!!")
}

We’re all lazy, and that’s why we use print-driven debugging (and I’m sure I’ll expand on the laziness at another time). (It’s funny that acutely lazy evaluation is an acutely brilliant solution.) All the other solutions mentioned above require a little more effort, whereas adding a print() call to the code is almost effortless. Almost. As it usually turns out, the devil is in the detail.

Since we all do it and we all consider it to be a kind of inappropriate practice, perhaps it would be worth taking some steps to make it more appropriate after all? And at the same time, pay even more tribute to our laziness? That was the idea behind the developer of the Python library IceCream. And this is the idea that guided me and Lewin Appleton-Fox in creating R version of icecream.

How to eat ice cream and why you should do it.

The most basic usage is straightforward. The library’s primary function is ic(), which takes an argument of any kind, prints its value to the screen and returns it invisibly.

library(icecream)

ic(12)

## ℹ ic| `12`: num 12

We can immediately see the first advantage over print: typing ic() is faster than typing “print”. Since we are here because we like to make our life more convenient, that might be important to one.

That would be useless, though, if we could not include it within a function. And of course, we can do it. We will reuse the previous example but replace the print() calls with ic(). And let’s pack the function and other functions into an external file called external.R.

# external.R

other_function <- function(x) x * 5 - 7
more_calculations <- function() NULL

some_advanced_function <- function(x) {
  ic("HERE")
  y <- other_function(x)
  ic(y)
  more_calculations()
  ic("I HOPE IT GETS THERE!!!!")
}

source("external.R")
some_advanced_function(42)

## ℹ ic| `"HERE"`: chr "HERE"
## ℹ ic| `y`: num 203
## ℹ ic| `"I HOPE IT GETS THERE!!!!"`: chr "I HOPE IT GETS THERE!!!!"

That works well. And you can also see another reason to use ic(): it automatically prints expression alongside its value. That might be very useful, especially if we have many prints and don’t want to check which one corresponds to which value.

But, as you have probably noticed, this verbose printing introduces redundancy when printing HERE etc. Well, that is on purpose. With icecream you don’t need to provide a message to get the context. Let’s modify the function slightly and remove those locator messages.

# external_2.R

some_advanced_function_2 <- function(x) {
  ic()
  y <- other_function(x)
  ic(y)
  more_calculations()
  ic()
}

source("external_2.R")
some_advanced_function_2(42)

ℹ ic| `some_advanced_function_2()` in external_2.R:3:2
ℹ ic| `y`: num 203
ℹ ic| `some_advanced_function_2()` in external_2.R:7:2

Now you can see that ic() is even more clever! It includes as precise information about the file and position in the file as possible if you call it without arguments. This context inclusion makes debugging even more seamless!

When the context of the file is not available (e.g. when calling a function created from a console), an environment of the source is printed:

a_function <- function(x) {
  x + 2
  ic()
}

a_function(0)

## ℹ ic| `a_function()` in <env: global>

(Side note: I need to admit that I lied a little with the above code: compiling this code with R markdown does not preserve the file names when sourcing. The output is pasted manually because that would call it from the console. knitr removes references when calling source, and we have not found a way around it. But that is not a concern since Rmd documents are usually final results.)

What is the biggest problem with print-driven debugging? Our memory. It might be a problem if we forget to remove those print()s. Especially when, being irritated, we include some curse words here. And then send it to the client (anticipating questions: I have not done that, but my colleague has…). Thankfully, ic() is help here! It is enough to call:

ic_disable()

to disable all the ic() calls. They behave just like a regular identity() now. There is a counterpart, ic_enable(), which does the opposite. If you are, for example, building a shiny app, you can enable and disable ic() conditionally depending on whether your app is in development or production mode. Additionally, using ic() when building a package is a safety net. If you don’t import the ic() function and rely solely on loading the library externally, the package will not pass checks.

Last and not least, there is a dose of customization possibilities. You can e.g. change the function used to peek and the number of lines printed, as you can see in the chunk below:

# standard settings:
ic(mtcars)

## ℹ ic| `mtcars`: data.frame [32 x 11]: $'mpg': dbl [32], $'cyl': dbl [32], ...

# modified settings:
options(icecream.max.lines = 5)
options(icecream.peeking.function = head)

ic(mtcars)

## ℹ ic| `mtcars`: 
## mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1

A few words on ice cream production.

The inner workings of icecream are pretty curious, and I encourage you heavily to explore the code if you want to see rlang in action. I will comment on a few code snippets to give you a glimpse.

Using ic() is possible because we can suspend the evaluation of arguments:

ic <- function(x) {
  # capture the input to allow us to work with the expression and value separately
  q <- rlang::enquo(x)
  ...
}

Now, q holds the expression alongside with its evaluation environment. We can quickly transform it into a string by deparsing it:

deparsed_expression <- rlang::expr_deparse(rlang::quo_get_expr(q))

To get the value of the expression, we can enforce evaluation:

x <- rlang::eval_tidy(q)

Getting the precise location of the ic() call is more tricky. We can do it, however, with the usage of rlang functions for analyzing the stack of calls:

# this code is simplified and not guaranteed always to work

# inspecting the traceback and extracting the call stack:
trace <- rlang::trace_back()
call_stack <- trace$call

# getting length of the call stack:
num_calls <- length(call_stack)

# accessing the second-to-last call (to omit the `ic` call) and extracting parent reference:
parent_ref <- call_stack[[num_calls - 1]][[1]]

# extracting location from the reference:
ref <- attr(call_stack[[num_calls]], "srcref")
loc <- rlang:::src_loc(ref)

Notice that rlang:::src_loc() is not an exported function, as it is not meant for end users of the package. We allowed ourselves to borrow the code of the function as it is crucial for our package.

Plans for the ice cream industry and conclusion.

If you practice using print to debug your programs, do it with class. That is what icecream offers. Its main advantages are less typing, convenient information printing, the inclusion of context, easy disabling and customization. The package is still under development, and our current goal is to fully implement the original Python version’s functionalities. I encourage you to give it a try!

If you have any suggestions for the icecream, do not hesitate to leave an issue on our repo.

Overview: Sorting Version Codes

Sat, 16 Jul 2022 00:00:00 +0000

Problem statement

Different versions of software are traditionally described using version codes. These codes contain usually numerical values and subsequent versions increment these codes, allowing them to be sorted. The standard is to use “semantic versioning”, where codes are in the form of “major.minor.patch”; the bigger the update, the more important number is incremented.

However, not all software obey semantic versioning standard.

Suppose we have a vector of version codes:

codes <- c("1.2", "1.10", "1.1")

We’d expect them to be ordered as 1.1, 1.2, 1.10. If you try to order version codes using base sort() functionality…

sort(codes)
## [1] "1.1"  "1.10" "1.2"

…oh. Where did something go wrong?

This is what’s called “alphabetical sorting”. It checks characters one by one, including digits. “1” is before “2”, so “10” goes before “2”, no matter the value. Thus, it cannot compare numbers, if they have a different number of digits.

Numerous software uses the following trick to have it sort correctly: it pads the numbers with leading 0’s until a certain length. For example, your old camera could name photos like 00034.jpg, 00164.jpg, etc. But this is not the approach to use here. We need a different one.

Sorting version codes requires using something called “natural sort”, where numbers are detected and ordered according to their value (with letters still sorted alphabetically, thus placing e.g. “a99” before “b51”). This is a more intuitive sorting for humans than pure alphabetical sort.

Solutions

Several packages allow natural sorting of character vectors. There’s even a base R solution! However, not all natural sorting algorithms are adapted to handle version codes, which (usually) have a particular structure with dots and/or dashes. Let’s have a look at the options.

Using base R

There’s a facility in base R allowing the user to handle (numeric) version codes. While there is no simple “sort these strings like they were version codes” function, you can wrap your vector in numeric_version() and use associated methods, like comparison operators (<, ==, >=…), as well as sort() method for this class. This is what it looks like:

sort(numeric_version(codes))
## [1] '1.1'  '1.2'  '1.10'

There are two main problems with this approach, however. But first, let’s quote the documentation:

Numeric versions are sequences of one or more non-negative integers, usually (…) represented as character strings with the elements of the sequence concatenated and separated by single . or - characters.

This means that version codes cannot have any string components or non-standard separators. As described in the semantic versioning standard, letters are allowed in pre-release identifiers and build metadata, so the base solution cannot even handle all valid cases of semantic versioning, much less the variety of version codes that don’t fit these rules.

Using gtools

gtools is a collection of functions to help with simple tasks when writing R packages. It has a mixedsort() function that detects embedded numbers in strings. However…

gtools::mixedsort(codes)
## [1] "1.10" "1.1"  "1.2"

…it is not suited to sorting version codes. The use case here is different, it’s detecting numbers like these in “Aspirin 50mg” and “Aspirin 100mg” (examples taken from the documentation). Dots in version codes are treated as decimal separators, that’s why "1.10" and "1.1" are considered equal.

Using naturalsort

naturalsort isn’t written specifically for sorting version codes either, but…

naturalsort::naturalsort(codes)
## [1] "1.1"  "1.2"  "1.10"

…it gets the job done. The difference lies in not treating dots as decimal separators (in fact, naturalsort doesn’t recognize decimals at all). Instead, the code detects numbers and non-numbers, then splits the strings into continuous pieces of numbers and non-numbers. If we were to For example, "1.10-a" would be split like that:

split_like_naturalsort("1.10-a")
## [1] "1"  "."  "10" "-a"

These pieces are then used to sort version codes either alphabetically or by value, depending on whether it’s a number or not. Accidentally, this works well for almost all version codes. The only case where it’s working a little weird is when there’s an inconsistency in separators:

weird_codes <- c("1.3-6", "1.3-2", "1.3.4")
naturalsort::naturalsort(weird_codes)
## [1] "1.3-2" "1.3-6" "1.3.4"

I like that naturalsort has decreasing and na.last parameters, implemented just like in base sort(). There’s also a naturalfactor() function that could be useful for long vectors of repeated version codes.

naturalsort isn’t developed since 2016, but this shouldn’t be considered a problem, since the package is in an already stable state. There’s one pet peeve of mine about this package and it’s the implicit coercion of parameters. Let’s say we accidentally made this call:

naturalsort::naturalsort(codes,
                         decreasing = c("true", "dat"))
## [1] "1.10" "1.2"  "1.1"

Turns out c("true", "dat") was coerced to logical, then the first element was taken. It should raise an error saying that the arguments are wrong, but – instead – it tries to work at all costs, over-interpreting the parameters. A similar thing would happen with a vector of logical values.

Using stringr/stringi

You’ve probably already installed stringr and/or stringi if you’ve been using R for a while. Both these packages can do way, way more to strings than just some natural sorting, but we’ll focus on just that. There’s a parameter called numeric in stri_sort(), where we can pass TRUE to sort digits numerically:

# There's analogous stringr::str_sort() function
stringi::stri_sort(codes, numeric = TRUE)
## [1] "1.1"  "1.2"  "1.10"

The overall implementation seems to base on the same idea of separating numbers and non-numbers, and it has the same behavior on inconsistent separators:

stringi::stri_sort(weird_codes, numeric = TRUE)
## [1] "1.3-2" "1.3-6" "1.3.4"

There are decreasing and na_last parameters just like in naturalsort() and base sort(); and again, there’s silent argument coercion I despise. At least I can give half a point for raising a warning when only the first element is used.

Using versionsort

versionsort is our solution to this problem. I’ve implemented, documented, tested, and submitted it to CRAN within 24 hours when working on implementing a feature in deepdep; obviously, it has changed a bit since then.

There’s a ver_sort() function that’s of the main interest here:

versionsort::ver_sort(codes)
## [1] "1.1"  "1.2"  "1.10"

It works a little differently than other sorts I’ve mentioned before, as it splits the string on separators first (which are sequences of anything that is not a number or a letter), only then separating numbers and non-numbers (i.e. letters). All separators are equal, which gives a little more predictable behavior with inconsistent separators:

versionsort::ver_sort(weird_codes)
## [1] "1.3-2" "1.3.4" "1.3-6"

versionsort is still a Work-In-Progress, which means that it lacks decreasing and na_last parameters, but this whole analysis serves partially to define requirements for future development (more in soon-to-be-published versionsort Dev Plan).

There’s one convenient utility versionsort has, though – the pair of functions named ver_latest() and ver_oldest(). They return max() and min() for version codes, respectively, without the need for sorting the whole vector:

versionsort::ver_latest(codes)
## [1] "1.10"
versionsort::ver_oldest(codes)
## [1] "1.1"

Stay tuned for the new, improved versionsort!

Semantic version codes

All these solutions have one thing in common – they don’t detect semantic version codes. There are, however, R packages with this functionality; I’ve found two, described below.

Using semver

semver is the most used R package for handling semantic version codes. It relies heavily on C++ semver implementation under the hood, but it results in a wide range of functionalities: there are many methods for parsed semantic codes, sort() included:

semver_codes <- semver::parse_version(
  c("1.6.0-dev", "1.5.10", "1.5.1", "1.6.0")
)
sort(semver_codes)
## [1] Maj: 1 Min: 5 Pat: 1
## 
## [2] Maj: 1 Min: 5 Pat: 10
## 
## [3] Maj: 1 Min: 6 Pat: 0 Pre: dev
## 
## [4] Maj: 1 Min: 6 Pat: 0

Operators as well:

semver_codes[1:3] < semver_codes[2:4]
## [1] FALSE FALSE  TRUE

We can use max() and min() too:

max(semver_codes)
## Maj: 1 Min: 6 Pat: 0

This package has the most functionality of them all…

semver::parse_version("1.0-10")
## Error in parse_ptr(version): invalid character encountered: -

…but is nonetheless limited to semantic version codes.

Using semverutils

Sorting numeric codes is not possible in semverutils. You can only check if a version code is higher than others by calling:

v_code <- semverutils::semVer$new(c("1.6.0-dev"))
v_code$higherThanAll(c("1.5.10", "1.6.0", "1.5.1"))
## [1] TRUE

Unfortunately, the answer isn’t even correct, since "1.6.0-dev" precedes "1.6.0" due to being a pre-release version.

Conclusion

There are a few ways to sort version codes. If your version codes are simple and well-behaved, base::numeric_version() should be enough. If you need to handle semantic version codes, use semver. Otherwise, there is no best solution, but versionsort is the closest you can get, especially after the next big update is published.

Missed a package? Described one incorrectly? Contact us on Twitter or make an issue on Github.

Shiny Reactivity Tricks, pt. I: Creating Observers in Loop

Sat, 09 Jul 2022 00:00:00 +0000

Preface

I’ve been recently working a lot in Shiny, and I’ve constantly been stumbling across new problems. Many of them seemed interesting enough for me to want to write an article. However, in the course of the work, I found that I had too much to write about… so I decided to break it down into a series of smaller articles linked to each other.

I’ll start with something simple: creating some objects in a loop. If you think it’s too easy, wait a minute; you may be surprised! This seemingly easy task will be an excuse to understand how certain things in R and shiny work underneath, even though we ignore it daily.

Use case and naive solution

Let’s start with some simple examples. Like all simple examples, it may feel highly artificial, but at least it is not very complex. Let’s say that we want to have an app that has four numeric inputs. The first input allows the user to select an Important Number. Other inputs… well, they are not used. But we want them to have their values updated every time the Important Number is updated. And we want to have separate observers to handle them (yes, I know that one observer would work here as well… but I told you this might feel artificial!). Their values are calculated via some calculations which are entirely irrelevant right now.

library(shiny)
# side note: all functions which are not prepended with scope operator 
# are either from base R or from the `shiny` package

do_calculations <- function(important, other) {
  # this is totally random
  (important * other * 2^5 + important - 7 * other) %% 101
}

ui <- fluidPage(
  numericInput("important", "Select important number", 
               min = 0, max = 100, value = 42),
  numericInput("var_1", "Select input 1", 
               min = 0, max = 100, value = 0),
  numericInput("var_2", "Select input 2", 
               min = 0, max = 100, value = 0),
  numericInput("var_3", "Select input 3", 
               min = 0, max = 100, value = 0)
)

server <- function(input, output, session) {
  observe({
    updateSliderInput(
      session = session,
      inputId = "var_1",
      value = do_calculations(input[["important"]], 1)
    )
  })

  observe({
    updateSliderInput(
      session = session,
      inputId = "var_2",
      value = do_calculations(input[["important"]], 2)
    )
  })

  observe({
    updateSliderInput(
      session = session,
      inputId = "var_3",
      value = do_calculations(input[["important"]], 3)
    )
  })
}

shinyApp(ui, server)

Simple version of the app.

It works like a charm! But we can easily see that there is a lot of copypasting. And copypasting is something we should avoid. (Some other day, I would love to say something more about it in general.) If you have gone so far into the article, you probably know what I want to do: a loop. There are two prominent places to use it – in UI and server. UI first, since it seems more manageable. We are R users, so we should make real loops only when there are no other means. Thus we are using good old lapply, as it is even simpler than trying to use for.

ui <- fluidPage(
  numericInput("important", "Select important number", 
               min = 0, max = 100, value = 42),
  lapply(1:3, function(i) {
    numericInput(inputId = paste0("var_", i), paste0("Select input ", i), 
                 min = 0, max = 100, value = 0)
  })
)

Okay, this is simple, indeed. It works as it did without the loop; you can check on your own if you don’t believe me.

Now it’s time for the server. Here we are calling observer mainly for its side effect (i.e. registration of observers), so the for loop might be more appropriate.

server <- function(input, output, session) {
  for (i in 1:3) {
    observe({
      updateSliderInput(
        session = session,
        inputId = paste0("var_", i),
        value = do_calculations(input[["important"]], i)
      )
    })
  }
}

And now – something strange happens! Only the last of the observers works as expected; others seem off. If you know at first glance what happens there – congrats; you are good at R and Shiny. If you don’t know, then we need to understand what environments are relevant for Shiny.

A quick recap of environments

Each R expression is evaluated in some environment. An environment can be considered a list of names with associated values. There are, however, some differences, of which two are the most important:

Firstly, each environment contains a reference to its parent, so they create a tree-like structure. One exception to this rule is the empty environment which does not have a parent and serves as the root of the whole hierarchy.

Secondly, environments are mutable. If you pass an environment to a function and mutate some objects inside, the values in the environments are also changed for the external viewer.

Every time we create an object in R and assign its value to some time, we bind this object within some environment. Calls from the console or scripts at the top-level assign objects to the global environment. Creating variables inside a function assigns them to an execution environment. Each package also has its environment, where all its functions and objects are available.

When we want to access some variable, it is first sought in the evaluation environment, i.e. the environment where this call is performed. If no object of this name is found, it is sought down the search path: in the parent environment of the calling env, then in the parent of the parent, etc., down to the empty env. At the start of the R session, base packages are inserted into this hierarchy, so those objects are accessible from the global environment. Additionally, every time we use library, the environment of the loaded package is inserted into the path. That explains why we have access to all the functions and objects from the console. That also explains why objects in the global environment are visible from within the function called in the global environment.

One more thing worth mentioning – in R, we can manage the time of evaluation of the code. If a function is provided with an expression, there are means to suspend the evaluation of its arguments. Then we can modify it or evaluate it later.

I won’t go into the details of how to do it right now (but don’t worry, I like the topic too much not to tell more about it). If you want to get new knowledge or refresh the one that you’ve got, you can see a chapter in Advanced R by Hadley Wickham

Environments in Shiny

Environments are super crazy functional. The fact that they exist does Shiny work at all. And they are also the reason why our for loop did not work. That is, of course, only a part of the whole picture, but such complicated topics are easier to swallow in smaller bits.

We need to focus on what happens when the app starts (e.g. when function shinyApp(ui, server) is called). Somewhere in the guts of Shiny, the server function is called. Just as with any other function, it has its execution environment. This environment is provided with input and output lists of reactive values; it also has the global environment as its ancestor (and therefore, it has access to any loaded packages). When the code inside this function is executed, objects specified by the user are created. Significantly, all reactive objects are made, and the user can modify input and output reactive values. Also, our objects of focus – observers – are registered.

The most crucial point is: when the server function is executed, and R stumbles upon the observer call, it does not evaluate the code passed as the first argument. Instead, it registers the observer and stores the code for later evaluation. We are still at the start-up stage, and now Shiny only creates net reactive dependencies. No inputs are available; they appear just after somebody opens the browser and initializes the page. That also applies to all expressions enclosed in reactive’s, render’s, etc., but we skip them in the other part as they do not appear in our example.

Now goes the neat part: during the application runtime, code provided to observer is evaluated as is in the server function execution environment. So, when seeking a variable, it looks for its value in this environment. It does not magically replace the i variable with the value used in the loop iteration, as might be expected. I made a diagram (included below) which illustrates how it does not work.

Expected situation

Stating it once again: every time the observer is called, i is being sought in the current or enclosing environments. But that introduces the question: shouldn’t that mean that i is missing and calling i raises an error? Why does it work for one of the observers? The answer is: actually, all three observers work. But they all modify the input number 3. And that is because of one quite peculiar property of R. Namely, the variable used to iterate is not destroyed after finishing the loop execution, and it stays in the environment with the value of the last value used in the loop. In our case, i exists in the server environment and has the value 3. See the diagram below.

Actual situation

As a side note, I want to add that there is also the topic of visibility of Shiny objects between users’ sessions. It is covered by an article on the official Shiny webpage. I strongly recommend reading it.

Since we already know what happens, we must learn how to get around it. There are at least a few solutions.

Solution 1: Create an environment for each observer

I told you that the code in observers etc., is evaluated in the server environment. It is true by default. There are ways of changing it, even straightforward ways. observer function has a parameter env, which allows us to set the evaluation environment.

for (i in 1:3) {
  env <- new.env()
  env[["i"]] <- i
    
  observe({
    updateSliderInput(
      session = session,
      inputId = paste0("var_", i),
      value = do_calculations(input[["important"]], i)
    )
  }, env = env)
}

In each loop iteration, we create a new environment whose parent is the server environment. Then we assign the current value of the i to the environment and provide the env as a parameter for the observer. As a result, each observer has its environment with its own i object. Those envs refer to server env as a parent, so all other variables are accessible. The trick is possible because those numerous envs are not used inside the observers’ quoted code; they are kept together with code. The concept is illustrated in the diagram below.

Solution with the creation of environments

We can also do the same with a small change. new.env() call can be replaced with rlang::env(). rlang is a package that overhauls the interface of operating on expressions and environments in R, as base R ways of doing it are messy. A minor change, but on this blog, we will demonstrate rlang features heavily, so you should get used to it. Additional source is the project webpage.

Solution 2: Inject the constants

Speaking of rlang… The second solution employs rlang’s features and does something very clever: it replaces a variable with a constant.

for (i in 1:3) {
  rlang::inject({
    observe({
      updateSliderInput(
        session = session,
        inputId = paste0("var_", !!i),
        value = do_calculations(input[["important"]], !!i)
      )
    })
  })
}

Beautiful, isn’t it? This code looks almost the same as the code from the not-working approach of simple iteration. The first difference is prepending i variable appearances with the !! symbol (bang-bang, as ones say, or more professionally: unquotation symbol). The second is wrapping the whole observer with the rlang::inject function. This function replaces everything prepended by an unquotation symbol with the value of the symbol. That operates on the level of expression. Before observer is called, both appearances of i are replaced with the current value of i. Only after that the observer is called, and modified code is saved instead. You have to admit: this is elegant. It also makes the code work as we expected the original code to work, that is, without creating additional environments.

This solution is possible with base R, but the code is significantly less appealing and easy to understand, so we will only stick to it.

Solution 3 (or solution suggestion): Utilize Shiny modules

Finally, there is one more solution. If you are already familiar with shiny modules then it should be clear that you can use them. I will not provide the code yet, mainly because it seems to me that many things have already happened today. But don’t worry – I have a lot to say about the modules too. We will come back to it. Those that know how they work can write code using modules as homework. Those that do not know should read some introduction to shiny modules.

Summary

When environments and modification of expressions come into play, even as simple as loops may get complicated, I wanted to show you that coming across them in R (especially in Shiny) is easy. Without a slightly more profound understanding of it, they might seem illogical and not rarely impossible to overcome. Having the additional knowledge and ability to use such tools as rlang might allow you to make your code better and cleaner. And sometimes working at all.

Hello world

Fri, 01 Jul 2022 00:00:00 +0000

Hi all!

I think it would be some aberration to start a programming blog with a post with any other title than “Hello world”…

I am Dominik. I am a software developer and Data Science student (with the time of graduation approaching with high velocity). My friend Laura and I are passionate about programming in R. We love creating tools and playing with language to please others and ourselves (in any order).

This blog is meant for us to write about R.

(Probably not only R, but hey, this is the language we have the most experience with)

We will publish here information about the open-source stuff we make. We will show some obstacles we have stumbled upon and our ways around them. We will praise the language and complain about it. We will share other cool toys we have found and love using. ~~We will rock you~~.

You can find some of our packages in our GitHub organization. Soon there will be many rearrangements, and we will focus on finally turning our recent ideas into flesh.

Stay tuned!

About

Thu, 05 May 2016 21:48:51 -0700

Hey, Laura and Dominik here!

We’ve studied and worked together on several R projects in many domains over the last few years. There were shiny apps, machine learning, bioinformatics, Rcpp, data visualization… we can’t even list it all. And the time has come to share that knowledge.

We want to have a few types of blog posts: some talking about our experiences with certain packages/frameworks (looking at you, shiny!), some sharing obscure knowledge and R facts, some showing how to implement something in a given framework, and some giving an overview of existing solutions to certain problems. And maybe an occasional rambling about other languages (especially C++ due to the existence of Rcpp) as well.

If you have an idea for a post (especially How To and Overview post types), tag us on Twitter or make an issue on Github.

This blog uses a blogdown framework.