### Artwork Search API

Loading packages:
  
```{r}
library("httr")
library("jsonlite")
library("tidyverse")
library("jpeg") #to let us read .jpegs/.jpgs
library("grid") #to let us plot images
```

Our cat facts API is cute but it's not terribly useful unless we want to pull "random" objects out of an API pipeline, which probably is not the case. So instead, let's explore a more useful public API, from the [Art Institute of Chicago](https://api.artic.edu/docs/#quick-start). This API has multiple **models** or "resources" (essentially, representations of the underlying data that exist in some relational databases somewhere -- more next week), each of which can be queried via three **endpoints**. 

Much like our cat facts API, we can just do a direct call to the base URL, which corresponds in this case to the **listings endpoint**. Queries to this endpoint return pages from all listings of the AIC collection. In this case, by default get back the first page of results only, and we get a lot of data for each artwork or each artist (depending on which model we query):

```{r}
artworks_url <- "https://api.artic.edu/api/v1/artworks"

fromJSON(artworks_url)

artists_url <- "https://api.artic.edu/api/v1/artists"

fromJSON(artists_url)
```

Let's focus, for now, on the artworks model. As we just saw, our query produced a large number of columns ("fields"), many of which we don't really want or need. Consulting the documentation, and using what we know about the structure of URLs, we see that we can specify fields for our query:

```{r}
artworks_url_fields <- "https://api.artic.edu/api/v1/artworks?fields=id,title,artist_display,date_display"

fromJSON(artworks_url_fields)
```

Now let's switch to a different endpoint, the **detail endpoint** where we can request information on specific artworks. We're sitll using the artworks model, and again we'll only query specific fields for the artwork(s) of interest. We'll start to build this up in a slightly more principled fashion, using `paste0()`, which concatenates strings. Below, the first string in our `paste0()` function is the `artworks_url` model URL we defined above, the second string is some required formatting, the third string is the specific `artwork` of interest (can you figure out which artwork it is?), and the third string is the specific set of `fields` we want. 

```{r}
# define our fields of interest
fields <- "?fields=id,title,artist_display,date_display"

# provide an artwork to study
artwork <- "28560"

# build the query and retrieve JSON
artwork_detail_url <- paste0(artworks_url, "/", artwork, fields)

fromJSON(artwork_detail_url)

# to show only the data we want
fromJSON(artwork_detail_url)$data
```

The next endpoint is perhaps the most interesting for us: the **search endpoint**. This allows us to search the model of interest, and return only the data that results from that search. This is great because it lets us narrow down our requests and not overload the AIC's servers, and because it lets us look for specific types of art (you can imagine how useful this would be in a social science application). In furtherance of our feline API efforts, let's start by searching for artwork about cats:

```{r}
# artworks model, search endpoint url:
artworks_search_url <- "https://api.artic.edu/api/v1/artworks/search?q="

# define search terms. we use gsub(" ", "%20", "x") here to replace spaces between search terms with "%20" which is how we often represent spaces in a URL. 
search_terms <- gsub(" ", "%20", "cat")

# build the query:
cat_search_url <- paste0(artworks_search_url, search_terms)

fromJSON(cat_search_url)
```

What we've done above is to use the logic of the AIC API to build particular URLs of interest, and then query them directly with `fromJSON`. Now let's start writing queries in a slightly more elegant fashion, using the `httr` package (you can also use `httr2`, which is a newer re-write of `httr` that is in early versioning). These packages are high-level interfaces of [`curl`](https://github.com/jeroen/curl), developed for flexible and customisable querying of web resources from R. For now, we will use the `GET` function from `httr`. Among other things, the `GET` function allows us to input the `url` of interest, an additional `path` (which we won't use in this case), and a detailed `query` list which can take as many elements as there are parameters for our API. 

```{r}
# build the API GET request
cat_search <- GET(artworks_search_url, # the API endpoint of interest
                  query = list(q = search_terms, 
                               fields = "id,title,artist_display,date_display",
                               size = 10)) # query allows us to specify parameters, which we find in the API documentation

# parse the content returned from our GET request
json_cat_search <- content(cat_search, "parsed")

# let's inspect our content
json_cat_search

# not so useful! so let's see what we got in a slightly easier way...
names(json_cat_search) 

# $data is what we want. so let's use do.call, rbind, and lapply to extract all the data from our returned content, and format it as a tidy tibble
cat_art <- do.call(rbind, lapply(json_cat_search$data, as_tibble, stringsAsFactors = FALSE)) %>%
  select(- '_score') # removing the search score, but you can keep it if interesting to you

# let's look at our tibble
cat_art
```

So far we have used the AIC API to extract information about the collection and its artworks. That's nice, but there's more interesting things we can do. The AIC supports a second -- different -- API that allows us to download `.jpeg` copies of their artwork. We're now going to learn how to download and visualise these images in R. 

First, we have to retrieve from the default API (but using the images model) the image id (not the same as the artwork id!) for the pieces of interest. Then we can query the alternative API to retrieve the actual images. 

```{r}
# query the API:
cat_image_search <- GET(artworks_search_url, # the API endpoint of interest
                    query = list(q = search_terms, 
                                 fields = "title, image_id",
                                 size = 1)) # query allows us to specify parameters, which we find in the API documentation
json_cat_image_search <- content(cat_image_search, "parsed")

# directly extract the image id (as we are just working with one request, we don't need to worry about flattening the data)
cat_image_id <- json_cat_image_search$data[[1]]$image_id

# now, we introduce our alternative API, the AIC's IIIF (International Image Interoperability Framework) API
iiif_url <- "https://www.artic.edu/iiif/2"

# using our iiif_url and our cat_image_id, plus some formatting as provided by the AIC API documentation, we get
iiif_url_artwork <- paste0(iiif_url, "/", cat_image_id, "/full/843,/0/default.jpg")

# assign an empty temporary file to store our downloaded image in this R session (in a moment we will save these locally, when we do a retrieve of images)
temp <- tempfile()

# download the file from our API URL
download.file(iiif_url_artwork, temp, mode="wb")

#Reading the file from the temp object
image_to_plot <- readJPEG(temp)

class(image_to_plot)

# plot our image, using ggplot (can also use base R)
ggplot() +
  annotation_custom(rasterGrob(image_to_plot), xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=Inf) +
  theme_void() +
  theme(plot.margin = unit(rep(0, 4), "null"))

```

Finally, let's build a piece of code for requesting and plotting artwork from the AIC, using any set of search terms we want. 

```{r}
# let's build a function
art_image_search <- function(search_term, n_images = 5, output_dir = "temp_images", clear_directory = TRUE, plot_images = TRUE) {

  search_term <- gsub(" ", "%20", search_term)
  
  images_search_url <- "https://api.artic.edu/api/v1/artworks/search?q="
  images_search_out <- GET(images_search_url, # the API endpoint of interest
                       query = list(q = search_term, 
                                    fields = "id, title, artist_display, image_id",
                                    size = n_images)) # query allows us to specify parameters, which we find in the API documentation
  json_images_search_out <- content(images_search_out, "parsed")
  
  # replace NULL values with NA values
  json_images_search_out$data <- eval(parse(text = gsub("NULL", "NA", deparse(json_images_search_out$data))))
  
  image_ids <- do.call(rbind, lapply(json_images_search_out$data, as_tibble, stringsAsFactors = FALSE)) %>%
                dplyr::select('id', 'title', 'artist_display', 'image_id') 

  # we now check if our output directory exists. if not, we create it. if it does and we want to clear the directory, we do so. else, proceed.
  if (!dir.exists(paste0("./",output_dir))) {
    dir.create(paste0("./",output_dir))
  } else if(dir.exists(paste0("./",output_dir)) & clear_directory == TRUE) {
      unlink(paste0("./",output_dir), recursive = TRUE, force = TRUE)
      dir.create(paste0("./",output_dir))
  } else {}

    # now move to image API query
  iiif_url <- "https://www.artic.edu/iiif/2"
  
  # now work through the image ids, with api queries:
  for(i in 1:nrow(image_ids)){
    
    file <- paste0("./", output_dir, "/", image_ids$id[i], ".jpg")
    
    # try() here allows our request to fail without interrupting the run
    try(download.file(paste0(iiif_url, "/", image_ids$image_id[i], "/full/843,/0/default.jpg"), 
                  file, mode="wb"))
    
    # take a breath
    Sys.sleep(1)
    
  }
  
  # enumerate our successfully downloaded files
  downloads <- list.files(paste0("./", output_dir))
  
  # now, if we want to plot images, we save them to a list of ggplots
  if (plot_images == TRUE){
      
    images <- list()
    
      for(j in 1:length(downloads)){
     
      image_to_plot <- readJPEG(paste0("./", output_dir,"/", downloads[j]))
      
      id <- gsub(".jpg", "", downloads[j])
        
      artist <- image_ids$artist_display[image_ids$id==id]
        
      title <- image_ids$title[image_ids$id==id]

      title_for_image <- paste0(title, " by ", artist)
      
      images[[j]] <- ggplot() +
                      annotation_custom(rasterGrob(image_to_plot), xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=Inf) + 
                      ggtitle(str_wrap(title_for_image, 80)) +
                      theme_void() 
    
      } 
    
    # return the object
    return(images)
    
  } else {}
  
}

modern_art_images <- art_image_search("modern art", 10)

modern_art_images
``` 