# Author keywords and references

Keyword co-occurrence and citation-network analysis both need something the Search API alone does not return: a document’s author-supplied keywords, and its own reference list. This walks through retrieving both, what each costs, and what your ‘Scopus’ entitlement needs to cover.

The API calls below are shown exactly as you would run them with a configured key. No key is available when the site and the package are built, and Elsevier’s terms do not permit redistributing retrieved records in any case, so each call is paired with its output assembled offline from one representative example, the 2015 Nature review ‘Deep learning’. The corpus of real articles that the other articles run on carries neither author keywords nor reference lists, the two fields this one is about, so it cannot stand in here. The vignette therefore renders the same with or without a key, and `scopus_has_key()` is the switch you would use to run the calls for real.

    library(scopusflow)

## Author keywords from a search

The Search API’s `COMPLETE` view carries an `authkeywords` field alongside the usual title, DOI and date. Requesting it costs no extra request beyond `COMPLETE`’s own smaller page size (25 records per page, against 200 for `STANDARD`), which already means more requests, and so more quota, for the same number of records.

    recs <- scopus_fetch("DOI(10.1038/nature14539)", view = "COMPLETE")
    recs$authkeywords

    recs$authkeywords

    [1] "deep learning | neural networks | representation learning | backpropagation"

In development, this field came back `NA` even on a live, otherwise fully-entitled key, for documents that do carry author keywords in ‘Scopus’ itself. Those documents do have keywords, so what this points to is an entitlement gap specific to this one field. If your own keywords come back all `NA`, it is worth raising with your Scopus/Elsevier account contact.

## References via Abstract Retrieval

The reference list is not available from Search under any view. It needs Abstract Retrieval’s `FULL` or `REF` view, an entitlement separate from ordinary abstract access and from Search access. This is a per-document endpoint, so retrieving references for n documents costs n requests against Abstract Retrieval’s own, smaller weekly quota, separate from Search’s. Each row of the returned `references` data frame is one cited work.

    ab <- scopus_abstract(
      "10.1038/nature14539",
      view = "FULL", include = c("references", "keywords")
    )
    ab$references[[1]][, c("title", "authors", "source", "year")]

    ab$references[[1]][, c("title", "authors", "source", "year")]

title | authors | source | year
--- | --- | --- | ---
Gradient-based learning applied to document recognition | LeCun Y.; Bottou L.; Bengio Y.; Haffner P. | Proceedings of the IEEE | 1998
Human-level control through deep reinforcement learning | Mnih V.; Kavukcuoglu K.; Silver D. | Nature | 2015
ImageNet classification with deep convolutional neural networks | Krizhevsky A.; Sutskever I.; Hinton G. | Advances in Neural Information Processing Systems | 2012
Learning representations by back-propagating errors | Rumelhart D.; Hinton G.; Williams R. | Nature | 1986

`view = "FULL"` is the recommended default. In development, it returned a complete, correctly counted reference list for every document tried, while `view = "REF"` returned an inconsistent, sometimes-truncated subset, on an otherwise identical request made moments apart. `scopus_abstract()` warns when the number of references returned does not match the document’s own reported count, so a partial list never arrives unannounced.

The number of requests spent and the remaining Abstract Retrieval quota are attached as attributes, since this endpoint draws on a separate, smaller quota that is easy to exhaust unnoticed.

    attr(ab, "n_requests")        # requests spent so far

    [1] 1

    attr(ab, "quota")$remaining   # Abstract Retrieval quota left

    [1] 24999

A key or subscription tier that does not cover the requested view raises a `scopus_error_forbidden` condition that names the view, where a generic HTTP failure would leave you guessing. It stops the whole batch too, since entitlement is a property of the account, so the same refusal would only recur on every remaining identifier.

For more than a handful of identifiers, pass `cache_dir` so an interrupted or quota-limited batch resumes without re-spending quota already spent.

    ab <- scopus_abstract(
      dois, view = "FULL", include = c("references", "keywords"),
      cache_dir = "abstract-cache"
    )

## A minimal, cross-tool corpus

`scopus_corpus()` combines a search result with this Abstract Retrieval step, returning a minimal shape close to what OpenAlex’s `works` API already returns: `id`, `title`, `year`, `keywords` (a list-column of character vectors) and `references` (a list-column of data frames), where bibliometrix would give you semicolon-joined citation strings. It does not replace [`as_bibliometrix()`], which keeps its own field-mapping convention for users who want that.

    recs <- scopus_fetch("DOI(10.1038/nature14539)", max_results = 1)
    corpus <- scopus_corpus(recs, view = "FULL")
    corpus$keywords[[1]]
    nrow(corpus$references[[1]])

    corpus$keywords[[1]]

    [1] "deep learning"           "neural networks"
    [3] "representation learning" "backpropagation"

    nrow(corpus$references[[1]])

    [1] 4

The keywords list-column is ready for the co-occurrence analysis this article is named for, unnested and tallied into a per-keyword document count across the corpus.

    sort(table(unlist(corpus$keywords)), decreasing = TRUE)

            backpropagation           deep learning         neural networks
                          1                       1                       1
    representation learning
                          1

This costs one Abstract Retrieval request per record in `recs`, on top of whatever retrieved `recs` in the first place.