Updated / printed: 2023-11-01


Introduction

This script continues the data preparation methods needed for the Condition of Benthic Communities indicator (BH2b) in the UKMS Celtics Seas areas. Please see Data Preparation & Data Cleaning for previous data preparation steps.

This script focuses on;

  • Combining individual datasets into a single data frame
  • Checking and matching taxa names to WoRMS
    • Updating taxa names to standardised WoRMS list
  • Applying a taxonomic standardisation to the data
    • Removing inconsistent taxa
    • Truncating taxa names
    • standardising record type (presence/absence or count)
  • Exploratory analysis of potential outliers
  • Calculating sample Margalef

Data Preparation Part 3

  • Read in the data from the previous data cleaning script Script 2
## Load in the processed data from the previous script
# OneBenthic
OB_samples_count <- readRDS("Data/Processed/R_data/C20230831_OB_samples_count")
# Marine Recorder
MR_samples_count <- readRDS("Data/Processed/R_data/C20230906_MR_samples_count")
# EA WFD
WFD_samples_count <- readRDS("Data/Processed/R_data/C20230531_WFD_samples_count")
# MERMAN
merman_samples_count <- readRDS("Data/Processed/R_data/C20230831_merman_samples_count")

Taxonomic Standardisation

For the QSR2023, standards for species/taxa recordings in line with the provided data (participating countries of the Greater North Sea region) were agreed. Those standards were proposed as the lowest acceptable data standards for future data provision and assessments.

One can decide to opt for other standard levels of identification in other regions, for other monitoring techniques or with regards to assessments at sub-regional scale. But it has to be considered that other choices might lead to (slight) deviations in assessment results and have consequences with regards to comparability as the focus might be on other parts of the benthic diversity.

The highest level of taxonomic detail that can be reached in a standardised way is determined by the dataset(s) with the lowest level of taxonomic detail.

— OSPAR CEMP guidelines

The lowest level of taxonomic detail provided by participating countries for the QSR2023 may under represent the data available for the Celtic Seas. There is also the potential for taxa present in the Celtic Seas that might not be represented in the Greater North Sea region taxa list. For these reasons it’s not practical or desirable to join the Celtic Seas data with the Taxonomic standardisation for BH2b application region II v220222.xlsx list. However, the species list will be used to guide taxonomic standardisation for the Celtic Seas data alongside the BH2b CEMP guidelines:

  • Use the internationally standardised WoRMS taxa name list
  • Duplicates due to adults and juveniles will be aggregated by summation
  • Presence (P) records given a count value of ‘1’

Combine Datasets

First we need to reduce the fields to only those needed, standardise the field names, and bind the datasets together.

  • Take the common fields and give standard names
    • msfd will be the primary broad-scale habitat field (broad-scale habitat classifications based on the North Atlantic definitions)
    • eunis_l3 and hab_type will be retained for reference (composite habitat map / MR EUNIS codes)
  • Add an identifier of dataset source (dataset)
  • Bind into one benthos dataset
    • Note, NRW data is combine in the next script
  • Check some of the records
# select and standardise MR fields
MR_data <- MR_samples_count %>%
  select(samplekey, sampledate, sampleyear, geartype, preferredspeciesname, preferredaphiaid, count_n, au, msfd, eunis_l3_mr, eunis_l3, hab_type, geometry) %>%
  mutate(sampleref = samplekey,
         taxon = preferredspeciesname,
         aphia = preferredaphiaid,
         eunis_l3 = if_else(is.na(eunis_l3_mr), eunis_l3, eunis_l3_mr),
         dataset = "MR")

# select and standardise OB fields
OB_data <- OB_samples_count %>%
  select(samplecode, samplecode2, sampledate, year, gearname, validnameworms, aphiaid, count_n, au, msfd, eunis_l3, hab_type, geometry) %>%
  mutate(sampleref = paste(samplecode, samplecode2, sep = "_"),
         taxon = validnameworms,
         aphia = aphiaid,
         sampleyear = year,
         geartype = gearname,
         dataset = "OB")

# select and standardise WFD fields
WFD_data <- WFD_samples_count %>%
  select(samplecode, sampledate, sampleyear, samplemethod, preferredtaxonname, count_n, au, msfd, eunis_l3, hab_type, geometry) %>%
  mutate(sampleref = samplecode,
         taxon = preferredtaxonname,
         aphia = NA_real_,
         geartype = samplemethod,
         dataset = "WFD")

# select and standardise MERMAN fields
merman_data <- merman_samples_count %>%
  select(samplecode, samplestartdate, monitoringyear, samplertype, specieslatinname, result, au, msfd, eunis_l3, hab_type, geometry) %>%
  mutate(sampleref = samplecode,
         sampleyear = monitoringyear,
         sampledate = lubridate::dmy(samplestartdate),
         taxon = specieslatinname,
         aphia = as.integer(NA),
         count_n = result,
         geartype = samplertype) %>%
  mutate(dataset = "MERMAN")

# bring the data together and just keep the standardised names 
benthos <- bind_rows(MR_data, OB_data, WFD_data, merman_data) %>%
  select(sampleref, sampledate, sampleyear, geartype, taxon, aphia, count_n, msfd, eunis_l3, hab_type, au, dataset, geometry)

# qc check a sample 
slice_sample(benthos, n = 50)
# tidy up temp data
rm(MR_data, OB_data, WFD_data, merman_data)
  • 5083 samples

Aphia ID

Taxa records from the EA WFD dataset were matched to the NHM nomenclature of preferred names and currently do not have Aphia IDs (WoRMS) assigned. MERMAN records are also without Aphia IDs.

  • Find all taxa without Aphia ID
aphia_na <- benthos %>%
  as.data.frame() %>%
  filter(is.na(aphia)) %>%
  select(taxon) %>%
  unique() %>%
  arrange(taxon)
  • Match taxon name to Aphia ID using WoRMS
# Add aphia ID from WoRMS
aphia_id <- aphia_na %>%
  pull(taxon) %>%
  wormsbynames(ids = TRUE) %>%
  select(name, AphiaID)
## REQUESTING 1386 ITEMS BY NAME from World Register of Marine Species (www.marinespecies.org), 01/11/2023 14:21:15 (CC-BY)
##                                                     chunk 1/28 
##                                                     chunk 2/28 
##                                                     chunk 3/28 
## Aphelochaeta "species A"                             no match                                 
## Araphura brevimana                                   no match                                 
##                                                     chunk 4/28 
##                                                     chunk 5/28 
## Bugula purpurocincta                                 no match                                 
## Callopora dumerilli                                  no match                                 
##                                                     chunk 6/28 
## Caulleriella "species B"                             no match                                 
## Chaetozone "species D"                               no match                                 
## Chorizopora brongniarti                              no match                                 
##                                                     chunk 7/28 
## Cossura pygodactyla                                  no match                                 
## Crassicorophium bonnellii                            no match                                 
## Cumella pygmaea                                      no match                                 
##                                                     chunk 8/28 
## Diastyloides biplicata                               no match                                 
## DIPLOPODA                                            no match                                 
##                                                     chunk 9/28 
##                                                    chunk 10/28 
## Euclymene "species A"                                no match                                 
##                                                    chunk 11/28 
##                                                    chunk 12/28 
##                                                    chunk 13/28 
## Heteromysis microps                                  no match                                 
##                                                    chunk 14/28 
## Lemnaceae                                            no match                                 
##                                                    chunk 15/28 
##                                                    chunk 16/28 
## Mangelia brachystoma                                 no match                                 
## Metal splatter                                       no match                                 
##                                                    chunk 17/28 
## Nebalia herbsti                                      no match                                 
##                                                    chunk 18/28 
## Ocenebra erinacea                                    no match                                 
##                                                    chunk 19/28 
## Opisthodonta "species A"                             no match                                 
##                                                    chunk 20/28 
##                                                    chunk 21/28 
## PHOLADOMYOIDA                                        no match                                 
## Pholoe baltica (sensu Petersen)                      no match                                 
## Pholoe inornata (sensu Petersen)                     no match                                 
##                                                    chunk 22/28 
## Polinices catena                                     no match                                 
## Polinices montagui                                   no match                                 
## Polinices pulchellus                                 no match                                 
## Polyides rotundus                                    no match                                 
##                                                    chunk 23/28 
## Pseudocuma similis                                   no match                                 
##                                                    chunk 24/28 
## Rhopalomenia aglaophenae                             no match                                 
##                                                    chunk 25/28 
## Siphonoecetes striatus                               no match                                 
## Sphaerodoropsis balticum                             no match                                 
##                                                    chunk 26/28 
## Syllis "species A"                                   no match                                 
## Syllis "species D"                                   no match                                 
## Syllis "species H"                                   no match                                 
##                                                    chunk 27/28 
## Terebellomorpha                                      no match                                 
## Thyasira sarsi                                       no match                                 
##                                                    chunk 28/28 
## by names ........................................... DONE
  • Check those that didn’t match to an Aphia ID
aphia_id %>%
  filter(is.na(AphiaID))
  • Remove Metal splatter from dataset and taxa list
# remove from 'metal' the main dataset 
benthos <- benthos %>%
  filter(taxon != "Metal splatter")
# and aphia lists
aphia_id <- aphia_id %>%
  filter(name != "Metal splatter")
aphia_na <- aphia_na %>%
  filter(taxon != "Metal splatter")
  • Try to find some of the missing aphia codes from the main dataset
# remove "Species A" etc from names
aphia_na_temp <- aphia_id %>%
  filter(is.na(AphiaID)) %>%
  select(-c(AphiaID)) %>%
  mutate(alt_name = str_remove(name, " \"species [:upper:]\""))

# take a list of aphia from the dataset
benthos_aphia <- benthos %>%
    select(taxon, aphia) %>%
    filter(!is.na(aphia)) %>%
    unique()

# join to missing aphia list
aphia_na_temp <- left_join(aphia_na_temp, benthos_aphia, by = c("alt_name" = "taxon"))

# join to aphia taxa list
aphia_id <- left_join(aphia_id, aphia_na_temp, by = c("name" = "name"))
# update and remove fields
aphia_id <- aphia_id %>%
  mutate(AphiaID = ifelse(is.na(AphiaID), as.double(aphia), AphiaID)) %>%
  select(-c(alt_name, aphia))
  • Try a ‘fuzzy’ match with those still missing ID
# filter to those still missing ID
aphia_na_temp <- aphia_na_temp %>%
  filter(is.na(aphia)) %>%
  select(-c(alt_name, aphia))
# fuzzy match with 'worrms' package
aphia_id_fuzzy <- aphia_na_temp %>%
  pull(name) %>%
  wm_records_taxamatch(marine_only = FALSE)

# extract the matched names, aphia, and row number
extracted_data <- lapply(seq_along(aphia_id_fuzzy), function(i) {
  tbl <- aphia_id_fuzzy[[i]]
  if (nrow(tbl) > 0) {
    tbl %>%
      mutate(row_number = i,
             valid_AphiaID = first(valid_AphiaID),
             valid_name = first(valid_name)) %>%
      select(row_number, valid_AphiaID, valid_name)
  } else {
    tbl
  }
})

# Combine the extracted data into a single dataframe
result_df <- bind_rows(extracted_data)
# remove duplicates
result_df <- distinct(result_df)

# Print the matches
print(result_df)  
## # A tibble: 25 x 3
##    row_number valid_AphiaID valid_name                       
##         <int>         <int> <chr>                            
##  1          1        136344 Araphura brevimanus              
##  2          2        834041 Crisularia purpurotincta         
##  3          3        111196 Callopora dumerilii              
##  4          4        111304 Chorizopora brongniartii         
##  5          5        129985 Cossura pygodactylata            
##  6          6        237004 Crassicorophium bonellii         
##  7          7        110567 Cumella (Cumella) pygmaea        
##  8          8        110494 Diastyloides biplicatus          
##  9          9          1305 Diplopoda                        
## 10         10        120037 Heteromysis (Heteromysis) microps
## # ... with 15 more rows
# tidy up
rm(extracted_data)
  • Add successful matches to aphia_id list
# add row ID to join tables
aphia_na_temp <- aphia_na_temp %>%
  dplyr::mutate(row_id = row_number())
# join results to missing aphia list
aphia_na_temp_results <- left_join(aphia_na_temp, result_df, by = c("row_id" = "row_number"))

# check what's still missing an aphia
aphia_na_temp_results %>%
  filter(is.na(valid_AphiaID))
  • try matching these with a different function
# fuzzy match with a different call function
aphia_id_fuzzy_2 <- aphia_na_temp_results %>%
  filter(is.na(valid_AphiaID)) %>%
  pull(name) %>%
  wm_records_names(fuzzy = TRUE, marine_only = FALSE)

# check the matches
aphia_id_fuzzy_2 %>% bind_rows()

Only a match for PHOLADOMYOIDA

  • Add this Aphia ID for PHOLADOMYOIDA
  • and one for Terebellomorpha (152292 - Terebelliformia) manually
# update remaining missing aphia ids¥
aphia_na_temp_results <- aphia_na_temp_results %>%
  mutate(valid_AphiaID = case_when(name == "PHOLADOMYOIDA" ~ as.double(254),
                                   name == "Terebellomorpha" ~ as.double(152292),
                                   TRUE ~ as.double(valid_AphiaID)))
  • Join matched Aphia IDs to the initial list of missing IDs
# join IDs
aphia_id_matched <- left_join(aphia_id, select(aphia_na_temp_results, name, valid_AphiaID), by = c("name" = "name")) %>%
  mutate(AphiaID = ifelse(is.na(AphiaID), valid_AphiaID, AphiaID)) %>%
  select(-c(valid_AphiaID))

# tidy up (following qc)
rm(aphia_id, aphia_id_fuzzy, aphia_id_fuzzy_2, aphia_na, aphia_na_temp, aphia_na_temp_results, result_df)
  • Add these names and Aphia IDs to the list of taxa names that already had Aphia ID numbers
# add missing id rows to main taxa list
benthos_aphia_all <- aphia_id_matched %>%
  dplyr::rename(taxon = name, aphia = AphiaID) %>%
  bind_rows(benthos_aphia)
# remove any duplicates
benthos_aphia_all <- distinct(benthos_aphia_all)
  • Check for duplicated names with different Aphia IDs
# check for duplicate names
benthos_aphia_all %>%
  group_by(taxon) %>%
  tally() %>%
  filter(n>1)
# check the Aphia IDs
benthos_aphia_all %>%
  filter(taxon == "Mangelia costulata")

Aphia ID 154073 is the incorrect synonym for Mangelia costulata (Smithiella costulata)

  • remove 154073 and keep 1644044
  • update Mangelia costulata aphia in the dataset
# filter out 154073 from the taxa list
benthos_aphia_all <- benthos_aphia_all %>%
  filter(aphia != 154073)

# change to the correct ID in the data
benthos <- benthos %>%
  mutate(aphia = if_else(aphia == 154073, 1644044, aphia))

We now have 2984 unique taxa names with a corresponding Aphia ID. However, many will be duplicated names (e.g. synonyms or capitalisation of higher names)

  • Show duplicates with ‘different’ taxa names and the same Aphia ID
# find duplicates
aphia_qc <- benthos_aphia_all %>%
  group_by(aphia) %>%
  tally() %>%
  filter(n>1) %>%
  pull(aphia)
# filter duplicates
benthos_aphia_all %>%
  filter(aphia %in% aphia_qc) %>%
  arrange(aphia, taxon)
# tidy up
rm(aphia_qc)

These have the ‘correct’ Aphia ID but slight variation on the taxa name spelling, we’ll keep these duplicate IDs but assign the valid name and ID through the WoRMS match tool in the next stage.

  • Add Aphia IDs to the records that are missing aphia (WFD and MERMAN records), this is needed when we join taxonomic details later in the script
# join the matched aphia records to the dataset
benthos <- left_join(benthos, benthos_aphia_all, by = c("taxon" = "taxon"), suffix = c("", ".y"),)

# update missing aphia and remove joined field
benthos <- benthos %>%
  mutate(aphia = if_else(is.na(aphia), aphia.y, aphia)) %>%
  select(-c(aphia.y))
  • Check no records are missing aphia
benthos %>%
   as.data.frame() %>%
   filter(is.na(aphia))
## Warning in `[<-.data.frame`(`*tmp*`, is_list, value = list(`13` =
## "<s_GEOMET>")): replacement element 1 has 1 row to replace 0 rows

WoRMS Taxonomy Match

  • Use the Aphia ID (aphia) to match taxa through WoRMS
    • Just take the unique IDs to save time
  • Join results to the list of names and IDs from the dataset
# take the aphia field and call match through WoRMS
wormsid <- benthos_aphia_all %>%
  distinct(aphia) %>%
  pull(aphia) %>%
  wormsbyid()
# join the WoRMS call results to the taxa list
matchednames <- left_join(benthos_aphia_all, wormsid, by = c("aphia" = "AphiaID"), suffix=c("",".y"))
  • Check the names have matched by ID where there are differences between the taxa name in the dataset and that returned by WoRMS from the ID
matchednames %>%
  select(taxon, aphia, valid_name, valid_AphiaID) %>%
  filter(taxon != valid_name)

As expected records with different spellings, synonyms, sub-genus names etc.

  • Check name status
matchednames %>%
  group_by(status) %>%
  tally() %>%
  ungroup()
  • Check any taxa that are not “accepted” by WoRMS
matchednames %>%
  filter(status != "accepted" | is.na(status)) %>%
  select(taxon, class, aphia, status, unacceptreason, valid_name, valid_AphiaID) %>%
  arrange(status, class, valid_name)  %>%
  datatable(
        caption = "Unaccepted taxa names",
        escape = FALSE,
        rownames = FALSE,
        extensions = 'Buttons',
        options = list(pageLength = 10, 
                       buttons = c("csv"),
                       dom = 'Brtip'),
        class = "cell-border")

Those with valid alternative names we can change, those that are not accepted but have no alternative we will keep, those that are quarantined with no alternative we will remove from the dataset.

  • First we’ll remove the quarantined name
    • the quarantined ID 505605 had data witheld as taxa name
# find the invalid
invalid_name <- matchednames %>%
  filter(status == "quarantined" & is.na(valid_name)) %>%
  pull(taxon)

# remove from taxa list 
matchednames <- matchednames %>%
  filter(taxon != invalid_name)
# remove from dataset
benthos <- benthos %>%
  filter(taxon != invalid_name)
# tidy up
rm(invalid_name)
  • Take subsets of the taxa list that does and doesn’t have valid Aphia IDs
    • those with aphia changes will need updating
matchednames_valid_id <- matchednames %>%
  filter(aphia == valid_AphiaID)

matchednames_id_update <- matchednames %>%
  filter(aphia != valid_AphiaID | is.na(valid_AphiaID))
  • First find all those that don’t have a valid name / Aphia ID
matchednames_id_update %>%
  filter(is.na(valid_name) | is.na(valid_AphiaID)) 

Following BH2b standardisation used for the Greater North Sea (BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx), Magelona and Platyhelminthes are to be aggregated to genus and phylum respectively.

  • The paraphyletic suborder Gammaridea can be raised to order (Amphipoda),

  • Golfingiida can be reasonably be transfered to Golfingiidae,

  • and the no longer used superfamily name Aphroditoidea can be raised to suborder (Aphroditiformia).

  • update the valid ID for these taxa (for these it’s just the parent ID)

matchednames_id_update <- matchednames_id_update %>%
  mutate(valid_AphiaID = case_when(taxon == "Magelona minuta" ~ parentNameUsageID,
                                   taxon == "TURBELLARIA" ~ parentNameUsageID,
                                   taxon == "Turbellaria" ~ parentNameUsageID,
                                   taxon == "Golfingiida" ~ as.integer(2032),
                                   taxon == "Gammaridea" ~ parentNameUsageID,
                                   taxon == "Aphroditoidea" ~ parentNameUsageID,
                                   TRUE ~ valid_AphiaID))
  • Re-match through WoRMS with the valid IDs
# take the valid IDs from the invalid names
worms_valid_id <- matchednames_id_update %>%
  pull(valid_AphiaID) %>%
  wormsbyid()
# take the original invalid names, IDs and Valid IDs
matchednames_invalid_tmp <- matchednames_id_update %>%
  select(taxon, aphia, valid_AphiaID)
# join with the valid names from WoRMS call of valid IDs
matchednames_invalid_fixed <- left_join(matchednames_invalid_tmp, worms_valid_id, by = c("valid_AphiaID" = "AphiaID"), suffix=c("",".y")) %>%
  select(-c(valid_AphiaID.y))
# remove duplicates
matchednames_invalid_fixed <- distinct(matchednames_invalid_fixed)

# tidy up
rm(matchednames_invalid_tmp, worms_valid_id)
  • Bring together the valid names and now fixed alternative names
    • This will make a ‘look up’ table to be used later on the dataset
  • Take only the taxonomic information/fields needed
# bind the valid names and the fixed unaccepted names
taxonomy_accepted <- bind_rows(matchednames_valid_id, matchednames_invalid_fixed)

# check those that have alternative names
taxonomy_accepted %>%
  filter(taxon != scientificname) %>%
  select(taxon, scientificname, aphia, valid_AphiaID, rank) %>%
  arrange(taxon) %>%
  datatable(
        caption = "Taxa name changes",
        colnames = c("original aphia" = "aphia"),
        escape = FALSE,
        rownames = FALSE,
        extensions = 'Buttons',
        options = list(pageLength = 10, 
                       buttons = c("csv"),
                       dom = 'Brtip'),
        class = "cell-border")
# take the fields needed
taxonomy_accepted <- taxonomy_accepted %>%
  select(taxon, aphia, scientificname, rank, valid_AphiaID, kingdom, phylum, class, order, family, genus)

# tidy up the temporary tables (following qc)
rm(matchednames, matchednames_valid_id, matchednames_invalid_fixed, matchednames_id_update, wormsid, benthos_aphia_all, benthos_aphia, aphia_id_matched)

We’ll now investigate the taxa list (taxonomy_accepted) and start to define the truncation to be applied to the dataset.

  • Check the major groups present (and those the can be easily removed)
taxonomy_accepted %>%
  group_by(kingdom, phylum, class) %>%
  tally() %>%
  arrange(kingdom, phylum, class)

Taxonomic Reduction

A summary of the taxa standardisation used for the QSR BH2b Greater North Sea assessment are given in the BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx document.

The reason to remove some major groups for the BH2b QSR2023 assessment in the Greater North Sea is likely because they were not consistently recorded by other CPs. For the assessment of BH2b in the Celtic Seas, groups that are commonly recorded in UK waters have been retained (e.g. epifauna taxa).

Some major groups are not recorded consistently in grab/core samples and can be easily marked for removal from the dataset, these were;

  • Bacteria
  • Algae
  • Dinoflagellates
  • Foraminifera
  • Plants
  • Protozoans
    • All the above are encompassed in the Bacteria, Chromista, Plantae, and Protozoa kingdoms
  • Assign as not to be retained with a new retain logical field
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(grepl("Bacteria|Chromista|Plantae|Protozoa", kingdom), FALSE, TRUE))
  • Remove records only recorded at kingdom level (“Animalia”) and with the taxa name “Cnidaria”
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(is.na(phylum), FALSE, retain),
         retain = if_else(phylum == "Cnidaria" & is.na(class), FALSE, retain))
  • Remove Chordata recorded at phylum or higher (taxa names Pisces, Tunicata, Actinopterygii, and Chordata)
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(phylum == "Chordata"& is.na(class), FALSE, retain))
  • Remove phyla that are not to be kept
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(grepl("Chaetognatha|Ctenophora|Entoprocta|Nematoda", phylum), FALSE, retain))
  • Remove classes that are not to be kept
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(grepl("Arachnida|Copepoda|Diplopoda|Hexapoda|Scyphozoa|Staurozoa|Cephalopoda|Teleostei", class), FALSE, retain))
  • Remove orders that are not to be kept
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(retain = if_else(grepl("Arhynchobdellida|Rhynchobdellida|Euphausiacea|Ingolfiellida|Mysida", order), FALSE, retain))

Summary of major groups to be removed:

taxonomy_accepted %>%
  filter(retain == FALSE) %>%
  group_by(phylum, class, order) %>%
  tally(name = "taxa") %>%
  arrange(phylum, class, order)

Taxonomic Truncation

Changes or aggregations can be summarised into a few different categories

  • Aggregation to a higher taxonomic level (e.g. species to genus)
  • Aggregation to an alternative genus level (e.g. Monocorophium to Corophium )
  • Aggregation to a genus level or species level complex (e.g. species with problematic or uncertain status)

A summary of the standardisation and truncation applied to the BH2b assessment in the Celtic Seas at different taxonomic levels is summarised below (Taxa truncation changes to be applied).

First add a field for any higher aggregation level (agglevel)

taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = as.character(NA))
  • Add those that are to be aggregated to phylum
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(grepl("Nemertea|Platyhelminthes", phylum) & !is.na(phylum), "phylum", agglevel))
  • Add those that are to be aggregated to class
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(grepl("Clitellata", class) & !is.na(class), "class", agglevel))
  • Add those that are to be aggregated to order
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(grepl("Cephalaspidea|Nudibranchia", order) & !is.na(order), "order", agglevel))
  • Add those that are to be aggregated to family
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(grepl("Edwardsiidae|Cerianthidae", family) & !is.na(family), "family", agglevel))
  • Add those that are to be aggregated to genus
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(grepl("Magelona|Owenia|Ophryotrocha|Arenicola|Capitella|Dasybranchus|Notomastus|Microphthalmus|Harmothoe|Pholoe|Syllides|Exogone|Polydora|Streblospio|Aphelochaeta|Tharyx|Polycirrus|Streblosoma|Terebellides|Corophium|Gammarus|Pseudocuma|Gnathia|Callipallene", genus) & !is.na(genus), "genus", agglevel))
# Spio separately, some others include 'spio' in their name
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(agglevel = if_else(genus == "Spio", "genus", agglevel))

For those that will be recorded with an alternative name we’ll add an alternative name field (altname)

taxonomy_accepted <- taxonomy_accepted %>%
  mutate(altname = as.character(NA))
  • Add an alternative name for genus level changes
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(altname = if_else(grepl("Pseudopolydora", genus), "Polydora", altname)) %>%
  mutate(altname = if_else(grepl("Monocorophium", genus), "Corophium", altname)) %>%
  mutate(altname = if_else(grepl("Heteromastus|Mediomastus", genus), "Heteromastus-Mediomastus", altname))
  • Add an alternative name for species level changes
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(altname = if_else(grepl("Glycera alba|Glycera tridactyla", scientificname), "Glycera alba-tridactyla", altname)) %>%
  mutate(altname = if_else(grepl("Phyllodoce maculata|Phyllodoce mucosa", scientificname), "Phyllodoce maculata-mucosa", altname)) %>%
  mutate(altname = if_else(grepl("Eteone flava|Eteone longa", scientificname), "Eteone flava-longa", altname))

Summary of the 407 taxon changes:

  • Note some taxa might be listed to aggregate but are already at the appropriate level, or are already marked to remove
taxonomy_accepted %>%
  filter(!is.na(altname) | !is.na(agglevel) | retain == FALSE) %>%
  select(taxon, retain, agglevel, altname, phylum, class, order, family) %>%
  arrange(phylum, class, order, family, taxon) %>%
  datatable(
        caption = "Taxa truncation changes to be applied",
        escape = FALSE,
        rownames = FALSE,
        extensions = 'Buttons',
        options = list(pageLength = 10, 
                       buttons = c("csv"),
                       dom = 'Brtip'),
        class = "cell-border")

Recording Policy

To enable standardisation of BH2b for the QSR2023 assessment in Region II all taxa that were potentially recorded with presence/absence were removed (BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx). Whilst there is the potential for some inconsistent recording policies with some of these major groups (e.g. algae) in the Celtic Seas, we will try to retain epifaunal groups (e.g. bryozoans) as they are significant part of benthic communities in mixed and coarse sediments.

Three taxa were marked to be recorded as ‘present’ for the Region II QSR assessment (Hirudinea, Nemertea, and Malacostraca). For the BH2b Celtic Seas assessment we will maintain the policy for nemerteans, these are often recorded as ‘fragments/present’. Records of ‘Malacostraca’ and leeches can be left as density records, there no reason to believe they would be recorded differently in grab or core samples in UK waters.

Major groups to be recorded as ‘present’ for the current assessment were Nemertea, Porifera, Phoronida, and Bryozoa.

Some major groups contain both solitary and colonial taxa, or policies might vary between labs, in these instances the precautionary assignment of ‘Presence’ will be applied (e.g. barnacles).

  • Add a field for recording policy (presence)
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(presence = as.logical(FALSE))
  • Assign presence for phylum level groups
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(presence = if_else(grepl("Nemertea|Porifera|Phoronida|Bryozoa", phylum) & !is.na(phylum), TRUE, as.logical(presence)))
  • Assign presence for class level groups
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(presence = if_else(class == "Thecostraca" & !is.na(class), TRUE, as.logical(presence)))
  • ‘Colonial’ cnidaria (e.g. Alcyonium digitatum)
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(presence = if_else(grepl("Malacalcyonacea|Octocorallia incertae sedis|Zoantharia", order) & !is.na(order), TRUE, as.logical(presence)),
         presence = if_else(taxon == "Anthozoa", TRUE, as.logical(presence)),
         presence = if_else(taxon == "Octocorallia", TRUE, as.logical(presence)),
         presence = if_else(taxon == "Hexacorallia", TRUE, as.logical(presence)))
  • Ascidiacea recorded at class level (potentially a mix of presence and count)
taxonomy_accepted <- taxonomy_accepted %>%
  mutate(presence = if_else(taxon == "Ascidiacea", TRUE, as.logical(presence)))

Summary of major groups to be represented as ‘presence’ records:

taxonomy_accepted %>%
  filter(presence == TRUE) %>%
  group_by(phylum, class, order, family) %>%
  tally() %>%
  arrange(phylum, class, order, family) %>%
  datatable(
        caption = "Major groups to record as presence only",
        escape = FALSE,
        colnames = c('number of taxa' = 'n'),
        rownames = FALSE,
        extensions = 'Buttons',
        options = list(pageLength = 10, 
                       buttons = c("csv"),
                       dom = 'Brtip'),
        class = "cell-border")

Apply Taxonomic Standardisation

At this stage we have a list of all the taxa in the dataset and the type of standarisation to be applied to each taxa in the dataset (taxonomy_accepted ‘look up’ table). We will now apply these policies to the dataset before interrogating further for amalgamations based on frequencies of occurrence records at a less detailed taxonomic level (e.g. juvenile or incomplete specimens that could not be identified to a more detailed level).

First we will apply the truncation and reduction policies defined in taxonomy_accepted.

  • Join benthos data to the ‘look up’ table
  • Update the taxon and aphia fields to the accepted names and remove the WoRMS equivalents
# join benthos data to the look up table
benthos_t <- left_join(benthos, taxonomy_accepted, by = c("taxon" = "taxon", "aphia" = "aphia"), suffix=c("",".y"))
# update the taxon and aphia fields (we could keep the WoRMS fields but the originals are quicker to type!)
benthos_t <- benthos_t %>%
  mutate(taxon = scientificname,
         aphia = valid_AphiaID) %>%
  select(-c(scientificname, valid_AphiaID))
  • Remove inconsistent taxa records (retain == FALSE, 4470 records)
benthos_t <- benthos_t %>%
  filter(retain == TRUE)
  • Update taxa records with alternative names (altname, 3943 records)
    • Change aphia of alternative names to 0 to avoid misleading IDs
benthos_t <- benthos_t %>%
  mutate(taxon = if_else(!is.na(altname), altname, taxon),
         rank = if_else(!is.na(altname), "Complex", rank),
         aphia = if_else(!is.na(altname), 0, as.numeric(aphia)),
         genus = if_else(grepl("Pseudopolydora", genus) & !is.na(genus), "Polydora", genus),
         genus = if_else(grepl("Monocorophium", genus) & !is.na(genus), "Corophium", genus),
         genus = if_else(grepl("Heteromastus|Mediomastus", genus) & !is.na(genus), "Heteromastus-Mediomastus", genus))
  • Update taxa names to higher levels where specified (agglevel, 17940 records)
    • Change aphia of aggregated names to 0 to avoid misleading IDs
    • We’ll also update rank, although we my not need it later
benthos_t <- benthos_t %>%
  mutate(taxon = case_when(agglevel == "phylum" ~ phylum,
                           agglevel == "class" ~ class,
                           agglevel == "order" ~ order,
                           agglevel == "family" ~ family,
                           agglevel == "genus" ~ genus,
                           TRUE ~ taxon),
         rank = case_when(agglevel == "phylum" ~ "Phylum",
                           agglevel == "class" ~ "Class",
                           agglevel == "order" ~ "Order",
                           agglevel == "family" ~ "Family",
                           agglevel == "genus" ~ "Genus",
                           TRUE ~ rank),
         class = if_else(agglevel == "phylum" & !is.na(agglevel), as.character(NA), class),
         order = if_else(grepl("phylum|class", agglevel) & !is.na(agglevel), as.character(NA), order),
         family = if_else(grepl("phylum|class|order", agglevel) & !is.na(agglevel), as.character(NA), family),
         genus = if_else(grepl("phylum|class|order|family", agglevel) & !is.na(agglevel), as.character(NA), genus),
         aphia = if_else(!is.na(agglevel), 0, as.numeric(aphia)))

Some taxa that have been aggregated will be missing aphia IDs (aphia == 0) whilst other original records of the names will have correct aphia IDs. This causes some problems later when grouping data. To avoid this we’ll add in missing aphia IDs where we can.

  • Take a list of any taxa with the aphia placeholder of zero
  • Get the correct AphiaID from WoRMS
  • Join the correct IDs to the dataset and update where possible
# get those without ID
aphia_updates <- benthos_t %>%
  as.data.frame() %>%
  filter(aphia == 0) %>%
  group_by(taxon) %>%
  tally()
# match through WoRMS
aphia_updates_matched <- aphia_updates %>%
  pull(taxon) %>%
  wormsbynames(ids = TRUE) %>%
  select(name, AphiaID)
## REQUESTING 35 ITEMS BY NAME from World Register of Marine Species (www.marinespecies.org), 01/11/2023 14:23:55 (CC-BY)
##                                                      chunk 1/1 
## Eteone flava-longa                                   no match                                 
## Glycera alba-tridactyla                              no match                                 
## Heteromastus-Mediomastus                             no match                                 
## Phyllodoce maculata-mucosa                           no match                                 
## by names ........................................... DONE
# join WoRMS IDs to dataset
benthos_t <- left_join(benthos_t, aphia_updates_matched, by = c("taxon" = "name"))
# update changes and drop new field
benthos_t <- benthos_t %>%
  mutate(aphia = if_else(aphia == 0 & !is.na(AphiaID), as.numeric(AphiaID), aphia)) %>%
  select(-c(AphiaID))
# tidy up
rm(aphia_updates, aphia_updates_matched)

Aggregate Sample Abundance Records

Following aggregations and standardisation of taxa names there are now multiple records of the same taxa in a sample (e.g. species records now aggregated to the same genus).

  • Find the duplicate records
  • Group the data to sum counts of duplicate taxa in the same sample
  • Update the counts to the sum values where needed
  • remove the duplicate entries
# find duplicate taxa records
dups <- benthos_t %>%
  as.data.frame() %>%
  select(-c(count_n, geometry)) %>%
  duplicated()

# extract the fields to filter the data
dup_list <- benthos_t %>%
  as.data.frame() %>%
  filter(dups) %>%
  select(sampleref, sampledate, taxon)

# qc the duplicates
benthos_t %>%
  as.data.frame() %>%
  right_join(dup_list) %>%
  select(sampleref, taxon, count_n) %>%
  slice_sample(n = 100)
# sum duplicate records
qc_temp <- benthos_t %>%
  as.data.frame() %>%
  right_join(dup_list) %>%
  group_by(sampleref, sampledate, taxon) %>%
  dplyr::summarise(count_sum = sum(count_n))

# add the sum values and remove duplicates
benthos_t_agg <- benthos_t %>%
  left_join(qc_temp) %>%
  filter(!dups) %>%
  mutate(count_n = if_else(is.na(count_sum), count_n, count_sum)) %>%
  select(-c(count_sum))

# tidy up
rm(dups, dup_list, qc_temp)

2136 records aggregated

Investigate taxa following initial standarisation

High taxon abundance

Investigate the abundance records for any unusually high values following aggregation.

  • Check high abundance values
    • Note, some taxa will be assigned ‘Presence’ in later stages (e.g. Verruca stroemia)
benthos_t_agg %>%
  as.data.frame() %>%
  select(sampleref, taxon, count_n, sampleyear, msfd, rank, dataset, geometry) %>%
  mutate(coords = as.character(geometry)) %>%
  mutate(sample = str_sub(sampleref, 0, 15)) %>%
  select(-c(sampleref)) %>%
  filter(count_n >500) %>%
  arrange(desc(count_n))
  • Check high abundances of taxa that aren’t recorded at species or genus
benthos_t_agg %>%
  as.data.frame() %>%
  select(sampleref, taxon, count_n, sampleyear, msfd, rank) %>%
  mutate(sample = str_sub(sampleref, 0, 15)) %>%
  select(-c(sampleref)) %>%
  filter(rank != "Species", rank != "Genus", rank != "Complex", count_n >100) %>%
  arrange(desc(count_n))
  • Check high abundances of bivalves that might have juveniles occurring in high abundances
benthos_t_agg %>%
  as.data.frame() %>%
  select(sampleref, family, taxon, count_n, sampleyear, msfd, family) %>%
  mutate(sample = str_sub(sampleref, 0, 15)) %>%
  select(-c(sampleref)) %>%
  filter(family == "Cardiidae"| family == "Mytilidae" | family == "Veneridae", count_n >100) %>%
  arrange(desc(count_n))

High abundance taxa look relatively normal, there aren’t any extreme abundances from juvenile bivalves (e.g. thousands of M. edule spat) although there are some high records of what are likely juvenile Acanthocardia echinata and Mytilus edulis. There are a couple of high records of ‘Ascidiacea’ recorded at class, these will be assigned ‘Presence’ in later stages.

Rare taxa names

  • Check records at major groups that are not commonly recorded (<5 uses of the name in the dataset)
benthos_t_agg %>%
  as.data.frame() %>%
  select(sampleref, phylum, class, order, family, taxon, rank, count_n) %>%
  filter(rank != "Species", rank != "Genus", rank != "Complex", rank != "Subspecies") %>%
  group_by(phylum, class, order, rank, taxon) %>%
  dplyr::summarise(sample_count = n_distinct(sampleref)) %>%
  filter(sample_count <5) %>%
  arrange(phylum, class, order)

Apply additional standardisation

Following exploration of the dataset we can see some taxa that can be updated, for example there are records of Magelonidae that can be assigned to Magelona (Magelonidae is a monophyletic group). However, most aggregations (e.g. the classes Calcarea and Demospongiae to phylum Porifera) would make no difference to the Margalef diversity score. Many don’t occur with the taxon they would be aggregated with, therefore the number of ‘species’ in a sample would not change. If the aim was to calculate community similarity then there would be reasons to standardise the names further. For the purpose of BH2b (Margalef diversity) we can apply the main aggregations identified. However, there is little benefit standardising names completely at this stage.

  • Make the changes to those identified
    • Ampharetinae to Ampharetidae
    • Terebellinae to Terebellidae
    • Terebelliformia to Terebellida
    • Magelonidae to Magelona
benthos_t_agg <- benthos_t_agg %>%
  mutate(taxon = if_else(taxon == "Ampharetinae", "Ampharetidae", taxon),
         rank = if_else(aphia == 152252, "Family", rank),
         aphia = if_else(aphia == 152252, 981, as.numeric(aphia))) %>%
  mutate(taxon = if_else(taxon == "Terebellinae", "Terebellidae", taxon),
         rank = if_else(aphia == 322588, "Family", rank),
         aphia = if_else(aphia == 322588, 982, as.numeric(aphia))) %>%
  mutate(taxon = if_else(taxon == "Terebelliformia", "Terebellida", taxon),
         rank = if_else(aphia == 152292, "Order", rank),
         aphia = if_else(aphia == 152292, 900, as.numeric(aphia))) %>%
  mutate(taxon = if_else(taxon == "Magelonidae", "Magelona", taxon),
         rank = if_else(aphia == 914, "Genus", rank),
         aphia = if_else(aphia == 914, 129341, as.numeric(aphia)))
  • Aggregate abundance records again across the dataset (sum abundances of the taxa in each sample)
# pull a list of column names except for the count and geometry
fields <- colnames(benthos_t_agg)[colnames(benthos_t_agg) != 'count_n' & colnames(benthos_t_agg) != 'geometry']
# aggregate abundance records for duplicate names in a sample
benthos_t_agg_tmp <- benthos_t_agg %>%
  group_by(across(all_of(fields))) %>%
  dplyr::summarise(count_n = sum(count_n)) %>%
  ungroup()
# note geometry needs rejoining

# Take a data frame of the coordinates per sample
coordinates <- benthos_t_agg %>%
  select(sampleref, geometry) %>%
  distinct()
# Join to the aggregated sample data to the geometry
benthos_t_agg <- left_join(benthos_t_agg_tmp, coordinates, by = c("sampleref" = "sampleref"))

# tidy up 
rm(benthos_t_agg_tmp, fields, coordinates)

Apply Abundance Recording Policy

Following all taxonomic aggregations we can now ensure any taxa to be represented as “Present” are converted to nominal abundance of 1

benthos_t_agg <- benthos_t_agg %>%
  mutate(count_n = if_else(presence == TRUE, 1, count_n))
  • Following standardisation of the dataset there are 174412 records in 5077 samples (6657 records removed or aggregated)

  • Save a final summary of the taxa from the dataset
  • Note, the taxonomic truncation for NRW data was undertaken separately (OFFICIAL SENSITIVE), but followed the steps outlined above.
benthos_taxonomy <- benthos_t_agg %>%
  as.data.frame() %>%
  select(taxon, aphia, rank, kingdom, phylum, class, order, family, genus, agglevel, altname, presence) %>%
  group_by(taxon, aphia, rank, kingdom, phylum, class, order, family, genus, agglevel, altname, presence) %>%
  dplyr::summarise(records = n()) %>%
  arrange(kingdom, phylum, class, order, family, genus, taxon)

write.csv(benthos_taxonomy, paste0("Data/Processed/Taxonomy/", creation_date(), "_benthos_taxonomy.csv"))
  • Save the final dataset for reference
# csv copy
#write.csv(benthos_t_agg, paste0("Data/Processed/Benthos/", creation_date(), "_benthos_t_agg.csv"))
# R copy if needed again
write_rds(benthos_t_agg, paste0("Data/Processed/R_data/", creation_date(), "_benthos_t_agg"))

Summarise Sample Data and Margalef

Following truncation and standardisation community data can be summarised to the key diversity metrics. The Margalef diversity score is calculated using the benthos package.

samples <- benthos_t_agg %>%
  group_by(sampleref, sampledate, sampleyear, msfd, eunis_l3, hab_type) %>%
  dplyr::summarise(D = margalef(., taxon = taxon, count = count_n), S = n_distinct(taxon), N = sum(count_n), .groups = "drop")

Save Processed Data

Save the sample data for analysis in the next script (Data Analysis)

# save processed data
write_rds(samples, paste0("Data/Processed/R_data/", creation_date(), "_samples"))

Session Info

details::details(sessionInfo(), summary = 'sessionInfo')
sessionInfo

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server x64 (build 14393)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] RColorBrewer_1.1-2 benthos_1.3-6      details_0.3.0      DT_0.26           
 [5] sf_0.9-8           forcats_0.5.1      stringr_1.4.0      dplyr_1.0.7       
 [9] purrr_0.3.4        readr_2.1.0        tidyr_1.1.4        tibble_3.1.6      
[13] ggplot2_3.3.5      tidyverse_1.3.1    rprojroot_2.0.3    worrms_0.4.2      
[17] worms_0.2.2        plyr_1.8.6         httr_1.4.2        

loaded via a namespace (and not attached):
 [1] sass_0.4.4         jsonlite_1.7.2     modelr_0.1.8       bslib_0.4.1       
 [5] assertthat_0.2.1   triebeard_0.3.0    urltools_1.7.3     cellranger_1.1.0  
 [9] yaml_2.2.1         pillar_1.6.4       backports_1.3.0    glue_1.5.0        
[13] digest_0.6.28      rvest_1.0.2        colorspace_2.0-2   htmltools_0.5.2   
[17] clipr_0.7.1        pkgconfig_2.0.3    httpcode_0.3.0     broom_0.7.10      
[21] haven_2.4.3        scales_1.1.1       tzdb_0.2.0         proxy_0.4-27      
[25] generics_0.1.1     ellipsis_0.3.2     cachem_1.0.6       withr_2.5.0       
[29] lazyeval_0.2.2     cli_3.1.0          magrittr_2.0.1     crayon_1.4.2      
[33] readxl_1.3.1       evaluate_0.18      fs_1.5.0           fansi_0.5.0       
[37] xml2_1.3.2         class_7.3-15       tools_3.6.1        hms_1.1.1         
[41] lifecycle_1.0.1    munsell_0.5.0      reprex_2.0.1       compiler_3.6.1    
[45] jquerylib_0.1.4    e1071_1.7-11       rlang_0.4.12       classInt_0.4-3    
[49] units_0.8-0        grid_3.6.1         rstudioapi_0.13    htmlwidgets_1.5.4 
[53] crosstalk_1.2.0    rmarkdown_2.18     gtable_0.3.0       curl_4.3.2        
[57] DBI_1.1.3          R6_2.5.1           lubridate_1.8.0    knitr_1.41        
[61] fastmap_1.1.0      utf8_1.2.2         KernSmooth_2.23-15 desc_1.4.2        
[65] stringi_1.7.5      crul_1.2.0         Rcpp_1.0.10        vctrs_0.3.8       
[69] png_0.1-7          dbplyr_2.1.1       tidyselect_1.1.1   xfun_0.35