Updated / printed: 2023-11-01
This script continues the data preparation methods needed for the Condition of Benthic Communities indicator (BH2b) in the UKMS Celtics Seas areas. Please see Data Preparation & Data Cleaning for previous data preparation steps.
This script focuses on;
## Load in the processed data from the previous script
# OneBenthic
OB_samples_count <- readRDS("Data/Processed/R_data/C20230831_OB_samples_count")
# Marine Recorder
MR_samples_count <- readRDS("Data/Processed/R_data/C20230906_MR_samples_count")
# EA WFD
WFD_samples_count <- readRDS("Data/Processed/R_data/C20230531_WFD_samples_count")
# MERMAN
merman_samples_count <- readRDS("Data/Processed/R_data/C20230831_merman_samples_count")
For the QSR2023, standards for species/taxa recordings in line with the provided data (participating countries of the Greater North Sea region) were agreed. Those standards were proposed as the lowest acceptable data standards for future data provision and assessments.
One can decide to opt for other standard levels of identification in other regions, for other monitoring techniques or with regards to assessments at sub-regional scale. But it has to be considered that other choices might lead to (slight) deviations in assessment results and have consequences with regards to comparability as the focus might be on other parts of the benthic diversity.
The highest level of taxonomic detail that can be reached in a standardised way is determined by the dataset(s) with the lowest level of taxonomic detail.
— OSPAR CEMP guidelines
The lowest level of taxonomic detail provided by participating countries for the QSR2023 may under represent the data available for the Celtic Seas. There is also the potential for taxa present in the Celtic Seas that might not be represented in the Greater North Sea region taxa list. For these reasons it’s not practical or desirable to join the Celtic Seas data with the Taxonomic standardisation for BH2b application region II v220222.xlsx list. However, the species list will be used to guide taxonomic standardisation for the Celtic Seas data alongside the BH2b CEMP guidelines:
First we need to reduce the fields to only those needed, standardise the field names, and bind the datasets together.
msfd will be the primary broad-scale habitat field
(broad-scale habitat classifications based on the North Atlantic
definitions)eunis_l3 and hab_type will be retained for
reference (composite habitat map / MR EUNIS codes)dataset)benthos dataset
# select and standardise MR fields
MR_data <- MR_samples_count %>%
select(samplekey, sampledate, sampleyear, geartype, preferredspeciesname, preferredaphiaid, count_n, au, msfd, eunis_l3_mr, eunis_l3, hab_type, geometry) %>%
mutate(sampleref = samplekey,
taxon = preferredspeciesname,
aphia = preferredaphiaid,
eunis_l3 = if_else(is.na(eunis_l3_mr), eunis_l3, eunis_l3_mr),
dataset = "MR")
# select and standardise OB fields
OB_data <- OB_samples_count %>%
select(samplecode, samplecode2, sampledate, year, gearname, validnameworms, aphiaid, count_n, au, msfd, eunis_l3, hab_type, geometry) %>%
mutate(sampleref = paste(samplecode, samplecode2, sep = "_"),
taxon = validnameworms,
aphia = aphiaid,
sampleyear = year,
geartype = gearname,
dataset = "OB")
# select and standardise WFD fields
WFD_data <- WFD_samples_count %>%
select(samplecode, sampledate, sampleyear, samplemethod, preferredtaxonname, count_n, au, msfd, eunis_l3, hab_type, geometry) %>%
mutate(sampleref = samplecode,
taxon = preferredtaxonname,
aphia = NA_real_,
geartype = samplemethod,
dataset = "WFD")
# select and standardise MERMAN fields
merman_data <- merman_samples_count %>%
select(samplecode, samplestartdate, monitoringyear, samplertype, specieslatinname, result, au, msfd, eunis_l3, hab_type, geometry) %>%
mutate(sampleref = samplecode,
sampleyear = monitoringyear,
sampledate = lubridate::dmy(samplestartdate),
taxon = specieslatinname,
aphia = as.integer(NA),
count_n = result,
geartype = samplertype) %>%
mutate(dataset = "MERMAN")
# bring the data together and just keep the standardised names
benthos <- bind_rows(MR_data, OB_data, WFD_data, merman_data) %>%
select(sampleref, sampledate, sampleyear, geartype, taxon, aphia, count_n, msfd, eunis_l3, hab_type, au, dataset, geometry)
# qc check a sample
slice_sample(benthos, n = 50)
# tidy up temp data
rm(MR_data, OB_data, WFD_data, merman_data)
Taxa records from the EA WFD dataset were matched to the NHM nomenclature of preferred names and currently do not have Aphia IDs (WoRMS) assigned. MERMAN records are also without Aphia IDs.
aphia_na <- benthos %>%
as.data.frame() %>%
filter(is.na(aphia)) %>%
select(taxon) %>%
unique() %>%
arrange(taxon)
# Add aphia ID from WoRMS
aphia_id <- aphia_na %>%
pull(taxon) %>%
wormsbynames(ids = TRUE) %>%
select(name, AphiaID)
## REQUESTING 1386 ITEMS BY NAME from World Register of Marine Species (www.marinespecies.org), 01/11/2023 14:21:15 (CC-BY)
## chunk 1/28
## chunk 2/28
## chunk 3/28
## Aphelochaeta "species A" no match
## Araphura brevimana no match
## chunk 4/28
## chunk 5/28
## Bugula purpurocincta no match
## Callopora dumerilli no match
## chunk 6/28
## Caulleriella "species B" no match
## Chaetozone "species D" no match
## Chorizopora brongniarti no match
## chunk 7/28
## Cossura pygodactyla no match
## Crassicorophium bonnellii no match
## Cumella pygmaea no match
## chunk 8/28
## Diastyloides biplicata no match
## DIPLOPODA no match
## chunk 9/28
## chunk 10/28
## Euclymene "species A" no match
## chunk 11/28
## chunk 12/28
## chunk 13/28
## Heteromysis microps no match
## chunk 14/28
## Lemnaceae no match
## chunk 15/28
## chunk 16/28
## Mangelia brachystoma no match
## Metal splatter no match
## chunk 17/28
## Nebalia herbsti no match
## chunk 18/28
## Ocenebra erinacea no match
## chunk 19/28
## Opisthodonta "species A" no match
## chunk 20/28
## chunk 21/28
## PHOLADOMYOIDA no match
## Pholoe baltica (sensu Petersen) no match
## Pholoe inornata (sensu Petersen) no match
## chunk 22/28
## Polinices catena no match
## Polinices montagui no match
## Polinices pulchellus no match
## Polyides rotundus no match
## chunk 23/28
## Pseudocuma similis no match
## chunk 24/28
## Rhopalomenia aglaophenae no match
## chunk 25/28
## Siphonoecetes striatus no match
## Sphaerodoropsis balticum no match
## chunk 26/28
## Syllis "species A" no match
## Syllis "species D" no match
## Syllis "species H" no match
## chunk 27/28
## Terebellomorpha no match
## Thyasira sarsi no match
## chunk 28/28
## by names ........................................... DONE
aphia_id %>%
filter(is.na(AphiaID))
Metal splatter from dataset and taxa list# remove from 'metal' the main dataset
benthos <- benthos %>%
filter(taxon != "Metal splatter")
# and aphia lists
aphia_id <- aphia_id %>%
filter(name != "Metal splatter")
aphia_na <- aphia_na %>%
filter(taxon != "Metal splatter")
# remove "Species A" etc from names
aphia_na_temp <- aphia_id %>%
filter(is.na(AphiaID)) %>%
select(-c(AphiaID)) %>%
mutate(alt_name = str_remove(name, " \"species [:upper:]\""))
# take a list of aphia from the dataset
benthos_aphia <- benthos %>%
select(taxon, aphia) %>%
filter(!is.na(aphia)) %>%
unique()
# join to missing aphia list
aphia_na_temp <- left_join(aphia_na_temp, benthos_aphia, by = c("alt_name" = "taxon"))
# join to aphia taxa list
aphia_id <- left_join(aphia_id, aphia_na_temp, by = c("name" = "name"))
# update and remove fields
aphia_id <- aphia_id %>%
mutate(AphiaID = ifelse(is.na(AphiaID), as.double(aphia), AphiaID)) %>%
select(-c(alt_name, aphia))
# filter to those still missing ID
aphia_na_temp <- aphia_na_temp %>%
filter(is.na(aphia)) %>%
select(-c(alt_name, aphia))
# fuzzy match with 'worrms' package
aphia_id_fuzzy <- aphia_na_temp %>%
pull(name) %>%
wm_records_taxamatch(marine_only = FALSE)
# extract the matched names, aphia, and row number
extracted_data <- lapply(seq_along(aphia_id_fuzzy), function(i) {
tbl <- aphia_id_fuzzy[[i]]
if (nrow(tbl) > 0) {
tbl %>%
mutate(row_number = i,
valid_AphiaID = first(valid_AphiaID),
valid_name = first(valid_name)) %>%
select(row_number, valid_AphiaID, valid_name)
} else {
tbl
}
})
# Combine the extracted data into a single dataframe
result_df <- bind_rows(extracted_data)
# remove duplicates
result_df <- distinct(result_df)
# Print the matches
print(result_df)
## # A tibble: 25 x 3
## row_number valid_AphiaID valid_name
## <int> <int> <chr>
## 1 1 136344 Araphura brevimanus
## 2 2 834041 Crisularia purpurotincta
## 3 3 111196 Callopora dumerilii
## 4 4 111304 Chorizopora brongniartii
## 5 5 129985 Cossura pygodactylata
## 6 6 237004 Crassicorophium bonellii
## 7 7 110567 Cumella (Cumella) pygmaea
## 8 8 110494 Diastyloides biplicatus
## 9 9 1305 Diplopoda
## 10 10 120037 Heteromysis (Heteromysis) microps
## # ... with 15 more rows
# tidy up
rm(extracted_data)
aphia_id list# add row ID to join tables
aphia_na_temp <- aphia_na_temp %>%
dplyr::mutate(row_id = row_number())
# join results to missing aphia list
aphia_na_temp_results <- left_join(aphia_na_temp, result_df, by = c("row_id" = "row_number"))
# check what's still missing an aphia
aphia_na_temp_results %>%
filter(is.na(valid_AphiaID))
# fuzzy match with a different call function
aphia_id_fuzzy_2 <- aphia_na_temp_results %>%
filter(is.na(valid_AphiaID)) %>%
pull(name) %>%
wm_records_names(fuzzy = TRUE, marine_only = FALSE)
# check the matches
aphia_id_fuzzy_2 %>% bind_rows()
Only a match for PHOLADOMYOIDA
# update remaining missing aphia ids¥
aphia_na_temp_results <- aphia_na_temp_results %>%
mutate(valid_AphiaID = case_when(name == "PHOLADOMYOIDA" ~ as.double(254),
name == "Terebellomorpha" ~ as.double(152292),
TRUE ~ as.double(valid_AphiaID)))
# join IDs
aphia_id_matched <- left_join(aphia_id, select(aphia_na_temp_results, name, valid_AphiaID), by = c("name" = "name")) %>%
mutate(AphiaID = ifelse(is.na(AphiaID), valid_AphiaID, AphiaID)) %>%
select(-c(valid_AphiaID))
# tidy up (following qc)
rm(aphia_id, aphia_id_fuzzy, aphia_id_fuzzy_2, aphia_na, aphia_na_temp, aphia_na_temp_results, result_df)
# add missing id rows to main taxa list
benthos_aphia_all <- aphia_id_matched %>%
dplyr::rename(taxon = name, aphia = AphiaID) %>%
bind_rows(benthos_aphia)
# remove any duplicates
benthos_aphia_all <- distinct(benthos_aphia_all)
# check for duplicate names
benthos_aphia_all %>%
group_by(taxon) %>%
tally() %>%
filter(n>1)
# check the Aphia IDs
benthos_aphia_all %>%
filter(taxon == "Mangelia costulata")
Aphia ID 154073 is the incorrect synonym for Mangelia costulata (Smithiella costulata)
aphia in the
dataset# filter out 154073 from the taxa list
benthos_aphia_all <- benthos_aphia_all %>%
filter(aphia != 154073)
# change to the correct ID in the data
benthos <- benthos %>%
mutate(aphia = if_else(aphia == 154073, 1644044, aphia))
We now have 2984 unique taxa names with a corresponding Aphia ID. However, many will be duplicated names (e.g. synonyms or capitalisation of higher names)
# find duplicates
aphia_qc <- benthos_aphia_all %>%
group_by(aphia) %>%
tally() %>%
filter(n>1) %>%
pull(aphia)
# filter duplicates
benthos_aphia_all %>%
filter(aphia %in% aphia_qc) %>%
arrange(aphia, taxon)
# tidy up
rm(aphia_qc)
These have the ‘correct’ Aphia ID but slight variation on the taxa name spelling, we’ll keep these duplicate IDs but assign the valid name and ID through the WoRMS match tool in the next stage.
aphia
(WFD and MERMAN records), this is needed when we join taxonomic details
later in the script# join the matched aphia records to the dataset
benthos <- left_join(benthos, benthos_aphia_all, by = c("taxon" = "taxon"), suffix = c("", ".y"),)
# update missing aphia and remove joined field
benthos <- benthos %>%
mutate(aphia = if_else(is.na(aphia), aphia.y, aphia)) %>%
select(-c(aphia.y))
aphiabenthos %>%
as.data.frame() %>%
filter(is.na(aphia))
## Warning in `[<-.data.frame`(`*tmp*`, is_list, value = list(`13` =
## "<s_GEOMET>")): replacement element 1 has 1 row to replace 0 rows
aphia) to match taxa through WoRMS
# take the aphia field and call match through WoRMS
wormsid <- benthos_aphia_all %>%
distinct(aphia) %>%
pull(aphia) %>%
wormsbyid()
# join the WoRMS call results to the taxa list
matchednames <- left_join(benthos_aphia_all, wormsid, by = c("aphia" = "AphiaID"), suffix=c("",".y"))
matchednames %>%
select(taxon, aphia, valid_name, valid_AphiaID) %>%
filter(taxon != valid_name)
As expected records with different spellings, synonyms, sub-genus names etc.
matchednames %>%
group_by(status) %>%
tally() %>%
ungroup()
matchednames %>%
filter(status != "accepted" | is.na(status)) %>%
select(taxon, class, aphia, status, unacceptreason, valid_name, valid_AphiaID) %>%
arrange(status, class, valid_name) %>%
datatable(
caption = "Unaccepted taxa names",
escape = FALSE,
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Brtip'),
class = "cell-border")
Those with valid alternative names we can change, those that are not
accepted but have no alternative we will keep, those that are
quarantined with no alternative we will remove from the
dataset.
quarantined name
data witheld as taxa
name# find the invalid
invalid_name <- matchednames %>%
filter(status == "quarantined" & is.na(valid_name)) %>%
pull(taxon)
# remove from taxa list
matchednames <- matchednames %>%
filter(taxon != invalid_name)
# remove from dataset
benthos <- benthos %>%
filter(taxon != invalid_name)
# tidy up
rm(invalid_name)
matchednames_valid_id <- matchednames %>%
filter(aphia == valid_AphiaID)
matchednames_id_update <- matchednames %>%
filter(aphia != valid_AphiaID | is.na(valid_AphiaID))
matchednames_id_update %>%
filter(is.na(valid_name) | is.na(valid_AphiaID))
Following BH2b standardisation used for the Greater North Sea (BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx), Magelona and Platyhelminthes are to be aggregated to genus and phylum respectively.
The paraphyletic suborder Gammaridea can be raised to order (Amphipoda),
Golfingiida can be reasonably be transfered to Golfingiidae,
and the no longer used superfamily name Aphroditoidea can be raised to suborder (Aphroditiformia).
update the valid ID for these taxa (for these it’s just the parent ID)
matchednames_id_update <- matchednames_id_update %>%
mutate(valid_AphiaID = case_when(taxon == "Magelona minuta" ~ parentNameUsageID,
taxon == "TURBELLARIA" ~ parentNameUsageID,
taxon == "Turbellaria" ~ parentNameUsageID,
taxon == "Golfingiida" ~ as.integer(2032),
taxon == "Gammaridea" ~ parentNameUsageID,
taxon == "Aphroditoidea" ~ parentNameUsageID,
TRUE ~ valid_AphiaID))
# take the valid IDs from the invalid names
worms_valid_id <- matchednames_id_update %>%
pull(valid_AphiaID) %>%
wormsbyid()
# take the original invalid names, IDs and Valid IDs
matchednames_invalid_tmp <- matchednames_id_update %>%
select(taxon, aphia, valid_AphiaID)
# join with the valid names from WoRMS call of valid IDs
matchednames_invalid_fixed <- left_join(matchednames_invalid_tmp, worms_valid_id, by = c("valid_AphiaID" = "AphiaID"), suffix=c("",".y")) %>%
select(-c(valid_AphiaID.y))
# remove duplicates
matchednames_invalid_fixed <- distinct(matchednames_invalid_fixed)
# tidy up
rm(matchednames_invalid_tmp, worms_valid_id)
# bind the valid names and the fixed unaccepted names
taxonomy_accepted <- bind_rows(matchednames_valid_id, matchednames_invalid_fixed)
# check those that have alternative names
taxonomy_accepted %>%
filter(taxon != scientificname) %>%
select(taxon, scientificname, aphia, valid_AphiaID, rank) %>%
arrange(taxon) %>%
datatable(
caption = "Taxa name changes",
colnames = c("original aphia" = "aphia"),
escape = FALSE,
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Brtip'),
class = "cell-border")
# take the fields needed
taxonomy_accepted <- taxonomy_accepted %>%
select(taxon, aphia, scientificname, rank, valid_AphiaID, kingdom, phylum, class, order, family, genus)
# tidy up the temporary tables (following qc)
rm(matchednames, matchednames_valid_id, matchednames_invalid_fixed, matchednames_id_update, wormsid, benthos_aphia_all, benthos_aphia, aphia_id_matched)
We’ll now investigate the taxa list (taxonomy_accepted)
and start to define the truncation to be applied to the dataset.
taxonomy_accepted %>%
group_by(kingdom, phylum, class) %>%
tally() %>%
arrange(kingdom, phylum, class)
A summary of the taxa standardisation used for the QSR BH2b Greater North Sea assessment are given in the BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx document.
The reason to remove some major groups for the BH2b QSR2023 assessment in the Greater North Sea is likely because they were not consistently recorded by other CPs. For the assessment of BH2b in the Celtic Seas, groups that are commonly recorded in UK waters have been retained (e.g. epifauna taxa).
Some major groups are not recorded consistently in grab/core samples and can be easily marked for removal from the dataset, these were;
retain logical
fieldtaxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(grepl("Bacteria|Chromista|Plantae|Protozoa", kingdom), FALSE, TRUE))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(is.na(phylum), FALSE, retain),
retain = if_else(phylum == "Cnidaria" & is.na(class), FALSE, retain))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(phylum == "Chordata"& is.na(class), FALSE, retain))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(grepl("Chaetognatha|Ctenophora|Entoprocta|Nematoda", phylum), FALSE, retain))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(grepl("Arachnida|Copepoda|Diplopoda|Hexapoda|Scyphozoa|Staurozoa|Cephalopoda|Teleostei", class), FALSE, retain))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(retain = if_else(grepl("Arhynchobdellida|Rhynchobdellida|Euphausiacea|Ingolfiellida|Mysida", order), FALSE, retain))
Summary of major groups to be removed:
taxonomy_accepted %>%
filter(retain == FALSE) %>%
group_by(phylum, class, order) %>%
tally(name = "taxa") %>%
arrange(phylum, class, order)
Changes or aggregations can be summarised into a few different categories
A summary of the standardisation and truncation applied to the BH2b assessment in the Celtic Seas at different taxonomic levels is summarised below (Taxa truncation changes to be applied).
First add a field for any higher aggregation level
(agglevel)
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = as.character(NA))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(grepl("Nemertea|Platyhelminthes", phylum) & !is.na(phylum), "phylum", agglevel))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(grepl("Clitellata", class) & !is.na(class), "class", agglevel))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(grepl("Cephalaspidea|Nudibranchia", order) & !is.na(order), "order", agglevel))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(grepl("Edwardsiidae|Cerianthidae", family) & !is.na(family), "family", agglevel))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(grepl("Magelona|Owenia|Ophryotrocha|Arenicola|Capitella|Dasybranchus|Notomastus|Microphthalmus|Harmothoe|Pholoe|Syllides|Exogone|Polydora|Streblospio|Aphelochaeta|Tharyx|Polycirrus|Streblosoma|Terebellides|Corophium|Gammarus|Pseudocuma|Gnathia|Callipallene", genus) & !is.na(genus), "genus", agglevel))
# Spio separately, some others include 'spio' in their name
taxonomy_accepted <- taxonomy_accepted %>%
mutate(agglevel = if_else(genus == "Spio", "genus", agglevel))
For those that will be recorded with an alternative name we’ll add an
alternative name field (altname)
taxonomy_accepted <- taxonomy_accepted %>%
mutate(altname = as.character(NA))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(altname = if_else(grepl("Pseudopolydora", genus), "Polydora", altname)) %>%
mutate(altname = if_else(grepl("Monocorophium", genus), "Corophium", altname)) %>%
mutate(altname = if_else(grepl("Heteromastus|Mediomastus", genus), "Heteromastus-Mediomastus", altname))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(altname = if_else(grepl("Glycera alba|Glycera tridactyla", scientificname), "Glycera alba-tridactyla", altname)) %>%
mutate(altname = if_else(grepl("Phyllodoce maculata|Phyllodoce mucosa", scientificname), "Phyllodoce maculata-mucosa", altname)) %>%
mutate(altname = if_else(grepl("Eteone flava|Eteone longa", scientificname), "Eteone flava-longa", altname))
Summary of the 407 taxon changes:
taxonomy_accepted %>%
filter(!is.na(altname) | !is.na(agglevel) | retain == FALSE) %>%
select(taxon, retain, agglevel, altname, phylum, class, order, family) %>%
arrange(phylum, class, order, family, taxon) %>%
datatable(
caption = "Taxa truncation changes to be applied",
escape = FALSE,
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Brtip'),
class = "cell-border")
To enable standardisation of BH2b for the QSR2023 assessment in Region II all taxa that were potentially recorded with presence/absence were removed (BH2-Appendix3a_Taxonomic standardization for BH2b application in region 2.xlsx). Whilst there is the potential for some inconsistent recording policies with some of these major groups (e.g. algae) in the Celtic Seas, we will try to retain epifaunal groups (e.g. bryozoans) as they are significant part of benthic communities in mixed and coarse sediments.
Three taxa were marked to be recorded as ‘present’ for the Region II QSR assessment (Hirudinea, Nemertea, and Malacostraca). For the BH2b Celtic Seas assessment we will maintain the policy for nemerteans, these are often recorded as ‘fragments/present’. Records of ‘Malacostraca’ and leeches can be left as density records, there no reason to believe they would be recorded differently in grab or core samples in UK waters.
Major groups to be recorded as ‘present’ for the current assessment were Nemertea, Porifera, Phoronida, and Bryozoa.
Some major groups contain both solitary and colonial taxa, or policies might vary between labs, in these instances the precautionary assignment of ‘Presence’ will be applied (e.g. barnacles).
presence)taxonomy_accepted <- taxonomy_accepted %>%
mutate(presence = as.logical(FALSE))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(presence = if_else(grepl("Nemertea|Porifera|Phoronida|Bryozoa", phylum) & !is.na(phylum), TRUE, as.logical(presence)))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(presence = if_else(class == "Thecostraca" & !is.na(class), TRUE, as.logical(presence)))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(presence = if_else(grepl("Malacalcyonacea|Octocorallia incertae sedis|Zoantharia", order) & !is.na(order), TRUE, as.logical(presence)),
presence = if_else(taxon == "Anthozoa", TRUE, as.logical(presence)),
presence = if_else(taxon == "Octocorallia", TRUE, as.logical(presence)),
presence = if_else(taxon == "Hexacorallia", TRUE, as.logical(presence)))
taxonomy_accepted <- taxonomy_accepted %>%
mutate(presence = if_else(taxon == "Ascidiacea", TRUE, as.logical(presence)))
Summary of major groups to be represented as ‘presence’ records:
taxonomy_accepted %>%
filter(presence == TRUE) %>%
group_by(phylum, class, order, family) %>%
tally() %>%
arrange(phylum, class, order, family) %>%
datatable(
caption = "Major groups to record as presence only",
escape = FALSE,
colnames = c('number of taxa' = 'n'),
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Brtip'),
class = "cell-border")
At this stage we have a list of all the taxa in the dataset and the
type of standarisation to be applied to each taxa in the dataset
(taxonomy_accepted ‘look up’ table). We will now apply
these policies to the dataset before interrogating further for
amalgamations based on frequencies of occurrence records at a less
detailed taxonomic level (e.g. juvenile or incomplete specimens that
could not be identified to a more detailed level).
First we will apply the truncation and reduction policies defined in
taxonomy_accepted.
benthos data to the ‘look up’ tabletaxon and aphia fields to the
accepted names and remove the WoRMS equivalents# join benthos data to the look up table
benthos_t <- left_join(benthos, taxonomy_accepted, by = c("taxon" = "taxon", "aphia" = "aphia"), suffix=c("",".y"))
# update the taxon and aphia fields (we could keep the WoRMS fields but the originals are quicker to type!)
benthos_t <- benthos_t %>%
mutate(taxon = scientificname,
aphia = valid_AphiaID) %>%
select(-c(scientificname, valid_AphiaID))
retain == FALSE,
4470 records)benthos_t <- benthos_t %>%
filter(retain == TRUE)
altname,
3943 records)
aphia of alternative names to 0 to
avoid misleading IDsbenthos_t <- benthos_t %>%
mutate(taxon = if_else(!is.na(altname), altname, taxon),
rank = if_else(!is.na(altname), "Complex", rank),
aphia = if_else(!is.na(altname), 0, as.numeric(aphia)),
genus = if_else(grepl("Pseudopolydora", genus) & !is.na(genus), "Polydora", genus),
genus = if_else(grepl("Monocorophium", genus) & !is.na(genus), "Corophium", genus),
genus = if_else(grepl("Heteromastus|Mediomastus", genus) & !is.na(genus), "Heteromastus-Mediomastus", genus))
agglevel, 17940 records)
aphia of aggregated names to 0 to
avoid misleading IDsrank, although we my not need it
laterbenthos_t <- benthos_t %>%
mutate(taxon = case_when(agglevel == "phylum" ~ phylum,
agglevel == "class" ~ class,
agglevel == "order" ~ order,
agglevel == "family" ~ family,
agglevel == "genus" ~ genus,
TRUE ~ taxon),
rank = case_when(agglevel == "phylum" ~ "Phylum",
agglevel == "class" ~ "Class",
agglevel == "order" ~ "Order",
agglevel == "family" ~ "Family",
agglevel == "genus" ~ "Genus",
TRUE ~ rank),
class = if_else(agglevel == "phylum" & !is.na(agglevel), as.character(NA), class),
order = if_else(grepl("phylum|class", agglevel) & !is.na(agglevel), as.character(NA), order),
family = if_else(grepl("phylum|class|order", agglevel) & !is.na(agglevel), as.character(NA), family),
genus = if_else(grepl("phylum|class|order|family", agglevel) & !is.na(agglevel), as.character(NA), genus),
aphia = if_else(!is.na(agglevel), 0, as.numeric(aphia)))
Some taxa that have been aggregated will be missing aphia IDs
(aphia == 0) whilst other original records of the names
will have correct aphia IDs. This causes some problems later when
grouping data. To avoid this we’ll add in missing aphia IDs where we
can.
aphia placeholder of
zero# get those without ID
aphia_updates <- benthos_t %>%
as.data.frame() %>%
filter(aphia == 0) %>%
group_by(taxon) %>%
tally()
# match through WoRMS
aphia_updates_matched <- aphia_updates %>%
pull(taxon) %>%
wormsbynames(ids = TRUE) %>%
select(name, AphiaID)
## REQUESTING 35 ITEMS BY NAME from World Register of Marine Species (www.marinespecies.org), 01/11/2023 14:23:55 (CC-BY)
## chunk 1/1
## Eteone flava-longa no match
## Glycera alba-tridactyla no match
## Heteromastus-Mediomastus no match
## Phyllodoce maculata-mucosa no match
## by names ........................................... DONE
# join WoRMS IDs to dataset
benthos_t <- left_join(benthos_t, aphia_updates_matched, by = c("taxon" = "name"))
# update changes and drop new field
benthos_t <- benthos_t %>%
mutate(aphia = if_else(aphia == 0 & !is.na(AphiaID), as.numeric(AphiaID), aphia)) %>%
select(-c(AphiaID))
# tidy up
rm(aphia_updates, aphia_updates_matched)
Following aggregations and standardisation of taxa names there are now multiple records of the same taxa in a sample (e.g. species records now aggregated to the same genus).
# find duplicate taxa records
dups <- benthos_t %>%
as.data.frame() %>%
select(-c(count_n, geometry)) %>%
duplicated()
# extract the fields to filter the data
dup_list <- benthos_t %>%
as.data.frame() %>%
filter(dups) %>%
select(sampleref, sampledate, taxon)
# qc the duplicates
benthos_t %>%
as.data.frame() %>%
right_join(dup_list) %>%
select(sampleref, taxon, count_n) %>%
slice_sample(n = 100)
# sum duplicate records
qc_temp <- benthos_t %>%
as.data.frame() %>%
right_join(dup_list) %>%
group_by(sampleref, sampledate, taxon) %>%
dplyr::summarise(count_sum = sum(count_n))
# add the sum values and remove duplicates
benthos_t_agg <- benthos_t %>%
left_join(qc_temp) %>%
filter(!dups) %>%
mutate(count_n = if_else(is.na(count_sum), count_n, count_sum)) %>%
select(-c(count_sum))
# tidy up
rm(dups, dup_list, qc_temp)
2136 records aggregated
Investigate the abundance records for any unusually high values following aggregation.
benthos_t_agg %>%
as.data.frame() %>%
select(sampleref, taxon, count_n, sampleyear, msfd, rank, dataset, geometry) %>%
mutate(coords = as.character(geometry)) %>%
mutate(sample = str_sub(sampleref, 0, 15)) %>%
select(-c(sampleref)) %>%
filter(count_n >500) %>%
arrange(desc(count_n))
benthos_t_agg %>%
as.data.frame() %>%
select(sampleref, taxon, count_n, sampleyear, msfd, rank) %>%
mutate(sample = str_sub(sampleref, 0, 15)) %>%
select(-c(sampleref)) %>%
filter(rank != "Species", rank != "Genus", rank != "Complex", count_n >100) %>%
arrange(desc(count_n))
benthos_t_agg %>%
as.data.frame() %>%
select(sampleref, family, taxon, count_n, sampleyear, msfd, family) %>%
mutate(sample = str_sub(sampleref, 0, 15)) %>%
select(-c(sampleref)) %>%
filter(family == "Cardiidae"| family == "Mytilidae" | family == "Veneridae", count_n >100) %>%
arrange(desc(count_n))
High abundance taxa look relatively normal, there aren’t any extreme abundances from juvenile bivalves (e.g. thousands of M. edule spat) although there are some high records of what are likely juvenile Acanthocardia echinata and Mytilus edulis. There are a couple of high records of ‘Ascidiacea’ recorded at class, these will be assigned ‘Presence’ in later stages.
benthos_t_agg %>%
as.data.frame() %>%
select(sampleref, phylum, class, order, family, taxon, rank, count_n) %>%
filter(rank != "Species", rank != "Genus", rank != "Complex", rank != "Subspecies") %>%
group_by(phylum, class, order, rank, taxon) %>%
dplyr::summarise(sample_count = n_distinct(sampleref)) %>%
filter(sample_count <5) %>%
arrange(phylum, class, order)
Following exploration of the dataset we can see some taxa that can be updated, for example there are records of Magelonidae that can be assigned to Magelona (Magelonidae is a monophyletic group). However, most aggregations (e.g. the classes Calcarea and Demospongiae to phylum Porifera) would make no difference to the Margalef diversity score. Many don’t occur with the taxon they would be aggregated with, therefore the number of ‘species’ in a sample would not change. If the aim was to calculate community similarity then there would be reasons to standardise the names further. For the purpose of BH2b (Margalef diversity) we can apply the main aggregations identified. However, there is little benefit standardising names completely at this stage.
benthos_t_agg <- benthos_t_agg %>%
mutate(taxon = if_else(taxon == "Ampharetinae", "Ampharetidae", taxon),
rank = if_else(aphia == 152252, "Family", rank),
aphia = if_else(aphia == 152252, 981, as.numeric(aphia))) %>%
mutate(taxon = if_else(taxon == "Terebellinae", "Terebellidae", taxon),
rank = if_else(aphia == 322588, "Family", rank),
aphia = if_else(aphia == 322588, 982, as.numeric(aphia))) %>%
mutate(taxon = if_else(taxon == "Terebelliformia", "Terebellida", taxon),
rank = if_else(aphia == 152292, "Order", rank),
aphia = if_else(aphia == 152292, 900, as.numeric(aphia))) %>%
mutate(taxon = if_else(taxon == "Magelonidae", "Magelona", taxon),
rank = if_else(aphia == 914, "Genus", rank),
aphia = if_else(aphia == 914, 129341, as.numeric(aphia)))
# pull a list of column names except for the count and geometry
fields <- colnames(benthos_t_agg)[colnames(benthos_t_agg) != 'count_n' & colnames(benthos_t_agg) != 'geometry']
# aggregate abundance records for duplicate names in a sample
benthos_t_agg_tmp <- benthos_t_agg %>%
group_by(across(all_of(fields))) %>%
dplyr::summarise(count_n = sum(count_n)) %>%
ungroup()
# note geometry needs rejoining
# Take a data frame of the coordinates per sample
coordinates <- benthos_t_agg %>%
select(sampleref, geometry) %>%
distinct()
# Join to the aggregated sample data to the geometry
benthos_t_agg <- left_join(benthos_t_agg_tmp, coordinates, by = c("sampleref" = "sampleref"))
# tidy up
rm(benthos_t_agg_tmp, fields, coordinates)
Following all taxonomic aggregations we can now ensure any taxa to be represented as “Present” are converted to nominal abundance of 1
benthos_t_agg <- benthos_t_agg %>%
mutate(count_n = if_else(presence == TRUE, 1, count_n))
benthos_taxonomy <- benthos_t_agg %>%
as.data.frame() %>%
select(taxon, aphia, rank, kingdom, phylum, class, order, family, genus, agglevel, altname, presence) %>%
group_by(taxon, aphia, rank, kingdom, phylum, class, order, family, genus, agglevel, altname, presence) %>%
dplyr::summarise(records = n()) %>%
arrange(kingdom, phylum, class, order, family, genus, taxon)
write.csv(benthos_taxonomy, paste0("Data/Processed/Taxonomy/", creation_date(), "_benthos_taxonomy.csv"))
# csv copy
#write.csv(benthos_t_agg, paste0("Data/Processed/Benthos/", creation_date(), "_benthos_t_agg.csv"))
# R copy if needed again
write_rds(benthos_t_agg, paste0("Data/Processed/R_data/", creation_date(), "_benthos_t_agg"))
Following truncation and standardisation community data can be summarised to the key diversity metrics. The Margalef diversity score is calculated using the benthos package.
samples <- benthos_t_agg %>%
group_by(sampleref, sampledate, sampleyear, msfd, eunis_l3, hab_type) %>%
dplyr::summarise(D = margalef(., taxon = taxon, count = count_n), S = n_distinct(taxon), N = sum(count_n), .groups = "drop")
Save the sample data for analysis in the next script (Data Analysis)
# save processed data
write_rds(samples, paste0("Data/Processed/R_data/", creation_date(), "_samples"))
details::details(sessionInfo(), summary = 'sessionInfo')
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server x64 (build 14393)
Matrix products: default
locale:
[1] LC_COLLATE=English_United Kingdom.1252
[2] LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] RColorBrewer_1.1-2 benthos_1.3-6 details_0.3.0 DT_0.26
[5] sf_0.9-8 forcats_0.5.1 stringr_1.4.0 dplyr_1.0.7
[9] purrr_0.3.4 readr_2.1.0 tidyr_1.1.4 tibble_3.1.6
[13] ggplot2_3.3.5 tidyverse_1.3.1 rprojroot_2.0.3 worrms_0.4.2
[17] worms_0.2.2 plyr_1.8.6 httr_1.4.2
loaded via a namespace (and not attached):
[1] sass_0.4.4 jsonlite_1.7.2 modelr_0.1.8 bslib_0.4.1
[5] assertthat_0.2.1 triebeard_0.3.0 urltools_1.7.3 cellranger_1.1.0
[9] yaml_2.2.1 pillar_1.6.4 backports_1.3.0 glue_1.5.0
[13] digest_0.6.28 rvest_1.0.2 colorspace_2.0-2 htmltools_0.5.2
[17] clipr_0.7.1 pkgconfig_2.0.3 httpcode_0.3.0 broom_0.7.10
[21] haven_2.4.3 scales_1.1.1 tzdb_0.2.0 proxy_0.4-27
[25] generics_0.1.1 ellipsis_0.3.2 cachem_1.0.6 withr_2.5.0
[29] lazyeval_0.2.2 cli_3.1.0 magrittr_2.0.1 crayon_1.4.2
[33] readxl_1.3.1 evaluate_0.18 fs_1.5.0 fansi_0.5.0
[37] xml2_1.3.2 class_7.3-15 tools_3.6.1 hms_1.1.1
[41] lifecycle_1.0.1 munsell_0.5.0 reprex_2.0.1 compiler_3.6.1
[45] jquerylib_0.1.4 e1071_1.7-11 rlang_0.4.12 classInt_0.4-3
[49] units_0.8-0 grid_3.6.1 rstudioapi_0.13 htmlwidgets_1.5.4
[53] crosstalk_1.2.0 rmarkdown_2.18 gtable_0.3.0 curl_4.3.2
[57] DBI_1.1.3 R6_2.5.1 lubridate_1.8.0 knitr_1.41
[61] fastmap_1.1.0 utf8_1.2.2 KernSmooth_2.23-15 desc_1.4.2
[65] stringi_1.7.5 crul_1.2.0 Rcpp_1.0.10 vctrs_0.3.8
[69] png_0.1-7 dbplyr_2.1.1 tidyselect_1.1.1 xfun_0.35