Updated / printed: 2023-10-26
This script continues from the data preparation methods needed for the Condition of Benthic Communities indicator (BH2b) assessment in the UK EEZ Celtics Seas areas. Please see Data Preparation, Data Cleaning & Data Truncation for previous data preparation steps.
This script focuses on;
## Load in the processed sample data from the previous scripts
samples <- readRDS("Data/Processed/R_data/C20230906_samples")
## Load in the processed NRW sample data
samples_NRW_bht <- readRDS("Data/Processed/R_data/C20230831_samples_NRW_bht")
# Load in the benthos data that has geospatial point data
benthos <- readRDS("Data/Processed/R_data/C20230906_benthos_t_agg")
# Take the sample reference (to join to the sample data) and geometry
benthos <- benthos %>%
select(sampleref, geometry) %>%
distinct()
# join geometry to samples
samples <- left_join(samples, benthos, by = c("sampleref" = "sampleref"), suffix = c("", ".y"))
# make it a spatial object
samples <- st_as_sf(samples)
# tidy up
rm(benthos)
# bind datasets into one
samples <- samples_NRW_bht %>%
select(sampleref:eunis_l3, msfd) %>%
rename(D = d, S = s, N = n) %>%
bind_rows(samples)
To represent the assessment period and sample data, Swept Area Ratio (SAR) pressure data collated for the BH3 QSR2023 assessment will be used for BH2b in the Celtic Seas (ICES, 2021a). Fishing pressure for the BH2b indicator is used when assigning reference values, these take a percentile score of diversity depending on the ‘quality score’ of the samples and SAR.
ICES, 2021a. OSPAR request on the production of spatial data layers of fishing intensity / pressure. ICES Technical service.
The processed VMS data is taken from the BH3 pressure layer (unioined annual SAR layers).
# load in the VMS
QSR_VMS_Pressure <- st_read("Data/Raw/VMS/C20220608_QSR_MSFD_Year_VMS_Pressure_Map_20092020_Reduced.shp")
# Load UKMS assessment units
ukms <- st_read("Data/Raw/UKMS/C20230620_UKMS_final.shp")
# fix geometry
ukms <- st_make_valid(ukms)
# filter to Celtic Seas
ukms_cs <- filter(ukms, UKMS_SubRe == "Celtic Seas")
For the BH2b assessment pressure was taken as average SAR across the available years. The categories of SAR follow those supplied in the ICES EUTRADE 2021 data products ICES, 2021b as used in the Greater North Sea assessment:
Only the first three categories are used when determining reference values (mean SAR 0 to 1)
Processed VMS data from BH3 did not categories SAR in the same format so we first need to create the equivalent categories from the VMS data
-1) to true
NAVMS_pressure <- QSR_VMS_Pressure %>%
select(starts_with("sar"))
# Drop any Z or M dimensions in the pressure layer
VMS_pressure <- st_zm(VMS_pressure, drop = TRUE, what = "ZM")
# cut to Celtic Seas area of assessment
VMS_pressure <- st_intersection(VMS_pressure, ukms_cs)
# update field names to remove "_R"
names(VMS_pressure) <- gsub(x = names(VMS_pressure), pattern = "_R", replacement = "")
# remove the '-1's added during BH3 analysis and return to true 'NA'
VMS_pressure <- VMS_pressure %>%
mutate_at(vars(sar09:sar20), ~replace(., . == -1, NA))
# remove any c-squares where no SAR has been recorded
VMS_pressure <- VMS_pressure %>%
filter(rowSums(is.na(select(., sar09:sar20))) < 12)
# remove unnecessary fields
VMS_pressure <- VMS_pressure %>%
select(-c("OBJECTID", "Shape_Leng", "Shape_Area"))
# tidy up
rm(QSR_VMS_Pressure)
mean_SAR across the year range (2009 to 2020)
NA removed# mean SAR from 2009-2020
VMS_pressure <- VMS_pressure %>%
as.data.frame() %>%
mutate(mean_SAR = rowMeans(select(., sar09:sar20), na.rm = TRUE))
# return to sf
VMS_pressure <- st_as_sf(VMS_pressure, crs = st_crs(4326))
cat_SAR as above)
# categories of SAR
VMS_pressure <- VMS_pressure %>%
mutate(cat_SAR = case_when(
mean_SAR <= 0.1 ~ "0-0.1",
mean_SAR <= 0.5 ~ "0.1-0.5",
mean_SAR <= 1 ~ "0.5-1",
mean_SAR <= 5 ~ "1-5",
mean_SAR <= 10 ~ "5-10",
mean_SAR > 10 ~ ">10")) %>%
mutate(cat_SAR = factor(cat_SAR, levels = c("0-0.1", "0.1-0.5", "0.5-1", "1-5", "5-10", ">10")))
VMS_pressure %>%
as.data.frame() %>%
select(-c("geometry")) %>%
slice_sample(n = 50)
Map of SAR categories for the years 2009 to 2020
# Read in world landmass shapefile for creating maps
land <- st_read(paste0(NEA_PANACEA_folder, "Benthic Habitat AUs and Land Shapefiles/Benthic Habitat AUs and Land Shapefiles/c20210824_World_Outline_WGS84_LAEA.shp"))
# transform to WGS84
land <- st_transform(land, 4326)
# fix geometry
land <- st_make_valid(land)
# clip the land to Celtic Seas area
sf::sf_use_s2(FALSE)
land_cs <- st_crop(land, ukms_cs)
sf::sf_use_s2(TRUE)
# plot SAR categories across the Celtic Seas
ggplot() +
geom_sf(data = land_cs, fill = "grey", colour = NA) +
geom_sf(data = ukms_cs, aes(colour = UKMS_SubRe), fill = NA) +
geom_sf(data = VMS_pressure, mapping = aes(geometry = geometry, fill = cat_SAR), colour = NA) +
scale_fill_viridis_d(name = "SAR") +
scale_color_manual(name = "UKMS Sub-region", values = c("grey20")) +
ggtitle("SAR Categories in the Celtic Seas") +
theme_bw() +
ylim(48.19, 63.4) +
xlim(-14.21, 1.1)
Before determining ‘reference’ values we need to join the fishing pressure data to the sample data
# Join the samples (with geometry) to the fishing pressure data (SAR),
# turn off s2 (problems with s2 geometry in area layers)
sf::sf_use_s2(FALSE)
samples_SAR <- st_join(samples, VMS_pressure, left = TRUE)
sf::sf_use_s2(TRUE)
# plot SAR categories across the Celtic Seas samples
ggplot() +
geom_sf(data = land_cs, fill = "grey", colour = NA) +
geom_sf(data = ukms_cs, lwd = 0.5, colour = "grey", fill = NA) +
geom_sf(data = samples_SAR, mapping = aes(geometry = geometry, colour = cat_SAR), size = 0.6) +
scale_colour_viridis_d(name = "SAR", na.value = "tomato") +
theme_bw() +
ggtitle("SAR Categories for sample locations in the Celtic Seas") +
guides(color = guide_legend(override.aes = list(size=3))) +
ylim(48.19, 63.4) +
xlim(-14.21, 1.1)
To better aid the spatial representation of the BH2b indicator sample data will be grouped by Charting Progress Regions (CP2). These regions have been drafted on the basis of key environmental characteristics, grouping samples by these regions will help avoid some of the biogeographic variations inherent across the Celtic Seas area.
# load in CP2 layer
cp2_regions <- st_read("Data/Raw/CP2_Regions/C071222_CP2_reporting_regions.shp")
# select the regions needed
cp2 <- cp2_regions %>%
filter(Region == "Atlantic North-West Approaches, Rockall Trough and Faroe-Shetland Channel" |
Region == "Scottish Continental Shelf"|
Region == "Minches & Western Scotland"|
Region == "Irish Sea"|
Region == "Western Channel & Celtic Sea") %>%
mutate(Region = if_else(Region == "Atlantic North-West Approaches, Rockall Trough and Faroe-Shetland Channel",
as.character("Atlantic North-West Approaches,\nRockall Trough and Faroe-Shetland Channel"), as.character(Region)))
# Select the fields needed
cp2 <- cp2 %>%
select(Region, RegionID)
# transform to WGS84
cp2 <- st_transform(cp2, 4326)
# Join samples (with geometry) to CP2 Regions
samples_SAR_cp2 <- st_join(samples_SAR, cp2, left = TRUE)
There are some areas of the Celtic Seas assessment units that are not covered by the CP2 regions layer.
samples_SAR_cp2 %>%
as.data.frame() %>%
filter(is.na(Region))
samples_SAR_cp2 %>%
filter(is.na(Region)) %>%
ggplot() +
geom_sf(data = land_cs, fill = "grey", colour = NA) +
geom_sf(data = cp2, aes(fill = Region), colour = NA) +
geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
geom_sf(mapping = aes(geometry = geometry, colour = msfd), size = 3) +
scale_fill_manual(name = "CP2 Regions", values = cp2_palette) +
scale_colour_manual(name = "Broad-scale Habitat", values = cb_palette) +
coord_sf(xlim = c(-6.5, -3.5), ylim = c(53.5, 54.5)) +
theme_bw()
1 sample that did not intersect CP2 regions, one recorded in Isle of Man waters (2011)
# filter samples without CP2 region
samples_SAR_cp2 <- samples_SAR_cp2 %>%
filter(!is.na(Region))
The BH2b indicator method has not yet been fully tested with data from transnational waters. Additionally, diversity is likely to be impacted in reduced salinity habitats. To avoid potentially including transitional data, sample locations will be checked against classified transitional water bodies.
Transitional water body Shapefiles were sourced from:
* Note, Whitstable Bay (water body ID GB640604290000) was recategorized from being a coastal water body in Cycle 2, to a transitional water body in Cycle 3. However, this change is not relevant to the Celtic Seas
All transitional water body layers were available through the Open Government Licence (nationalarchives.gov.uk)
# Load in the transitional waterbodies
EA_trans <- st_read("Data/Processed/Spatial/C20230712_EA_WFD_Transitional_Updated.shp")
SEPA_trans <- st_read("Data/Raw/Transitional_waters/C20230712_SEPA_Transitional_waters.shp")
NIEA_trans <- st_read("Data/Processed/Spatial/C20230712_NIEA_Estuaries.shp")
NRW_trans <- st_read("Data/Raw/Transitional_waters/C20230712_NRW_Transitional_Waterbodies_Cycle_3.shp")
# fix geometry in NIEA layer
NIEA_trans <- st_make_valid(NIEA_trans)
### Check for spatial overlaps with transitional water bodies
st_filter(samples_SAR_cp2, EA_trans)
st_filter(samples_SAR_cp2, SEPA_trans)
st_filter(samples_SAR_cp2, NIEA_trans)
st_filter(samples_SAR_cp2, NRW_trans)
Only overlaps with English and Welsh transitional waters (Irish Sea and Western Channel & Celtic Sea CP2 region)
# filter out English transitional samples
samples_SAR_cp2 <- samples_SAR_cp2 %>%
st_filter(., st_union(EA_trans), .predicate = st_disjoint)
# filter out Welsh transitional samples
samples_SAR_cp2 <- samples_SAR_cp2 %>%
st_filter(., st_union(NRW_trans), .predicate = st_disjoint)
# tidy up
rm(EA_trans, SEPA_trans, NRW_trans, NIEA_trans)
Reference values are calculated for each broad-scale habitat, for the QSR assessment cases were additionally split between OSPAR benthic assessment units and country to allow for variations introduced by biogeography and national monitoring techniques and protocols. For the Celtic Seas assessment reference values will be calculated for broad-scale habitats and Charting Progress Regions (CP2) to maintain spatial representativity across the Celtic Seas area. Reference values are calculated form all available data (2009 to 2021).
Assignment of reference values are a multistage process, first up to three percentile values are chosen from ‘low fishing pressure’ areas (mean SAR >0 and ≤1), secondly the highest of these percentile values is taken as the final reference value (Dref).
For each assessment case (broad-scale habitat) samples are grouped by
average SAR category (cat_SAR) and given a preliminary
reference value from the percentile dependent on the:
| Average SAR | Samples (n) | Years (n) | Reference Percentile | Quality Code |
|---|---|---|---|---|
| 0 - 0.1 | ≥50 | ≥3 | 75% | 1 |
| ≥30 | ≥2 | 95% | 2 | |
| ≥20 | 1 | 99% | 3 | |
| ≥10 & <20 | - | - | 4 | |
| <10 | - | - | 5 | |
| 0.1 - 0.5 | ≥50 | ≥3 | 95% | 1 |
| ≥30 | ≥2 | 99% | 2 | |
| ≥10 & <20 | - | - | 4 | |
| <10 | - | - | 5 | |
| 0.5 - 1 | ≥50 | ≥3 | 99% | 1 |
| >1 | ≥10 & <20 | - | - | 4 |
| <10 | - | - | 5 |
For example a broad-scale habitat may have samples in different ‘low pressure’ SAR categories:
In this example the the 99% percentile Margalef from each group would be considered, with the highest of the three taken as the final reference value (Dref).
Alternatively, there could be more than 50 samples across more than 3 years for all 3 ‘low pressure’ categories and the highest Margalef of the corresponding 75%, 95% and 99% would be taken as the final reference value.
Note; reference values are only taken from samples where VMS data has been reported (i.e. SAR recorded in any year), in accordance with the OSPAR CEMP guidelines.
In case of no information available (indicated as ‘NA’) related samples are not used to estimate the reference as there might be fisheries after all of which we are not aware.
— OSPAR CEMP guidelines
We will create a Margalef diversity reference table for the broad-scale habitats in the dataset following the method above
# group samples by SAR categories and calculate percentiles etc.
d_ref_prelim <- samples_SAR_cp2 %>%
as.data.frame() %>%
filter(!is.na(cat_SAR)) %>%
group_by(msfd, cat_SAR, Region) %>%
dplyr::summarise(n_samples = n(),
n_years = length(unique(sampleyear)),
D75 = quantile(D, probs = 0.75),
D95 = quantile(D, probs = 0.95),
D99 = quantile(D, probs = 0.99),
.groups = "drop") %>%
mutate(quality_code = if_else(
(n_samples >= 50) & (n_years >= 3),
1L,
if_else(
(n_samples >= 30) & (n_years >= 2),
2L,
if_else(
n_samples >= 20,
3L,
if_else(
n_samples >= 10,
4L,
5L
)))),
reference = if_else(
(quality_code == 3) & (cat_SAR == "0-0.1"),
D99,
if_else(
(quality_code == 2) & (cat_SAR == "0-0.1"),
D95,
if_else(
(quality_code == 2) & (cat_SAR == "0.1-0.5"),
D99,
if_else(
(quality_code == 1) & (cat_SAR == "0-0.1"),
D75,
if_else(
(quality_code == 1) & (cat_SAR == "0.1-0.5"),
D95,
if_else(
(quality_code == 1) & (cat_SAR == "0.5-1"),
D99,
NA_real_)))))))
# summary of percentile reference values in <1 SAR areas
d_ref_prelim %>%
filter(!is.na(reference)) %>%
mutate(across(starts_with("D"), ~ round(.x, digits = 1))) %>%
mutate(across(starts_with("ref"), ~ round(.x, digits = 1))) %>%
arrange(Region, msfd, cat_SAR) %>%
datatable(rownames = FALSE,
colnames = c("Broad-scale habitat" = "msfd"))
We will now take the highest of the ‘preliminary reference’ values for each habitat and CP2 region as the final Dref
# broad-scale habitat and cp2 reference
d_ref <- d_ref_prelim %>%
filter(!is.na(reference)) %>%
group_by(msfd, Region) %>%
dplyr::summarise(D_reference = max(reference), .groups = "drop")
d_ref %>%
select(msfd:D_reference) %>%
mutate(msfd = factor(msfd, levels = BHT_order),
Region = factor(Region, levels = region_order)) %>%
arrange(desc(msfd)) %>%
pivot_wider(names_from = Region, values_from = D_reference) %>%
datatable(
caption = "Reference Margalef for Broad-scale Habitat and Charting Progress 2 Region",
colnames = c("Broad-scale habitat" = "msfd"),
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Bfrtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border")
# tidy up
rm(d_ref_prelim)
In the QSR there was a higher level of groupings (twelve BHTs across five assessment units for seven countries) which resulted in many combinations without reference values. Where this was the case reference values were derived from averages of all other reference values split into sub-groups. In brief averages were taken from combinations of country, subregion, depth zone, and sediment type, then an average of these averages was given as the reference value.
For the assessment of BH2b in the Celtic Seas there are far fewer group combinations (i.e. no grouping by country). Therefore, reference values for a broad-scale habitat that could not be calculated in a CP2 region can be taken as the average of reference values of that broad-scale habitat type in other CP2 regions. For example, Dref for Infralittoral sand in the Minches & Western Scotland will be the mean of Dref for Infralittoral sand in the Irish Sea and Western Channel & Celtic Sea.
# calculate the average reference value by habitat
d_ref_avg <- d_ref %>%
group_by(msfd) %>%
dplyr::summarise(D_reference_avg = mean(D_reference))
# make a summary of samples in CP2 regions where BHT refs are avilaible
sample_summary <- samples_SAR_cp2 %>%
as.data.frame() %>%
filter(msfd %in% d_ref$msfd) %>%
group_by(Region, msfd) %>%
dplyr::summarise(samples_n = n(), .groups = "drop")
# add reference averages for all broad-scale habitats in CP2 regions
d_ref_avg_prelim <- left_join(sample_summary, d_ref_avg, by = "msfd")
# add CP2 reference values
d_ref_final <- left_join(d_ref_avg_prelim, d_ref, by = c("msfd" = "msfd", "Region" = "Region"))
# give regions a factor order, give final ref (region ref or avg) and give rounded label for plots
d_ref_final <- d_ref_final %>%
mutate(Region = factor(Region, levels = region_order),
msfd = factor(msfd, level = BHT_order),
D_reference_final = if_else(is.na(D_reference), D_reference_avg, D_reference),
D_ref_label = if_else(is.na(D_reference), paste0("(", as.character(round(D_reference_final, 1)), ")"), as.character(round(D_reference_final, 1))))
# show in table
d_ref_final %>%
select(msfd, Region, D_ref_label) %>%
group_by(msfd, Region) %>%
arrange(desc(msfd), Region) %>%
pivot_wider(names_from = Region, values_from = D_ref_label, values_fill = "-") %>%
datatable(
caption = "Reference Margalef for Broad-scale Habitat and Charting Progress 2 Region",
colnames = c("Broad-scale habitat" = "msfd"),
rownames = FALSE,
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Bfrtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border")
# tidy up
rm(d_ref_avg, d_ref_avg_prelim)
636 samples were assigned broad-scale habitats that had insufficient data in ‘low pressure’ areas to calculate Dref in any CP2 region
samples_SAR_cp2 %>%
as.data.frame() %>%
group_by(msfd, Region) %>%
dplyr::summarise(total_samples = n(), .groups = "drop") %>%
filter(!msfd %in% d_ref$msfd)
To try and include these broad-scale habitats in the assessment we’ll investigate alternative ways to calculate a reference value.
# group samples by SAR categories only and calculate percentiles etc.
samples_SAR_cp2 %>%
as.data.frame() %>%
filter(!is.na(cat_SAR), !msfd %in% d_ref$msfd) %>%
group_by(msfd, cat_SAR) %>%
dplyr::summarise(n_samples = n(),
n_years = length(unique(sampleyear)),
D75 = quantile(D, probs = 0.75),
D95 = quantile(D, probs = 0.95),
D99 = quantile(D, probs = 0.99),
.groups = "drop") %>%
mutate(quality_code = if_else(
(n_samples >= 50) & (n_years >= 3),
1L,
if_else(
(n_samples >= 30) & (n_years >= 2),
2L,
if_else(
n_samples >= 20,
3L,
if_else(
n_samples >= 10,
4L,
5L
)))),
reference = if_else(
(quality_code == 3) & (cat_SAR == "0-0.1"),
D99,
if_else(
(quality_code == 2) & (cat_SAR == "0-0.1"),
D95,
if_else(
(quality_code == 2) & (cat_SAR == "0.1-0.5"),
D99,
if_else(
(quality_code == 1) & (cat_SAR == "0-0.1"),
D75,
if_else(
(quality_code == 1) & (cat_SAR == "0.1-0.5"),
D95,
if_else(
(quality_code == 1) & (cat_SAR == "0.5-1"),
D99,
NA_real_))))))) %>%
mutate(across(starts_with("D"), ~ round(.x, digits = 1))) %>%
mutate(across(starts_with("ref"), ~ round(.x, digits = 1))) %>%
arrange(msfd, cat_SAR) %>%
datatable(rownames = FALSE, colnames = c("Broad-scale habitat" = "msfd"))
There are insufficient samples in ≤1 SAR areas
(quality_code 1-3 dependent on mean SAR) even when not
grouped by CP2 region.
To estimate reference values for these broad-scale habitats we can use the grouped approach outlined for Bh2b in the Greater North Sea (step-2-eda.html#reference-values). This approach uses a combination of average reference values for each sub-region, depth zone, and sediment type
For example, Offshore circalittoral mud has no refer◘ence value, we can estimate a value by using the average reference values of each subgroup:
For Offshore circalittoral mud in the Irish Sea we can estimate by the average of these subgroups: Dref = 12.5 (13.2 + 15.5 + 8.9)/3
# add depth zone and sediment types from BHT descriptions
d_ref <- d_ref %>%
mutate(depth_zone = case_when(str_detect(msfd, "Infralittoral") ~ "Infralittoral",
str_detect(msfd, "Circalittoral") ~ "Circalittoral",
str_detect(msfd, "Offshore") ~ "Offshore circalittoral",
str_detect(msfd, "Upper bathyal") ~ "Upper bathyal",
TRUE ~ "ERROR"),
sediment_type = case_when(str_detect(msfd, "sand") ~ "sand",
str_detect(msfd, "mud") ~ "mud",
str_detect(msfd, "coarse sediment and/or Circalittoral sand") ~
"coarse sediment and/or Circalittoral sand",
str_detect(msfd, "coarse sediment and/or Circalittoral mixed sediment") ~
"coarse sediment and/or mixed sediment",
str_detect(msfd, "coarse sediment and/or Offshore circalittoral mixed sediment") ~
"coarse sediment and/or mixed sediment",
str_detect(msfd, "coarse sediment") ~ "coarse sediment",
str_detect(msfd, "mixed sediment") ~ "mixed sediment",
str_detect(msfd, "bathyal sediment") ~ "sediment",
TRUE ~ "ERROR"))
# Average reference value by cp2 region
region_avg <- d_ref %>%
ungroup() %>%
group_by(Region) %>%
dplyr::summarise(Region_mean = mean(D_reference))
# Average reference value by depth zone
depth_avg <- d_ref %>%
ungroup() %>%
group_by(depth_zone) %>%
dplyr::summarise(depth_mean = mean(D_reference))
# Average reference value by sediment type
sediment_avg <- d_ref %>%
ungroup() %>%
group_by(sediment_type) %>%
dplyr::summarise(sediment_mean = mean(D_reference))
# Combine the averages into a long format
combined_avg <- region_avg %>%
full_join(depth_avg, by = character()) %>%
full_join(sediment_avg, by = character()) %>%
mutate(D_reference_grp_avg = rowMeans(.[c("Region_mean", "depth_mean", "sediment_mean")], na.rm = TRUE),
bht = paste(depth_zone, sediment_type),
D_ref_label = paste0("(", as.character(round(D_reference_grp_avg, 1)), ")"))
# Make a summary of samples in CP2 regions where BHTs references are missing
ref_na <- samples_SAR_cp2 %>%
as.data.frame() %>%
filter(!msfd %in% d_ref$msfd) %>%
group_by(Region, msfd) %>%
dplyr::summarise(samples_n = n(), .groups = "drop")
# Find grouped average reference values for those missing reference values
ref_na_avg <- left_join(ref_na, combined_avg, by = c("msfd" = "bht", "Region" = "Region"))
# Show the additional reference values in a table
ref_na_avg %>%
select(Region, msfd, D_ref_label) %>%
arrange(msfd, Region) %>%
pivot_wider(names_from = Region, values_from = D_ref_label, values_fill = "-") %>%
datatable(
caption = "Grouped Average Margalef Reference Values",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = 'Buttons',
options = list(pageLength = 10,
buttons = c("csv"),
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
# Tidy up
rm(region_avg, depth_avg, sediment_avg, ref_na)
# Select the fields and rename to match
# bind together, populate the final ref value and add factor for plots
d_ref_final_grp <- ref_na_avg %>%
select(Region, msfd, samples_n, depth_zone, sediment_type, D_reference_grp_avg, D_ref_label) %>%
dplyr::rename(D_reference_avg = D_reference_grp_avg) %>%
bind_rows(d_ref_final) %>%
mutate(D_reference_final = if_else(is.na(D_reference_final), D_reference_avg, D_reference_final),
msfd = factor(msfd, levels = BHT_order))
# Tidy up
rm(ref_na_avg, d_ref_final, combined_avg)
# Remove "Upper bathyal sediment" insufficient data to calculate a reference
d_ref_final_grp <- d_ref_final_grp %>%
filter(!is.na(D_reference_final))
# Drop levels no in the final reference groups
d_ref_final_grp <- droplevels(d_ref_final_grp)
# Modify the long BHT labels for better display in plots
d_ref_final_plot <- as.data.frame(d_ref_final_grp)
levels(d_ref_final_plot$msfd)[levels(d_ref_final_plot$msfd)=="Offshore circalittoral coarse sediment and/or Offshore circalittoral mixed sediment"] <- "Offshore circalittoral coarse and/or mixed sediment"
levels(d_ref_final_plot$msfd)[levels(d_ref_final_plot$msfd)=="Circalittoral coarse sediment and/or Circalittoral mixed sediment"] <- "Circalittoral coarse and/or mixed sediment"
# ad Region order
d_ref_final_plot <- d_ref_final_plot %>%
mutate(Region = factor(Region, levels = region_order))
# Show the final ref D in a column plot
d_ref_final_plot %>%
select(msfd, Region, D_reference_final, D_ref_label) %>%
ggplot() +
geom_col(mapping = aes(x = msfd, y = D_reference_final, fill = msfd), alpha = 1) +
scale_fill_viridis_d() +
scale_y_continuous(breaks = seq(0, 25, 5), limits = c(0, 26), name = bquote(D[ref])) +
geom_text(
aes(x = msfd, y = D_reference_final, label = D_ref_label),
position = position_dodge(0.9),
vjust = - 0.3,
size = 3) +
facet_wrap(~ Region) +
theme_bw() +
labs(title = "Reference Margalef for broad-scale habitat and Charting Progress 2 Region") +
theme(legend.position = "bottom",
axis.title.x = element_blank(),
axis.text.x = element_blank(),
legend.text = element_text(size = 7)) +
guides(fill = guide_legend(nrow = 5, byrow = FALSE, title = NULL, reverse = TRUE)) +
scale_x_discrete(limits = rev(levels(d_ref_final_plot$msfd)))
#scale_x_discrete(limits = filtered_levels)
# Show reference vales in table, parentheses indicate averages
d_ref_final_grp %>%
select(msfd, Region, D_ref_label) %>%
group_by(msfd, Region) %>%
arrange(desc(msfd), Region) %>%
pivot_wider(names_from = Region, values_from = D_ref_label, values_fill = "-") %>%
datatable(
caption = "Reference Margalef for Broad-scale Habitat and Charting Progress 2 Region",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = "Buttons",
escape = FALSE,
rownames = FALSE,
options = list(dom = "Blfrtip", buttons = c("csv")))
# Tidy up
rm(d_ref_final_plot)
Here we will investigate sample Margalef diversity (DM) and the reference Margalef values (Dref) in relation to the reported SAR values.
# Find the BHT factor levels present in the data (to help with plotting)
filtered_levels <- intersect(BHT_order, levels(d_ref_final_grp$msfd))
# Join D ref to samples
samples_SAR_ref <- left_join(samples_SAR_cp2, d_ref_final_grp, by = c("msfd" = "msfd", "Region" = "Region")) %>%
mutate(Region = factor(Region, levels = region_order),
msfd = factor(msfd, level = filtered_levels))
# Make a reference label
label <- samples_SAR_ref %>%
as.data.frame() %>%
filter(!is.na(D_reference_final)) %>%
group_by(Region, msfd) %>%
dplyr::summarise(D_ref_label = first(D_ref_label),
D_reference_final = max(D_reference_final), .groups = "drop")
# Modify the long BHT labels/levels for better display in plots
samples_SAR_plot <- samples_SAR_ref %>%
as.data.frame()
levels(samples_SAR_plot$msfd)[levels(samples_SAR_plot$msfd)=="Offshore circalittoral coarse sediment and/or Offshore circalittoral mixed sediment"] <- "Offshore circalittoral coarse and/or mixed sediment"
levels(samples_SAR_plot$msfd)[levels(samples_SAR_plot$msfd)=="Circalittoral coarse sediment and/or Circalittoral mixed sediment"] <- "Circalittoral coarse and/or mixed sediment"
levels(label$msfd)[levels(label$msfd)=="Offshore circalittoral coarse sediment and/or Offshore circalittoral mixed sediment"] <- "Offshore circalittoral coarse and/or mixed sediment"
levels(label$msfd)[levels(label$msfd)=="Circalittoral coarse sediment and/or Circalittoral mixed sediment"] <- "Circalittoral coarse and/or mixed sediment"
# Infralittoral box plot with low SAR categories ("0" SAR removed)
samples_SAR_plot %>%
filter(!is.na(D_reference_final), grepl("Infralittoral", msfd), !is.na(cat_SAR)) %>%
group_split(Region, .keep = T) %>%
walk(function(x) {
g <- x %>%
ggplot(aes(x = cat_SAR, y = D)) +
geom_boxplot() +
geom_jitter(color = "red", size = 0.6, alpha = 0.3) +
scale_y_continuous(breaks = seq(0, 25, 5)) +
geom_hline(aes(yintercept = D_reference_final), color = "blue", linetype="dashed") +
geom_text(data = (filter(label, Region %in% x$Region,
grepl("Infralittoral", msfd))),
aes(x = 1, y = D_reference_final + 2,
label = D_ref_label),
colour = "blue",
size = 4) +
facet_wrap(~ msfd, ncol = 2) +
ggtitle(paste0(x$Region, " infralittoral"), subtitle = "Sample Margalef (2009 to 2021)") +
labs(x = "SAR Category", y = bquote(D[M]))
print(g)
})
# Circalittoral box plot with low SAR categories ("0" SAR removed)
samples_SAR_plot %>%
filter(!is.na(D_reference_final), grepl("Circalittoral", msfd), !is.na(cat_SAR)) %>%
group_split(Region, .keep = T) %>%
walk(function(x) {
g <- x %>%
ggplot(aes(x = cat_SAR, y = D)) +
geom_boxplot() +
geom_jitter(color = "red", size = 0.6, alpha = 0.3) +
scale_y_continuous(breaks = seq(0, 25, 5)) +
geom_hline(aes(yintercept = D_reference_final), color = "blue", linetype="dashed") +
geom_text(data = (filter(label, Region %in% x$Region,
grepl("Circalittoral", msfd))),
aes(x = 1, y = D_reference_final + 2,
label = D_ref_label),
colour = "blue",
size = 4) +
facet_wrap(~ msfd, ncol = 2) +
ggtitle(paste0(x$Region, " circalittoral"), subtitle = "Sample Margalef (2009 to 2021)") +
labs(x = "SAR Category", y = bquote(D[M]))
print(g)
})
# Offshore box plot with low SAR categories ("0" SAR removed)
samples_SAR_plot %>%
filter(!is.na(D_reference_final), grepl("Offshore", msfd), !is.na(cat_SAR)) %>%
group_split(Region, .keep = T) %>%
walk(function(x) {
g <- x %>%
ggplot(aes(x = cat_SAR, y = D)) +
geom_boxplot() +
geom_jitter(color = "red", size = 0.6, alpha = 0.3) +
scale_y_continuous(breaks = seq(0, 25, 5)) +
geom_hline(aes(yintercept = D_reference_final), color = "blue", linetype="dashed") +
geom_text(data = (filter(label, Region %in% x$Region,
grepl("Offshore", msfd))),
aes(x = 1, y = D_reference_final + 2,
label = D_ref_label),
colour = "blue",
size = 4) +
facet_wrap(~msfd, ncol = 2) +
ggtitle(paste0(x$Region, " offshore circalittoral"), subtitle = "Sample Margalef (2009 to 2021)") +
labs(x = "SAR Category", y = bquote(D[M]))
print(g)
})
rm(label, samples_SAR_plot)
So far we have calculated sample diversity (DM) and broad-scale habitat reference values (Dref, using the percentile values from ‘low pressure’ areas). However, to make an assessment of benthic habitat quality status between habitats for the assessment period (2009 to 2021) a ‘relative Margalef index of diversity’ needs to be given. In other publications, this measure is referred to as ecological quality ratio (EQR).
The Relative Margalef Diversity (DM’) is given as:
\(\displaystyle D_{M}{'} = \frac{D_{ass}-D_{bad}}{D_{ref}-D_{bad}}\)
Dass = Assessed Margalef diversity (sample DM)
Dref = Reference Margalef diversity for good ecological state
Dbad = Margalef diversity for bad ecological state
For Margalef, bad ecological state equals 0, therefore DM’ can be simplified to:
\(\displaystyle D_{M}{'} = \frac{D_{ass}}{D_{ref}}\)
D_rel) to the sample
data# Calculate relative D with the benthos package (i.e. D/ref, but we'll use the benthos package for consistency)
samples_SAR_rel <- samples_SAR_ref %>%
filter(!is.na(D_reference_final)) %>%
mutate(D_rel = eqr(D, bad = 0, ref = D_reference_final))
As reference values are calculated from percentile values it is not unusual for relative Margalef values (DM’) to be greater than 1 (reference status)
D_rel_greater <- samples_SAR_rel %>%
as.data.frame() %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021")) %>%
filter(D_rel >1) %>%
group_by(msfd, year_cat) %>%
dplyr::summarise(D_rel_1 = n(), .groups = "drop")
samples_SAR_rel %>%
as.data.frame() %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021")) %>%
group_by(msfd, year_cat) %>%
dplyr::summarise(n = n(), .groups = "drop") %>%
left_join(D_rel_greater, by = c("msfd", "year_cat")) %>%
filter(!is.na(D_rel_1)) %>%
mutate(DM_over1 = scales::percent((D_rel_1/n), accuracy = 0.1)) %>%
arrange(desc(msfd)) %>%
datatable(
colnames = c("Broad-scale habitat" = "msfd", `D<sub>M</sub>' > 1` = "DM_over1",
"total n" = "n", `n > 1` = "D_rel_1"),
extensions = "Buttons",
caption = paste0("Percentage of samples with sample Margalef greater than reference Margalef"),
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Bfrtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
escape = FALSE,
class = "cell-border",
rownames = FALSE)
# BHT across regions
samples_SAR_rel %>%
as.data.frame() %>%
ggplot(aes(x = D_rel, colour = msfd, fill = msfd)) +
geom_histogram(alpha = 0.6, position = 'identity',
breaks = seq(from = 0, to = 2.8, by = 0.05)) +
scale_colour_viridis_d() +
scale_fill_viridis_d() +
geom_vline(xintercept = 1, colour = "red", linetype = "dashed", alpha = 0.5) +
scale_y_continuous(name = "frequency") +
scale_x_continuous(name = bquote("D"[M]*"'"), breaks = seq(0, 2.5, 0.5)) +
theme_minimal() +
ggtitle("Relative Margalef in each broad-scale habitat") +
theme(legend.position="none",
panel.spacing = unit(0.1, "lines"),
strip.text.x = element_text(size = 8),
axis.text.x = element_text(size = 7)) +
facet_wrap(~msfd, ncol = 5, scales = "free_y", labeller = label_wrap_gen())
# Regions despite BHTs
samples_SAR_rel %>%
as.data.frame() %>%
ggplot(aes(x = D_rel, colour = Region, fill = Region)) +
geom_histogram(alpha=0.6, position = 'identity',
breaks = seq(from = 0, to = 2.8, by = 0.05)) +
geom_vline(xintercept = 1, colour = "red", linetype = "dashed", alpha = 0.5) +
scale_y_continuous(name = "frequency") +
scale_x_continuous(name = bquote("D"[M]*"'"), breaks = seq(0, 2.5, 0.5)) +
theme_minimal() +
ggtitle("Relative Margalef by CP2 region") +
theme(legend.position="none", panel.spacing = unit(0.1, "lines"), strip.text.x = element_text(size = 8)) +
facet_wrap(~Region, scales = "free_y", labeller = label_wrap_gen())
Now check for any variations in DM’ that may relate to the underlying metadata
# Load in the processed aggregated sample data (from the previous script)
benthos_t_agg <- readRDS("Data/Processed/R_data/C20230906_benthos_t_agg")
# Take the metadata variables needed and reduce to samples (currently has species records)
benthos_meta <- benthos_t_agg %>%
as.data.frame() %>%
select(sampleref, geartype, au, dataset) %>%
unique()
# add the NRW sample details
NRW_meta <- samples_NRW_bht %>%
as.data.frame() %>%
select(sampleref, au) %>%
mutate(geartype = "Day grab", # Note, NRW were mostly Day but not all
dataset = "NRW")
benthos_meta_all <- bind_rows(benthos_meta, NRW_meta)
# Join to the relative margalef sample data
samples_rel_meta <- left_join(samples_SAR_rel, benthos_meta_all, by = c("sampleref" = "sampleref"))
# Tidy up
rm(benthos_t_agg, benthos_meta, NRW_meta)
Plot DM’ against some of the metadata variables. Note, points are grouped by year and not strictly plotted on a continuous date scale (their horizontal position above the given year is not related to the time of year collected), this is to avoid all the points overlapping when taken on the same date. Dashed line added to distinguish samples collected prior 2016.
# Modify the long BHT labels for better display in plots
samples_meta <- samples_rel_meta %>%
as.data.frame()
levels(samples_meta$msfd)[levels(samples_meta$msfd)=="Offshore circalittoral coarse sediment and/or Offshore circalittoral mixed sediment"] <- "Offshore circalittoral coarse and/or mixed sediment"
levels(samples_meta$msfd)[levels(samples_meta$msfd)=="Circalittoral coarse sediment and/or Circalittoral mixed sediment"] <- "Circalittoral coarse and/or mixed sediment"
# Gear type plots
samples_meta %>%
mutate(D_rel = if_else(D_rel >1, 1, D_rel),
geartype = case_when(grepl("hamon", geartype, ignore.case = TRUE) ~ "Hamon grab",
grepl("day", geartype, ignore.case = TRUE) ~ "Day grab",
grepl("veen", geartype, ignore.case = TRUE) ~ "Van Veen grab",
TRUE ~ geartype)) %>%
group_split(geartype) %>%
walk(function(x) {
g <- x %>%
ggplot() +
geom_vline(xintercept = 2015.5, colour = "white", linetype = "dashed", alpha = 0.5) +
geom_jitter(mapping = aes(x = sampleyear, y = Region, colour = D_rel), size = 0.5) +
scale_colour_viridis(begin = 0, end = 1,
name = bquote("D"[M]*"'"),
breaks = 0:5 / 5,
labels = c(0:4 / 5, "\u2265 1"),
limits = c(0, 1)) +
scale_y_discrete(name = "") +
scale_x_continuous(limits = c(2008, 2021), breaks = seq(2008, 2021, 4)) +
labs(title = x$geartype, x = "Year") +
facet_wrap(vars(msfd), labeller = label_wrap_gen()) +
theme_dark() +
theme(legend.position="right", panel.spacing = unit(0.1, "lines"),
strip.text.x = element_text(size = 8, margin = margin(0.1,0.1,0.1,0.1)))
print(g)
})
# CP2 region and BHT
samples_meta %>%
mutate(D_rel = if_else(D_rel >1, 1, D_rel)) %>%
group_split(Region) %>%
walk(function(x) {
g <- x %>%
ggplot() +
geom_vline(xintercept = 2015.5, colour = "white", linetype = "dashed", alpha = 0.5) +
geom_jitter(mapping = aes(x = sampleyear, y = Region, colour = D_rel), size = 0.5) +
scale_colour_viridis(begin = 0, end = 1,
name = bquote("D"[M]*"'"),
breaks = 0:5 / 5,
labels = c(0:4 / 5, "\u2265 1"),
limits = c(0, 1)) +
scale_y_discrete(name = "", limits = rev, labels = NULL) +
scale_x_continuous(limits = c(2008, 2021), breaks = seq(2008, 2021, 4)) +
labs(title = x$Region, x = "Year")+
theme_dark() +
facet_wrap(vars(msfd), labeller = label_wrap_gen()) +
theme(legend.position="none", panel.spacing = unit(0.1, "lines"),
strip.text.x = element_text(size = 8, margin = margin(0.1,0.1,0.1,0.1)))
print(g)
})
# Dataset
samples_meta %>%
as.data.frame() %>%
mutate(D_rel = if_else(D_rel >1, 1, D_rel)) %>%
group_split(Region) %>%
walk(function(x) {
g <- x %>%
ggplot() +
geom_vline(xintercept = 2015.5, colour = "white", linetype = "dashed", alpha = 0.5) +
geom_jitter(mapping = aes(x = sampleyear, y = dataset, colour = D_rel), size = 0.5) +
scale_colour_viridis(begin = 0, end = 1,
name = bquote("D"[M]*"'"),
breaks = 0:5 / 5,
labels = c(0:4 / 5, "\u2265 1"),
limits = c(0, 1)) +
scale_y_discrete(name = "") +
scale_x_continuous(limits = c(2008, 2021), breaks = seq(2008, 2021, 4)) +
labs(title = paste0("Dataset source in ", x$Region),
subtitle = "Marine Recorder (MR), OneBenthic (OB), MERMAN, NRW or EA WFD (WFD)", x = "Year") +
theme_dark() +
facet_wrap(vars(msfd), labeller = label_wrap_gen()) +
theme(legend.position="none", panel.spacing = unit(0.1, "lines"),
strip.text.x = element_text(size = 8, margin = margin(0.1,0.1,0.1,0.1)))
print(g)
})
# Add year groups and limit relative D to 1 for plots
year_rel <- samples_SAR_rel %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021"),
D_rel = if_else(D_rel > 1, 1, D_rel))
# plot relative D for all samples
year_rel %>%
mutate(D_rel = if_else(D_rel >1, 1, D_rel)) %>%
ggplot() +
geom_sf(data = land_cs, fill = "grey", colour = NA) +
geom_sf(data = cp2, aes(fill = Region), colour = NA, alpha = 0.3) +
geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
scale_fill_manual(values = cp2_palette, name = "CP2 Region") +
new_scale("fill") +
geom_sf(mapping = aes(geometry = geometry, colour = D_rel), size = 0.9) +
scale_colour_viridis(begin = 0, end = 1,
name = bquote("D"[M]*"'"),
breaks = 0:5 / 5,
labels = c(0:4 / 5, "\u2265 1"),
limits = c(0, 1)) +
ggtitle("Relative Margalef 2009 to 2021") +
theme_bw() +
ylim(48.19, 63.4) +
xlim(-14.21, 1.1)
You can view and filter by year all relative Margalef values (DM’) using this interactive map of results:
The median of relative Margalef (DM‘) for each assessment group (CP2 region, broad-scale habitat, and assessment period) is calculated as the indicator of benthic habitat quality status. To avoid confusion with relative Margalef (DM’) we will use the term aggregated Margalef (DMagg) to label these median values.
The DMagg is estimated as follows:
Note: This qualification definitely is not related to threshold values (TVs) for Good Environmental Status (GES)
— OSPAR CEMP guidelines
Confidence scores are based on the number of years and samples covered by each assessment group (habitat and assessment period). These confidence scores are defined as follows;
| Years | Samples (n) | Years (n) | Confidence |
|---|---|---|---|
| 2016 to 2021 | ≥50 | ≥3 | High |
| ≥30 | ≥2 | Good | |
| ≥20 | 1 | Sufficient | |
| ≥10 & <20 | - | Low | |
| <10 | - | Poor | |
| 2009 to 2015 | ≥50 | ≥3 | Good |
| ≥30 | ≥2 | Sufficient | |
| ≥20 | 1 | Low | |
| <20 | - | Poor |
Note; confidence scores were developed for the assessment of BH2 in the Greater North Sea, they are used here following the OSPAR CEMP guidelines. However, the number of samples/years considered to give equivalent confidence for a higher aggregation (Greater North Sea assessment included additional groups for country) might need to be considered.
Confidence of spatial representativity is estimated in an additional Spatial Representation script
# Aggregate relative Margalef by CP2 region, BHT, and Year groups
d_assessment <- samples_SAR_rel %>%
as.data.frame() %>%
mutate(coords = as.character(geometry),
year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021")) %>%
select(-c(geometry)) %>%
group_by(Region, msfd, sampledate, coords, year_cat) %>%
dplyr::summarise(n = n(), D_rel = median(D_rel), .groups = "drop") %>%
group_by(Region, msfd, year_cat) %>%
dplyr::summarise(n = n(), D_agg = median(D_rel), .groups = "drop")
# Determine the number of years sampled for each habitat
years_habitat <- samples_SAR_rel %>%
as.data.frame() %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021")) %>%
select(-c(geometry)) %>%
group_by(Region, msfd, year_cat) %>%
dplyr::summarise(n_years = length(unique(sampleyear)), .groups = "drop")
# Summarise relative diversity and confidence for assessment groups
d_assessment_conf <- d_assessment %>%
left_join(y = years_habitat, by = c("msfd", "year_cat", "Region")) %>%
mutate(D_agg = round(D_agg, 2),
state = case_when(D_agg <= 0.6 ~ "Low Diversity",
D_agg <= 0.8 ~ "Moderate Diversity",
TRUE ~ "High Diversity"),
confidence = case_when(n >= 50 & year_cat == "2016-2021" & n_years >=3 ~ "High",
n >= 50 & year_cat == "2009-2015" & n_years >=2 ~ "Good",
n >= 30 & year_cat == "2016-2021" & n_years >=2 ~ "Good",
n >= 30 & year_cat == "2009-2015" & n_years >=2 ~ "Sufficient",
n >= 20 & year_cat == "2016-2021" ~ "Sufficient",
n >= 20 & year_cat == "2009-2015" ~ "Low",
n >= 10 & year_cat == "2016-2021" ~ "Low",
TRUE ~ "Poor")) %>%
arrange(desc(msfd), Region)
# show in table
d_assessment_conf %>%
datatable(
caption = "Median values of relative Margalef diversity (D M agg)",
colnames = c("Broad-scale habitat" = "msfd", "Period" = "year_cat",
"Samples (n)" = "n", "D M agg" = "D_agg", "Year (n)" = "n_years"),
extensions = "Buttons",
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Bfrtip',
columnDefs = list(list(className = 'dt-center', targets = 1:7))),
class = "cell-border",
rownames = FALSE)
Here we will summarise the BH2b results for the Celtic Seas in an equivalent format to those reported for the Greater North Sea in the QSR BH2b assessment. Where data is available assessments are based on 2016-2021 samples with sufficient or better confidence, otherwise results represent 2009-2015 data with sufficient or better confidence.
In case of lack of data, older data can be used to do assessment of the current quality status on best available data, however with lower confidence… with a (one category) lower confidence.
— OSPAR CEMP guidelines
# Split the assessment periods
# 2009-2015
assessment_a <- d_assessment_conf %>%
filter(year_cat == "2009-2015") %>%
select(Region, msfd, D_agg_a = D_agg, confidence_a = confidence, n_a = n)
# 2016-2021
assessment_b <- d_assessment_conf %>%
filter(year_cat == "2016-2021") %>%
select(Region, msfd, D_agg_b = D_agg, confidence_b = confidence, n_b = n)
# Join across common fields
assessment_ukms <- full_join(assessment_a, assessment_b, by = c("Region", "msfd"))
# Tidy up
rm(assessment_a, assessment_b)
# Show only confidence sufficient or better results (use values from sufficient or better period)
assessment_ukms_sufficient <- assessment_ukms %>%
filter(grepl("High|Good|Sufficient", confidence_a)|grepl("High|Good|Sufficient", confidence_b)) %>%
mutate(D_agg = if_else(is.na(D_agg_b), D_agg_a,
if_else(grepl("High|Good|Sufficient", confidence_b), D_agg_b, D_agg_a)),
period = if_else(is.na(D_agg_b), "2009-2015",
if_else(grepl("High|Good|Sufficient", confidence_b), "2016-2021", "2009-2015")),
confidence = if_else(is.na(D_agg_b), confidence_a,
if_else(grepl("High|Good|Sufficient", confidence_b), confidence_b, confidence_a)),
n = if_else(is.na(n_b), n_a,
if_else(grepl("High|Good|Sufficient", confidence_b), n_b, n_a)))
# Wide format final D agg
assessment_ukms_sufficient %>%
select(Region, msfd, D_agg) %>%
pivot_wider(
names_from = Region,
values_from = D_agg) %>%
datatable(
caption = "Median values of relative Margalef diversity",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Brtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
# Wide form assessment period
assessment_ukms_sufficient %>%
select(Region, msfd, period) %>%
pivot_wider(
names_from = Region,
values_from = period) %>%
datatable(
caption = "Assessment period used for 'current' quality status",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Brtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
# wide format number of samples
assessment_ukms_sufficient %>%
select(Region, msfd, n) %>%
pivot_wider(
names_from = Region,
values_from = n) %>%
datatable(
caption = "Samples (n) for assessment period",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Brtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
# Wide format confidence
assessment_ukms_sufficient %>%
select(Region, msfd, confidence) %>%
pivot_wider(
names_from = Region,
values_from = confidence) %>%
datatable(
caption = "Confidence of current quality status",
colnames = c("Broad-scale habitat" = "msfd"),
extensions = 'Buttons',
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Brtip',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
samples_SAR_rel %>%
as.data.frame() %>%
mutate(Region = factor(Region, levels = region_order)) %>%
ggplot() +
geom_histogram(data = samples_SAR_rel, aes(x = sampledate, colour = Region, fill = Region), bins = 12) +
scale_y_continuous(name = "Sample Frequency") +
scale_x_date(date_breaks = "2 year", date_labels = "%Y", name = "") +
scale_color_manual(values = cp2_palette) +
scale_fill_manual(values = cp2_palette) +
theme_bw() +
theme(legend.position="none", panel.spacing = unit(0.1, "lines"), strip.text.x = element_text(size = 8)) +
facet_wrap(~Region)
# Modify the long BHT labels for better display in plots
sample_freq <- samples_SAR_rel %>%
as.data.frame()
levels(sample_freq$msfd)[levels(sample_freq$msfd)=="Offshore circalittoral coarse sediment and/or Offshore circalittoral mixed sediment"] <- "Offshore circalittoral coarse and/or mixed sediment"
levels(sample_freq$msfd)[levels(sample_freq$msfd)=="Circalittoral coarse sediment and/or Circalittoral mixed sediment"] <- "Circalittoral coarse and/or mixed sediment"
# Plot frequency of sample collection
sample_freq %>%
mutate(Region = factor(Region, levels = region_order)) %>%
group_split(Region) %>%
walk(function(x) {
p <- x %>%
ggplot() +
geom_histogram(data =, aes(sampledate, fill = msfd), bins = 12) +
scale_fill_viridis_d() +
scale_y_continuous(name = "Sample Frequency",
limits = c(0, 200, 50)) +
scale_x_date(date_breaks = "2 year",
date_labels = "%y",
limits = as.Date(c("2007-12-30", "2022-01-01")),
name = "Year ('YY)") +
theme_bw() +
theme(legend.position="none", panel.spacing = unit(0.1, "lines"), strip.text.x = element_text(size = 7,
margin = margin(0.1,0.1,0.1,0.1))) +
ggtitle(x$Region) +
facet_wrap(~ msfd, ncol = 4, labeller = label_wrap_gen())
print(p)
})
For the Greater North Sea the results (DM‘) of the three assessment periods (1998-2008, 2009-2015, and 2016-2020) were tested for significant differences. A two-sided independent t-testing at p<0,05 was applied for all combinations of assessment periods for each case specific aggregated Relative Margalef diversity (mean DM’ for country, assessment unit, and broad-scale habitat).
We will investigate if it’s possible to determine any significant difference of DM’ values between 2009-2015 and 2016-2021 in the Celtic Seas.
# plot frequency of Relative DM' for each habitat and region
samples_SAR_rel %>%
as.data.frame() %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021"),
msfd = factor(msfd, levels = rev(levels(msfd)))) %>%
group_split(Region, .keep = T) %>%
walk(function(x) {
g <- x %>%
ggplot() +
geom_histogram(aes(x = D_rel, fill = year_cat),
alpha = 0.6, position = 'identity',
binwidth = 0.05) +
scale_x_continuous(name = bquote("D"[M]*"'"), breaks = seq(0, 2.5, 1)) +
scale_y_continuous(breaks = seq(0, 250, 10), name = NULL) +
scale_fill_manual(values=c("#440154FF", "#5DC863FF")) +
theme_bw(base_size = 9) +
theme(legend.position = "bottom",
axis.text.y=element_blank(),
axis.ticks.y = element_blank(),
panel.spacing = unit(0.1, "lines")) +
guides(fill = guide_legend(title = NULL)) +
ggtitle(paste0(x$Region, " frequency of relative sample Margalef")) +
facet_wrap(~msfd, scales = "free_y",
labeller = label_wrap_gen(width = 48))
print(g)
})
# Plot by habitat (regions combined)
samples_SAR_rel %>%
as.data.frame() %>%
mutate(year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021"),
msfd = factor(msfd, levels = rev(levels(msfd)))) %>%
ggplot() +
geom_histogram(aes(x = D_rel, fill = year_cat),
alpha = 0.6, position = 'identity',
binwidth = 0.05) +
scale_x_continuous(name = bquote("D"[M]*"'"), breaks = seq(0, 2.5, 1)) +
scale_y_continuous(breaks = seq(0, 250, 10), name = NULL) +
scale_fill_manual(values=c("#440154FF", "#5DC863FF")) +
theme_bw(base_size = 9) +
theme(legend.position = "bottom",
axis.text.y=element_blank(),
axis.ticks.y = element_blank(),
panel.spacing = unit(0.1, "lines")) +
guides(fill = guide_legend(title = NULL)) +
ggtitle("Frequency of relative sample Margalef across all regions") +
facet_wrap(~msfd, scales = "free_y",
labeller = label_wrap_gen(width = 48))
As the values of DM’ in the Celtic Seas do not meet the criteria for a t-test (normal distribution and that the variances of the two groups are equal) and we would prefer to compare the medians rather than the means we will use Mann-Whitney U test (also known as the Wilcoxon rank-sum test). This non-parametric test allows us to compare the distributions of two independent groups without assuming normality or equal sample sizes.
# group replicates first (median for reps)
samples_SAR_rel_no_rep <- samples_SAR_rel %>%
as.data.frame() %>%
mutate(coords = as.character(geometry),
year_cat = if_else(sampleyear <2016, "2009-2015", "2016-2021")) %>%
select(-c(geometry)) %>%
group_by(Region, msfd, sampledate, coords, year_cat) %>%
dplyr::summarise(n = n(), D_rel = median(D_rel), .groups = "drop")
# Select the fields needed
samples_SAR_rel_df <- samples_SAR_rel_no_rep %>%
as.data.frame() %>%
select(msfd, year_cat, D_rel, Region)
# Find those habitats that have samples in both periods
filtered_data <- samples_SAR_rel_df %>%
group_by(year_cat, msfd) %>%
count() %>%
group_by(msfd) %>%
count() %>%
filter(n == 2)
# filter data to those habitats
samples_SAR_rel_df_bht <- samples_SAR_rel_df %>%
mutate(msfd = as.character(msfd)) %>%
filter(msfd %in% filtered_data$msfd)
# Create a function
run_wilcox_test <- function(data, response_variable) {
categories <- unique(data$msfd)
results <- lapply(categories, function(category) {
subset_data <- subset(data, msfd == category)
subset_2009_2015 <- subset_data[subset_data$year_cat == "2009-2015", response_variable]
subset_2016_2021 <- subset_data[subset_data$year_cat == "2016-2021", response_variable]
wilcox_result <- wilcox.test(subset_2009_2015, subset_2016_2021, paired = FALSE)
})
names(results) <- categories
return(results)
}
# set variable to test
response_variable <- "D_rel"
# Call the function to run the test
test_results <- run_wilcox_test(samples_SAR_rel_df_bht, response_variable)
# Access and show the individual test results where significant (<0.05)
for (msfd_value in names(test_results)) {
p_value <- test_results[[msfd_value]]$p.value
if (p_value < 0.05) {
cat("BHT:", msfd_value, "\n")
cat("p-value:", p_value, "\n")
cat("\n")
}
}
## BHT: Offshore circalittoral sand
## p-value: 0.0175917
##
## BHT: Offshore circalittoral coarse sediment
## p-value: 1.828577e-06
##
## BHT: Circalittoral coarse sediment
## p-value: 0.000404195
##
## BHT: Infralittoral sand
## p-value: 0.006174184
##
## BHT: Circalittoral mud
## p-value: 0.02763441
##
## BHT: Circalittoral sand
## p-value: 0.0001085262
# Create an empty data frame to store the results
result_df <- data.frame(bht = character(),
statistic = numeric(),
p_value = numeric(),
stringsAsFactors = FALSE)
# Iterate over each element in the results list
for (i in seq_along(test_results)) {
# Extract the values for each test
bht <- names(test_results)[i]
statistic <- test_results[[i]]$statistic
p_value <- test_results[[i]]$p.value
# Create a temporary data frame for each result
temp_df <- data.frame(bht = bht,
statistic = statistic,
p_value = round(p_value, 7),
stringsAsFactors = FALSE)
# Append the temporary data frame to the main result data frame
result_df <- rbind(result_df, temp_df)
}
# summarise medians of DM' tested
tested_medians <- samples_SAR_rel_df_bht %>%
group_by(msfd, year_cat) %>%
summarise(median_D = median(D_rel),
obs = n()) %>%
arrange(msfd, desc(year_cat)) %>%
mutate(period_median_D = paste0(round(median_D, 2), " (", obs, ")")) %>%
mutate(tested = paste0(period_median_D, " - ", lag(period_median_D, k = 1))) %>%
filter(year_cat == "2009-2015") %>%
select(msfd, tested)
# Show significant results alongside median DM' for each habitat/assessment period
result_df %>%
filter(p_value <0.05) %>%
left_join(tested_medians, by = c("bht" = "msfd")) %>%
arrange(bht) %>%
datatable(caption = "Significant differences between median relative Marglef diversity (Wilcoxon rank-sum test)",
colnames = c("Broad-scale habitat" = "bht",
"Relative D 2009-2015 & 2016-2021 (n)" = "tested"),
extensions = "Buttons",
options = list(pageLength = 10,
buttons = c("csv"),
dom = 'Brti',
columnDefs = list(list(className = 'dt-center', targets = 1:3))),
class = "cell-border",
rownames = FALSE)
# tidy up
rm(test_results, tested_medians, temp_df, statistic, p_value, i, response_variable)
Significant differences in the median relative Margalef (DM’) shown for some habitats when considered together across all of the UKMS sub-region. However, the results need to be considered with relatively low number of observations in some of the groups and the spatial variation in sample locations.
# Find those habitats that have samples in both periods and regions
filtered_data <- samples_SAR_rel_df %>%
group_by(year_cat, msfd, Region) %>%
count() %>%
group_by(msfd, Region) %>%
count() %>%
filter(n == 2)
# filter data to those habitats in those regions
samples_SAR_rel_df_bht <- inner_join(samples_SAR_rel_df_bht, filtered_data, by = c("msfd", "Region")) %>%
mutate(msfd = as.character(msfd))
# Create a function to include region too
run_wilcox_test_region <- function(data, response_variable) {
combinations <- unique(data[, c("msfd", "Region")])
results <- lapply(1:nrow(combinations), function(i) {
subset_data <- subset(data, msfd == combinations$msfd[i] & Region == combinations$Region[i])
subset_2009_2015 <- subset_data[subset_data$year_cat == "2009-2015", response_variable]
subset_2016_2021 <- subset_data[subset_data$year_cat == "2016-2021", response_variable]
wilcox_result <- wilcox.test(subset_2009_2015, subset_2016_2021, paired = FALSE)
wilcox_result
})
names(results) <- paste(combinations$msfd, combinations$Region, sep = " - ")
return(results)
}
# set variable to test
response_variable <- "D_rel"
# Call the function to run the test including regions splits
test_results_region <- run_wilcox_test_region(samples_SAR_rel_df_bht, response_variable)
# Create an empty data frame to store the results
result_region_df <- data.frame(cp2_bht = character(),
statistic = numeric(),
p_value = numeric(),
stringsAsFactors = FALSE)
# Iterate over each element in the results list
for (i in seq_along(test_results_region)) {
# Extract the values for each test
cp2_bht <- names(test_results_region)[i]
statistic <- test_results_region[[i]]$statistic
p_value <- test_results_region[[i]]$p.value
# Create a temporary data frame for each result
temp_df <- data.frame(cp2_bht = cp2_bht,
statistic = statistic,
p_value = round(p_value, 7),
stringsAsFactors = FALSE)
# Append the temporary data frame to the main result data frame
result_region_df <- rbind(result_region_df, temp_df)
}
# summarise medians of DM' tested
tested_medians_region <- samples_SAR_rel_df_bht %>%
group_by(Region, msfd, year_cat) %>%
summarise(median_D = median(D_rel),
obs = n()) %>%
arrange(Region, msfd, desc(year_cat)) %>%
mutate(period_median_D = paste0(round(median_D, 2), " (", obs, ")")) %>%
mutate(tested = paste0(period_median_D, " - ", lag(period_median_D, k = 1))) %>%
filter(year_cat == "2009-2015") %>%
select(Region, msfd, tested)
# Show significant results alongside median DM' for each habitat/assessment period
result_region_df %>%
filter(p_value <0.05) %>%
separate(cp2_bht, into = c("bht", "Region"), sep = " - ") %>%
left_join(tested_medians_region, by = c("bht" = "msfd", "Region" = "Region")) %>%
relocate(bht, .after = Region) %>%
arrange(Region, bht) %>%
datatable(caption = "Significant differences between median relative Marglef diversity (Wilcoxon rank-sum test)",
colnames = c("Broad-scale habitat" = "bht",
"Relative D 2009-2015 & 2016-2021 (n)" = "tested"),
extensions = "Buttons",
options = list(pageLength = 15,
buttons = c("csv"),
dom = 'Brti',
columnDefs = list(list(className = 'dt-center', targets = 1:4))),
class = "cell-border",
rownames = FALSE)
# tidy up
rm(test_results_region, tested_medians_region, temp_df, cp2_bht, statistic, p_value, i, response_variable)
Again, some significant differences in the median relative Margalef (DM’) shown for some habitats within CP2 regions. However, as before the results need to be considered with the relatively low number of observations in some of the groups and the spatial variation/aggregation in sample locations.
details::details(sessionInfo(), summary = 'sessionInfo')
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server x64 (build 14393)
Matrix products: default
locale:
[1] LC_COLLATE=English_United Kingdom.1252
[2] LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] viridis_0.6.2 viridisLite_0.4.0 RColorBrewer_1.1-2 benthos_1.3-6
[5] details_0.3.0 DT_0.26 ggnewscale_0.4.8 sf_0.9-8
[9] forcats_0.5.1 stringr_1.4.0 dplyr_1.0.7 purrr_0.3.4
[13] readr_2.1.0 tidyr_1.1.4 tibble_3.1.6 ggplot2_3.3.5
[17] tidyverse_1.3.1 rprojroot_2.0.3
loaded via a namespace (and not attached):
[1] httr_1.4.2 sass_0.4.4 jsonlite_1.7.2 modelr_0.1.8
[5] bslib_0.4.1 assertthat_0.2.1 highr_0.9 cellranger_1.1.0
[9] yaml_2.2.1 pillar_1.6.4 backports_1.3.0 glue_1.5.0
[13] digest_0.6.28 rvest_1.0.2 colorspace_2.0-2 htmltools_0.5.2
[17] clipr_0.7.1 pkgconfig_2.0.3 broom_0.7.10 haven_2.4.3
[21] scales_1.1.1 tzdb_0.2.0 proxy_0.4-27 farver_2.1.0
[25] generics_0.1.1 ellipsis_0.3.2 cachem_1.0.6 withr_2.5.0
[29] lazyeval_0.2.2 cli_3.1.0 magrittr_2.0.1 crayon_1.4.2
[33] readxl_1.3.1 evaluate_0.18 fs_1.5.0 fansi_0.5.0
[37] xml2_1.3.2 class_7.3-15 tools_3.6.1 hms_1.1.1
[41] lifecycle_1.0.1 munsell_0.5.0 reprex_2.0.1 compiler_3.6.1
[45] jquerylib_0.1.4 e1071_1.7-11 rlang_0.4.12 classInt_0.4-3
[49] units_0.8-0 grid_3.6.1 rstudioapi_0.13 htmlwidgets_1.5.4
[53] crosstalk_1.2.0 labeling_0.4.2 rmarkdown_2.18 gtable_0.3.0
[57] DBI_1.1.3 R6_2.5.1 gridExtra_2.3 lubridate_1.8.0
[61] knitr_1.41 fastmap_1.1.0 utf8_1.2.2 KernSmooth_2.23-15
[65] desc_1.4.2 stringi_1.7.5 Rcpp_1.0.10 vctrs_0.3.8
[69] png_0.1-7 dbplyr_2.1.1 tidyselect_1.1.1 xfun_0.35