Updated / printed: 2023-08-16


Introduction

This script details the data preparation methods used to compile sample data suitable to run the Condition of Benthic Communities indicator (BH2b) for the UK EEZ Celtics Seas areas (OSPAR Northern and Southern Celtic Sea assessment units within the UK EEZ). Once data is standardised, stratified to habitat type, and joined with fishing pressure data, Margalef’s index of diversity (DM) is used as an indicator of the benthic habitat quality status. Margalef’s index of diversity (DM) is given by:

\(\displaystyle D_{M} = \frac{S-1}{ln(N)}\)

S = Species richness

N = Total sample abundance

To account for ‘natural’ variability of diversity between different broad-scale habitats the Relative Margalef Diversity (DM’) is used to assess habitat condition:

\(\displaystyle D_{M}{'} = \frac{D_{ass}}{D_{ref}}\)

Dass = Assessed Margalef diversity

Dref = Reference Margalef diversity for good ecological state

Reference values for Margalef diversity are determined from samples in location of low reported fishing pressure and are habitat and area specific (where possible).

The methods used to prepare data and assess Relative Margalef Diversity in the Celtic Seas areas will mostly follow those used for BH2b in OSPAR Region II for the QSR 2023 (Van Loon et. al., in prep).

Data Preparation Part 1

Analysis was primarily undertaken using R version 3.6.1. See Session Info for full details.

Area of Interest

The ‘North Celtic Sea’ and ‘South Celtic Sea’ OSPAR assessment units were clipped to the UK EEZ as part of the MSPACE project. The WGS84 Shapefile of these two assessment was saved to the ‘Data/Processed/Spatial’ folder and will be used to clip available sample data for the Celtic Seas assessment.

  • Load assessment unit layer
# Load UKMS assessment units 
ukms <- st_read("Data/Raw/UKMS/C20230620_UKMS_final.shp")
# fix geometry
ukms <- st_make_valid(ukms)
# filter to Celtic Seas
ukms_cs <- filter(ukms, UKMS_SubRe == "Celtic Seas")
ukms_cs_2 <- filter(ukms, grepl("Celtic Seas", UKMS_SubRe)) # including 'seabed & sub-soil only'

# Load in OSPAR assessment units clipped to the UK EEZ for the Celtic Seas 
celtic_sea_au <- st_read("Data/Processed/Spatial/C20221212_Celtic_Sea_Assessmen_Units.shp")
# fix geometry
celtic_sea_au <- st_make_valid(celtic_sea_au)

# add abbreviation for assessment units and only keep neccessary fields 
celtic_sea_au <- celtic_sea_au %>%
  mutate(AU = case_when(Label == 'Southern Celtic Sea' ~ 'SCS',
                        Label == 'Northern Celtic Sea' ~ 'NCS',
                        TRUE ~ 'error')) %>%
  select(AU, Label)

Areas are additionally divided by the Charting Progress Regions (CP2). These regions have been drafted on the basis of key environmental characteristics, grouping samples by these regions will help avoid some of the biogeographic variations inherent across the whole Celtic Seas area (JNCC, 2022).

  • Load in the CP2 layer
  • Select the Regions in the Celitic Seas
# load in CP2 layer
cp2_regions <- st_read("Data/Raw/CP2_Regions/C071222_CP2_reporting_regions.shp")

# select the regions needed
cp2 <- cp2_regions %>%
  filter(Region == "Atlantic North-West Approaches, Rockall Trough and Faroe-Shetland Channel" |
         Region == "Scottish Continental Shelf"|
         Region == "Minches & Western Scotland"|
         Region == "Irish Sea"|
         Region == "Western Channel & Celtic Sea") %>%
  mutate(Region = if_else(Region == "Atlantic North-West Approaches, Rockall Trough and Faroe-Shetland Channel",
                          as.character("Atlantic North-West Approaches,\nRockall Trough and Faroe-Shetland Channel"), as.character(Region)),
         Region = factor(Region, levels = region_order,)) %>%
  left_join(cp2_palette_df, by = c("Region")) %>%
  mutate(colour_code = as.character(colour_code))
  
# transform to WGS84
cp2 <- st_transform(cp2, 4326)
  • clip the CP2 regions to the limits of the Celtic Seas assessment units (to remove the Hatton bank area in Region V from plots)
# crop to limits of Celtic Seas
cp2_cs <- st_crop(cp2, ukms_cs)
  • Map the regions
    • First load in the land mass layer for the maps
# Read in world landmass shapefile for creating maps
land <- st_read(paste0(land_reference_folder, "c20210824_World_Outline_WGS84_LAEA.shp"))
# transform to WGS84
land <- st_transform(land, 4326)

# clip the land to Celtic Seas area
land_cs <- st_crop(land, ukms_cs)
  • Plot the regions to check the match with UK EEZ
# plot assessment units and regions
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = cp2, aes(fill = Region), colour = NA) +
  geom_sf(data = ukms_cs_2, aes(colour = UKMS_SubRe), fill = NA) +
  scale_color_manual(name = "UKMS Sub-region", values = c("grey20", "blue")) +
  scale_fill_manual(name = "CP2 Regions", values = cp2_palette) +                     
  theme(legend.position="right") +
  labs(title = "UKMS Celtic Seas Sub-region and Charting Progress 2 Regions") +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-25.05, 1.05)

# tidy up
rm(cp2, cp2_regions, ukms_cs_2)

Note, no sample data were available in the Atlantic North-West Approaches, Rockall Trough and Faroe-Shetland Channel CP2 region or the UKMS sub-region ‘Celtic Seas (seabed and subsoil only)’

One Benthic Data

The CEFAS OpenScience OneBenthic database was used as a source of additional data to Marine Recorder. The ‘OneBenthic Data Extraction Tool: Grab/Core’ was used to extract industry sample data from the Celtic Seas regions (OneBenthic, 2020)

  • Exported from the portal (csv) and saved to the ‘Data/Raw’ folder
  • csv loaded into R and filtere to the relevant years (2009-2021)
  • Field names formatted and data type specified
  • Checked for duplicate rows (considered valid duplicates for summation later)
  • Coverage checked and filtered to the Celtic Seas assessment units

OneBenthic database (2020). Available from https://openscience.cefas.co.uk/OneBenthicExtraction/. Accessed: 09/12/2022.

# load in the One Benthic data
OB_data <- read.csv("Data/Raw/C20221209_OneBenthic_download.csv", header = T, sep = ",", stringsAsFactors = FALSE)

# check field names etc.
#str(OB_data)

# filter to only the years needed
OB_data <- OB_data %>%
  filter(grepl("2009|2010|2011|2012|2013|2014|2015|2016|2017|2018|2019|2020|2021", Date))

## Tidy up the data 
# change all field names to lower case
OB_data_tidy <- names_to_lowercase(OB_data)
# remove "."s, "_"'s, and " "
OB_data_tidy <- tidy_names(OB_data_tidy)

# redefine fields to the correct type and fix date, also add sample year
OB_data_tidy <- OB_data_tidy %>%
        mutate(
            date = as.character(date),
            surveyname = as.character(surveyname),
            samplecode = as.character(samplecode),
            samplecode2 = as.character(samplecode2),
            latitude = as.double(latitude),
            longitude = as.double(longitude),
            waterdepth = as.double(waterdepth),
            gearname = as.character(gearname),
            samplesize = as.double(samplesize),
            macrosievesizemm = as.double(macrosievesizemm),
            validnameworms = as.character(validnameworms),
            aphiaid = as.integer(aphiaid),
            rank = as.character(rank),
            taxonqualifier = as.character(taxonqualifier),
            abund = as.character(abund),
            dataowner = as.character(dataowner),
            sampledate = lubridate::ymd(date),
            year = lubridate::year(sampledate))

# qc a sample
#slice_sample(OB_data_tidy, n = 5)

# Check for any duplicates in the data
OB_data_tidy %>%
  subset(duplicated(OB_data_tidy)) %>%
  select(validnameworms, abund) %>%
  slice_sample(n = 5) %>%
  knitr::kable(caption = "Example of duplicate records in OneBenthic data")
Example of duplicate records in OneBenthic data
validnameworms abund
Syllis 1
Syllis 1
Spirobranchus triqueter 2
Syllis 1
Fabulina fabula 3
# Duplicates look like genuine aggregations of records (from qualifiers of uncertain species e.g., Syllis, Decapoda)
# (to be aggregated by summation at later stage)

# convert to a spatial object6
OB_data_cs <- st_as_sf(OB_data_tidy, coords = c("longitude", "latitude"), crs = 4326)

# plot data points
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = OB_data_cs, size = 0.2, colour = "red") +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

# filter to only points in the Celtic seas
OB_data_cs <- st_filter(OB_data_cs, ukms_cs)

# check what's in the Celtic seas
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = OB_data_cs, size = 0.2, colour = "red") +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

  • Check some OneBenthic records for QC
# Check 
OB_data_cs %>%
  as.data.frame() %>%
  slice_sample(n = 5) 
  • Check sample gears
# Check what sample methods are included 
OB_data_cs %>%
  as.data.frame() %>%
  select(samplecode, gearname) %>%
  group_by(samplecode, gearname) %>%
  dplyr::summarise(record_count = n()) %>%
  select(samplecode, gearname, record_count) %>%
  group_by(gearname) %>%
  dplyr::summarise(total = n(), .groups = "drop") %>%
  knitr::kable(caption = "Sample number and gear in OneBenthic data")
Sample number and gear in OneBenthic data
gearname total
0.1m2 Day grab 3977
0.1m2 Hamon grab 2971
0.1m2 Nioz Box Core 44
0.1m2 Van Veen grab 479
unspecified grab 14
# remove raw/interim data (after QC)
rm(OB_data, OB_data_tidy)
  • 7485 OneBenthic grab or core samples in the Celtic Seas

Summary of years sampled:

gearlist <- OB_data_cs %>%
    chuck("gearname") %>%
    unique %>%
    sort

# summary of samples and years
OB_data_cs %>%
  as.data.frame() %>%
  distinct(sampledate, gearname) %>%
    ggplot() +
        geom_jitter(
            mapping = aes(x = sampledate, y = match(gearname, gearlist)),
            alpha = 0.3, width = 0, height = 0.2) +
        scale_x_date(name = "", date_breaks = "1 years", date_labels = "%Y") +
        scale_y_continuous(name = "", breaks = seq_len(length(gearlist)), labels = gearlist)

Marine Recorder Data

The most recent UK Marine Recorder (Public) snapshot [https://jncc.gov.uk/our-work/marine-recorder/] (version “2022-08-05”) was interrogated for suitable data.

Sample Information

Potential samples of interest (sample date 2009 onwards) were extracted from Marine Recorder (SnapshotDatav52_PUBLIC_20220805) using the equivalent query:

SELECT SurveyEvent.SURVEY_KEY, Sample.SURVEY_EVENT_KEY, SampleReplicate.SAMPLE_REFERENCE, SampleReplicate.SAMPLE_Key, Sample.Date, Sample.LatWGS84, Sample.LongWGS84, SampleBiotopeAll.BiotopeCode, EUNISCorrelationTable.[JNCC 1503 code], EUNISCorrelationTable.[EUNIS code 2007], SampleReplicate.Method, SampleReplicate.MeshSize, SampleReplicate.NumTaken, SampleReplicate.SampleSize, SampleReplicate.TotalArea, SampleReplicate.Volume INTO BH2b_Sample_Data_all
FROM (Survey INNER JOIN (SurveyEvent INNER JOIN ((SampleBiotopeAll RIGHT JOIN Sample ON SampleBiotopeAll.SAMPLE_REFERENCE = Sample.SAMPLE_REFERENCE) LEFT JOIN EUNISCorrelationTable ON SampleBiotopeAll.BiotopeCode = EUNISCorrelationTable.[JNCC 1503 code]) ON SurveyEvent.SURVEY_EVENT_KEY = Sample.SURVEY_EVENT_KEY) ON Survey.SurveyKey = SurveyEvent.SURVEY_KEY) INNER JOIN SampleReplicate ON Sample.SAMPLE_REFERENCE = SampleReplicate.SAMPLE_REFERENCE
WHERE (((Sample.Date)>#12/31/2008#));
  • Sample data were exported from MR to Excel for import into R (Data/Processed/Marine_Recorder/)
  • Sample data were loaded into R for processing
  • Duplicate rows were removed
  • Field names standardised
  • Sample date formatted and sample year added
  • Sample methods (geartype) interrogated and filtered

There were many duplicate sample rows (identical values in each field), this was from a join in the SQL code used to extract sample data. All duplicates were removed before processing further.

# Load in the MR sample data exports  
MR_samples <- read_excel("Data/Processed/Marine_Recorder/C20230112_BH2b_Data_Extract.xlsx",
                          col_types = c("guess", "guess", "guess", "guess", "guess", "guess", 
                                        "guess", "guess", "guess", "guess", "guess", "numeric", 
                                        "numeric", "numeric", "numeric", "numeric"))

# there were some duplicate rows (likely from a join in the SQL export query)
# remove any duplicates and keep only unique rows 
MR_samples <- MR_samples %>%
    distinct() # 208,280 rows -> 207,994 rows

## Tidy up the data 
# change all field names to lower case
MR_samples_tidy <- names_to_lowercase(MR_samples)
# remove "."s, "_"'s, and " "
MR_samples_tidy <- tidy_names(MR_samples_tidy)

# redefine fields to the correct type and fix date, also add sample year
MR_samples_tidy <- MR_samples_tidy %>%
        mutate(
            date = as.character(date),
            surveykey = as.character(surveykey),
            surveyeventkey = as.character(surveyeventkey),
            samplereference = as.character(samplereference),
            samplekey = as.character(samplekey),
            latwgs84 = as.double(latwgs84),
            longwgs84 = as.double(longwgs84),
            biotopecode = as.character(biotopecode),
            eunis = as.character(euniscode2007),
            geartype = as.character(method),
            sampledate = lubridate::ymd(date),
            sampleyear = lubridate::year(sampledate),
            meshsize = as.double(meshsize),
            numtaken = as.double(numtaken),
            samplesize = as.double(samplesize),
            totalarea = as.double(totalarea),
            volume = as.double(volume))

# qc a sample
slice_sample(MR_samples_tidy, n = 5)
# see what gear types have been used
MR_samples_tidy %>%
  select(geartype) %>%
  group_by(geartype) %>%
  summarise(count = n(), .groups = "drop") %>%
    datatable(
        extensions = "Buttons",
        escape = FALSE,
        options = list(dom = "Blfrtip", buttons = c("csv")))
# filter to only grabs, cores, dredges, or trawls and add group identifier
# take a df of those to be excluded
MR_samples_excluded <- MR_samples_tidy %>%
  filter(!grepl("grab|core|trawl|dredge", geartype, ignore.case = TRUE)) %>%
  mutate(gearcat = "Other")
# filter those to be kept
MR_samples_tidy <- MR_samples_tidy %>%
  filter(grepl("grab|core|trawl|dredge", geartype, ignore.case = TRUE)) %>%
  mutate(gearcat = case_when(grepl("grab|core", geartype, ignore.case = TRUE) == TRUE ~ "grab_core",
                             grepl("trawl|dredge", geartype, ignore.case = TRUE) == TRUE ~ "trawl_dredge",
                             TRUE ~ "ERROR"))

# summary of grabs or cores
MR_samples_tidy %>%
    group_by(gearcat, geartype) %>%
    summarise(n = n(), .groups = "drop") %>%
    knitr::kable(caption = "MR grab/core/trawl/dredge samples")
MR grab/core/trawl/dredge samples
gearcat geartype n
grab_core Box-corer 24
grab_core Core - box 223
grab_core Core - hand-held 5483
grab_core Core - unspecified 412
grab_core Day grab 2860
grab_core Diver-operated corer 86
grab_core Ekman grab 93
grab_core Grab - Birge Eckman 24
grab_core Grab - Day 2613
grab_core Grab - Hamon 1943
grab_core Grab - Ponar 1
grab_core Grab - unspecified 1874
grab_core Grab - Van Veen 322
grab_core Hamon grab 1093
grab_core Van Veen grab 214
trawl_dredge 1.5 m Beam trawl 15
trawl_dredge Anchor dredge 10
trawl_dredge Beam trawl 314
trawl_dredge Dredge - anchor 1
trawl_dredge Dredge - biological 285
trawl_dredge Dredge - pipe 2
trawl_dredge Dredge - scallop 418
trawl_dredge Dredge - unspecified 5426
trawl_dredge Trawl - Beam 7
trawl_dredge Trawl - Otter 1321
trawl_dredge Trawl - unspecified 312

Sample Size and Area

Although the index of Margalef diversity is specifically developed to minimize the impact of possible size variability (sampled surface area or volume) of samples on assessment results, sampling methodologies should be comparable to a certain extent.

— OSPAR CEMP guidelines

For Grab and Core samples these should have a sampled surface area around 0.1 m2. For Marine Recorder records the sample area or volume are not consistently recorded, however for most sample methods (e.g. Day grabs) sample area is reasonably standard.

  • Check recorded sample area for grab and core sample methods
# recorded sample size/area for grabs and cores
geartypes_total <- MR_samples_tidy %>%
  filter(grepl("core|grab", geartype, ignore.case = T)) %>%
  group_by(geartype) %>%
  summarise(total = n()) %>%
  arrange(geartype, desc(total))

geartypes_size <- MR_samples_tidy %>%
  filter(!is.na(samplesize), grepl("core|grab", geartype, ignore.case = T)) %>%
  group_by(geartype, samplesize) %>%
  count() %>%
  arrange(geartype, desc(n))

geartypes <- left_join(geartypes_size, geartypes_total, by = c("geartype"))

geartypes %>%
  mutate(percent = scales::percent(n/total))
# tidy up
rm(geartypes_total, geartypes_size, geartypes)

Except “Box-corer”, almost all “core” samples with area recorded are less than 0.1 m2. Except “Ekman grab”, the majority of “grab” samples with a recorded area are 0.1 m2

  • Remove “Core” samples (except Box-corer)
  • Remove “Ekman grab” samples
# filter out hand cores and ekman
MR_samples_tidy <- MR_samples_tidy %>%
  filter(geartype != "Ekman grab", !grepl("Core|Diver", geartype))

Filter to Celtic Seas Area

  • Convert the sample data to a spatial object and filter to only points in the Celtic Seas UKMS sub-region
    • Make a separate dataset of samples removed
# convert to a spatial object
MR_samples_cs <- st_as_sf(MR_samples_tidy, coords = c("longwgs84", "latwgs84"), crs = 4326)

# filter to only points in the Celtic seas
MR_samples_cs <- st_filter(MR_samples_cs, ukms_cs)

# check what's in the Celtic seas
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = MR_samples_cs,  mapping = aes(geometry = geometry, colour = gearcat), size = 0.5) +
  scale_colour_manual(values = cb_palette) +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

# summary of samples in Celtic Seas
MR_samples_cs %>%
  as.data.frame() %>%
  group_by(gearcat, geartype) %>%
  summarise(n = n(), .groups = "drop") %>%
  knitr::kable(caption = "MR samples in Celtic Seas area")
MR samples in Celtic Seas area
gearcat geartype n
grab_core Day grab 1872
grab_core Grab - Day 2306
grab_core Grab - Hamon 877
grab_core Grab - Ponar 1
grab_core Grab - unspecified 915
grab_core Grab - Van Veen 303
grab_core Hamon grab 562
grab_core Van Veen grab 214
trawl_dredge Anchor dredge 10
trawl_dredge Beam trawl 6
trawl_dredge Dredge - biological 285
trawl_dredge Dredge - pipe 2
trawl_dredge Dredge - scallop 414
trawl_dredge Dredge - unspecified 192
trawl_dredge Trawl - Beam 1
trawl_dredge Trawl - Otter 726
trawl_dredge Trawl - unspecified 8
### Find those excluded (not in Celtic Seas area)
MR_samples_not_cs <- anti_join(MR_samples_tidy, MR_samples_cs, by = c("surveykey" = "surveykey", "surveyeventkey" = "surveyeventkey", "samplereference" = "samplereference"))
# bring together with those excluded by gear type
MR_samples_removed <- rbind(MR_samples_excluded, MR_samples_not_cs)
# convert to a spatial object
MR_samples_removed_sf <- st_as_sf(MR_samples_removed, coords = c("longwgs84", "latwgs84"), crs = 4326)

# remove raw/interim data (after QC)
rm(MR_samples, MR_samples_tidy, MR_samples_not_cs, MR_samples_excluded, MR_samples_removed)
  • 8445 grab, core, trawl, or dredge MR samples in the Celtic Seas from 2009 onwards.

Check Trawl and Dredge Sample Size

To calculate a representative Margalef diversity sample size needs to be relatively similar between samples, otherwise sample size will have too large an influence on diversity (species number being proportional to sample size).

# check trawl/dredge meta data
MR_samples_cs %>%
  as.data.frame() %>%
  filter(gearcat == "trawl_dredge") %>%
  group_by(geartype, meshsize, volume, totalarea) %>%
  count()

There is no metadata associated with the dredge or trawl samples (sample area, or sample volume). Any estimate of diversity would be unreliable without standardisation of sample size or methods.

assessment results for the category ‘Dredge-Trawl’ were strongly related to the specific methodology used, more than the expected (fishing) pressure level.

— OSPAR CEMP guidelines

  • Remove Trawl/Dredge samples from the dataset
# pull a list of those to be removed
MR_samples_cs_trawls <- MR_samples_cs %>%
  filter(gearcat == "trawl_dredge")
# add to the other removed samples
MR_samples_removed_sf <- rbind(MR_samples_removed_sf, MR_samples_cs_trawls)

# filter the trawl dredge samples
MR_samples_cs <- MR_samples_cs %>%
  filter(gearcat != "trawl_dredge")
# tidy up
rm(MR_samples_cs_trawls)
  • 6834 grab or core MR samples in the Celtic Seas from 2009 onwards

Summary of Sample Collection Over Years

# pull the list of gear types
gearlist_MR <- MR_samples_cs %>%
    chuck("geartype") %>%
    unique %>%
    sort

# summary of samples and years
MR_samples_cs %>%
  as.data.frame() %>%
  distinct(sampledate, geartype) %>%
    ggplot() +
        geom_jitter(
            mapping = aes(x = sampledate, y = match(geartype, gearlist_MR)),
            alpha = 0.3, width = 0, height = 0.2) +
        scale_x_date(name = "", date_breaks = "1 years", date_labels = "%Y") +
        scale_y_continuous(name = "", breaks = seq_len(length(gearlist_MR)), labels = gearlist_MR)

# tidy up 
rm(gearlist, gearlist_MR)

Multiple Habitat Codes

Some samples in the Marine Recorder dataset have multiple records of habitat type (more than one biotope code / EUNIS code). To avoid assigning taxa data to multiple rows of the same sample (with a different habitat for each), the duplicate samples need amalgamating with a mosaic habitat code.

  • Find the duplicates
  • Concatenate EUNIS codes where multiples are present
  • Join mosaic habitats to the sample data
  • Remove redundant habitat information
  • Remove any duplicate rows
# pull out a df list of duplicate samples (to use in a join)
samples <- MR_samples_cs %>%
  as.data.frame() %>%
  select(c(samplekey)) %>%
  group_by_all() %>%
  summarise(n = n()) %>%
  filter(n>1) %>%
  select(samplekey)

# build a df of duplicate samples and their eunis codes
sample_multiples <- left_join(samples, select(MR_samples_cs, samplekey, eunis), by = c("samplekey" = "samplekey"), suffix=c("",".y"))

# remove any duplicates from NA eunis codes
sample_multiples <- sample_multiples %>%
  as.data.frame() %>%
  filter(!is.na(eunis))

# some single sample hab codes still need filtering out
sample_multiples <- sample_multiples %>%
  as.data.frame() %>%
  unique()

# concatenate the eunis codes
sample_multiples <- sample_multiples %>% 
  group_by(samplekey) %>% 
  summarise(mosaic = paste0(eunis,collapse = "&"))

# join the new mosaic habitat to the sample data
MR_samples_cs <- left_join(MR_samples_cs, sample_multiples, by = c("samplekey" = "samplekey"), suffix=c("",".y"))

# update the eunis field if the sample has multiple codes
MR_samples_cs <- MR_samples_cs %>%
  mutate(eunis = if_else(is.na(mosaic), eunis, mosaic, mosaic))

# remove all other habitat fields (so duplicates can be removed)
MR_samples_cs <- MR_samples_cs %>%
  select(-c(biotopecode, jncc1503code, euniscode2007, mosaic))

# remove any duplicate rows
MR_samples_cs <- MR_samples_cs %>%
  distinct()

# tidy up
rm(samples, sample_multiples)

Species Data

Import and Tidy Data

  • The species data table was exported from MR to csv for import into R (Data/Processed/C20221213_BH2b_Species_Data.csv)
  • Species data were loaded into R for processing
  • Removed any records marked as ‘dead’ or ‘confidential’
  • Removed unnecessary fields
  • Tidied column names
  • Defined data types
# Load in the MR species data export  
MR_species <- read.csv("Data/Processed/Marine_Recorder/C20221213_BH2b_Species_Data.csv", header = T, sep = ",")

# filter any 'dead' or 'confidential' rows
MR_species_tidy <- MR_species %>%
  filter(IsDead != 1) %>%
  filter(Confidential != 1)

# select only relevant fields
MR_species_tidy <- MR_species_tidy %>%
  select(SAMPLE_REFERENCE, SAMPLE_Key, SpeciesName, SpQual, Uncertain, RepId, SACFORN, SpPercent, SpCount, PA, NumPerSqm, AphiaId, PreferredSpeciesName, PreferredAphiaId)

# change all field names to lower case
MR_species_tidy <- names_to_lowercase(MR_species_tidy)
# remove "."s, "_"'s, and " "
MR_species_tidy <- tidy_names(MR_species_tidy)

# redefine fields to the correct type
MR_species_tidy <- MR_species_tidy %>%
        mutate(
          samplereference = as.character(samplereference),
          speciesname = as.character(speciesname),
          samplekey = as.character(samplekey),
          spqual = as.character(spqual),
          repid = as.character(repid),
          sacforn = as.character(sacforn),
          spcount = as.integer(spcount),
          pa = as.character(pa),
          numpersqm = as.double(numpersqm),
          aphiaid = as.integer(aphiaid),
          preferredspeciesname = as.character(preferredspeciesname),
          preferredaphiaid = as.integer(preferredaphiaid)
          )

# remove raw data (after QC)
rm(MR_species)

Join Marine Recorder Sample And Species Data

Join the tidied Marine Recorder species data to the tidied and filtered MR sample data (grabs or cores in the Celtic Seas)

  • Output a quick summary table of taxa in samples
  • 3868037 taxa records joined to 6834 samples
# join the tidied species data to the filtered sample records
MR_data_cs <- left_join(MR_samples_cs, MR_species_tidy, by = c("samplekey" = "samplekey"), suffix=c("",".y"))

# summary of taxa in samples
MR_data_cs %>%
  as.data.frame() %>%
  group_by(geartype, samplekey) %>%
  summarise(n = n(), .groups = "drop") %>%
  group_by(geartype) %>%
  summarise(median = round(median(n)), mean = round(mean(n)), min = min(n), max = max(n), .groups = "drop") %>%
  knitr::kable(caption = "MR sample type summary of 'taxa' records")
MR sample type summary of ‘taxa’ records
geartype median mean min max
Day grab 37 50 1 299
Grab - Day 34 40 1 251
Grab - Hamon 31 39 1 186
Grab - Ponar 1 1 1 1
Grab - unspecified 3 27 1 210
Grab - Van Veen 32 37 1 141
Hamon grab 30 35 1 120
Van Veen grab 14 21 1 166
# remove the separate MR sample and species data frames (after QC)
rm(MR_species_tidy, MR_samples_cs)

Grab - Ponar sample method has only 1 record, remove as outlier.

MR_data_cs <- MR_data_cs %>%
  filter(geartype != "Grab - Ponar")

EA 2009 to 2019 WFD Data

Biological and environmental data (sediment and salinity) for the EA’s WFD programme from 2009 onwards, provided by Graham Phillips (e-mail 16/03/23).

  • The data and taxon list are from the EA’s BioSys database. This is based on the Natural History Museum nomenclature and coding.
  • This is likely to include duplicate data from the OneBenthic dataset, so you may need to check this.
  • Habitat data is provided in the ‘Environmental data’ worksheet as PSA data and derived Folk sediment description.
  • Refer to the Open Government Licence for the use of this data Open Government Licence (nationalarchives.gov.uk).
    • You are free to:
      • copy, publish, distribute and transmit the Information;
      • adapt the Information;
      • exploit the Information commercially and non-commercially for example, by combining it with other Information, or by including it in your own product or application.
    • You must
      • acknowledge the source of the Information in your product or application by including or linking to any attribution statement specified by the Information Provider(s) and, where possible, provide a link to this licence;
      • If the Information Provider does not provide a specific attribution statement, you must use the following:
      • Contains public sector information licensed under the Open Government Licence v3.0.
  • Load in the raw data and check for duplicate rows
# Load in the MR sample data exports  
WFD_samples <- read_excel("Data/Raw/EA_WFD_230315/EA 2009-2019 WFD Data 2023-03-15.xlsx")

# check for any initial duplicates in the data
dups <- WFD_samples %>%
  duplicated()
# Check for duplicates
WFD_samples %>%
  filter(dups)
# tidy up
rm(dups)

No duplicates in raw data

  • Tidy variable names
## Tidy up the data 
# change all field names to lower case
WFD_samples_tidy <- names_to_lowercase(WFD_samples)
# remove "."s, "_"'s, " " etc.
WFD_samples_tidy <- tidy_names(WFD_samples_tidy)
  • Standardise sample date field and add sampleyear field
# mutate sample date field and add sample year
WFD_samples_tidy <- WFD_samples_tidy %>%
  mutate(sampledate = lubridate::ymd(sampledate),
            sampleyear = lubridate::year(sampledate))
  • Check for missing location data (easting & northing)
# records missing coordinates
WFD_samples_tidy %>%
  select(surveyname, surveycodesurveykey, easting, northing)%>%
  filter(is.na(easting)|is.na(northing)) %>%
  group_by(surveyname, surveycodesurveykey) %>%
  count()
# check of records missing coordinates
WFD_samples_tidy %>%
  filter(is.na(easting)|is.na(northing)) %>%
  group_by(surveyname, surveycodesurveykey) %>%
  slice_sample(n=20)
  • Remove records without coordinates
# filter out missing coordinate records
WFD_samples_tidy <- WFD_samples_tidy %>%
  filter(!is.na(easting) | !is.na(northing)) 
  • Check for missing taxa details
# records missing coordinates
WFD_samples_tidy %>%
  select(surveyname, surveycodesurveykey, taxonname ,abundance) %>%
  filter(is.na(taxonname )|taxonname == "") %>%
  group_by(surveyname, surveycodesurveykey) %>%
  count()
  • Check for missing abundance records or strange characters
# Check abundance records
WFD_samples_tidy %>%
  select(abundance) %>%
  distinct(abundance) %>%
  arrange(abundance) %>%
  slice_head(n=20)
WFD_samples_tidy %>%
  select(abundance) %>%
  distinct(abundance) %>%
  arrange(desc(abundance)) %>%
  slice_head(n=20)

No obvious signs of unusual data to remove at this stage

Filter to Celtic Seas Area

  • Check where the sample points are distributed
    • Convert to spatial feature
    • Transform to WGS84
# convert ot a spatial object
WFD_data_sf <- st_as_sf(WFD_samples_tidy, coords = c("easting", "northing"), crs = 27700)

# transform to WGS84
WFD_data_sf <- st_transform(WFD_data_sf, 4326)
  • Plot WFD sample points
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = WFD_data_sf, mapping = aes(geometry = geometry), colour = "red", size = 0.5) +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

  • Filter to those in the area of interest (Celtic Seas)
# filter to only points in the Celtic seas
WFD_data_cs <- st_filter(WFD_data_sf, ukms_cs)

# map
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = WFD_data_cs, mapping = aes(geometry = geometry), colour = "red", size = 0.5) +
  coord_sf(xlim = c(-6.5, -2), ylim = c(50, 54)) +
  theme_bw()

# tidy up
rm(WFD_data_sf)
  • Check the sample method types and mesh size
# Check sample methods and mesh size
WFD_data_cs %>%
  as.data.frame() %>%
  select(sampleidsamplekey, samplemethod, sievemeshum) %>%
  unique() %>%
  group_by(samplemethod, sievemeshum) %>%
  summarise(count = n(), .groups = "drop") 

Most samples are at 0.5 mm mesh size and collected with 0.01 m2 cores or pooled 0.01 m2 cores

  • Remove core samples, they will not be representative of other sample methods with approx 0.1 m2 sampling area. They are also likely to be collected intertidally and not suitable for the sublittoral indicator.
# filter out cores
WFD_data_cs <- WFD_data_cs %>%
  filter(!grepl("core", samplemethod, ignore.case = TRUE))

340 samples in the Celtic Seas area from the original ~ 1837.

MERMAN Data

Benthic sample data from the MERMAN database was investigated for use in the assessment. Despite the OneBenthic database containing record from the MERMAN database, these only go up to 2013. To include more recent data the MERMAN data is being incorporated into the assessment (and filtered from the OneBenthic data).

These data are a snapshot of the data held within MERMAN obtained on the (17/08/2022). The data were supplied by the British Oceanographic Data Centre on behalf of the Clean Safe Seas Evidence Group. Data were collected by the Agri-Food and Biosciences Institute, Centre for Environment, Fisheries and Aquaculture Science, Department of Agriculture, Environment and Rural Affairs, Environment Agency, Food Standards Scotland, Marine Scotland Science, Natural Resource Wales and Scottish Environment Protection Agency. The data were funded by Agri-Food Biosciences institute, Department of Agriculture, Environment and Rural Affairs, Department for Environment, Food and Rural Affairs and Scottish Government. These data contain public sector information licensed under the Open Government Licence v3.0 (nationalarchives.gov.uk).

  • Load in the MERMAN data
# load in the MERMAN data
merman_data <- read.csv("Data/Raw/MERMAN/BODCREQ-3275_ZB/BODCREQ-3275_ZB.csv", header = T, sep = ",", stringsAsFactors = FALSE)

# check field names etc.
#str(merman_data)

# tidy field names
merman_data_tidy <-  names_to_lowercase(merman_data) %>%
  tidy_names()
# rename staiton number
merman_data_tidy <- merman_data_tidy %>%
  rename(stationnumber = ïstationnumber)

# check data types
#glimpse(merman_data_tidy)
  • Check sample locations in the Celtic Seas
# group to samples
merman_samples <- merman_data_tidy %>%
  group_by(stationnumber, samplelatitude, monitoringyear, samplelongitude, samplestartdate, samplenumber, subsamplenumber, samplertype, meshsize) %>%
  summarise(records = n(), .groups = "drop") %>%
  filter(monitoringyear >2008) 

# convert to a spatial object
merman_samples_sf <- st_as_sf(merman_samples, coords = c("samplelongitude", "samplelatitude"), crs = 4326)

# plot data points
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = merman_samples_sf, colour = "red", size = 0.4) +
  theme_bw() +
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

# tidy up
rm(merman_samples_sf, merman_samples)
  • Filter to the Celtic Sea region and years of the assessment (2009 onwards)
  • Filter to only 1 mm mesh size (0.5 mm mesh is excluded from analysis at later stages)
# convert benthos data to a spatial object
merman_data_sf <- st_as_sf(merman_data_tidy, coords = c("samplelongitude", "samplelatitude"), crs = 4326)

# filter to only points in the Celtic seas, 1mm and 2009+
merman_data_cs <- st_filter(merman_data_sf, ukms_cs) %>%
  filter(monitoringyear >2008, meshsize == 1000) 
  • Biomass data is also included in the dataset
  • Filter to abundance records
# record types
merman_data_cs %>%
  as.data.frame() %>%
  distinct(determinandcode)
# filter to just abundance data
merman_data_cs <- merman_data_cs %>%
  filter(determinandcode == "ABUNDNR")
  • Remove surplus variables
# remove variables
merman_data_cs <- merman_data_cs %>%
  select(-c(samplestarttime, datatype, determinandcode, determinandqualifierqflag, unitofmeasuremunit, basiscodebasis, sexcodesexco, maximumdepthofsamplebiology, minimumdepthofsamplebiology, samplertypeidsmtyp))

Example of data

# example of data
merman_data_cs %>%
  as.data.frame() %>%
  slice_sample(n = 20)
  • Concatenate station, sample number, and sub-sample number into a single sampleref variable
# give a sample reference code
merman_data_cs <- merman_data_cs %>%
  unite("sampleref", stationnumber, samplenumber, subsamplenumber)
  • 594 MERMAN 1 mm Day grab samples in the Celtic Seas

NRW Data

Following a request for additional data for use in the BH2b indicator assessment NRW supplied benthic grab data to support monitoring data already included in Marine Recorder.

NRW supplied an extract from the KIECO dataset showing sample Benthic grab sites across Celtic Sea. Information provided in the dataset include samples details, species found in the area and the type of sediment on the sea bed. The dataset contains Natural Resources Wales information © Natural Resources Wales and database right. All rights reserved.

NRW deem the detailed location of certain habitat and the detailed location and numbers/density of certain species to be environmentally sensitive (Official – sensitive). Any published outputs must be disguised to a coarser resolution.

As these habitat and species data need to be treated as official sensitive and only published at a coarse resolution all processing was undertaken in a separate script. These processing steps mirrored those undertaken for the other datasets (covered in this and following scripts). However, the NRW samples still required habitat information adding from the combined map (steps below) and are therefore included here.

Summary of NRW data processing:

  • Select relevant fields
  • Standardise data
  • Filter to only 1 mm mesh size
  • Filter to locations inside the UKMS Celtic Seas sub-region
  • Taxonomic and record standardisation (see Data Truncation for details)
  • Check against other sample points for duplication
  • Summarise as samples and add Margalef diversity

Following processing the NRW dataset only contains:

  • sampleref (sample ID)
  • sampledate
  • sampleyear
  • D Margalef diversity
  • S species richness
  • N sample abundance
  • geometry

Load the sample data

# Load processed NRW sample data
samples_NRW <- readRDS("OFFICIAL SENSITIVE_NRW_data/OFF_SEN_NRW_Data_Prep/Processed/C20230815_samples_NRW")
  • Summary of the 232 NRW sample locations
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_NRW, mapping = aes(geometry = geometry), colour = "red", size = 1.5) +
  theme_bw() +
  ylim(51.2, 53.5) +
  xlim(-5.5, -2.5)

Habitat Data

Habitat and fishing pressure data are taken from already processed BH3 Shapefiles. BH3 output layers for the Northern Celtic Sea and Southern Celtic Sea were exported to the ‘Data/Processed/Spatial’ folder (C20221215_VMS_Habitat_SCS.shp and C20221215_VMS_Habitat_NCS.shp) with only the combined map habitat information and ICES VMS SAR data attributes (annual SAR 2009 to 2020). These output layers were clipped to the UK EEZ (initially for the MSPACE project), topological repairs were then processed in QGIS (fix geometries, delete duplicate geometries, and remove null geometries) and the final layers saved to a geopackage (BH2b_Processed.gpkg).

  • Load in the Shapefiles
  • Add assessment unit identifier to each Shapefile
  • Bind the two assessment units into one
  • Run QC checks on the EUNIS habitat field
  • Change true NA records to “Na” for consistency
# Load in BH3/MSPACE layers that contain EUNIS data and VMS pressure clipped to the UK EEZ 
hab_vms_SCS <- st_read("Data/Processed/Spatial/BH2b_Processed.gpkg",  layer = "C20221229_VMS_Habitat_SCS")
hab_vms_NCS <- st_read("Data/Processed/Spatial/BH2b_Processed.gpkg",  layer = "C20221229_VMS_Habitat_NCS")

# add assessment unit identifier (possibly needed later?)
hab_vms_NCS <- hab_vms_NCS %>%
  mutate(au = "NCS")
hab_vms_SCS <- hab_vms_SCS %>%
  mutate(au = "SCS")

# bind together the two assessment units
hab_vms_cs <- bind_rows(hab_vms_NCS, hab_vms_SCS)

# remove the individual assessment units to free up space
rm(hab_vms_NCS, hab_vms_SCS)

# run some quick checks on the EUNIS habitat field (any '+'s, 'or's, 'NA's in the habitat field)
hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(grepl("\\+", HAB_TYPE)) %>%
  distinct(HAB_TYPE) # zero rows

hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(grepl("or", HAB_TYPE, ignore.case = TRUE)) # zero rows

hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(grepl(" ", HAB_TYPE)) # zero rows

hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(grepl("na", HAB_TYPE, ignore.case = TRUE)) %>%
  distinct(HAB_TYPE) # "Na" charcters in HAB_TYPE

hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(is.na(HAB_TYPE)) # True "NA" missing values in HAB_TYPE

# replace nulls with "Na" characters for consistency  
hab_vms_cs <- hab_vms_cs %>%
  mutate(HAB_TYPE = as.character(HAB_TYPE)) %>%
  mutate(HAB_TYPE = if_else(is.na(HAB_TYPE), "Na", HAB_TYPE))

Add EUNIS Level 3 & 4 Habitat Codes

Habitat information from the combined map is at the best available EUNIS level and includes ‘mosaic/combined’ habitat codes (e.g., A5.25&A5.23). For assessing Relative Margalef Diversity habitats will be stratified at Broad-Scale Habitat type. Before joining habitat information to sample data EUNIS Level 3 & 4 habitat fields will be added.

Note, the final assessment grouped samples by broad scale habitat classifications based on the North Atlantic definitions, not EUNIS 2007 level 3 or 4 habitat definitions

Convert EUNIS habitat codes in the HAB_TYPE field into the EUNIS Level 3 and 4 equivalents:

  • Check the number of habitats codes (count the &s)
    • take a list of mosaics
    • split mosaics
    • convert codes to L3 and L4
    • check for matches at L3 and L4 among mosaics
    • take a list of any that are all the same L3 (none at L4)
    • convert mosaics that can be changed to L3
    • leave any mosaics that can’t be summarised to L3
  • Convert single habitat codes to L3 and L4
# check the number of EUNIS codes in mosaic habitats (number of "&"s in field)
hab_vms_cs %>% 
  as.data.frame() %>%
  select(HAB_TYPE) %>% 
  mutate(mosaic_n = str_count(HAB_TYPE, "&")) %>%
  select(mosaic_n) %>%
  filter(mosaic_n >0) %>%
  group_by(mosaic_n) %>%  
  count(mosaic_n) %>%
  ungroup() %>%
  print()

# subset mosaic habitats (2 x "&" in HAB_TYPE field so only a max of 3 EUNIS codes)
distinct_mosaic <- hab_vms_cs %>%
  as.data.frame() %>%
  select(HAB_TYPE) %>%
  filter(grepl("&", HAB_TYPE)) %>%
  group_by(HAB_TYPE) %>%
  count(HAB_TYPE) %>%
  arrange(desc(n)) %>%
  ungroup()

# split mosaics, convert to L3 and make a new L3 mosaic field
L3_mosaics <- distinct_mosaic %>%
  select(HAB_TYPE) %>%
  separate(HAB_TYPE,  sep = "&", into = c("HAB1", "HAB2", "HAB3"), remove = FALSE) %>%
  mutate(HAB_1_L3 = str_sub(HAB1, 0, 4)) %>%
  mutate(HAB_2_L3 = str_sub(HAB2, 0, 4)) %>%
  mutate(HAB_3_L3 = str_sub(HAB3, 0, 4)) %>%
  unite("HAB_mosaic_L3", HAB_1_L3, HAB_2_L3, HAB_3_L3, sep = "&", remove = FALSE, na.rm = TRUE) 
# print those that can be aggregated
L3_mosaics %>%
  group_by(HAB_mosaic_L3) %>%
  count(HAB_mosaic_L3) %>%
  arrange(HAB_mosaic_L3) %>%
  ungroup() %>%
  filter(n >1) %>%
  print()

# separate mosaic habitats codes and check if any are the same at L3  
L3_mosaic_duplicates <- L3_mosaics %>%
  mutate(duplicate_mosaic_L3 = if_else(is.na(HAB_3_L3), 
                                       (if_else(HAB_1_L3 == HAB_2_L3, HAB_1_L3, "Na")),
                                       (if_else(HAB_1_L3 == HAB_2_L3 & HAB_2_L3 == HAB_3_L3, HAB_1_L3, "Na")))) %>%
  select(HAB_TYPE, duplicate_mosaic_L3) %>%
  filter(duplicate_mosaic_L3 != "Na") %>%
  arrange(duplicate_mosaic_L3)

# make a 'list' of mosaics that can be summarised at L3 
mosaic_duplicate_list_L3 <- pull(L3_mosaic_duplicates, HAB_TYPE, duplicate_mosaic_L3)

# split mosaics, convert to L4 and make a new L4 mosaic field
L4_mosaics <- distinct_mosaic %>%
  select(HAB_TYPE) %>%
  separate(HAB_TYPE,  sep = "&", into = c("HAB1", "HAB2", "HAB3"), remove = FALSE) %>%
  mutate(HAB_1_L4 = str_sub(HAB1, 0, 5)) %>%
  mutate(HAB_2_L4 = str_sub(HAB2, 0, 5)) %>%
  mutate(HAB_3_L4 = str_sub(HAB3, 0, 5)) %>%
  unite("HAB_mosaic_L4", HAB_1_L4, HAB_2_L4, HAB_3_L4, sep = "&", remove = FALSE, na.rm = TRUE) 
# print those that can be aggregated
L4_mosaics%>%
  group_by(HAB_mosaic_L4) %>%
  count(HAB_mosaic_L4) %>%
  arrange(HAB_mosaic_L4) %>%
  ungroup() %>%
  filter(n >1) # No duplicates at L4

# separate mosaic habitats codes and check if any are the same at L4  
L4_mosaic_duplicates <- L4_mosaics %>%
  mutate(duplicate_mosaic_L4 = if_else(is.na(HAB_3_L4), 
                                       (if_else(HAB_1_L4 == HAB_2_L4, HAB_1_L4, "Na")),
                                       (if_else(HAB_1_L4 == HAB_2_L4 & HAB_2_L4 == HAB_3_L4, HAB_1_L4, "Na")))) %>%
  select(HAB_TYPE, duplicate_mosaic_L4) %>%
  filter(duplicate_mosaic_L4 != "Na") %>%
  arrange(duplicate_mosaic_L4) # No aggregates at L4

# create a L3 and L4 EUNIS code field, change mosaics that can be aggregated to L3 to single L3 code,
# change those that can't be aggregated to L3 to L3 and L4 mosaics, and change any single EUNIS codes to L3 and L4
hab_vms_cs <- hab_vms_cs %>%
  left_join(select(L3_mosaics, HAB_TYPE, HAB_mosaic_L3), by = c("HAB_TYPE")) %>%
  left_join(select(L4_mosaics, HAB_TYPE, HAB_mosaic_L4), by = c("HAB_TYPE")) %>%
  mutate(EUNIS_L3 = substr(HAB_TYPE, 1, 4),
         EUNIS_L4 = substr(HAB_TYPE, 1, 5)) %>%
  mutate(EUNIS_L3 = if_else(is.na(HAB_mosaic_L3), EUNIS_L3, HAB_mosaic_L3)) %>%
  mutate(EUNIS_L3 = if_else(HAB_TYPE %in% mosaic_duplicate_list_L3, substr(HAB_TYPE, 1, 4), EUNIS_L3)) %>%
  mutate(EUNIS_L3 = if_else(EUNIS_L3 == "Na", "No EUNIS Data", EUNIS_L3)) %>%
  mutate(EUNIS_L4 = if_else(is.na(HAB_mosaic_L4), EUNIS_L4, HAB_mosaic_L4)) %>%
  select(!c(HAB_mosaic_L3, HAB_mosaic_L4))

# remove temporary data
rm(distinct_mosaic, L3_mosaic_duplicates, L3_mosaics, mosaic_duplicate_list_L3, L4_mosaic_duplicates, L4_mosaics)
  • Create layer with only habitat information for joining to sample data (where habitat data is missing)
# select only the fields needed
hab_cs <- hab_vms_cs %>%
  select(HAB_TYPE, EUNIS_L3, EUNIS_L4, au)

Marine Recorder Habitat

  • Check the EUNIS habitat information in MR sample data
# check EUNIS codes used in the MR data and if there are any strange symbols etc.
print(unique(unlist(MR_data_cs$eunis)))
##   [1] NA                       "A5"                     "A5.25"                 
##   [4] "A5.261"                 "A5.44"                  "A5.233"                
##   [7] "A5.35"                  "A5.26"                  "A5.231"                
##  [10] "A5.242"                 "A5.24"                  "A5.14"                 
##  [13] "A5.43"                  "A5.334"                 "A5.133"                
##  [16] "A5.142"                 "A5.355"                 "A5.23"                 
##  [19] "A5.252"                 "A5.33"                  "A5.13"                 
##  [22] "A5.233&A5.26"           "A5.26&A5.233"           "A5.244"                
##  [25] "A5.33&A5.242"           "A5.34"                  "A5.242&A5.33"          
##  [28] "A5.142&A5.43"           "A5.44&A5.142"           "A5.36"                 
##  [31] "A2.2&A2.3"              "A2.323"                 "A2.321"                
##  [34] "A2.421"                 "A2.42"                  "A2.3&A2.2"             
##  [37] "A2.24"                  "A2.244"                 "A2.3&A2.24"            
##  [40] "A5.221"                 "A2.22"                  "A2.231"                
##  [43] "A2.23"                  "A2.32"                  "A5.22"                 
##  [46] "A4.131"                 "A4.214"                 "A4.112"                
##  [49] "A4.13"                  "A5.2"                   "A5.3"                  
##  [52] "A5.4"                   "A5.1"                   "A5.451"                
##  [55] "A5.251"                 "A5.27"                  "A5.45"                 
##  [58] "A5.27&A5.45"            "A5.27&A5.37"            "A5.45&A5.27"           
##  [61] "A5.27&A5.15"            "A5.15&A5.27"            "A5.37&A5.27"           
##  [64] "A5.15"                  "A5.51&A5.511"           "A5.511&A5.51"          
##  [67] "A5.51&A5.512"           "A5.611"                 "A5.621"                
##  [70] "A5.625"                 "A5.134"                 "A5.135"                
##  [73] "A4.221"                 "A5.25&A5.14"            "A5.234"                
##  [76] "A5.323"                 "A5.325"                 "A5.321"                
##  [79] "A5.32"                  "A5.136"                 "A5.432"                
##  [82] "A5.336&A5.325&A5.513"   "A5.513&A5.336&A5.325"   "A5.336&A5.513&A5.325"  
##  [85] "A5.325&A5.336"          "A5.513&A5.325&A5.336"   "A5.42"                 
##  [88] "A5.422"                 "A5.145"                 "A5.513&A5.42"          
##  [91] "A5.513&A5.422"          "A5.42&A5.513"           "A5.322"                
##  [94] "A5.421"                 "A5.422&A5.513"          "A5.336&A5.325"         
##  [97] "A5.421&A5.322"          "A5.322&A5.422"          "A5.243"                
## [100] "A5.612"                 "A5.137"                 "A5.431"                
## [103] "A5.262"                 "A5.511"                 "A5.513&A5.334"         
## [106] "A5.51"                  "A5.221&A5.612"          "A5.324"                
## [109] "A5.37"                  "A5.622"                 "A5.272"                
## [112] "A5.143"                 "A5.434"                 "A5.52"                 
## [115] "A5.361"                 "A5.527"                 "A5.3611"               
## [118] "A5.343"                 "A5.5331"                "A5.362"                
## [121] "A5.332"                 "A5.362&A5.361"          "A5.361&A5.362"         
## [124] "A5.35&A5.36"            "A5.445"                 "A5.363"                
## [127] "A5.241"                 "A3.223&A5.5331&A5.5111" "A5.7211"               
## [130] "A5.5112"                "A5.142&A5.145"          "A5.145&A5.142"         
## [133] "A5.142&A5.252"          "A5.252&A5.142"          "A5.242&A5.142"         
## [136] "A5.142&A5.242"
  • Convert habitat codes to L3 and L4 following the similar principle used for the combined map
# subset mosaic habitats
distinct_mosaic <- MR_data_cs %>%
  as.data.frame() %>%
  select(eunis) %>%
  filter(grepl("&", eunis)) %>%
  group_by(eunis) %>%
  count(eunis) %>%
  arrange(desc(n)) %>%
  ungroup()

# check the number of "&"s, and therefore +1 = the number of codes 
str_count(distinct_mosaic$eunis, pattern = "&") %>%
  max()
## [1] 2
# split mosaics, convert to L3 and make a new L3 mosaic field
L3_mosaics <- distinct_mosaic %>%
  select(eunis) %>%
  separate(eunis,  sep = "&", into = c("HAB1", "HAB2", "HAB3", "HAB4", "HAB5"), remove = FALSE) %>%
  mutate(HAB_1_L3 = str_sub(HAB1, 0, 4)) %>%
  mutate(HAB_2_L3 = str_sub(HAB2, 0, 4)) %>%
  mutate(HAB_3_L3 = str_sub(HAB3, 0, 4)) %>%
  mutate(HAB_4_L3 = str_sub(HAB4, 0, 4)) %>%
  mutate(HAB_5_L3 = str_sub(HAB5, 0, 4)) %>%
  unite("HAB_mosaic_L3", HAB_1_L3, HAB_2_L3, HAB_3_L3, HAB_4_L3, HAB_5_L3, sep = "&", remove = FALSE, na.rm = TRUE) 


# check if any mosaics are the same at L3  
L3_mosaic_duplicates <- L3_mosaics %>%
  mutate(duplicate_mosaic_L3 = case_when(
    HAB_1_L3 == HAB_2_L3 & HAB_2_L3 == HAB_3_L3 & HAB_3_L3 == HAB_4_L3 & HAB_4_L3 == HAB_5_L3 ~ HAB_1_L3,
    HAB_1_L3 == HAB_2_L3 & HAB_2_L3 == HAB_3_L3 & HAB_3_L3 == HAB_4_L3 & is.na(HAB_5_L3) ~ HAB_1_L3,
    HAB_1_L3 == HAB_2_L3 & HAB_2_L3 == HAB_3_L3 & is.na(HAB_4_L3) & is.na(HAB_5_L3)  ~ HAB_1_L3,
    HAB_1_L3 == HAB_2_L3 & is.na(HAB_3_L3) & is.na(HAB_4_L3) & is.na(HAB_5_L3)   ~ HAB_1_L3,
    TRUE ~ "not aggregated"
    )) %>%
  select(eunis, duplicate_mosaic_L3) %>%
  filter(duplicate_mosaic_L3 != "not aggregated") %>%
  arrange(duplicate_mosaic_L3) 

# make a 'list' of mosaics that can be summarised at L3 
mosaic_duplicate_list_L3 <- pull(L3_mosaic_duplicates, eunis, duplicate_mosaic_L3)

# split mosaics, convert to L4 and make a new L4 mosaic field
L4_mosaics <- distinct_mosaic %>%
  select(eunis) %>%
  separate(eunis,  sep = "&", into = c("HAB1", "HAB2", "HAB3", "HAB4", "HAB5"), remove = FALSE) %>%
  mutate(HAB_1_L4 = str_sub(HAB1, 0, 5)) %>%
  mutate(HAB_2_L4 = str_sub(HAB2, 0, 5)) %>%
  mutate(HAB_3_L4 = str_sub(HAB3, 0, 5)) %>%
  mutate(HAB_4_L4 = str_sub(HAB4, 0, 5)) %>%
  mutate(HAB_5_L4 = str_sub(HAB5, 0, 5)) %>%
  unite("HAB_mosaic_L4", HAB_1_L4, HAB_2_L4, HAB_3_L4, HAB_4_L4, HAB_5_L4, sep = "&", remove = FALSE, na.rm = TRUE) 

# check if any mosaics are the same at L4  
L4_mosaic_duplicates <- L4_mosaics %>%
  mutate(duplicate_mosaic_L4 = case_when(
    HAB_1_L4 == HAB_2_L4 & HAB_2_L4 == HAB_3_L4 & HAB_3_L4 == HAB_4_L4 & HAB_4_L4 == HAB_5_L4 ~ HAB_1_L4,
    HAB_1_L4 == HAB_2_L4 & HAB_2_L4 == HAB_3_L4 & HAB_3_L4 == HAB_4_L4 & is.na(HAB_5_L4) ~ HAB_1_L4,
    HAB_1_L4 == HAB_2_L4 & HAB_2_L4 == HAB_3_L4 & is.na(HAB_4_L4) & is.na(HAB_5_L4)  ~ HAB_1_L4,
    HAB_1_L4 == HAB_2_L4 & is.na(HAB_3_L4) & is.na(HAB_4_L4) & is.na(HAB_5_L4)   ~ HAB_1_L4,
    TRUE ~ "not aggregated"
    )) %>%
  select(eunis, duplicate_mosaic_L4) %>%
  filter(duplicate_mosaic_L4 != "not aggregated") %>%
  arrange(duplicate_mosaic_L4)

# make a 'list' of mosaics that can be summarised at L3 
mosaic_duplicate_list_L4 <- pull(L4_mosaic_duplicates, eunis, duplicate_mosaic_L4)

# create a L3 & L4 EUNIS code field, change mosaics that can be aggregated to L3 and L4,
# change those that can't be aggregated to L3 or L4 mosaics, and change any single EUNIS codes to L3 & L4
MR_data_cs <- MR_data_cs %>%
  left_join(select(L3_mosaics, eunis, HAB_mosaic_L3), by = c("eunis")) %>%
  left_join(select(L4_mosaics, eunis, HAB_mosaic_L4), by = c("eunis")) %>%
  mutate(EUNIS_L3_MR = substr(eunis, 1, 4),
         EUNIS_L4_MR = substr(eunis, 1, 5)) %>%
  mutate(EUNIS_L3_MR = if_else(is.na(HAB_mosaic_L3), EUNIS_L3_MR, HAB_mosaic_L3)) %>%
  mutate(EUNIS_L3_MR = if_else(eunis %in% mosaic_duplicate_list_L3, substr(eunis, 1, 4), EUNIS_L3_MR)) %>%
  mutate(EUNIS_L4_MR = if_else(is.na(HAB_mosaic_L4), EUNIS_L4_MR, HAB_mosaic_L4)) %>%
  mutate(EUNIS_L4_MR = if_else(eunis %in% mosaic_duplicate_list_L4, substr(eunis, 1, 4), EUNIS_L4_MR)) %>%
  select(!c(HAB_mosaic_L3, HAB_mosaic_L4))

# remove temporary data
rm(distinct_mosaic, L3_mosaic_duplicates, L3_mosaics, mosaic_duplicate_list_L3, L4_mosaic_duplicates, L4_mosaics, mosaic_duplicate_list_L4)
  • Check after aggregation to EUNIS L3
# Level 3
print(unique(unlist(MR_data_cs$EUNIS_L3_MR)))
##  [1] NA               "A5"             "A5.2"           "A5.4"          
##  [5] "A5.3"           "A5.1"           "A5.3&A5.2"      "A5.2&A5.3"     
##  [9] "A5.1&A5.4"      "A5.4&A5.1"      "A2.2&A2.3"      "A2.3"          
## [13] "A2.4"           "A2.3&A2.2"      "A2.2"           "A4.1"          
## [17] "A4.2"           "A5.2&A5.4"      "A5.4&A5.2"      "A5.2&A5.1"     
## [21] "A5.1&A5.2"      "A5.5"           "A5.6"           "A5.3&A5.3&A5.5"
## [25] "A5.5&A5.3&A5.3" "A5.3&A5.5&A5.3" "A5.5&A5.4"      "A5.4&A5.5"     
## [29] "A5.4&A5.3"      "A5.3&A5.4"      "A5.5&A5.3"      "A5.2&A5.6"     
## [33] "A3.2&A5.5&A5.5" "A5.7"

Join Habitat Data (combined map) to MR Data

  • Create a new habitat field that takes the information from MR and fills in any missing habitat with combined map data (sample_hab)
# spatial join the habitat data to the sample points
MR_data_hab_cs <- st_join(MR_data_cs, hab_cs, suffix=c("",".y"))

# add a new field to give final sample L3 and L4 habitat
MR_data_hab_cs <- MR_data_hab_cs %>%
  mutate(sample_hab_L3 = if_else(is.na(EUNIS_L3_MR), EUNIS_L3, EUNIS_L3_MR),
         sample_hab_L4 = if_else(is.na(EUNIS_L4_MR), EUNIS_L4, EUNIS_L4_MR))

Join Habitat Data (combined map) to OneBenthic Data

  • Spatially join habitat information to sample points
# spatial join the habitat data to the sample points
OB_data_hab_cs <- st_join(OB_data_cs, hab_cs, suffix=c("",".y"))

Join Habitat Data (combined map) to WFD Data

  • Spatially join habitat information to WFD records
# spatial join the habitat data to the sample points
WFD_data_hab_cs <- st_join(WFD_data_cs, hab_cs, suffix=c("",".y"))

Join Habitat Data (combined map) to MERMAN Data

  • Spatially join habitat information to MERMAN records
# spatial join the habitat data to the sample points
merman_data_hab_cs <- st_join(merman_data_cs, hab_cs, suffix=c("",".y"))

Join Habitat Data (combined map) to NRW Data

  • Spatially join habitat information to NRW records
# spatial join the habitat data to the sample points
samples_NRW_hab <- st_join(samples_NRW, hab_cs, suffix=c("",".y"))

Add MSFD Habitat Classification

The MSFD habitats classification from the 2017 Commission Decision is to be used to report results grouped as broad-scale habitat classifications based on the North Atlantic definitions. A translation table was created by habitat classification experts at JNCC and EMODnet to facilitate correlation between EUNIS habitat and MSFD Broad Habitat Types (BHTs). In some instances, translations require combinations of EUNIS code, biological zone and / or substrate information to correctly assign BHT where EUNIS habitats contained or partially overlapped with multiple BHTs. Biological zone and substrate information can be obtained from a spatial intersect with EUSeaMap 2021. EUNIS habitat codes, biological zone, and substrate type information can then be joined in a step-wise process to provide the corresponding MSFD BHT.

The approach to use translations from EUNIS rather than a direct spatial join with EUSeaMap 2021 allows the retention of any EUNIS habitat information attributed to a sample to be retained (Marine Recorder metadata).

  • Load EUSeaMap 2021 (transformed to WGS84, clipped to Celtic Seas, and reduced to the required fields in ArcGIS)
    • Update substrate and biological zone values to match translation table values
  • Join sample records to EUSeaMaps biological zone and substrate variables
  • Load in a prepare the translation table (supplied as an addition Annex csv. document in the BH3 CEMP guidelines ‘Table A1.1’)
  • Prepare the translation table for joining (subset and rename variables)
  • Create a function to join the subset translation table to the data
  • Apply the translations and QC the results
# load EUSeaMap21 (reduced to the necessary fields and clipped to Celtic Seas)
euseamap <- st_read("Data/Processed/EUSeaMap21/EUSM21_Reduced_WGS84_CelticSeas.shp")

# convert factors to character variables, remove shape length & area, remove uppercase
euseamap <- euseamap %>%
  mutate_if(is.factor, as.character) %>%
  select(-c(Shape_Leng, Shape_Area)) %>%
  names_to_lowercase()

# EUSeaMap definitions of substrate and biological zone vary slightly to those needed for translation
# convert the variables to the definitions needed
euseamap <- euseamap %>%
  mutate(substrate = case_when(grepl("Fine mud or Sandy mud or Muddy sand", substrate) ~ "Mud",
                               grepl("Fine mud", substrate) ~"Mud",
                               grepl("Muddy sand", substrate) ~"Mud",
                               grepl("Sandy mud", substrate) ~"Mud",
                               grepl("Coarse substrate", substrate) ~"Coarse sediment",
                               grepl("reef| bed| beds|reefs|rock|biogenic", substrate, ignore.case = TRUE) ~"Rock",
                               TRUE ~ substrate)) %>%
  mutate(biozone = case_when(grepl("Deep circalittoral", biozone) ~ "Offshore circalittoral",
                             grepl("Shallow circalittoral", biozone) ~ "Circalittoral",
                             grepl("upper bathyal|mid bathyal", biozone) ~"Upper bathyal",
                             grepl("lower bathyal", biozone) ~"Lower bathyal",
                             grepl("abyssal", biozone, ignore.case = TRUE) ~"Abyssal",
                             TRUE ~ biozone))

# spatial join EUSeaMap to the sample points
MR_data_hab_cs <- st_join(MR_data_hab_cs, euseamap, suffix=c("",".y"))
OB_data_hab_cs <- st_join(OB_data_hab_cs, euseamap, suffix=c("",".y"))
WFD_data_hab_cs <- st_join(WFD_data_hab_cs, euseamap, suffix=c("",".y"))
merman_data_hab_cs <- st_join(merman_data_hab_cs, euseamap, suffix=c("",".y"))
samples_NRW_hab <- st_join(samples_NRW_hab, euseamap, suffix=c("",".y"))

# remove any upper case variable names
MR_data_hab_cs <- names_to_lowercase(MR_data_hab_cs)
OB_data_hab_cs <- names_to_lowercase(OB_data_hab_cs)
WFD_data_hab_cs <- names_to_lowercase(WFD_data_hab_cs)
merman_data_hab_cs <- names_to_lowercase(merman_data_hab_cs)
samples_NRW_hab <- names_to_lowercase(samples_NRW_hab)

# load in the translation table
translation_table <- read.csv("Data/Raw/C20220801_EUNIS_MSFD_translation.csv")

# rename first column, convert factors to characters, change "" to NA, lowercase and select the fields needed
translation_table <- translation_table %>% 
  rename(EUNIS07code = 1) %>%
  mutate_if(is.factor, as.character) %>%
  mutate_if(is.character, list(~na_if(.,""))) %>%
  names_to_lowercase() %>%
  select(eunis07code, biological_zone, substrate_type, msfd)
# QC translation table 
#slice_sample(translation_table, n = 10)

# sub set translation table for the consecutive joins
bio_sub <- filter(translation_table, !is.na(biological_zone) & !is.na(substrate_type))
bio <- filter(translation_table, !is.na(biological_zone) & is.na(substrate_type))
sub <- filter(translation_table, is.na(biological_zone) & !is.na(substrate_type))
hab <- filter(translation_table, is.na(biological_zone) & is.na(substrate_type) & !is.na(msfd))

# rename variable (1-3) for easy joins with the matching euseamap variable names
bio_sub <- bio_sub %>% 
  transmute(hab_type = .[[1]], biozone = .[[2]], substrate = .[[3]], msfd)
bio <- bio %>% 
  transmute(hab_type = .[[1]], biozone = .[[2]], msfd)
sub <- sub %>% 
  transmute(hab_type = .[[1]], substrate = .[[3]], msfd)
hab <- hab %>% 
  transmute(hab_type = .[[1]], msfd)

# create a function to apply joins in a step-wise way to the datasets
stepwise_join <- function(x) {
  joined_bio_sub <- inner_join(x, bio_sub, by = c("biozone", "substrate", "hab_type"))   # Join on all 3 variables first
  unmatched_bio_sub <- anti_join(x, bio_sub, by = c("biozone", "substrate", "hab_type")) # Take those that don't join to the next step

  joined_bio <- inner_join(unmatched_bio_sub, bio, by = c("biozone", "hab_type"))        # Join on biozone
  unmatched_bio <- anti_join(unmatched_bio_sub, bio, by = c("biozone", "hab_type"))      # Take those that don't join to the next step
  
  joined_sub <- inner_join(unmatched_bio, sub, by = c("substrate", "hab_type"))          # Join on substrate
  unmatched_sub <- anti_join(unmatched_bio, sub, by = c("substrate", "hab_type"))        # Take those that don't join to the next step
  
  joined_hab <- inner_join(unmatched_sub, hab, by = "hab_type")                          # Join on EUNIS code alone
  unmatched_hab <- anti_join(unmatched_sub, hab, by = "hab_type")                        # Take those that don't join to the next step
  # Give a reason for those that don't join
  no_habitat <- mutate(unmatched_hab, msfd = as.character(case_when(hab_type == "Na" ~ "No habitat data",
                                                      is.na(hab_type) ~ "No habitat data",
                                                      TRUE ~ "No EUNIS to MSFD BBHT translation")))
  # bring all the translation back together
  translations <- bind_rows(joined_bio_sub, joined_bio, joined_sub, joined_hab, no_habitat)
  return(translations)
  }

# apply the translations to the datasets 
MR_data_hab_cs_bht <- stepwise_join(MR_data_hab_cs)
OB_data_hab_cs_bht <- stepwise_join(OB_data_hab_cs)
WFD_data_hab_cs_bht <- stepwise_join(WFD_data_hab_cs)
merman_data_hab_cs_bht <- stepwise_join(merman_data_hab_cs)
samples_NRW_bht <- stepwise_join(samples_NRW_hab)

# QC translations
MR_data_hab_cs_bht %>%
  as.data.frame() %>%
  select(hab_type, biozone, substrate, msfd) %>%
  slice_sample(n = 20)

OB_data_hab_cs_bht %>%
  as.data.frame() %>%
  select(hab_type, biozone, substrate, msfd) %>%
  slice_sample(n = 20)

WFD_data_hab_cs_bht %>%
  as.data.frame() %>%
  select(hab_type, biozone, substrate, msfd) %>%
  slice_sample(n = 20)

merman_data_hab_cs_bht %>%
  as.data.frame() %>%
  select(hab_type, biozone, substrate, msfd) %>%
  slice_sample(n = 20)

samples_NRW_bht %>%
  as.data.frame() %>%
  select(hab_type, biozone, substrate, msfd) %>%
  slice_sample(n = 20)

Sample Points Following Intial Processing

We’ll investigate the sample locations and check for duplicates between datasets. First we need to prepare the data for mapping.

  • Group individual records to samples
  • Make a dataset of the MR sample that have been excluded
  • Add MR survey names
  • Combine all dataset samples into a single common data frame
  • To make the data more manageable create five area boxes and crop data to those boxes
  • Make survey name labels
# group to samples
# OneBenthic
OB <- OB_data_hab_cs_bht %>%
  group_by(surveyname, year, samplecode) %>%
  dplyr::summarise(taxa = n())
# WFD data
WFD <- WFD_data_hab_cs_bht %>%
  group_by(surveyname, sampleyear, samplecode) %>%
  dplyr::summarise(taxa = n())
# MR data
MR <- MR_data_hab_cs_bht %>%
  group_by(surveykey, surveyeventkey, sampleyear, samplereference) %>%
  dplyr::summarise(taxa = n())
# MERMAN data
MERMAN <- merman_data_hab_cs_bht %>%
  group_by(sampleref, samplestartdate, monitoringyear) %>%
  dplyr::summarise(taxa = n())

# load in MR survey names
surveyname <- read.csv("Data/Processed/Marine_Recorder/SurveyName.csv")
# fix field name
surveyname <- surveyname %>%
  rename("SurveyKey" = "ï..SurveyKey")
# add survey names to MR samples
MR_surveys <- left_join(MR, surveyname, by = c("surveykey" = "SurveyKey"))

# standardise fields between datasets
MR_surveys <- MR_surveys %>%
  mutate(sample = as.character(samplereference),
         year = sampleyear,
         survey = as.character(SurveyName),
         dataset = "MR") %>%
  select(sample, year, survey, taxa, dataset)

OB_surveys <- OB %>%
  mutate(survey = as.character(surveyname),
         sample = as.character(samplecode),
         dataset = "OB") %>%
  select(sample, year, survey, taxa, dataset)

WFD_surveys <- WFD %>%
  mutate(survey = as.character(surveyname),
         sample = as.character(samplecode),
         dataset = "WFD") %>%
  select(sample, survey, taxa, dataset)

MERMAN_surveys <- MERMAN %>%
  mutate(survey = "MERMAN database",
         sample = as.character(sampleref),
         dataset = "MERMAN") %>%
  select(sample, survey, taxa, dataset)

NRW_surveys <- samples_NRW_bht %>%
  mutate(survey = "NRW database",
         sample = as.character(sampleref),
         dataset = "NRW") %>%
  rename(taxa = s) %>%
  select(sample, survey, taxa, dataset)

# Bind OB, MR, and WFD datasets into single df
survey_samples <- bind_rows(MR_surveys, OB_surveys, WFD_surveys, MERMAN_surveys, NRW_surveys)

# add factor order to dataset names
dataset_order <- c("OB", "WFD", "MR", "MERMAN", "NRW")

survey_samples <- survey_samples %>%
  mutate(datset = factor(dataset, levels = dataset_order))

# group to survey and year
surveys <- survey_samples %>%
  as.data.frame() %>%
  select(-c(geometry)) %>%
  group_by(survey, year, dataset) %>%
  summarise(samples = n())

# create some boxes for a close ups of sample locations (boxes a-e)
areas <- data.frame(x1=c(-1,-4.6,-2.7,-2.6,-2.6), 
                   x2=c(-6.5,-8.3,-6.5,-6.8,-10), 
                   y1=c(58.5,55,52.5,51,48.2), 
                   y2=c(60.9,58.5,55,52.5,51), 
                   t=c('a','b','c','d','e'))

# Change these into polygons
# first add a function that calculates angle with respect to the polygon centroid
# we need this to order the polygons correctly
calc_angle <- function(lon,lat) {
  cent_lon <- mean(lon)
  cent_lat <- mean(lat)
  ang <- atan2(lat - cent_lat, lon - cent_lon)
  
  return(ang)
}

# convert the box areas into polygons 
areas_poly <- areas %>%
  group_by(t) %>%
  summarise(xmin = x1, ymin = y1, xmax = x2,  ymax = y2) %>%
  gather(x,lon,c('xmin','xmax')) %>%
  gather(y,lat,c('ymin','ymax')) %>%
  st_as_sf(coords = c('lon','lat'),crs = 4326, remove = F) %>%
  group_by(t) %>%
  mutate(angle = calc_angle(lon,lat)) %>%
  arrange(angle) %>%
  summarise(do_union=FALSE) %>%
  st_cast('POLYGON')

## uses these boxes to cut the data to the areas

# create functions to cut to the area coordinates
box_a_crop <- function(x) {
  st_crop(x, xmin = areas[1,1], xmax = areas[1,2],
                  ymin = areas[1,3], ymax = areas[1,4])
}
box_b_crop <- function(x) {
  st_crop(x, xmin = areas[2,1], xmax = areas[2,2],
                  ymin = areas[2,3], ymax = areas[2,4])
}
box_c_crop <- function(x) {
  st_crop(x, xmin = areas[3,1], xmax = areas[3,2],
                  ymin = areas[3,3], ymax = areas[3,4])
}
box_d_crop <- function(x) {
  st_crop(x, xmin = areas[4,1], xmax = areas[4,2],
                  ymin = areas[4,3], ymax = areas[4,4])
}
box_e_crop <- function(x) {
  st_crop(x, xmin = areas[5,1], xmax = areas[5,2],
                  ymin = areas[5,3], ymax = areas[5,4])
}

## crop layers to the boxes
# assessment unit area polygons
area_a <- box_a_crop(ukms_cs)
area_b <- box_b_crop(ukms_cs)
area_c <- box_c_crop(ukms_cs)
area_d <- box_d_crop(ukms_cs)
area_e <- box_e_crop(ukms_cs)

# CP2 area polygons
cp2_a <- box_a_crop(cp2_cs)
cp2_b <- box_b_crop(cp2_cs)
cp2_c <- box_c_crop(cp2_cs)
cp2_d <- box_d_crop(cp2_cs)
cp2_e <- box_e_crop(cp2_cs)

# Land area polygons
land_a <- box_a_crop(land_cs)
land_b <- box_b_crop(land_cs)
land_c <- box_c_crop(land_cs)
land_d <- box_d_crop(land_cs)
land_e <- box_e_crop(land_cs)

# samples in each area box
samples_a <- box_a_crop(survey_samples)
samples_b <- box_b_crop(survey_samples)
samples_c <- box_c_crop(survey_samples)
samples_d <- box_d_crop(survey_samples)
samples_e <- box_e_crop(survey_samples)

## Make some label points for the surveys in each box crop box
labels <- function(x) {
  mutate(x, coords = as.character(geometry)) %>%
  select(-c(geometry)) %>%
  group_by(survey, year, dataset) %>%
  dplyr::summarise(geometry = first(coords))
}
# labels for each box
labels_a <- labels(samples_a)
labels_b <- labels(samples_b)
labels_c <- labels(samples_c)
labels_d <- labels(samples_d)
labels_e <- labels(samples_e)

Removal of Duplicates Between Marine Recorder, OneBenthic, and EA WFD

Many of the OneBenthic sample points are duplicates of Marine Recorder or EA WFD samples (overlapping sample points). There is also some duplication between Marine Recorder and the EA WFD dataset

Note: In previous drafts attempts were made to remove any duplicates between the datasets (e.g. removing specific surveys, dataowner, sample identifiers, shared geometries etc.). However, values are not consistent between the datasets (e.g. variations in survey name and coordinates) and there is no single method to remove duplicates. Below is a summary of case by case investigation to remove duplicates.

Filter out duplicate in the OneBenthic and WFD samples by mapping them alongside Marine Recorder samples. Where any potential duplicates were identified these were explored using sample metadata and individual surveys were filtered from the datasets.

  • Plot sample points by dataset origin
    • Filter OneBenthic and WFD duplicates on survey name (survey) to ensure no duplication with MR dataset

Note: plots show sample points following filtering

ggplot() +
  geom_sf(data = land_a, fill = "grey", colour = NA) +
  geom_sf(data = area_a, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_a, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_a, dataset == "MERMAN"), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_a, dataset == "OB", !grepl("draft|MERMAN", survey)), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "a"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange"),
                      breaks=c("OB", "WFD", "MR", "MERMAN"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area A", subtitle = "MERMAN and draft 2011 Marine Scotland Orkney benthic survey removed from OB data") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_a, dataset == "OB", !grepl("draft|MERMAN", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3)

OB survey to be removed:

  • “draft 2011 Marine Scotland Orkney benthic survey” and MERMAN (covered by the MERMAN dataset)

ggplot() +
  geom_sf(data = land_b, fill = "grey", colour = NA) +
  geom_sf(data = area_b, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_b, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_b, dataset == "MERMAN"), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_b, dataset == "OB", !grepl("SNH|Marine Scotland|SAC|MERMAN", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "b"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange"),
                      breaks=c("OB", "WFD", "MR", "MERMAN"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area B", subtitle = "SNH and Marine Scotland OB surveys removed") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_b, dataset == "OB", !grepl("SNH|Marine Scotland|SAC|MERMAN", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3)

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaB_after.jpg"), width = 25, height = 30, units = "cm", dpi = 300)

Additional OB surveys to be removed:

  • SNH, Marine Scotland, SEPA, and SAC surveys
ggplot() +
  geom_sf(data = land_c, fill = "grey", colour = NA) +
  geom_sf(data = area_c, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_c, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_c, dataset == "MERMAN"), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_c, dataset == "OB", !grepl("MERMAN|SEPA|SAC|SNH|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_c, dataset == "WFD"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(samples_c, dataset == "NRW"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "c"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange", "purple3"),
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area C", subtitle = "SAC, SNH, MCZ, 'Mud Hole', NE, CCW, NRW, & WFD OB surveys removed") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_c, dataset == "OB", !grepl("MERMAN|SEPA|SAC|SNH|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3)

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaC_after.jpg"), width = 25, height = 30, units = "cm", dpi = 300)

OB surveys to be removed:

  • MCZ, “Mud Hole”, NE, CCW, NRW, and WFD surveys
ggplot() +
  geom_sf(data = land_c, fill = "grey", colour = NA) +
  geom_sf(data = area_c, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_c, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_c, dataset == "MERMAN"), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_c, dataset == "OB", !grepl("MERMAN|SEPA|SAC|SNH|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_c, dataset == "WFD"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "c"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange"),
                      breaks=c("OB", "WFD", "MR", "MERMAN"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area C", subtitle = "WFD locations close up") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_c, dataset == "WFD"), aes(label = survey, geometry = geometry), 
                  stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3) +
  coord_sf(xlim = c(-3.4, -2.9), ylim = c(53.3, 53.9)) 

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaC_WFD.jpg"), width = 25, height = 30, units = "cm", dpi = 300)

No duplication of WFD samples in area C


ggplot() +
  geom_sf(data = land_d, fill = "grey", colour = NA) +
  geom_sf(data = area_d, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_d, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_d, dataset == "OB", !grepl("MERMAN|SEPA|SAC|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_d, dataset == "OB", grepl("North Celtic deep|South Celtic Deep", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_d, dataset == "WFD"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(samples_d, dataset == "NRW"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "d"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange", "purple3"),
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area D", subtitle = "MCZ*, SAC, NE, CCW, NRW, & WFD OB surveys removed") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_d, dataset == "OB", !grepl("MERMAN|SEPA|SAC|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3) +
  geom_text_repel(data = filter(labels_d, dataset == "OB", grepl("North Celtic deep|South Celtic Deep", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3)

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaD_after.jpg"), width = 30, height = 22, units = "cm", dpi = 300)

OB surveys to be removed:

  • as previous, however need to keep “North Celtic deep_MCZ_infauna” & “South Celtic Deep_MCZ_infauna_MPASCLD2012” MCZ OneBenthic surveys
ggplot() +
  geom_sf(data = land_d, fill = "grey", colour = NA) +
  geom_sf(data = area_d, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_d, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_d, dataset == "OB", !grepl("MERMAN|MM09|SEPA|SAC|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD", survey)), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_d, dataset == "WFD", 
                        !grepl("SEVERN LOWER TW SUBTIDAL BENTHIC INVERTEBRATE WFD SURVEY 2016|Bristol Channel Inner South WFD Benthic Invertebrate Survey 2016", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(samples_d, dataset == "NRW"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "d"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "dimgrey", "purple3"),
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area D", subtitle = "WFD locations close up") +
 # guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_d, dataset == "WFD", 
                                !grepl("SEVERN LOWER TW SUBTIDAL BENTHIC INVERTEBRATE WFD SURVEY 2016|Bristol Channel Inner South WFD Benthic Invertebrate Survey 2016", survey)), aes(label = survey, geometry = geometry), 
                  stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3) +
  coord_sf(xlim = c(-3.6, -2.7), ylim = c(51.2, 51.55)) 

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaD.jpg"), width = 30, height = 22, units = "cm", dpi = 300)

EA WFD surveys to be removed:

  • “SEVERN LOWER TW SUBTIDAL BENTHIC INVERTEBRATE WFD SURVEY 2016” & “Bristol Channel Inner South WFD Benthic Invertebrate Survey 2016”

ggplot() +
  geom_sf(data = land_e, fill = "grey", colour = NA) +
  geom_sf(data = area_e, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_e, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_e, dataset == "MERMAN"), aes(geometry = geometry, colour = dataset), size = 1.5) +
  geom_sf(data = filter(samples_e, dataset == "OB", !grepl("MERMAN|MM09|SEPA|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD|SAC|CSEMP E W Channel Re-Design", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_e, dataset == "OB", grepl("North Celtic deep|South Celtic Deep|South West Deeps|Canyons|MPAGHGF2019", survey)), 
          aes(geometry = geometry, colour = dataset), size = 1.2) +
  geom_sf(data = filter(samples_e, dataset == "WFD"), aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "e"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "darkorange", "purple3"),
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area E", subtitle = "MCZ*, NE, CCW, NRW, & WFD OB surveys removed") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_e, dataset == "OB", 
                                !grepl("MERMAN|MM09|SEPA|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD|SAC|CSEMP E W Channel Re-Design", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3) +
  geom_text_repel(data = filter(labels_e, dataset == "OB", grepl("North Celtic deep|South Celtic Deep|South West Deeps|Canyons|MPAGHGF2019", survey)), 
                  aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3)

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaE_after.jpg"), width = 30, height = 20, units = "cm", dpi = 300)

OB surveys to be removed:

  • as previous plus “CSEMP E W Channel Re-Design” and “2009 National Marine Monitoring Programme MM09-06” (included in MERMAN)
  • however need to keep “The Canyons MCZ 2017_MPACNYN2017”, “South Celtic Deep_MCZ_infauna_MPASCLD2012”, “South West Deeps”, “Canyons”, & “Greater Haig Fras MCZ 2019” OneBenthic surveys.
ggplot() +
  geom_sf(data = area_e, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = filter(samples_e, dataset == "MR"), aes(geometry = geometry, colour = dataset), size = 2) +
  geom_sf(data = filter(samples_e, dataset == "WFD",
                        !grepl("CARRICK ROADS WFD BENTHIC SURVEY 2009|CARRICK ROADS INNER BENTHIC WFD SURVEY 2016", survey)),
          aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "e"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = c("red", "blue", "green4", "dimgrey"),
                      breaks=c("OB", "WFD", "MR", "MR_REMOVED"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MR - removed")) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area E", subtitle = "WFD Carrick Roads samples") +
  guides(color=guide_legend(title="Dataset")) +
  geom_text_repel(data = filter(labels_e, dataset == "WFD", 
                                !grepl("CARRICK ROADS WFD BENTHIC SURVEY 2009|CARRICK ROADS INNER BENTHIC WFD SURVEY 2016", survey),
                                grepl("carrick", survey, ignore.case = T)), 
                  aes(label = survey, geometry = geometry), 
                  stat = "sf_coordinates", size = 2, segment.size = 0.1, segment.alpha = 0.3) +
  coord_sf(xlim = c(-5.08, -5.02), ylim = c(50.21, 50.17)) 

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_OB_MR_AreaE_Carrick_MR.jpg"), width = 30, height = 20, units = "cm", dpi = 300)

WFD surveys to be removed:

  • “CARRICK ROADS WFD BENTHIC SURVEY 2009” & “CARRICK ROADS INNER BENTHIC WFD SURVEY 2016”

  • Both WFD surveys near Plymouth (“Plymouth Outer WFD Benthic Invertebrate Survey 2011” & “Plymouth Outer WFD Benthic Invertebrate Survey 2015”)

No duplicate samples to be removed near Dartmouth

Note, samples in transitional waters are filtered out at later stages


Filter OneBenthic and WFD Surveys

Filter the OneBenthic and WFD datasets to remove the surveys identified as being duplicated.

# subset the mcz etc. agency surveys to keep
OB_data_hab_cs_mczs <- OB_data_hab_cs_bht %>%
  filter(grepl("North Celtic deep|South Celtic Deep|South West Deeps|Canyons|MPAGHGF2019", surveyname))

# filter out the agency surveys to remove
OB_data_hab_cs_filtered <- OB_data_hab_cs_bht %>%
  filter(!grepl("MERMAN|MM09|SEPA|SNH|Marine Scotland|MCZ|Mud Hole|NE|CCW|NRW|WFD|SAC|CSEMP E W Channel Re-Design", surveyname))

# bring together the two subsets
OB_data_hab_cs_bht <- rbind(OB_data_hab_cs_filtered, OB_data_hab_cs_mczs)

# WFD surveys
WFD_data_hab_cs_bht <- WFD_data_hab_cs_bht %>%
  filter(!grepl("SEVERN LOWER TW SUBTIDAL BENTHIC INVERTEBRATE WFD SURVEY 2016|Bristol Channel Inner South WFD Benthic Invertebrate Survey 2016|CARRICK ROADS WFD BENTHIC SURVEY 2009|CARRICK ROADS INNER BENTHIC WFD SURVEY 2016|Plymouth Outer WFD Benthic Invertebrate Survey", surveyname))

# tidy up
rm(OB_data_hab_cs_filtered, OB_data_hab_cs_mczs)
  • Re-group sample data (as done previously but with the now filtered OB dataset)
# group to samples
# OneBenthic
OB <- OB_data_hab_cs_bht %>%
  select(surveyname, samplecode, year) %>%
  group_by(surveyname, year, samplecode) %>%
  dplyr::summarise(taxa = n())

OB_surveys <- OB %>%
  mutate(survey = as.character(surveyname),
         sample = as.character(samplecode),
         dataset = "OB") %>%
  select(sample, year, survey, taxa, dataset)

# WFD
WFD <- WFD_data_hab_cs_bht %>%
  select(surveyname, samplecode, year = sampleyear) %>%
  group_by(surveyname, year, samplecode) %>%
  dplyr::summarise(taxa = n())

WFD_surveys <- WFD %>%
  mutate(survey = as.character(surveyname),
         sample = as.character(samplecode),
         dataset = "WFD") %>%
  select(sample, year, survey, taxa, dataset)

# Bind OB, MR, and WFD datasets into single df
survey_samples <- bind_rows(MR_surveys, OB_surveys, WFD_surveys, MERMAN_surveys, NRW_surveys) 

survey_samples <- survey_samples %>%
  mutate(datset = factor(dataset, levels = dataset_order))

# samples in each area box
samples_a <- box_a_crop(survey_samples)
samples_b <- box_b_crop(survey_samples)
samples_c <- box_c_crop(survey_samples)
samples_d <- box_d_crop(survey_samples)
samples_e <- box_e_crop(survey_samples)

Map Sample Locations

  • Plot the sample points
# overview
ggplot() +
  geom_sf(data = land_cs, fill = "grey", colour = NA) +
  geom_sf(data = cp2_cs, aes(fill = Region), colour = NA, alpha = 0.3) +
  geom_sf(data = ukms_cs, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = survey_samples, aes(geometry = geometry, colour = dataset), size = 0.1) +
  geom_sf(data = areas_poly, mapping = aes(geometry = geometry), fill = NA, lwd = 0.5, ) +
  geom_sf_text(data = areas_poly, aes(label = t), size = 8) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(axis.title.x = element_blank(), axis.title.y = element_blank()) + #legend.position = "none",
  ggtitle("Celtic Seas Areas") +
  guides(color=guide_legend(title= "Dataset", override.aes = list(size=3)), fill =guide_legend(title="CP2 Region")) + 
  ylim(48.19, 63.4) +
  xlim(-14.21, 1.1)

#ggsave(paste0("Outputs/Figures/DRAFT/", creation_date(),"_Overview.jpg"), width = 22, height = 25, units = "cm", dpi = 300)
# labels commented out
ggplot() +
  geom_sf(data = land_a, fill = "grey", colour = NA) +
  geom_sf(data = cp2_a, aes(fill = Region), colour = NA, alpha = 0.3, lwd = 0.5) +
  geom_sf(data = area_a, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_a, aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "a"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area A") +
  guides(color=guide_legend(title="Dataset", override.aes = list(size=3)), fill =guide_legend(title="CP2 Region"))

#  geom_text_repel(data = labels_a, aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, nudge_y = 0.1, segment.size = 0.1, segment.alpha = 0.3)
# labels commented out
ggplot() +
  geom_sf(data = land_b, fill = "grey", colour = NA) +
  geom_sf(data = cp2_b, aes(fill = Region), colour = NA, alpha = 0.3) +
  geom_sf(data = area_b, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_b, aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "b"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area B") +
  guides(color=guide_legend(title="Dataset", override.aes = list(size=3)), fill =guide_legend(title="CP2 Region"))

#  geom_text_repel(data = labels_b, aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, nudge_y = 0.1, segment.size = 0.1, segment.alpha = 0.3)
# labels commented out
ggplot() +
  geom_sf(data = land_c, fill = "grey", colour = NA) +
  geom_sf(data = cp2_c, aes(fill = Region), colour = NA, alpha = 0.3, lwd = 0.5) +
  geom_sf(data = area_c, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_c, aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "c"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(legend.position = "right", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area C") +
  guides(color=guide_legend(title="Dataset", override.aes = list(size=3)), fill =guide_legend(title="CP2 Region"))

#  geom_text_repel(data = labels_c, aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, nudge_y = 0.1, segment.size = 0.1, segment.alpha = 0.3)
# labels commented out
ggplot() +
  geom_sf(data = land_d, fill = "grey", colour = NA) +
  geom_sf(data = cp2_d, aes(fill = Region), colour = NA, alpha = 0.3, lwd = 0.5) +
  geom_sf(data = area_d, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_d, aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "d"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(legend.position = "bottom", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area D") +
  guides(color=guide_legend(nrow=3, byrow=TRUE, title="Dataset", override.aes = list(size=3)), 
         fill=guide_legend(nrow=3, byrow=TRUE, title="CP2 Region"))

#  geom_text_repel(data = labels_d, aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, nudge_y = 0.1, segment.size = 0.1, segment.alpha = 0.3)
# labels commented out
ggplot() +
  geom_sf(data = land_e, fill = "grey", colour = NA) +
  geom_sf(data = cp2_e, aes(fill = Region), colour = NA, alpha = 0.3, lwd = 0.5) +
  geom_sf(data = area_e, mapping = aes(geometry = geometry), lwd = 0.5, colour = "grey20", fill = NA) +
  geom_sf(data = samples_e, aes(geometry = geometry, colour = dataset), size = 1) +
  geom_sf(data = filter(areas_poly, t == "e"), mapping = aes(geometry = geometry), color = "black", lwd = 0.2, fill = NA) +
  scale_colour_manual(values = cb_palette,
                      breaks=c("OB", "WFD", "MR", "MERMAN", "NRW"),
                      labels=c("OneBenthic", "WFD", "Marine Recorder", "MERMAN", "NRW")) +
  scale_fill_manual(values = cp2_palette) +
  theme_bw() +
  theme(legend.position = "bottom", axis.title.x = element_blank(), axis.title.y = element_blank(), panel.border = element_blank()) +
  ggtitle("Area E") +
  guides(color=guide_legend(nrow=3, byrow=TRUE, title="Dataset", override.aes = list(size=3)), 
         fill=guide_legend(nrow=3, byrow=TRUE, title="CP2 Region"))

#  geom_text_repel(data = labels_e, aes(label = survey, geometry = geometry), stat = "sf_coordinates", size = 2, nudge_y = 0.1, segment.size = 0.1, segment.alpha = 0.3)

Save Processed Data

Save the data for processing in the next script: Data Cleaning

# Processed Marine Recorder data
saveRDS(MR_data_hab_cs_bht, paste0("Data/Processed/R_data/", creation_date(), "_MR_data_hab_cs_bht"))
# Processed OneBenthic data
saveRDS(OB_data_hab_cs_bht, paste0("Data/Processed/R_data/", creation_date(), "_OB_data_hab_cs_bht"))
# Processed WFD data
saveRDS(WFD_data_hab_cs_bht, paste0("Data/Processed/R_data/", creation_date(), "_WFD_data_hab_cs_bht"))
# Processed MERMAN data
saveRDS(merman_data_hab_cs_bht, paste0("Data/Processed/R_data/", creation_date(), "_merman_data_hab_cs_bht"))
# Processed NRW data
saveRDS(samples_NRW_bht, paste0("Data/Processed/R_data/", creation_date(), "_samples_NRW_bht"))

Session Info

details::details(sessionInfo(), summary = 'sessionInfo')
sessionInfo

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server x64 (build 14393)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ggrepel_0.9.3      RColorBrewer_1.1-2 details_0.3.0      DT_0.26           
 [5] sf_0.9-8           forcats_0.5.1      stringr_1.4.0      dplyr_1.0.7       
 [9] purrr_0.3.4        readr_2.1.0        tidyr_1.1.4        tibble_3.1.6      
[13] ggplot2_3.3.5      tidyverse_1.3.1    rprojroot_2.0.3    readxl_1.3.1      

loaded via a namespace (and not attached):
 [1] httr_1.4.2         sass_0.4.4         jsonlite_1.7.2     modelr_0.1.8      
 [5] bslib_0.4.1        assertthat_0.2.1   highr_0.9          cellranger_1.1.0  
 [9] yaml_2.2.1         pillar_1.6.4       backports_1.3.0    glue_1.5.0        
[13] digest_0.6.28      rvest_1.0.2        colorspace_2.0-2   htmltools_0.5.2   
[17] clipr_0.7.1        pkgconfig_2.0.3    broom_0.7.10       haven_2.4.3       
[21] scales_1.1.1       tzdb_0.2.0         proxy_0.4-27       generics_0.1.1    
[25] farver_2.1.0       ellipsis_0.3.2     cachem_1.0.6       withr_2.5.0       
[29] cli_3.1.0          magrittr_2.0.1     crayon_1.4.2       evaluate_0.18     
[33] fs_1.5.0           fansi_0.5.0        xml2_1.3.2         class_7.3-15      
[37] tools_3.6.1        hms_1.1.1          lifecycle_1.0.1    munsell_0.5.0     
[41] reprex_2.0.1       compiler_3.6.1     jquerylib_0.1.4    e1071_1.7-11      
[45] rlang_0.4.12       classInt_0.4-3     units_0.8-0        grid_3.6.1        
[49] rstudioapi_0.13    htmlwidgets_1.5.4  crosstalk_1.2.0    rmarkdown_2.18    
[53] gtable_0.3.0       DBI_1.1.3          R6_2.5.1           lubridate_1.8.0   
[57] knitr_1.41         fastmap_1.1.0      utf8_1.2.2         KernSmooth_2.23-15
[61] desc_1.4.2         stringi_1.7.5      Rcpp_1.0.10        vctrs_0.3.8       
[65] png_0.1-7          dbplyr_2.1.1       tidyselect_1.1.1   xfun_0.35